Custom Silicon Eats Inference
NickyAds@nickyads
· 1
−11.1%
since Oct 2
The idea
I believe AI spending is shifting from training to inference, where cost per token matters most, and hyperscalers will lean on custom chips over general-purpose GPUs to get there. This basket leans into that shift broadly: ASIC designers Broadcom and Marvell anchor it, backed by design software, TSMC's foundry and packaging chain, the interconnect and memory suppliers inference depends on, and a smaller slice of the hyperscalers building their own silicon. No single name dominates, and I keep 8% in cash to add on dips.
What it owns
- Inference workloads at scale are bottlenecked by data movement, making memory bandwidth, optical interconnects, and cluster networking critical to every deployment. · Credo TechnologyIt makes high-speed interconnect chips and cables linking servers inside AI clusters, a requirement for both custom and general-purpose accelerators., Astera LabsIt makes connectivity chips that connect accelerators and memory inside AI servers, creating a critical bottleneck as inference scales., Micron TechnologyIt supplies the high-bandwidth memory sitting adjacent to inference chips, a requirement driven by the billions of daily model serving queries., Arista NetworksIt sells Ethernet switches for AI cluster interconnection, the networking choice preferred by hyperscalers building custom silicon deployments., CoherentIt makes optical components moving data between racks, a cost that grows with every inference cluster deployment regardless of chip source.23%
- Custom inference chips require manufacturing, advanced packaging, and testing regardless of whose design wins, so this layer captures revenue from the entire shift to custom silicon. · Taiwan Semiconductor ManufacturingIt manufactures nearly every custom inference chip regardless of design, earning per-chip revenue from the entire move to custom silicon., Amkor TechnologyIt provides advanced packaging capacity outside TSMC, a critical resource as custom inference chips require sophisticated logic-and-memory packaging., KLA CorporationIt supplies process control equipment for chipmakers, becoming more valuable as complex custom inference chips demand tighter yield control., ASE Technology HoldingIt assembles and tests chips at scale, directly benefiting from rising volumes of custom inference silicon production., CamtekIt sells defect inspection tools for packaging plants, serving the fastest-growing segment of advanced packaging for custom AI chips.23%
- Hyperscalers hire specialized design shops to build their own inference chips rather than rely on general-purpose processors, creating a structural shift in chip spending. · BroadcomIt is the largest designer of custom inference chips for hyperscalers including Google, capturing the shift from general-purpose processors to specialized silicon., Marvell TechnologyIt designs custom inference chips for major cloud providers including Amazon, winning business as hyperscalers build their own silicon.20%
- Every new custom chip project requires design software licenses and reusable building blocks, so growth in in-house chip programs drives recurring revenue regardless of which designs succeed. · Cadence Design SystemsIt supplies design software and simulation tools used in every custom chip project, earning licenses independent of which designs succeed., SynopsysIt sells chip design software and reusable building blocks that every custom silicon project requires, growing with the number of in-house chip programs., Arm HoldingsIt licenses CPU cores to most custom chip designers, collecting royalties as hyperscalers shift to in-house silicon development.14%
- Hyperscalers deploying their own inference silicon capture the cost savings directly in their margins and operational efficiency. · AlphabetIt pioneered the hyperscaler custom chip model with its TPU line and now runs significant inference workloads on its own silicon, demonstrating the business case., AmazonIt built its own inference chip to serve models more cost-effectively than GPUs, with AWS pushing higher volumes of inference onto that silicon., Meta PlatformsIt operates some of the largest inference workloads globally and is deploying its own chips to reduce the per-inference cost.12%
- Cash8%