Agents Run on Inference
Rafaborrajo@bor_torres
The idea
I believe AI agents are changing the shape of compute demand: instead of one quick reply, they loop through many model calls, tool use, and self checks, so each task burns far more inference than a chatbot ever did. This take owns the hardware chain that absorbs that load, platforms like Nvidia and AMD, custom inference silicon, networking and optics, memory for long context, foundry and packaging, and the CPUs handling agent tool calls.
What it owns
- Inference clusters require fast data movement between chips, making switches, connectors and lasers a key constraint. · Arista Networks, Inc.It sells the Ethernet switches that move data between thousands of chips in AI clusters, and agent-scale inference needs much more of that traffic., Astera Labs, Inc.It makes connectivity chips that let GPUs, CPUs and memory inside an AI server communicate at full speed., Credo Technology Group Holding LtdIt makes the active copper cables and signal chips that wire AI servers together inside a rack, a low-power link agents' traffic depends on., Lumentum Holdings Inc.It makes the lasers inside data center optical links, and those lasers are among the tighter supply spots as AI clusters grow., Coherent Corp.It makes the optical transceivers that convert electrical signals into light so data can move between racks in AI data centers.25%
- These chipmakers run most AI inference today and serve as steady ballast for the take. · Advanced Micro Devices, Inc.It sells both AI GPUs for inference and EPYC server CPUs that run agents' tool calls, capturing value on two layers of agent demand., NVIDIA CorporationMost AI models run on its chips and software, so when agents multiply model calls per task, a large share of inference spending lands here first.16%
- Hyperscalers design their own cheaper inference chips as agent workloads grow, and these companies build them. · Broadcom Inc.It designs custom inference chips hyperscalers build to cut per-task costs, plus the networking chips that link them., Marvell Technology, Inc.It builds custom AI chips and the optical signal chips inside data center links for the largest cloud companies.15%
- Long agent sessions require huge amounts of context stored in memory, making memory and storage chips scarcer and more expensive. · Micron Technology, Inc.Every AI accelerator ships with high-bandwidth memory stacks, and agents holding long context in memory push that need higher., Sandisk CorporationLong agent sessions spill context and data out of memory onto fast flash storage, raising how much flash each AI server needs.14%
- Nearly every inference chip, regardless of designer, gets manufactured and packaged by these companies. · Taiwan Semiconductor Manufacturing Company LimitedIt manufactures nearly every advanced AI chip in this take and does much of the advanced packaging that connects chips to memory., Amkor Technology, Inc.It is the largest outside company doing advanced chip packaging, the step that joins AI chips with memory and is often a bottleneck.13%
- Agents rely on server processors to run tools and code between model calls, lifting overall CPU demand. · Arm Holdings plcAgents spend much of each task calling tools and running code, which is CPU work, and newer server CPUs are increasingly built on Arm designs that pay it royalties., Intel CorporationIts Xeon chips are still the most installed server CPUs, so more agent tool calls mean more demand for the machines it already serves.9%
- Cash8%