Somebody outside Google finally measured a TPU, and the headline is not the interesting number
SemiAnalysis puts Ironwood ahead of Blackwell on cost per token, right up until you serve inference the way large operators actually serve it.
For more than a decade, everybody who could tell you what a TPU costs per token worked at Google.
That ended on 7 September, when SemiAnalysis published the first third-party inference results for Google's TPUv7 Ironwood. At an interactivity of 100 tokens per second per user, Ironwood comes out at $0.181 per million total tokens. A Nvidia B200 on the same runs costs $0.222, and a B300 costs $0.276.
Turn the dial down to 20 tokens per second per user and Ironwood leads on raw throughput too, at 9,364 tokens per second per chip, roughly 5 percent above either Blackwell part.
Those ratios will move. The existence of the measurement will not.
The promotional post undersells its own result
An alert SemiAnalysis put out the next morning says Ironwood "achieves 50% better perf per dollar than Blackwell Ultra." Read the article and that 50.4 percent is measured against the B200. Against Blackwell Ultra itself, the same run shows 96.0 percent.
So the post names the wrong chip and halves its own headline in the process. We mention it because that is the figure people will repeat for a year, and somebody ought to say which comparison it came out of.
The part that matters more is how big operators actually serve
Every number above is aggregated serving against aggregated serving, on Qwen3.5 397B in FP8, chosen as a bring-up model. Large operators do not run that way. Instead they split the prefill and decode stages across separate pools of chips, which the trade calls disaggregated serving, and SemiAnalysis is direct about what happens there: "today, GB200/300 NVL72 is more competitive on perf per dollar in a disagg-vs-disagg comparison."
In the mixed comparison they do publish, a GB300 NVL72 running disaggregated holds about a 30 percent advantage on performance per dollar over TPUv7 running aggregated, through the middle of the latency curve.
And there is a latency cost, which is not small. At a concurrency of 256, mean time to first token on the TPU is 5.41 seconds, more than double what Blackwell Ultra managed in the same runs.
So which chip is cheaper? Depends where on the curve your product lives. That is the honest answer, and it is also why vendor benchmarks pick one point on the curve and stop there (the point is never chosen at random). If you serve a chat box where the first token has to land inside three seconds, the Ironwood advantage at that concurrency is not available to you at all.
This is a software release wearing a silicon headline
TorchTPU lets developers treat a TPU as a native PyTorch device instead of translating through JAX, with StableHLO, XLA and TPU-tuned Pallas kernels underneath. The stack is expected to leave private beta and be open sourced around October, which is the actual news here.
TPUv7 has no native FP4 arithmetic, so Nvidia keeps the lead wherever a customer will accept the quality loss FP4 brings. Google's answer is TPUv8i, which does not yet exist in any form anyone outside the company can benchmark.
The conflict is worth stating plainly
Dylan Patel's own announcement of the work shouts out "Google @inferact and the InferenceX team at SemiAnalysis." Google supplied the stack under test. And the cost figures come from SemiAnalysis's own subscription model rather than from anybody's invoice.
None of that is a scandal. You cannot benchmark a chip nobody will sell you without help from the seller (ask anyone who has tried). But it is a long way from an independent lab, and anybody quoting that cost figure should hold it a little loosely.
Our expectation is that the promised disaggregated-versus-disaggregated follow-up, whenever it lands, still shows Nvidia ahead through the middle of the Pareto curve, and that the gap in Google's favour survives only at the high-latency end. A first published disaggregated run where the TPU matches Nvidia across the whole curve would settle it against us.
Anthropic has committed to more than a million of these chips, most of them for training. Somebody at Anthropic already knew all of this.
The rest of us finally have a number to argue with.
