vLLM 0.29 makes Model Runner V2 the default for every model

vLLM shipped v0.29.0 on the morning of September 9th with 594 commits from 277 contributors, 91 of them new. Which line in those highlights should worry you? Not any of the new models. It is this one: Model Runner V2 is now the default for all models.

That is a change of engine, not a feature.

MRV2 has been rolling out since pooling models moved over, and 0.29 finishes the job. It arrives with CUDA graph memory profiling for KV cache auto-sizing, batch-sharded sampling that cuts per-step logits memory by 1/TP, prompt embeds, and padded FULL cudagraph dispatch for uniform decode under speculative decoding. And the old runner survives only for a handful of ROCm models and the features MRV2 has not caught up on yet (the notes do not say which ones). If you run vLLM on AMD, that sentence is probably the one to read twice.

The rest of the breaking list is short and expensive:

1. Ten deprecated model architectures removed. Not renamed. Removed.
2. FlexOlmo, Olmo3 and Hunyuan V1/VL now run through the Transformers modeling backend instead of native implementations.
3. The PyAV video decoder backend is gone.
4. python -m vllm.entrypoints.openai.api_server is deprecated in favour of vllm serve.

That last one is the quiet tax. Nearly every Dockerfile, Slurm script and tutorial written before this summer starts the server the old way:

# deprecated in 0.29.0
python -m vllm.entrypoints.openai.api_server --model openbmb/MiniCPM5-2B

# the supported form
vllm serve openbmb/MiniCPM5-2B

But there is a better change buried further down, and it is the sort of thing that only sounds boring. Prefix-cache NONE_HASH is now deterministic by default, so anyone running a distributed KV cache no longer has to pin PYTHONHASHSEED across every worker to get cache hits. If you have ever debugged a cluster where one node quietly refused to reuse anybody else's prefixes, you probably know how many hours that one environment variable has cost the field. FlashInfer all-reduce is also on by default for TP CUDA groups now, with VLLM_ALLREDUCE_USE_FLASHINFER=0 to back out.

The cadence is the context nobody puts in the notes, so we counted it. Six minor releases since late June, spaced 12, 16, 14, 16 and 14 days apart.

So that is roughly 42 commits a day landing in the serving layer under most open-weights inference. The previous release carried 584 commits from 270 contributors, which makes 0.29 not an unusual bulge (it is the ordinary rate, with a default flip in it).

Our read: vLLM's release notes have become an upgrade hazard list wearing a feature list's clothes, and the project does not present them that way. A pip install -U vllm in production this week changes the runner executing your model, the launcher your scripts call, and possibly whether your architecture still exists. Which is a fair thing for a fast-moving project to do at 0.x, and probably unavoidable. It is not a fair thing for a reader to meet in paragraph nine of the highlights.

So here is a falsifiable one. Two of the last five minor releases needed a patch within three days — 0.25.1 landed three days after 0.25.0, 0.27.1 one day after 0.27.0 — and this is the largest default change of the year. We expect a 0.29.1 before September 23rd. If it does not come, the MRV2 rollout was genuinely as careful as the phased plan suggests, and we will say so.

What is coming after is more interesting than what shipped. On September 8th the project showed Hybrid HiSparse, which lets a request keep decoding after its KV cache stops fitting in HBM by spilling its coldest pages to host memory. On one 8×H200 node running GLM 5.3 at full 1M context, plain KV offloading held 5-6 concurrent requests; HiSparse held 19-25. It is planned for v0.30 (a fortnight away, if the cadence holds).

vLLM@vllm_project

Sparse MLA only attends to the top-K tokens, so the rest of the KV need not live on the GPU. Hybrid HiSparse in vLLM builds on that, and a request keeps decoding after its KV stops fitting in HBM.

It keeps KV on the GPU while there is room. Under pressure a request releases its coldest pages to host memory, keeps a small hot buffer of what the indexer asks for, and keeps decoding instead of being preempted.

📊 Demonstrated on GLM 5.3, one 8× H200 node, full 1M context. Same host memory, configured concurrency 32: KV offloading kept 5-6 requests running. Hybrid HiSparse kept 19-25.

🔹 Hot pages are ordinary KV blocks from the same pool (Hybrid Memory Allocator)
🔹 One fused kernel resolves resident, hot and missing rows, CUDA-graph capturable
🔹 Prefix caching, OffloadingConnector, P/D imports and MTP keep working

Built by @RedHat_AI and @PrimeIntellect with the vLLM community. Planned for v0.30; pinned commit, flags and calculator are in the post👇
🔗 t.co/6y7u5Hfc8c

Fourteen days, then.