vLLM 0.32: KV Cache, Faster Startup and Breaking Changes

Q: What does the hf tool and reasoning parser do?

Passing --tool-call-parser hf and --reasoning-parser hf makes vLLM read the response_template that ships with a model checkpoint, so models that include one no longer need a model-specific parser. It works in both the Python and Rust front ends.

Last updated: November 2, 2026

vLLM 0.32, released October 18, 2026, is a large update to the open-source inference server. It brings NVFP4 and FP8 KV cache to more NVIDIA GPUs, a generic hf parser for tool calls and reasoning, faster startup, new models and security fixes, but it also drops Python 3.10 and closes default CORS access.

Contents

What’s new in vLLM 0.32
Low-precision KV cache
Faster startup and serving
API and tool-calling changes
New model support
Security fixes
Breaking changes
How to upgrade
Practical workflows
Frequently asked questions

vLLM is the open-source engine many teams use to serve open-weight models behind an OpenAI-compatible API. According to the official v0.32.0 release notes, this release contains 814 commits from 363 contributors, 121 of them new. Everything below comes from those notes and the official vLLM documentation. Performance figures are the benchmarks quoted in the linked pull requests, not independent tests.

What’s new in vLLM 0.32

Area	What changed in 0.32.0
KV cache	NVFP4 KV cache on SM8x, SM90 and SM12x; FP8 KV cache default for GLM-5.x on SM100; 4-bit ultraquant_4bit KV cache on ROCm
Startup	vllm download-kernels, vllm preload –preload-autotune, shared-memory tensor arena on by default, local tokenizer snapshots
Serving	Least-loaded API worker selection when –api-server-count is above 1
Parsers and endpoints	Generic hf tool and reasoning parser; new POST /v1/systemone structured-decision endpoint
Engine	Model Runner V2 gains a GPU n-gram speculator, Elastic EP and a stock torch.compile mode; MRV2 is now the default on ROCm
Models	EmbeddingGemma 2, MiniCPM-V 4.7, Nemotron 3.5 ASR, video input in the Transformers backend
Security	Three GitHub security advisories fixed, CORS closed by default, raster-only image formats for untrusted images
Platform	Python 3.11 to 3.14 only; transformers 5.16.1 or later below 5.20.0, and tokenizers 0.23.2 or later

Low-precision KV cache

The KV cache is usually what limits how many long requests a GPU can hold at once, so smaller cache formats translate directly into more concurrent users or longer contexts. In 0.32, the NVFP4 KV cache runs on SM8x and SM12x GPUs and on SM90 (Hopper), including XQA decode on SM12x.

For GLM-5.x on SM100 (Blackwell), --kv-cache-dtype auto now selects an FP8 KV cache. The release notes say this halves KV memory per rank, with 2.3 to 5.5 percent more throughput at a 7 to 11 percent time-to-first-token cost in the pull request benchmark. If first-token latency matters more than capacity, set the KV cache type explicitly. On AMD ROCm, the new ultraquant_4bit KV cache offers about twice the density of FP8, according to the notes.

Faster startup and serving

Cold starts were a common complaint on newer GPUs. The vLLM quickstart now recommends running vllm download-kernels after install, because it installs precompiled kernels that vLLM would otherwise download and compile at startup, which can add several minutes on Hopper and newer GPUs. A second command, vllm preload --preload-autotune, keeps the FlashInfer autotune table warm across restarts.

When you run more than one API process with --api-server-count, each connection now goes to the least-loaded worker. The notes quote 43 to 85 percent more requests per second on short requests (Qwen3-0.6B, 32 input and 1 output token) in that pull request benchmark. Tokenizers and processors also load from the local model snapshot, and a shared-memory tensor arena, on by default, sends large multimodal tensors to workers with one copy.

API and tool-calling changes

The most useful change for people serving many models is the generic parser. With --tool-call-parser hf and --reasoning-parser hf, vLLM reads the response_template that ships with a checkpoint, so models that include one no longer need a model-specific parser. Other API changes in 0.32.0 include:

max_tokens behavior. max_tokens is now an upper bound on output rather than space reserved from the context window, so long prompts are no longer rejected.
Tool calls. parallel_tool_calls=false is enforced, and tool_choice="required" works with streaming.
Responses API. New min_p and stop_token_ids options, and reasoning stream events renamed to response.content_part.added and response.content_part.done with part.type="reasoning_text".
Structured decisions. A new POST /v1/systemone endpoint answers choice and score questions over a structured state and is served by default.

The server still speaks the OpenAI API at http://localhost:8000 by default, and the OpenAI-compatible server docs explain that --api-key or VLLM_API_KEY turns on key checks, with several keys accepted for rotation.

New model support

New architectures include EmbeddingGemma 2, an omni-modal text, image, audio and video embedding model; MiniCPM-V 4.7; and Nemotron 3.5 ASR for complete-audio transcription, though native realtime streaming for it is not yet supported. The Transformers modeling backend now accepts video input. For background on NVIDIA’s family, see our Nemotron 3.5 guide. GLM-5.3, DeepSeek-V4 and V4.1, Qwen, Kimi-K3 and MiniMax-M3 also received kernel and speculative decoding work; our DeepSeek V4.1 Flash guide covers that model’s API side.

Security fixes

Version 0.32.0 fixes three advisories: GHSA-4hhp-h66f-j5j7 (chat template resource exhaustion, now rendered under a timeout set by VLLM_CHAT_TEMPLATE_RENDER_TIMEOUT), GHSA-q97c-p368-75c4 (a tokenizer max_token_id off-by-one) and GHSA-frvg-qw88-hww9 (P2P ZMQ socket exhaustion, now bounded by VLLM_P2P_MAX_PEERS with a default of 128). Untrusted images are limited to raster formats, and JSON schemas nested more than 64 levels deep return a 400 error.

Breaking changes

Check these before you upgrade a production server:

Python 3.10 is no longer supported; the minimum is Python 3.11.
Default CORS origins changed from every origin to none. Browser clients need --allowed-origins.
Cascade attention was removed; --disable-cascade-attn only warns and will be removed in 0.33.0.
--mamba-cache-mode all now fails validation, and the torch.compile sequence parallelism and AsyncTP passes were removed.
Deprecated APIs such as AsyncLLM.pause_generation() and vllm bench throughput --dataset are gone.

How to upgrade

The release notes and quickstart give this path for a CUDA machine. The vllm 0.32.0 package on PyPI is the same build.

Create a Python 3.11 or newer environment, for example uv venv --python 3.12 --seed, and activate it.
Install the release: uv pip install vllm --torch-backend=auto (or pip install vllm for CUDA 13.0).
Run vllm download-kernels to install precompiled kernels before the first start.
Start the server with vllm serve <model>, adding --allowed-origins if a browser front end calls it directly and --api-key to require a key.
For containers, pull vllm/vllm-openai:v0.32.0, or the -cu129, ROCm, CPU or XPU image for your hardware.
Test tool calls and streaming clients, since reasoning stream event names changed in the Responses API.

Practical workflows

Serving many open models. Try the hf parsers first on models that ship a response template, and keep model-specific parsers only where needed.
Long-context capacity. On supported GPUs, test NVFP4 or FP8 KV cache and compare time to first token against the default before switching production traffic.
Autoscaling. Bake vllm download-kernels and autotune preloading into your image so new replicas start faster.

If you run models on a single workstation instead, compare vLLM with our Ollama new features guide and LM Studio Bionic guide. To put a chat interface in front of a vLLM server, see the Open WebUI 0.12 guide, and for downloading model weights, the hf CLI guide.

Frequently asked questions

What is new in vLLM 0.32?

vLLM 0.32.0, released on October 18, 2026, adds low-precision NVFP4 and FP8 KV cache on more GPUs, a generic hf tool and reasoning parser, faster startup through vllm download-kernels and autotune preloading, least-loaded API worker routing, new models such as EmbeddingGemma 2 and MiniCPM-V 4.7, and several security fixes.

Does vLLM 0.32 still support Python 3.10?

No. vLLM 0.32 dropped Python 3.10. The release notes list Python 3.11 to 3.14 as supported, so upgrade your environment before installing the new wheel.

Why does my browser client fail after upgrading to vLLM 0.32?

Default CORS origins changed from every origin to none in 0.32. Start the server with –allowed-origins set to the browser origins you need, such as the address of your web chat front end.

How do I install or upgrade to vLLM 0.32?

The release notes give pip install vllm for CUDA 13.0, or uv pip install vllm –torch-backend=auto. Docker users can pull vllm/vllm-openai:v0.32.0, with separate tags for CUDA 12.9, ROCm, CPU and Intel XPU.

What does the hf tool and reasoning parser do?

Passing –tool-call-parser hf and –reasoning-parser hf makes vLLM read the response_template that ships with a model checkpoint, so models that include one no longer need a model-specific parser. It works in both the Python and Rust front ends.

Sources

Image: vLLM logo from the vLLM GitHub repository, Apache License 2.0. Placed unchanged on a white background. FreeAnswers is not affiliated with the vLLM project.

vLLM 0.32: Power-User Guide to the October 2026 Release