vLLM 0.33: Power-User Guide to /v1/decisions, CUDA 13 and EFA Images

Last updated: November 16, 2026

vLLM 0.33, released November 2, 2026, is the current stable vLLM release. It adds an OpenAI-compatible /v1/decisions endpoint, upgrades to PyTorch 2.14, publishes CUDA 13.0-only wheels and images, adds an AWS EFA image and a vllm/vllm image alias, and removes Medusa and the MLP speculator.

Contents

What changed in vLLM 0.33

vLLM is an open-source inference and serving engine for large language models. Version 0.33.0 was published to GitHub and PyPI on November 2, 2026. The v0.33.0 release notes say it includes 927 commits from 389 contributors, 122 of them new. A v0.34.0 tag exists in the repository, but as of this update it has no published release notes or PyPI release, so 0.33.0 is the latest stable version.

Area Change in 0.33.0
Dependencies PyTorch 2.14.0, torchvision 0.29.0 and Triton 3.8.0
CUDA builds Wheels and images are CUDA 13.0 only; CUDA 12.9 artifacts are no longer published
Docker images New vllm/vllm-openai:v0.33.0-efa image for AWS EFA; every image is also published as vllm/vllm
API OpenAI-compatible /v1/decisions, token embeddings with optional late chunking, --reasoning-effort-map
Speculative decoding XPress refiner, opt-in FLy lossy verification, quantized draft lm_head
AMD ROCm ROCm 10.0 is the default image and wheel; ROCm 7.2 remains as -rocm72
Reliability Concurrent worker startup with TP above 1 and a watchdog that dumps stack traces from stuck processes

The new /v1/decisions endpoint

The biggest API addition is a text-only implementation of OpenAI’s POST /v1/decisions API. The release notes say it is on by default for supported generation models. According to the vLLM structured decisions documentation, a request carries an input and a list of questions of three types:

  • predicate for yes or no questions, such as whether a customer reports a damaged item
  • choice for picking one option from a list, such as which department should handle a complaint
  • score for placing the input on an ordered scale of levels

The documentation says to use OpenAI Python SDK 3.26.0 or newer for client.decisions.create, pointed at your vLLM server’s base URL. Answers keep question order, probabilities are normalized over the options you supply, and scores are probability-weighted averages of level indices.

Current limits are 1 to 64 questions, 2 to 26 choices and 2 to 10 score levels per request. Requests above those limits return HTTP 400, and unsupported models return 501. Image inputs, tools, streaming and other message roles are outside this first version. The older /v1/systemone structured decisions endpoint, added in vLLM 0.32, keeps its own request format.

How to install or upgrade

The PyPI page for vllm 0.33.0 requires Python 3.11 or newer and below 3.15. The release notes list these install paths:

  1. Create a fresh Python 3.11 to 3.14 virtual environment.
  2. For NVIDIA GPUs, run pip install vllm==0.33.0, or uv pip install vllm==0.33.0 --torch-backend=auto with uv. These wheels target CUDA 13.0.
  3. For AMD GPUs, add --extra-index-url https://wheels.vllm.ai/rocm/0.33.0/rocm100 for ROCm 10.0, or the rocm72 index for ROCm 7.2.
  4. For containers, run docker pull vllm/vllm-openai:v0.33.0, or the shorter vllm/vllm:v0.33.0. Use the -efa tag on EFA-enabled EC2 instances.
  5. Start the server with vllm serve and your model name, then test /v1/chat/completions or /v1/decisions on port 8000.

The vLLM Docker guide notes that containers need --ipc=host or a larger --shm-size so PyTorch can share memory between processes. It also says CUDA 13 images need an R580 or newer host driver for normal use, with a compatibility mode for some datacenter GPUs on older drivers.

Speculative decoding and performance

Version 0.33 adds several speculative decoding options. XPress is a new refiner for block-diffusion drafters such as DFlash and DSpark. FLy is an opt-in lossy verification policy that the release notes report at 1.04 to 1.31x decode throughput on MI355X and B300. DFlash and DSpark no longer reserve draft slots in the token budget, and a quantized draft lm_head option is reported at 18% faster decode at concurrency 1.

On NVIDIA hardware, the release adds opt-in latency-specialized decode on GB300 for Kimi-K3 and DeepSeek-V4.1-Flash, FA4 sparse MLA decode on Blackwell, and FlashInfer prefill changes that cut mean time to first token by 39 to 48% at 100k input on 8xB200 in the project’s benchmark. On AMD, --monokernel replaces the old VLLM_ROCM_MONO_DECODE variable, and a new 3 to 4 bit ROCM_OCTAVE KV cache is available. These figures come from individual pull request benchmarks, so measure your own workload before relying on them.

New model support

New architectures include Reflection Beam-1.0 (501B MoE, 23B active) with its DFlash2 drafter, Yandex AliceAI-Foundation-80B-A3B, Recursive Flint, Muse Spark Open, Step-5 Preview, NVIDIA Alpamayo 2 Super, Nemotron Parse 3.5, MiniCPM-OCR 1.0 and Domyn Edge. GLM-5.3-Flash now defaults to an FP8 KV cache. If you run open-weight models such as Inkling 2 Small, check the model card for the minimum vLLM version it needs.

vLLM 0.33 compared with 0.32

Topic vLLM 0.32 (October 18, 2026) vLLM 0.33 (November 2, 2026)
Decisions APIs Added /v1/systemone Adds OpenAI-compatible /v1/decisions
CUDA 12.9 artifacts Published as -cu129 images and wheels No longer published
Image names vllm/vllm-openai, plus -ubuntu2404 tags Adds vllm/vllm alias and -efa image
Speculative decoding GPU n-gram speculator in Model Runner V2 XPress and FLy added; Medusa and MLP speculator removed
Python Python 3.10 dropped PyPI lists Python 3.11 to 3.14

The 0.32 details come from the v0.32.0 release notes. For background on that release, see our vLLM 0.32 guide. If you are choosing a serving engine, compare it with SGLang 0.5.23, or with llama.cpp 1.0.0 and Ollama for local machines.

Breaking changes to check

  • Medusa and the MLP speculator were removed.
  • The legacy Ray compiled-graph executor was removed; RayExecutorV2 is the only Ray executor.
  • Requests sent while generation is paused now return HTTP 503.
  • The NIXL connector requires NIXL 1.5 or newer.
  • The Rust frontend rejects unsupported vllm serve flags and request fields instead of ignoring them, and an empty tools array returns 400.
  • --mamba-cache-mode is deprecated, --ir-op-priority is scheduled for removal in 0.34.0, and the LM Format Enforcer backend becomes opt-in in 0.34.0.

Limitations

  • Decisions MVP. The documentation describes /v1/decisions as a first version that is text-only, and it says confidence values are not calibrated to OpenAI’s models.
  • Driver requirements. Moving to CUDA 13.0-only builds means older host drivers may need an upgrade or compatibility mode.
  • Benchmarks. Speed gains are reported for specific models, GPUs and settings, not as general results.

Frequently asked questions

What is new in vLLM 0.33?

vLLM 0.33.0, released November 2, 2026, adds an OpenAI-compatible /v1/decisions endpoint, moves to PyTorch 2.14, ships CUDA 13.0-only wheels and images, adds an AWS EFA image and a shorter vllm/vllm image name, and brings new speculative decoding options and new model support.

Does vLLM 0.33 still support CUDA 12.9?

No prebuilt artifacts. The release notes say CUDA 12.9 wheels and the -cu129 and -cu129-ubuntu2404 images are no longer published, and recommend the default CUDA 13.0 artifacts instead.

What is the vLLM /v1/decisions endpoint?

It is a text-only implementation of OpenAI's POST /v1/decisions API. It answers predicate, choice and score questions about an input and is on by default for supported generation models. The docs list a limit of 1 to 64 questions per request.

Which Python versions does vLLM 0.33 support?

The PyPI listing for vllm 0.33.0 requires Python 3.11 or newer and below 3.15. Python 3.10 support was already dropped in vLLM 0.32.

What breaks when upgrading from vLLM 0.32 to 0.33?

Medusa and the MLP speculator are removed, the legacy Ray compiled-graph executor is gone, requests sent while generation is paused return HTTP 503, the NIXL connector needs NIXL 1.5 or newer, and the Rust frontend now rejects unsupported flags.

How do I install vLLM 0.33?

Run pip install vllm==0.33.0, or uv pip install vllm==0.33.0 –torch-backend=auto. For containers, pull vllm/vllm-openai:v0.33.0 or the alias vllm/vllm:v0.33.0.

Sources

Featured image: vLLM logo from the vLLM GitHub repository, Apache License 2.0. FreeAnswers is not affiliated with the vLLM project.

Related answers

Leave a Comment