llama.cpp logo banner with the LLaMA wordmark and an orange C++ llama icon on a dark background, for the llama.cpp 1.0.0 guide

llama.cpp 1.0.0: Power-User Guide to the Stable API, FP8 KV Cache and Server Security

by

Last updated: November 13, 2026

llama.cpp 1.0.0, released November 13, 2026, is the first semantically versioned release of the local inference engine. It removes the deprecated C API, adds FP8 KV caches, a MoE expert cache and external shard loading, hardens llama-server, adds an OpenAI-compatible speech endpoint and ships a redesigned web UI.

Our earlier llama.cpp 0.9.0 guide covered the November 4 release. This guide covers only what changed in 1.0.0 and what to check before you upgrade.

Contents

What is new in 1.0.0

The v1.0.0 release notes group the changes as follows.

Area What changed in 1.0.0
Public API Deprecated functions removed; NextN (speculative) API promoted from llama-ext.h to llama.h
KV cache OCP FP8 caches with -ctk/-ctv f8_e4m3, with optional static per-tensor scales
MoE models Host-memory expert cache with a configurable CPU share for cache misses
Model files External PLE/engram loading from separate GGUF shards via --shard or --ple
New models Kimi-K3.1-Pro, Ripple-65B-Preview, xing4_0 text models; drama-3 TTS; MageVL-Cowork and Kimi-K3.1 vision
Server SSRF, CORS and DNS rebinding fixes, MCP tool cancellation, new /v1/audio/speech endpoint
Web UI Chat groups, foldable sidebar, live-preview markdown editor, native phone layout
ggml Updated to ggml v1.0.0

The Hugging Face announcement says llama.cpp now supports more than 150 architecture variants and all mainstream speculative decoding methods (Hugging Face blog).

The new versioning rules

The practical headline is stability. Until now, downstream apps had to track a fast-moving library with frequent interface changes. From 1.0.0, both llama.cpp (libllama) and ggml (libggml) signal API and ABI incompatibilities with major-version bumps.

Bump Meaning
Major Breaking change to the public C API in ./include
Minor Backward-compatible feature, backend update, model support or API addition
Patch Bug fix with no API change

Numbered nightly builds such as b12366 continue alongside the tagged release. The blog lists distribution through pre-release artifacts for Linux, macOS, Windows, Android and iOS, plus Docker, Debian, Homebrew, conda-forge, MacPorts, Winget and an XCFramework for Apple platforms.

Breaking API changes

If you embed llama.cpp in your own code, review these items from the release notes before you rebuild:

  • llama_set_warmup, 35 deprecated functions and defrag_thold are removed from llama.h, along with the --defrag-thold flag.
  • llama_set_embd_state_src is renamed to llama_set_ctx_state_src.
  • llama_context_params gains moe_cache_size and moe_cache_cpu_share.
  • clip.h is removed, and the multimodal model code is consolidated into mtmd.
  • mtmd-helper.h adds mtmd_helper_resampler for streaming raw PCM at the model’s sample rate.

Projects that only call the command-line tools or the server API are less affected, but scripts that still pass --defrag-thold need updating.

FP8 KV cache, MoE cache and shards

FP8 KV cache. The server documentation now lists f8_e4m3 as an allowed value for -ctk and -ctv, next to f16 (the default), bf16 and the quantized types. The same values apply to the draft model’s cache flags for speculative decoding.

MoE expert cache. For mixture-of-experts models whose experts stay in system memory, --moe-cache-mib N sets a GPU cache in MiB for those experts (default 0, disabled). --moe-cache-cpu-share F, from 0 to 1, sets the share of cache misses the CPU computes instead of uploading experts to the GPU once the cache is full. The release notes add LRFU and ClassLRU replacement policies, and the CPU no longer waits on cached experts.

External shards. --shard (alias --ple) loads extra GGUF shards from any path, including a different drive, and can be given more than once. This lets per-layer embedding or engram files live apart from the main model file.

Windows also gets an unbuffered model load path that mirrors the Linux DirectIO path.

llama-server security and speech

Version 1.0.0 tightens what a networked llama-server exposes. Two new opt-in flags reverse the stricter defaults:

  • --allow-unauthenticated-tools allows server tools, MCP servers or the MCP proxy on a non-loopback address without an API key. The documentation says to use it only on trusted networks.
  • --media-allow-private-urls allows media URLs such as image_url that point to local or private addresses. By default, only public addresses are fetched.

The release also fixes --cors-origins handling, DNS rebinding and a CORS proxy SSRF issue, restricts HF_TOKEN to huggingface.co hosts, and cancels MCP tool calls when a client disconnects or times out.

The new POST /v1/audio/speech endpoint follows the OpenAI Speech API for models that generate audio. Per the server documentation, it returns WAV by default (not MP3) or raw PCM, keeps the model’s native sample rate, accepts only speed 1.0 and ignores instructions. OpenAI voice names such as alloy are not mapped; use a voice listed by GET /props.

How to upgrade safely

  1. Read the release notes and note which removed functions or flags your code or scripts use.
  2. Remove --defrag-thold from launch scripts and replace llama_set_embd_state_src calls with llama_set_ctx_state_src.
  3. If you build from source, check out the v1.0.0 tag; if you use prebuilt binaries, the matching nightly is b12359.
  4. If llama-server listens on a non-loopback address with tools or MCP enabled, set an API key rather than reaching for --allow-unauthenticated-tools.
  5. Test -ctk f8_e4m3 -ctv f8_e4m3 on one model with a fixed prompt and context length, and compare output quality and reported memory against your current cache type.
  6. For large MoE models offloaded to the CPU, start with a modest --moe-cache-mib value and adjust --moe-cache-cpu-share while watching throughput.
  7. Pin the major version in your package or build pipeline so a future 2.0.0 cannot break you unannounced.

Apps built on llama.cpp, such as Ollama and LM Studio Bionic, ship their own builds, so these changes reach them on their own schedules. For newly supported models, see our Ripple-65B Preview guide. For server-class serving, compare vLLM 0.32.

Frequently asked questions

When was llama.cpp 1.0.0 released?

The ggml-org team published llama.cpp v1.0.0 and ggml v1.0.0 on November 13, 2026, with an announcement on the Hugging Face blog. The release notes list nightly build b12359 as the matching asset.

Is llama.cpp 1.0.0 a breaking change?

Yes, for code that links against libllama. The release removes llama_set_warmup, 35 deprecated functions and the defrag_thold setting, renames llama_set_embd_state_src and removes clip.h. Users of the command-line tools mainly need to drop –defrag-thold.

What does the version number mean from now on?

Both llama.cpp and ggml now follow semantic versioning. A major bump marks a breaking change to the public C API, a minor bump adds backward-compatible features or model support, and a patch bump fixes bugs without an API change.

How do I use the FP8 KV cache in llama.cpp?

Pass -ctk f8_e4m3 and -ctv f8_e4m3 to llama-server or llama-cli. The server documentation lists f8_e4m3 alongside f16, bf16, q8_0 and other cache types, with f16 still the default.

Does llama-server 1.0.0 support text to speech?

Yes. It adds an OpenAI-compatible POST /v1/audio/speech endpoint for models that can generate audio. It returns WAV by default or raw PCM, accepts only speed 1.0, and does not map OpenAI voice names such as alloy.

Sources

Related answers

Leave a Comment