llama.cpp 1.0.0 Upgrade: Breaking Changes, FP8 KV Cache, and Server Security
Upgrade llama.cpp and ggml to 1.0.0 without breaking local coding agents. Remove obsolete C APIs and defragmentation flags, test FP8 KV cache memory savings, tune host-memory MoE expert caching, and protect llama-server tool access and private URLs. Learn what semantic versioning covers and when Ollama or LM Studio users need to act.
Contents

llama.cpp 1.0.0 is available as of November 14, 2026, but developers should check their C bindings, launch flags, and server security settings before upgrading. The November 13 release removes deprecated APIs, adds FP8 KV caching and new MoE expert-cache controls, and introduces security opt-ins for server tools and private media URLs. If you run a local coding model through llama-server, upgrade a staging instance first and verify both ordinary completion and tool calls.
The release also introduces a versioning commitment for libllama and libggml. Breaking public C API changes will now require a major-version increase.
That makes future dependency upgrades easier to assess. It does not guarantee that model output, performance, GPU backend behavior, or every server endpoint will remain identical.
What changed in llama.cpp 1.0.0?
The release affects developers embedding llama.cpp, operators running llama-server, and developers optimizing local inference memory.
| Area | Change in 1.0.0 | Required action |
|---|---|---|
| Versioning | Public C API and ABI incompatibilities use major versions | Pin a release or commit in production |
| C API | 35 deprecated functions and llama_set_warmup removed | Rebuild and migrate affected bindings |
| Context configuration | defrag_thold and --defrag-thold removed | Delete obsolete settings |
| Speculative decoding | NextN API moved into llama.h; one state function renamed | Update includes and function references |
| KV cache | f8_e4m3 support for K and V caches | Benchmark memory, quality, and backend support |
| MoE inference | GPU expert cache gains CPU execution for cache misses | Check GPU availability and tune cache size and CPU share |
| Server security | Tool authentication and private-media URL restrictions | Review network exposure and authentication |
| Model support | Kimi-K3.1-Pro, Ripple, xing4_0, and multimodal additions | Check model conversion and loading requirements |
| Speech API | OpenAI-compatible /v1/audio/speech endpoint | Test with a supported speech model |
These changes are documented in the v1.0.0 release notes and the project’s official versioning announcement.
The project reports support for more than 150 model architecture variants. That is architecture coverage, not a guarantee that every checkpoint or feature works on every backend.
What does semantic versioning mean for future upgrades?
The versioning commitment applies to public C interfaces under ./include, including those exposed by llama.cpp and ggml.
The maintainers define three version changes:
| Version component | Meaning | Example |
|---|---|---|
| MAJOR | Breaking change to the public C API | 1.x.x to 2.0.0 |
| MINOR | Backward-compatible features, new backends, models, or API additions | 1.0.0 to 1.1.0 |
| PATCH | Bug fix without a public API change | 1.0.0 to 1.0.1 |
The examples illustrate versioning rules, not announced future releases.
For applications built against libllama, this reduces the risk of unexpected public function-signature changes within a major version.
It does not promise that internal headers, undocumented symbols, CLI arguments, HTTP response details, or model behavior follow the same compatibility contract.
How should you pin llama.cpp?
Use an exact release tag or immutable commit when compiling llama.cpp into an application.
The official build guide documents the Git clone and CMake build process. Combine those instructions with the v1.0.0 release tag.
For example, in Bash:
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch --tags
git checkout v1.0.0
cmake -B build
cmake --build build --config Release
The CMake commands follow the project’s documented CPU build procedure. Platform defaults may enable additional backends. Check your configuration before assuming a particular runtime device.
The official server documentation identifies ./build/bin/llama-server as the server binary path for the standard build. Platform-specific generators can use different output layouts.
Add the documented CMake options for your GPU backend when needed.
For production, record the release tag, resolved Git commit, build options, and runtime backend. Test each dependency update before replacing the working binary.
If your application bundles llama.cpp, pin the wrapper or application release too. Its version may differ from the upstream llama.cpp revision.
Which APIs and command-line flags were removed?
The most immediate migration problem is the removal of deprecated C interfaces and the defragmentation threshold setting.
The API cleanup associated with PR #30565 removes 35 deprecated functions, llama_set_warmup, and the old defragmentation threshold configuration.
Other migration changes include:
- Remove calls to
llama_set_warmup. - Remove references to
defrag_tholdin context configuration. - Remove
--defrag-tholdfrom launch scripts. - Replace
llama_set_embd_state_srcwithllama_set_ctx_state_srcwhere applicable. - Update NextN/speculative decoding includes and declarations now exposed through
llama.h.
The 35-function removal is a migration category, not a list of 35 independent replacements. Use the tagged header changes to identify each symbol used by your binding.
Fix scripts using —defrag-thold
A previous server command might contain:
./build/bin/llama-server \
-m /path/to/model.gguf \
-c 32768 \
--defrag-thold 0.1
Remove the obsolete option:
./build/bin/llama-server \
-m /path/to/model.gguf \
-c 32768
The important change is removing the parameter, not selecting another threshold value. Do not replace --defrag-thold with an invented equivalent.
Search shell scripts, Docker configurations, service units, and application launch settings:
grep -RInE \
'defrag[_-]thold|llama_set_warmup|llama_set_embd_state_src' \
./scripts ./config ./src
Adjust the searched directories to your project. If a path does not exist, remove it from the command.
Also check environment variables or wrapper settings that still generate the removed argument.
Update C and Python bindings
If you compile directly against llama.h, compare your code with the v1.0.0 headers before rebuilding.
For CMake consumers, rebuild the dependent application against the new library and check the compiler and linker output.
For Python wrappers, first check whether the wrapper version explicitly supports llama.cpp 1.0.0. Installing a newer llama.cpp binary does not automatically update a Python binding’s compiled definitions.
If the wrapper bundles its own llama.cpp source, follow its dependency release process rather than replacing its shared library manually.
A successful import is only an initial check. Load a model, generate tokens, and test any context-state or speculative-decoding operations your application uses.
How do you enable FP8 KV cache in llama.cpp 1.0.0?
Use -ctk f8_e4m3 -ctv f8_e4m3 to request FP8 storage for the attention key and value caches.
The KV cache holds information from previous tokens so the model can continue generating without recomputing the full context. Larger context windows consume more KV memory.
FP8 uses eight bits per stored value instead of the 16 bits used by FP16. For equivalent tensor dimensions, this can approximately halve raw K/V storage.
That estimate excludes metadata, scaling information, alignment, model weights, compute buffers, and other runtime allocations.
Start with a local FP16 baseline
Use the same model and context size for both runs.
./build/bin/llama-server \
-m /path/to/model.gguf \
-c 32768 \
--host 127.0.0.1 \
--port 8080 \
-ctk f16 \
-ctv f16
Record the reported KV buffer allocation, available device memory, prompt processing speed, and generation speed.
Then stop the server and run:
./build/bin/llama-server \
-m /path/to/model.gguf \
-c 32768 \
--host 127.0.0.1 \
--port 8080 \
-ctk f8_e4m3 \
-ctv f8_e4m3
The release introduces FP8 KV caches with optional static per-tensor scales.
Check the startup logs for the selected cache types and allocation sizes. A recognized command-line value does not establish that your GPU backend supports every optimized execution path.
Does FP8 allow twice the context length?
Not necessarily.
FP8 reduces KV tensor storage, but the context window remains subject to the model’s supported context length and available memory.
For a hypothetical workload where FP16 KV consumes 8 GiB, an FP8 representation could reduce raw tensor storage to approximately 4 GiB.
This is a theoretical memory calculation, not a measured llama.cpp result.
The freed memory might allow a larger context or more concurrent requests. It cannot increase the model’s effective context capability beyond supported limits.
FP8 is also a lower-precision representation. The release documentation does not establish a universal coding-quality or tool-calling accuracy result for FP8 KV.
For coding agents, compare both cache types on the same repository tasks:
- Retrieve a relevant symbol from a long source file.
- Apply a multi-file change.
- Generate a structured tool call.
- Continue after several long conversation turns.
Check correctness and tool-call parsing, not only tokens per second.
How does the new MoE expert cache work?
The MoE cache helps when a Mixture-of-Experts model keeps expert weights in system RAM instead of GPU memory.
MoE models select a subset of experts for each token. Transferring expert weights from RAM to the GPU can become expensive during decoding.
The expert cache retains frequently used CPU-resident experts in GPU memory. The CPU-share control lets the CPU process some cache misses.
This is separate from FP8 KV caching. The MoE cache handles model expert weights, while the KV cache stores attention state.
The official MoE cache guide documents the cache’s scope and behavior.
| Option | Effect | Default |
|---|---|---|
-cmoe | Keep all MoE expert weights in CPU memory | Off unless selected |
--moe-cache-mib N | Allocate N MiB of GPU cache for CPU-resident experts | 0, disabled |
--moe-cache-cpu-share F | Compute share F of cache misses on the CPU once the cache fills | 0 |
The documented CPU-share range is 0 to 1.
The cache follows these rules:
- Cache hits execute on the GPU.
- Before the cache fills, misses are uploaded to the GPU.
- Once the cache fills, the configured CPU share determines which fraction of misses the CPU processes.
- Remaining misses are uploaded for GPU execution.
- Large batches can still upload the experts they need.
The official guide explains that CPU execution can help when reading host experts is faster than transferring them to the GPU.
It also explains why pinned host memory can change this tradeoff. When uploads already use the available transfer bandwidth efficiently, CPU execution may offer little benefit.
Example: cache experts from a CPU-resident MoE model
The example requires a supported GPU backend and available device memory for the expert cache.
A CPU-only build cannot provide the GPU expert cache described here.
For an NVIDIA system with the required CUDA development environment, the upstream CUDA build instructions use:
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release
Use the corresponding documented backend build instructions for other supported GPU platforms.
For a compatible MoE model and GPU-enabled build:
./build/bin/llama-server \
-m /path/to/moe-model.gguf \
-cmoe \
-c 32768 \
--host 127.0.0.1 \
--port 8080 \
--moe-cache-mib 2048 \
--moe-cache-cpu-share 0.5
Here, -cmoe keeps the MoE expert weights in CPU memory. The cache setting requests 2,048 MiB of GPU storage for selected experts.
The value 0.5 assigns a 50% CPU share to cache misses after the cache fills.
Before the cache fills, misses continue to be uploaded to the GPU. Cache hits still execute on the GPU.
These settings are an example, not a recommended configuration for every machine.
The GPU must have enough free memory for the expert cache and its other allocations. Cache size competes with model layers, KV storage, and compute buffers.
If the requested allocation does not fit, reduce the cache size or review model placement. Do not assume the cache makes an oversized model fit automatically.
Let the tuning script select a configuration
The project provides a tuning utility for this decision.
From the llama.cpp repository:
python3 scripts/moe-cache-tune.py -- \
./build/bin/llama-server \
-m /path/to/moe-model.gguf \
-cmoe \
-t 8 \
-c 32768
The tuning script runs the server with different cache sizes and CPU shares.
The official guide describes six measurement runs: two cache sizes combined with CPU shares of 0, 0.5, and 0.95.
The script uses those measurements to estimate other configurations. It reports measured speeds, predicted throughput, and recommended arguments.
Keep your actual model, CPU threads, context size, and expert-placement settings in the command. These affect free device memory and inference speed.
The documented cache primarily targets small-batch decoding. Large batches can still require expert uploads.
If your model is dense or its experts already fit entirely on the GPU, the host-memory cache is unlikely to help. Start with the default disabled setting instead.
What changed in llama-server security?
Version 1.0.0 introduces explicit security opt-ins for unauthenticated tool exposure and private media URLs.
These settings matter most when a server accepts connections beyond the local machine.
—allow-unauthenticated-tools
The new --allow-unauthenticated-tools flag permits server tools, MCP servers, or an MCP proxy on a non-loopback address without an API key.
The upstream server parameter reference lists this permission as disabled by default and limits its intended use to trusted networks.
For example, 127.0.0.1 restricts listening to the local machine. Binding to 0.0.0.0 can make the service reachable through other network interfaces.
A configuration that combines non-loopback access, enabled server tools or MCP, and no API key therefore needs explicit security review.
The flag is:
--allow-unauthenticated-tools
Do not add it as a routine upgrade fix. It explicitly permits unauthenticated access to security-sensitive server capabilities.
This setting concerns server-side tools and MCP exposure. A model returning a structured tool_calls response is a separate part of an agent workflow.
For a developer working on the same machine, prefer:
./build/bin/llama-server \
-m /path/to/model.gguf \
--host 127.0.0.1 \
--port 8080 \
--jinja
For remote clients, use the documented --api-key authentication mechanism and suitable network restrictions.
In Bash, collect a key without displaying it:
read -rs LLAMA_SERVER_API_KEY && export LLAMA_SERVER_API_KEY
Then start the server with a quoted environment-variable reference:
./build/bin/llama-server \
-m /path/to/model.gguf \
--host 0.0.0.0 \
--port 8080 \
--jinja \
--api-key "$LLAMA_SERVER_API_KEY"
Keep the server behind appropriate network controls. API-key authentication alone does not make an arbitrary public deployment safe.
—media-allow-private-urls
The second change concerns HTTP(S) media URLs submitted to the server, including image references.
By default, the hardened server does not permit media URLs targeting local or private network addresses.
The opt-in flag is:
--media-allow-private-urls
Enabling it permits HTTP(S) media access to local or private addresses.
This can help when a trusted application hosts images on a private service. It also increases server-side request forgery risk when untrusted users control media URLs.
Prefer supported direct image-data submission or controlled media hosting.
If private-media access is required, restrict request permissions and reachable destinations. Do not enable the flag solely to make an untrusted URL work.
Other security fixes
The release also includes changes affecting:
--cors-originsvalidation and origin matching.- DNS rebinding protection.
- Server-side request forgery through the CORS proxy.
- Direct browser access to child server instances.
- The destinations that receive
HF_TOKEN. - Cancellation of MCP tool calls after client disconnects or timeouts.
The release restricts HF_TOKEN transmission to huggingface.co hosts.
For MCP integrations, cancellation matters because a disconnected coding client should not leave unnecessary tool calls running.
Cancellation notifications do not guarantee that an external tool can undo an operation it has already performed.
Review the security changes in PR #32113, PR #32115, and the v1.0.0 release notes.
How do you verify a local coding-agent backend after upgrading?
Test model generation before reconnecting Claude Code, OpenCode, or another coding client.
First, start a loopback-only server with an appropriate instruction-following model.
./build/bin/llama-server \
-m /path/to/coding-model.gguf \
--alias local-coder \
-c 32768 \
--host 127.0.0.1 \
--port 8080 \
--jinja
The model must support the tool-calling behavior expected by your client. --jinja enables chat-template processing; it does not create tool-use capability in an unsuitable model.
Check the HTTP endpoint with curl:
curl --fail-with-body \
http://127.0.0.1:8080/v1/models
The returned model list should identify the configured model alias.
Then submit a simple completion request:
curl --fail-with-body \
http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"model": "local-coder",
"messages": [
{
"role": "user",
"content": "Explain what a Git diff shows in one sentence."
}
],
"max_tokens": 100
}'
Check for a response under choices.
Next, connect your coding tool using its documented custom-provider support. The local OpenAI-compatible API base URL is:
http://127.0.0.1:8080/v1
Do not assume that OpenAI-compatible chat completions also provide complete Anthropic Messages or OpenAI Responses API compatibility.
For Claude Code, a local integration may require a protocol adapter or another supported access path. OpenCode can use a compatible custom provider when its configured model and API match.
Verify one file-read task, one edit proposal, and one tool call before using the backend on a real repository.
A simple completion response does not establish that agent tools, structured arguments, or long-session context work correctly.
For a related model-access workflow, see the Kimi K3.1 Pro setup guide. The Kimi K3.1 Pro open-weights guide also explains why model weights, local hardware, and hosted APIs have different requirements.
Do Ollama and LM Studio users need to upgrade?
If you use Ollama or LM Studio without building or configuring llama.cpp directly, do not replace their internal inference libraries yourself.
These applications manage their own runtime versions, packaging, and integration tests.
The llama.cpp 1.0.0 release does not establish when every downstream application will bundle it.
Use each application’s update mechanism and release notes to identify the actual runtime version.
Check for these changes when they appear in your distribution:
- An upgrade explicitly mentions llama.cpp 1.0.0 or the corresponding upstream changes.
- Your application exposes new KV cache formats or MoE cache settings.
- Your application changes how server tools or MCP access is authenticated.
- Model loading or tool-calling behavior changes after an application update.
For ordinary desktop chat, keep using a working supported application version unless you need a specific fix or feature.
For developers using a local model as an agent backend, test the full client-server path after each application update.
When should you move to a hosted API instead?
Local inference remains useful when your hardware can run the selected model at the required context length and speed.
FP8 KV caching may reduce memory pressure, and MoE caching can improve some expert-offloading workloads. Neither solves every memory limit or guarantees faster coding tasks.
If model weights, attention cache, or concurrent workloads exceed your hardware budget, compare a smaller local model with a hosted model.
A hosted API can remove the need to manage local GPU memory, but introduces network, usage-cost, and provider-dependency considerations.
BetterToken is one pay-as-you-go multi-model API access option. It is separate from local llama.cpp execution and does not provide the llama.cpp runtime or its server-security features.
Compare model and workload costs on the current pricing page before choosing a hosted fallback.
For a staged migration, keep the local endpoint as one tested provider and maintain a separate hosted configuration. Do not assume the same model ID, tool format, or authentication settings work across both.
Safe upgrade checklist
For a production local coding backend, upgrade only after these checks pass:
- Pin the
v1.0.0source or an exact build artifact. - Remove
--defrag-tholdand obsolete C API calls. - Rebuild native bindings against the new public headers.
- Start with the previous FP16 KV configuration.
- Test FP8 KV separately and compare memory, speed, and coding correctness.
- Tune MoE caching only with a supported GPU backend and CPU-resident expert weights.
- Check tool authentication and private-media URL requirements.
- Verify
/v1/models, chat completion, tool calls, and the coding client. - Keep the previous working binary and configuration available for rollback.
Version 1.0.0 establishes a clearer public-library compatibility boundary. The practical upgrade decision still depends on your bindings, hardware, server exposure, and agent workflow.