llama.cpp 1.0.0 Upgrade: Breaking Changes, FP8 KV Cache, and Server Security

Upgrade llama.cpp and ggml to 1.0.0 without breaking local coding agents. Remove obsolete C APIs and defragmentation flags, test FP8 KV cache memory savings, tune host-memory MoE expert caching, and protect llama-server tool access and private URLs. Learn what semantic versioning covers and when Ollama or LM Studio users need to act.

Contents
llama.cpp 1.0.0 Upgrade: Breaking Changes, FP8 KV Cache, and Server Security

llama.cpp 1.0.0 is available as of November 14, 2026, but developers should check their C bindings, launch flags, and server security settings before upgrading. The November 13 release removes deprecated APIs, adds FP8 KV caching and new MoE expert-cache controls, and introduces security opt-ins for server tools and private media URLs. If you run a local coding model through llama-server, upgrade a staging instance first and verify both ordinary completion and tool calls.

The release also introduces a versioning commitment for libllama and libggml. Breaking public C API changes will now require a major-version increase.

That makes future dependency upgrades easier to assess. It does not guarantee that model output, performance, GPU backend behavior, or every server endpoint will remain identical.

What changed in llama.cpp 1.0.0?

The release affects developers embedding llama.cpp, operators running llama-server, and developers optimizing local inference memory.

AreaChange in 1.0.0Required action
VersioningPublic C API and ABI incompatibilities use major versionsPin a release or commit in production
C API35 deprecated functions and llama_set_warmup removedRebuild and migrate affected bindings
Context configurationdefrag_thold and --defrag-thold removedDelete obsolete settings
Speculative decodingNextN API moved into llama.h; one state function renamedUpdate includes and function references
KV cachef8_e4m3 support for K and V cachesBenchmark memory, quality, and backend support
MoE inferenceGPU expert cache gains CPU execution for cache missesCheck GPU availability and tune cache size and CPU share
Server securityTool authentication and private-media URL restrictionsReview network exposure and authentication
Model supportKimi-K3.1-Pro, Ripple, xing4_0, and multimodal additionsCheck model conversion and loading requirements
Speech APIOpenAI-compatible /v1/audio/speech endpointTest with a supported speech model

These changes are documented in the v1.0.0 release notes and the project’s official versioning announcement.

The project reports support for more than 150 model architecture variants. That is architecture coverage, not a guarantee that every checkpoint or feature works on every backend.

What does semantic versioning mean for future upgrades?

The versioning commitment applies to public C interfaces under ./include, including those exposed by llama.cpp and ggml.

The maintainers define three version changes:

Version componentMeaningExample
MAJORBreaking change to the public C API1.x.x to 2.0.0
MINORBackward-compatible features, new backends, models, or API additions1.0.0 to 1.1.0
PATCHBug fix without a public API change1.0.0 to 1.0.1

The examples illustrate versioning rules, not announced future releases.

For applications built against libllama, this reduces the risk of unexpected public function-signature changes within a major version.

It does not promise that internal headers, undocumented symbols, CLI arguments, HTTP response details, or model behavior follow the same compatibility contract.

How should you pin llama.cpp?

Use an exact release tag or immutable commit when compiling llama.cpp into an application.

The official build guide documents the Git clone and CMake build process. Combine those instructions with the v1.0.0 release tag.

For example, in Bash:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git fetch --tags
git checkout v1.0.0

cmake -B build
cmake --build build --config Release

The CMake commands follow the project’s documented CPU build procedure. Platform defaults may enable additional backends. Check your configuration before assuming a particular runtime device.

The official server documentation identifies ./build/bin/llama-server as the server binary path for the standard build. Platform-specific generators can use different output layouts.

Add the documented CMake options for your GPU backend when needed.

For production, record the release tag, resolved Git commit, build options, and runtime backend. Test each dependency update before replacing the working binary.

If your application bundles llama.cpp, pin the wrapper or application release too. Its version may differ from the upstream llama.cpp revision.

Which APIs and command-line flags were removed?

The most immediate migration problem is the removal of deprecated C interfaces and the defragmentation threshold setting.

The API cleanup associated with PR #30565 removes 35 deprecated functions, llama_set_warmup, and the old defragmentation threshold configuration.

Other migration changes include:

  • Remove calls to llama_set_warmup.
  • Remove references to defrag_thold in context configuration.
  • Remove --defrag-thold from launch scripts.
  • Replace llama_set_embd_state_src with llama_set_ctx_state_src where applicable.
  • Update NextN/speculative decoding includes and declarations now exposed through llama.h.

The 35-function removal is a migration category, not a list of 35 independent replacements. Use the tagged header changes to identify each symbol used by your binding.

Fix scripts using —defrag-thold

A previous server command might contain:

./build/bin/llama-server \
  -m /path/to/model.gguf \
  -c 32768 \
  --defrag-thold 0.1

Remove the obsolete option:

./build/bin/llama-server \
  -m /path/to/model.gguf \
  -c 32768

The important change is removing the parameter, not selecting another threshold value. Do not replace --defrag-thold with an invented equivalent.

Search shell scripts, Docker configurations, service units, and application launch settings:

grep -RInE \
  'defrag[_-]thold|llama_set_warmup|llama_set_embd_state_src' \
  ./scripts ./config ./src

Adjust the searched directories to your project. If a path does not exist, remove it from the command.

Also check environment variables or wrapper settings that still generate the removed argument.

Update C and Python bindings

If you compile directly against llama.h, compare your code with the v1.0.0 headers before rebuilding.

For CMake consumers, rebuild the dependent application against the new library and check the compiler and linker output.

For Python wrappers, first check whether the wrapper version explicitly supports llama.cpp 1.0.0. Installing a newer llama.cpp binary does not automatically update a Python binding’s compiled definitions.

If the wrapper bundles its own llama.cpp source, follow its dependency release process rather than replacing its shared library manually.

A successful import is only an initial check. Load a model, generate tokens, and test any context-state or speculative-decoding operations your application uses.

How do you enable FP8 KV cache in llama.cpp 1.0.0?

Use -ctk f8_e4m3 -ctv f8_e4m3 to request FP8 storage for the attention key and value caches.

The KV cache holds information from previous tokens so the model can continue generating without recomputing the full context. Larger context windows consume more KV memory.

FP8 uses eight bits per stored value instead of the 16 bits used by FP16. For equivalent tensor dimensions, this can approximately halve raw K/V storage.

That estimate excludes metadata, scaling information, alignment, model weights, compute buffers, and other runtime allocations.

Start with a local FP16 baseline

Use the same model and context size for both runs.

./build/bin/llama-server \
  -m /path/to/model.gguf \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080 \
  -ctk f16 \
  -ctv f16

Record the reported KV buffer allocation, available device memory, prompt processing speed, and generation speed.

Then stop the server and run:

./build/bin/llama-server \
  -m /path/to/model.gguf \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080 \
  -ctk f8_e4m3 \
  -ctv f8_e4m3

The release introduces FP8 KV caches with optional static per-tensor scales.

Check the startup logs for the selected cache types and allocation sizes. A recognized command-line value does not establish that your GPU backend supports every optimized execution path.

Does FP8 allow twice the context length?

Not necessarily.

FP8 reduces KV tensor storage, but the context window remains subject to the model’s supported context length and available memory.

For a hypothetical workload where FP16 KV consumes 8 GiB, an FP8 representation could reduce raw tensor storage to approximately 4 GiB.

This is a theoretical memory calculation, not a measured llama.cpp result.

The freed memory might allow a larger context or more concurrent requests. It cannot increase the model’s effective context capability beyond supported limits.

FP8 is also a lower-precision representation. The release documentation does not establish a universal coding-quality or tool-calling accuracy result for FP8 KV.

For coding agents, compare both cache types on the same repository tasks:

  1. Retrieve a relevant symbol from a long source file.
  2. Apply a multi-file change.
  3. Generate a structured tool call.
  4. Continue after several long conversation turns.

Check correctness and tool-call parsing, not only tokens per second.

How does the new MoE expert cache work?

The MoE cache helps when a Mixture-of-Experts model keeps expert weights in system RAM instead of GPU memory.

MoE models select a subset of experts for each token. Transferring expert weights from RAM to the GPU can become expensive during decoding.

The expert cache retains frequently used CPU-resident experts in GPU memory. The CPU-share control lets the CPU process some cache misses.

This is separate from FP8 KV caching. The MoE cache handles model expert weights, while the KV cache stores attention state.

The official MoE cache guide documents the cache’s scope and behavior.

OptionEffectDefault
-cmoeKeep all MoE expert weights in CPU memoryOff unless selected
--moe-cache-mib NAllocate N MiB of GPU cache for CPU-resident experts0, disabled
--moe-cache-cpu-share FCompute share F of cache misses on the CPU once the cache fills0

The documented CPU-share range is 0 to 1.

The cache follows these rules:

  • Cache hits execute on the GPU.
  • Before the cache fills, misses are uploaded to the GPU.
  • Once the cache fills, the configured CPU share determines which fraction of misses the CPU processes.
  • Remaining misses are uploaded for GPU execution.
  • Large batches can still upload the experts they need.

The official guide explains that CPU execution can help when reading host experts is faster than transferring them to the GPU.

It also explains why pinned host memory can change this tradeoff. When uploads already use the available transfer bandwidth efficiently, CPU execution may offer little benefit.

Example: cache experts from a CPU-resident MoE model

The example requires a supported GPU backend and available device memory for the expert cache.

A CPU-only build cannot provide the GPU expert cache described here.

For an NVIDIA system with the required CUDA development environment, the upstream CUDA build instructions use:

cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release

Use the corresponding documented backend build instructions for other supported GPU platforms.

For a compatible MoE model and GPU-enabled build:

./build/bin/llama-server \
  -m /path/to/moe-model.gguf \
  -cmoe \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080 \
  --moe-cache-mib 2048 \
  --moe-cache-cpu-share 0.5

Here, -cmoe keeps the MoE expert weights in CPU memory. The cache setting requests 2,048 MiB of GPU storage for selected experts.

The value 0.5 assigns a 50% CPU share to cache misses after the cache fills.

Before the cache fills, misses continue to be uploaded to the GPU. Cache hits still execute on the GPU.

These settings are an example, not a recommended configuration for every machine.

The GPU must have enough free memory for the expert cache and its other allocations. Cache size competes with model layers, KV storage, and compute buffers.

If the requested allocation does not fit, reduce the cache size or review model placement. Do not assume the cache makes an oversized model fit automatically.

Let the tuning script select a configuration

The project provides a tuning utility for this decision.

From the llama.cpp repository:

python3 scripts/moe-cache-tune.py -- \
  ./build/bin/llama-server \
  -m /path/to/moe-model.gguf \
  -cmoe \
  -t 8 \
  -c 32768

The tuning script runs the server with different cache sizes and CPU shares.

The official guide describes six measurement runs: two cache sizes combined with CPU shares of 0, 0.5, and 0.95.

The script uses those measurements to estimate other configurations. It reports measured speeds, predicted throughput, and recommended arguments.

Keep your actual model, CPU threads, context size, and expert-placement settings in the command. These affect free device memory and inference speed.

The documented cache primarily targets small-batch decoding. Large batches can still require expert uploads.

If your model is dense or its experts already fit entirely on the GPU, the host-memory cache is unlikely to help. Start with the default disabled setting instead.

What changed in llama-server security?

Version 1.0.0 introduces explicit security opt-ins for unauthenticated tool exposure and private media URLs.

These settings matter most when a server accepts connections beyond the local machine.

—allow-unauthenticated-tools

The new --allow-unauthenticated-tools flag permits server tools, MCP servers, or an MCP proxy on a non-loopback address without an API key.

The upstream server parameter reference lists this permission as disabled by default and limits its intended use to trusted networks.

For example, 127.0.0.1 restricts listening to the local machine. Binding to 0.0.0.0 can make the service reachable through other network interfaces.

A configuration that combines non-loopback access, enabled server tools or MCP, and no API key therefore needs explicit security review.

The flag is:

--allow-unauthenticated-tools

Do not add it as a routine upgrade fix. It explicitly permits unauthenticated access to security-sensitive server capabilities.

This setting concerns server-side tools and MCP exposure. A model returning a structured tool_calls response is a separate part of an agent workflow.

For a developer working on the same machine, prefer:

./build/bin/llama-server \
  -m /path/to/model.gguf \
  --host 127.0.0.1 \
  --port 8080 \
  --jinja

For remote clients, use the documented --api-key authentication mechanism and suitable network restrictions.

In Bash, collect a key without displaying it:

read -rs LLAMA_SERVER_API_KEY && export LLAMA_SERVER_API_KEY

Then start the server with a quoted environment-variable reference:

./build/bin/llama-server \
  -m /path/to/model.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  --jinja \
  --api-key "$LLAMA_SERVER_API_KEY"

Keep the server behind appropriate network controls. API-key authentication alone does not make an arbitrary public deployment safe.

—media-allow-private-urls

The second change concerns HTTP(S) media URLs submitted to the server, including image references.

By default, the hardened server does not permit media URLs targeting local or private network addresses.

The opt-in flag is:

--media-allow-private-urls

Enabling it permits HTTP(S) media access to local or private addresses.

This can help when a trusted application hosts images on a private service. It also increases server-side request forgery risk when untrusted users control media URLs.

Prefer supported direct image-data submission or controlled media hosting.

If private-media access is required, restrict request permissions and reachable destinations. Do not enable the flag solely to make an untrusted URL work.

Other security fixes

The release also includes changes affecting:

  • --cors-origins validation and origin matching.
  • DNS rebinding protection.
  • Server-side request forgery through the CORS proxy.
  • Direct browser access to child server instances.
  • The destinations that receive HF_TOKEN.
  • Cancellation of MCP tool calls after client disconnects or timeouts.

The release restricts HF_TOKEN transmission to huggingface.co hosts.

For MCP integrations, cancellation matters because a disconnected coding client should not leave unnecessary tool calls running.

Cancellation notifications do not guarantee that an external tool can undo an operation it has already performed.

Review the security changes in PR #32113, PR #32115, and the v1.0.0 release notes.

How do you verify a local coding-agent backend after upgrading?

Test model generation before reconnecting Claude Code, OpenCode, or another coding client.

First, start a loopback-only server with an appropriate instruction-following model.

./build/bin/llama-server \
  -m /path/to/coding-model.gguf \
  --alias local-coder \
  -c 32768 \
  --host 127.0.0.1 \
  --port 8080 \
  --jinja

The model must support the tool-calling behavior expected by your client. --jinja enables chat-template processing; it does not create tool-use capability in an unsuitable model.

Check the HTTP endpoint with curl:

curl --fail-with-body \
  http://127.0.0.1:8080/v1/models

The returned model list should identify the configured model alias.

Then submit a simple completion request:

curl --fail-with-body \
  http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "local-coder",
    "messages": [
      {
        "role": "user",
        "content": "Explain what a Git diff shows in one sentence."
      }
    ],
    "max_tokens": 100
  }'

Check for a response under choices.

Next, connect your coding tool using its documented custom-provider support. The local OpenAI-compatible API base URL is:

http://127.0.0.1:8080/v1

Do not assume that OpenAI-compatible chat completions also provide complete Anthropic Messages or OpenAI Responses API compatibility.

For Claude Code, a local integration may require a protocol adapter or another supported access path. OpenCode can use a compatible custom provider when its configured model and API match.

Verify one file-read task, one edit proposal, and one tool call before using the backend on a real repository.

A simple completion response does not establish that agent tools, structured arguments, or long-session context work correctly.

For a related model-access workflow, see the Kimi K3.1 Pro setup guide. The Kimi K3.1 Pro open-weights guide also explains why model weights, local hardware, and hosted APIs have different requirements.

Do Ollama and LM Studio users need to upgrade?

If you use Ollama or LM Studio without building or configuring llama.cpp directly, do not replace their internal inference libraries yourself.

These applications manage their own runtime versions, packaging, and integration tests.

The llama.cpp 1.0.0 release does not establish when every downstream application will bundle it.

Use each application’s update mechanism and release notes to identify the actual runtime version.

Check for these changes when they appear in your distribution:

  • An upgrade explicitly mentions llama.cpp 1.0.0 or the corresponding upstream changes.
  • Your application exposes new KV cache formats or MoE cache settings.
  • Your application changes how server tools or MCP access is authenticated.
  • Model loading or tool-calling behavior changes after an application update.

For ordinary desktop chat, keep using a working supported application version unless you need a specific fix or feature.

For developers using a local model as an agent backend, test the full client-server path after each application update.

When should you move to a hosted API instead?

Local inference remains useful when your hardware can run the selected model at the required context length and speed.

FP8 KV caching may reduce memory pressure, and MoE caching can improve some expert-offloading workloads. Neither solves every memory limit or guarantees faster coding tasks.

If model weights, attention cache, or concurrent workloads exceed your hardware budget, compare a smaller local model with a hosted model.

A hosted API can remove the need to manage local GPU memory, but introduces network, usage-cost, and provider-dependency considerations.

BetterToken is one pay-as-you-go multi-model API access option. It is separate from local llama.cpp execution and does not provide the llama.cpp runtime or its server-security features.

Compare model and workload costs on the current pricing page before choosing a hosted fallback.

For a staged migration, keep the local endpoint as one tested provider and maintain a separate hosted configuration. Do not assume the same model ID, tool format, or authentication settings work across both.

Safe upgrade checklist

For a production local coding backend, upgrade only after these checks pass:

  1. Pin the v1.0.0 source or an exact build artifact.
  2. Remove --defrag-thold and obsolete C API calls.
  3. Rebuild native bindings against the new public headers.
  4. Start with the previous FP16 KV configuration.
  5. Test FP8 KV separately and compare memory, speed, and coding correctness.
  6. Tune MoE caching only with a supported GPU backend and CPU-resident expert weights.
  7. Check tool authentication and private-media URL requirements.
  8. Verify /v1/models, chat completion, tool calls, and the coding client.
  9. Keep the previous working binary and configuration available for rollback.

Version 1.0.0 establishes a clearer public-library compatibility boundary. The practical upgrade decision still depends on your bindings, hardware, server exposure, and agent workflow.

Ready to optimize your LLM workflow?

Join thousands of developers building faster, smarter, and more cost-effective AI applications with BetterToken.

Get Started for Free