llmtrim MCP Server

llmtrim

llmtrim is a local proxy that compresses your LLM API requests so you pay less, with no change to the answers.
It sits between your AI tools and the provider, strips the wasted tokens out of every request, and forwards it on. You get the same answers for a smaller bill.

_{−31% input and −74% output tokens, measured live across 112 A/B cases, with no change in answer quality.}

_{Use it as a proxy, a CLI, an MCP server, or a library (Python · Ruby · Swift · Kotlin).}

What it does • See it in action • Get started • CLI, MCP & library • Works with • Configuration • Numbers

What it actually does

You run Claude Code, Codex, Cursor, or your own app. Every time it talks to an LLM, it sends a big blob of text: your system prompt, the tool definitions, the whole conversation history, and the raw output of every command it ran. You pay for every one of those tokens, on every single turn.

A lot of that text is waste. A 200-line build log where only 2 lines are errors. A tool schema resent identically 50 times. A JSON array with 500 near-identical rows. The model doesn't need the bulk of it to answer well, but you're billed for all of it.

llmtrim removes the waste before it's sent. It installs as a local proxy that sits between your tool and the LLM provider. Requests pass through it, get compressed, and continue to the provider. The reply comes back unchanged. Your tool doesn't know it's there; you just get a smaller bill.

  before:  your tool ───── full request ─────▶  OpenAI / Anthropic / …
                    ◀──────── reply ──────────

  after:   your tool ──▶ llmtrim ──smaller──▶  OpenAI / Anthropic / …
                            (on your machine)
                    ◀──────── reply ──────────  (same answer)

[!IMPORTANT] It can never make your bill bigger or break a request. Every compression step is re-measured with the provider's real tokenizer; if a step doesn't actually save tokens, it's reverted. If the provider rejects the compressed request, the original is resent verbatim. Worst case is zero savings, never a worse outcome.

Everything runs locally. Nothing is ever sent to us.

See it on real output

Here's one real thing llmtrim does, end to end. An AI agent ran a build, and the bash tool returned a 58-line log. Only two lines matter (the errors), but all 58 get sent to the model and billed.

Before, what the model would receive (58 lines, 4,662 chars):

[2026-06-13T10:02:00Z] INFO  compiling module core::worker::task_0 (incremental)
[2026-06-13T10:02:01Z] INFO  compiling module core::worker::task_1 (incremental)
[2026-06-13T10:02:02Z] INFO  compiling module core::worker::task_2 (incremental)
... 27 more near-identical INFO lines ...
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
... 25 more INFO lines ...
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

After, what llmtrim sends instead (5 lines, 978 chars, −79%):

[{}] INFO compiling module core::worker::task_{} (incremental) [×30: (10:02:00Z..10:02:29Z step 1s; 0..29)]
[2026-06-13T10:02:31Z] ERROR src/worker/pool.rs:214: mismatched types: expected `usize`, found `i64`
[{}] INFO compiling module core::net::conn_{} (incremental) [×25: 10:02:32Z..10:02:56Z; 0..24]
[2026-06-13T10:03:01Z] ERROR src/net/conn.rs:88: cannot borrow `buf` as mutable more than once
[2026-06-13T10:03:02Z] INFO  build failed, 2 errors

Both errors and the summary survive verbatim. The repetitive INFO lines fold into a template plus their values, losslessly, because the range is regular (task_0..task_29). The model still sees exactly what happened; it just costs a fifth as much.

Try it yourself on any request body:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai

Log-folding is just one of ten compressors. A different one re-encodes bulky JSON arrays into a compact table, with the same data in a third of the tokens:

before:  [{"id":1,"city":"Paris","ok":true},{"id":2,"city":"Lyon","ok":false}, … 200 rows]
after:   [200]{id,city,ok}: 1,Paris,true; 2,Lyon,false; …          (TOON encoding, lossless)

Each compressor fires only where it pays:

Get started

[!NOTE] Works with any tool that routes through HTTPS_PROXY: Claude Code, Codex, Cursor, Aider, your own app. GitHub Copilot pins its certificates and can't be intercepted (full list).

# 1. Install (any OS, prebuilt binary, no Rust needed)
npm install -g @llmtrim/cli && llmtrim setup

# 2. Open a new shell. Your AI tools now route through llmtrim automatically.

# 3. Watch the savings add up as you work
llmtrim status --watch

No Node? Use an installer instead:

# Linux / macOS
curl -fsSL https://raw.githubusercontent.com/fkiene/llmtrim/main/install.sh | sh

# Windows (PowerShell)
irm https://raw.githubusercontent.com/fkiene/llmtrim/main/install.ps1 | iex

Or your own package manager, same binary everywhere: brew install fkiene/tap/llmtrim · cargo binstall llmtrim · scoop install llmtrim · docker run ghcr.io/fkiene/llmtrim. Full options in INSTALL.md.

Is this safe to install?

setup is a local HTTPS proxy, the same technique as mitmproxy, scoped to LLM APIs. It changes exactly three things (a CA certificate in ~/.llmtrim/, a proxy setting in your shell profile, a login service), and llmtrim uninstall reverses all three. No API keys are stored (it forwards your tool's own auth), and your prompts never touch disk; only an anonymous count of tokens saved is kept. Full threat model: SECURITY.md.

llmtrim status      # health + savings dashboard (aliases: monitor, gain)
llmtrim doctor      # something off? end-to-end diagnosis; each check names its fix
llmtrim start       # start the background proxy
llmtrim stop        # stop it
llmtrim serve       # run in the foreground instead (Ctrl-C to quit)
llmtrim update      # update to the latest release + restart
llmtrim uninstall   # exact inverse of setup: removes all three changes

Use it as a CLI, MCP, or library

The same compression runs with no proxy and no setup, as a one-shot CLI, an embeddable Rust crate, or native bindings for Python, Ruby, Swift and Kotlin. No extra model calls, no network: the deterministic engine runs in your process.

CLI. Pipe a request in, get a compressed one out:

echo '{"model":"gpt-4o","messages":[...]}' | llmtrim compress --provider openai > out.json
echo '{"model":"gpt-4o","messages":[...]}' | llmtrim send     --provider openai   # compress, call, print

Rust. The engine is the llmtrim-core crate (no tokio, no network in its dependency tree):

use llmtrim_core::{compress, compress_with_config, config::DenseConfig, ir::ProviderKind};

// None auto-detects the provider from the request shape.
let out = compress(request_json, Some(ProviderKind::OpenAi))?;
println!("{} -> {} input tokens", out.input_tokens_before, out.input_tokens_after);

// …or pass an explicit preset/config:
let out = compress_with_config(request_json, Some(ProviderKind::OpenAi), &DenseConfig::preset("agent").unwrap())?;

Python / Ruby / Swift / Kotlin. One flat compress(input, provider, preset) call, generated from the same Rust engine via UniFFI. The compiled engine is bundled in each package, so there's no Rust toolchain to install:

import llmtrim

out = llmtrim.compress(request_json, llmtrim.Provider.OPEN_AI, "aggressive")
print(out.input_tokens_before, "->", out.input_tokens_after)

[!NOTE] Every binding returns the compressed request_json plus the before/after token counts, and maps errors to native exceptions. Per-language install and usage live in crates/llmtrim-uniffi.

MCP server. llmtrim mcp speaks the Model Context Protocol over stdin/stdout, so any MCP client can compress payloads and read your savings without the proxy. It exposes three tools: llmtrim_compress (compress a full request body, honoring your ~/.llmtrim config like the proxy), llmtrim_compress_text (shrink one text blob, lossless), and llmtrim_stats (your savings ledger). Every call records to the same ledger, so MCP traffic shows up in llmtrim status.

llmtrim mcp install          # register with Claude Code (one command)
llmtrim mcp install --print  # or print the config to paste into any other client

The printed block is the standard MCP config; for a client you edit by hand it looks like:

{
  "mcpServers": {
    "llmtrim": { "command": "llmtrim", "args": ["mcp"] }
  }
}

Works with

Any tool that honors HTTPS_PROXY and an env-provided CA, which is essentially every CLI agent and most Node apps:

Prefer no proxy? Any MCP client (Claude Code, Cursor, custom agents) can call llmtrim directly as tools instead: run llmtrim mcp install, or see CLI, MCP, or library.

Providers come from the llm_providers registry (OpenAI, Anthropic, Google, DeepSeek, Mistral, xAI, Moonshot, Zhipu, Qwen, OpenRouter, …) and update with it. Every non-LLM connection passes through untouched.

Configuration

Zero config needed. The default (auto) inspects each request and picks the right compressors for its shape: tool-heavy → agent, code → code, long context with a question → rag, otherwise → aggressive.

If you want to force a mode, set one line: LLMTRIM_PRESET=<name> or preset = "<name>" in $XDG_CONFIG_HOME/llmtrim/config.toml:

The numbers

The savings are measured live, not estimated. Each of 112 benchmark cases is sent twice (once original, once compressed), then both answers are scored and billed at real rates, so the saving and the answer quality are measured together.

The token cuts are model-independent (−31% input, −74% output); the dollar saving depends on the model's output-to-input price ratio: −66% on the benchmark model, projected −57–59% at GPT-4o / Claude Sonnet rates. On live Claude Code traffic, llmtrim cuts −68% of compressible input without ever touching the cached prefix, so your prompt-cache discount stays intact.

Accuracy preserved on standard benchmarks

The same A/B, run on the academic suites you already know, at a conservative shape-matched preset. Quality is the score on the original request vs the compressed one (qwen3-next-80b, n=20 each, paired 95% CI):

BFCL and SQuAD v2 are the compression wins at no quality cost: BFCL cuts 33% of input by dropping the tool schemas the query doesn't need (a multi-tool menu, 2 to 37 candidates per call), and SQuAD v2 cuts 11% while answering its unanswerable questions correctly (a right "no answer" counts as a hit). TruthfulQA MC1 is the quality-preservation row: a ~75-token prompt that is almost all answer text, so a safe preset finds nothing to cut and the factual accuracy is held exactly. GSM8K is the one measured dip: its reasoning preset trades −8pp for a large cost cut, so measure per workload before enabling it. ¹GSM8K input goes negative because that preset injects a Chain-of-Draft instruction whose payoff is output-side (see the frontier table). Evidence and a one-line reproduce are in the named-benchmark snapshot:

python3 crates/llmtrim-cli/bench/scripts/download.py 40 truthfulqa,squad2,bfcl
(cd crates/llmtrim-cli && cargo run -q --features live -- bench quality \
   --corpus bench/data/squad2.jsonl --preset rag \
   --model qwen/qwen3-next-80b-a3b-instruct --route "" --n 20)

Full methodology, per-corpus frontier, and confidence intervals are in crates/llmtrim-cli/bench/README.md. Reproduce it:

python3 crates/llmtrim-cli/bench/scripts/download.py 40   # pull real corpora (gsm8k, humaneval, dolly, hotpotqa, …)
(cd crates/llmtrim-cli && cargo run -q --features live -- bench suite)  # live A/B across all corpora (needs OPENROUTER_API_KEY)
python3 crates/llmtrim-cli/bench/scripts/chart.py         # regenerate the chart + table

Known limits

These are surfaced by the same A/B that proves the savings:

• Anthropic / Gemini token counts are approximate. There's no public exact tokenizer, so a BPE proxy is used and flagged in status. OpenAI is exact.
• Output savings aren't measured live. The proxy compresses input; an output saving needs the A/B counterfactual, which only the offline benchmark runs. status "saved" is input-side.
• The default is quality-gated, not lossless. Lossy stages run only where the eval shows quality holds. Want a byte-faithful round-trip? Use the safe preset.

Acknowledgments

Every compressor is a deterministic implementation of published research: the ideas are theirs, the engineering and the token gate are ours.

Found a problem?

Run llmtrim doctor for an end-to-end diagnosis; each failing check names its fix. Found a request it mangled? Set LLMTRIM_CAPTURE_DIR and open an issue with the before/after capture, since a repro is a fix. And if it saved you money, a ⭐ helps others find it.

_{Licensed under AGPL-3.0-only. Running llmtrim locally to compress your own traffic triggers no obligations; the copyleft applies only if you offer a modified llmtrim as a network service to others. Contributions via DCO sign-off.}