knowledge-rag

knowledge-rag: local RAG server for Claude Code, Cursor and other MCP clients

knowledge-rag is a local retrieval-augmented generation (RAG) server for the Model Context Protocol (MCP). It indexes the documents and code on your machine and gives Claude Code, Cursor, VS Code and other MCP clients 13 tools to search them, using hybrid search (BM25 keywords plus vector similarity) with cross-encoder reranking on by default. Everything runs in one Python process with embedded ChromaDB and FastEmbed ONNX models: no Docker, no database server, no API key. Indexing and search work offline once the models are cached.

• 11.1 ms median uncached hybrid query on CPU, 6.1 ms on CUDA: 36 queries over 149 chunks on one Windows host, reranker off (audit)
• 0.03 ms median for a repeated query served from the cache, same workload (audit)
• 75,016 chunks from 5,889 documents: full rebuild completed in a production reproduction (v4.8.3)
• 1,500+ tests collected on each of 9 OS × Python CI cells (platform-specific ones skip), 84% line coverage against an 80% floor (CI run, 2026-11-06)

Install and register with Claude Code (needs Python 3.11+ and Node.js 16+):

npx -y knowledge-rag --install-only
claude mcp add knowledge-rag -s user --env KNOWLEDGE_RAG_DIR=~/knowledge-rag -- npx -y knowledge-rag

Put files in ~/knowledge-rag/documents/ and ask your agent to search them. The folder is created on first start, new and changed files are indexed automatically, and the models (about 150 MB) download the first time they are needed.

Install · Results · How it works · Tools · Skills · Comparison · FAQ · Docs

Star history

_{Updated daily by the star-history workflow.}

Measured results

Measurements from the project's own audits. They come from one host and small corpora, so treat them as orders of magnitude for your planning, not as guarantees.

Workload: scripts/audit_retrieval.py indexing 7 public repository documents (149 chunks) into a temporary ChromaDB and FTS5 index; 36 uncached queries; cross-encoder disabled to isolate retrieval; Windows 11, RTX 3080 Ti, Python 3.12.7, FastEmbed 0.8.1, ChromaDB 1.5.9, model BAAI/bge-small-en-v1.5, Hugging Face offline mode. Source: audit 2026-10-03.

How it works

1. Ingest. Files under documents/ are parsed by format: code files get their function, class and import names as metadata, notebooks keep only Markdown and code cells, PDFs contribute their text. Markdown is split at ##/### sections; every other format is cut into 1,000-character chunks with 200 characters of overlap. Chunks are de-duplicated by SHA-256, embedded with FastEmbed (default BAAI/bge-small-en-v1.5, 384 dimensions) and stored in an embedded ChromaDB.
2. Watch. While the server runs, a watcher re-indexes changed files after 10 seconds without new changes. Full rebuilds fill a staging collection and swap it in after validation, so searches keep using the current index.
3. Search. search_knowledge checks a query cache (100 entries, 5-minute TTL), runs BM25 over an inverted index and vector search, fuses both rankings with weighted reciprocal rank fusion (hybrid_alpha, default 0.3, leaning towards keywords) and reranks the top candidates with the Xenova/ms-marco-MiniLM-L-6-v2 cross-encoder. With the opt-in FTS5 fast path, identifier-like queries such as CVE IDs, ticket keys and hashes go to SQLite FTS5 first.
4. Answer. Results go back to the MCP client with the file path, chunk index, snippet and scores. The client's model writes the answer; knowledge-rag never calls an LLM.
5. Serve. Over stdio for one client, or as one Streamable HTTP or SSE server for many. HTTP mode answers health probes first, then checks the bearer token. Every tool call passes through the optional rate limiter and metrics. config.yaml and five presets configure all of it.

flowchart TB
    subgraph CLIENTS["MCP clients"]
        direction LR
        C1[Claude Code]
        C2[Claude Desktop]
        C3[Cursor]
        C4[Windsurf]
        C5[VS Code · other stdio or HTTP clients]
    end

    subgraph TRANSPORT["Transport"]
        T1[stdio<br/>1 process per client]
        T2[Streamable HTTP / SSE<br/>1 server for N clients]
    end

    subgraph MIDDLEWARE["ASGI middleware (HTTP transports)"]
        M1[HealthMiddleware<br/>/health · /healthz]
        M2[BearerAuthMiddleware<br/>constant-time compare]
    end

    subgraph MCP["13 MCP tools (contract checked in CI)"]
        W[Tool wrappers<br/>optional rate limit · metrics]
        MT1[search_knowledge]
        MT2[get_document · search_similar]
        MT3[add_document · add_from_url · update · remove]
        MT4[reindex_documents · get_reindex_status]
        MT5[list_documents · list_categories · get_index_stats · evaluate_retrieval]
    end

    subgraph SEARCH["Retrieval pipeline"]
        QC[Query cache<br/>LRU 100 · 5-min TTL]
        R[Query router<br/>regex, FTS5 fast path only]
        F[FTS5 fast path<br/>opt-in lexical retrieval]
        H[Hybrid search]
        BM[BM25<br/>inverted index]
        SE[Vector search<br/>FastEmbed ONNX, lazy-loaded]
        RRF[Weighted reciprocal rank fusion]
        CE[Cross-encoder rerank<br/>MiniLM-L-6-v2]
    end

    subgraph STORAGE["Local storage (data_dir)"]
        CH[ChromaDB<br/>vectors + metadata<br/>WAL with HTTP transports]
        FT[SQLite FTS5<br/>lexical index<br/>WAL + busy timeout]
        MD[index_metadata.json<br/>durable state]
    end

    subgraph INGEST["Document ingestion"]
        FS[documents/ folder]
        WD[Watcher<br/>10 s debounce]
        PA[Format parsers<br/>MD · PDF · DOCX · code · IaC · IPYNB]
        CK[Chunker<br/>Markdown sections · 1,000-char windows]
        EM[FastEmbed ONNX<br/>default bge-small-en-v1.5, 384-d]
        DD[SHA-256 dedup]
        SW[Staging rebuild<br/>rollback on handled publication failure]
    end

    subgraph OBS["Observability (opt-in)"]
        PM[Prometheus /metrics<br/>separate port, HTTP transports]
        LG[Structured JSON logs]
        HC[Health payload<br/>version · uptime · cache stats]
    end

    subgraph CFG["Configuration"]
        YM[config.yaml<br/>+ 5 presets]
    end

    CLIENTS -->|MCP over stdio| T1
    CLIENTS -.->|MCP over HTTP| T2
    T1 --> W
    T2 --> M1 --> M2 --> W
    W --> MT1 & MT2 & MT3 & MT4 & MT5

    MT1 --> QC
    QC -->|cache miss| R
    R -->|identifier-like| F
    R -->|everything else| H
    H --> BM
    H --> SE
    F --> FT
    BM --> CH
    SE --> CH
    BM --> RRF
    SE --> RRF
    RRF --> CE
    CE --> QC

    MT2 --> CH
    MT3 --> INGEST
    MT4 --> SW
    MT5 --> CH

    FS --> WD --> PA
    PA --> CK --> DD --> EM --> CH
    SW -.->|validated swap| CH
    SW -.-> FT
    CH -.-> MD

    W -.->|instrumented| PM
    W -.->|logs| LG
    M1 --> HC

    YM -.-> SEARCH
    YM -.-> STORAGE
    YM -.-> OBS
    YM -.-> MIDDLEWARE

    classDef client fill:#3776AB,stroke:#1e5a8a,color:#fff
    classDef transport fill:#00A67E,stroke:#006e54,color:#fff
    classDef middleware fill:#6b46c1,stroke:#4c1d95,color:#fff
    classDef storage fill:#4b5563,stroke:#1f2937,color:#fff
    classDef obs fill:#dc2626,stroke:#7f1d1d,color:#fff
    classDef ingest fill:#b45309,stroke:#78350f,color:#fff

    class C1,C2,C3,C4,C5 client
    class T1,T2 transport
    class M1,M2 middleware
    class CH,FT,MD storage
    class PM,LG,HC obs
    class FS,WD,PA,CK,EM,DD,SW ingest

Component ownership, memory and concurrency boundaries and the hybrid_alpha weights: docs/ARCHITECTURE.md.

Install

Every path below runs the same server. Each claude mcp add sets KNOWLEDGE_RAG_DIR, the folder that holds config.yaml, documents/, data/ and models_cache/. Without it, the server picks a folder from its install location and the working directory the client starts it in, which can be a project you did not intend (config.py).

Claude Code with the npx launcher

npx -y knowledge-rag --install-only
claude mcp add knowledge-rag -s user --env KNOWLEDGE_RAG_DIR=~/knowledge-rag -- npx -y knowledge-rag

The npm package is a launcher. It finds Python 3.11+ (python3/python, or python/py -3 on Windows), creates a virtual environment in ~/.knowledge-rag/venv, installs the matching knowledge-rag version from PyPI and starts the server over stdio. --install-only does the installation once ahead of time, so the client's first start does not wait for pip. A new npm version reinstalls the package on the next start. If a client on native Windows fails to start npx, register cmd /c npx -y knowledge-rag instead.

pip in a virtual environment

python3 -m venv ~/knowledge-rag/venv
~/knowledge-rag/venv/bin/python -m pip install knowledge-rag
claude mcp add knowledge-rag -s user --env KNOWLEDGE_RAG_DIR=~/knowledge-rag \
  -- ~/knowledge-rag/venv/bin/python -m mcp_server.server

On Windows (PowerShell), use python and $HOME\knowledge-rag\venv\Scripts\python.exe in place of python3 and ~/knowledge-rag/venv/bin/python.

Optional: knowledge-rag init, run inside ~/knowledge-rag with the virtual environment active, copies config.example.yaml and the five presets and creates documents/. It does not create config.yaml or register the server: copy the template or a preset to config.yaml yourself. Without a config.yaml, the defaults apply, including all 33 default file types.

Docker

The image is published for linux/amd64 and linux/arm64. For stdio, the client starts one container per session:

claude mcp add knowledge-rag -s user -- docker run -i --rm \
  -v ~/knowledge-rag/documents:/app/documents \
  -v ~/knowledge-rag/data:/app/data \
  -v knowledge-rag-models:/app/models_cache \
  ghcr.io/lyonzin/knowledge-rag:latest

The image runs python -m mcp_server.server, which reads /app/config.yaml and defaults to stdio on 127.0.0.1. To serve HTTP from a container, mount a config that listens on all container interfaces and pass --transport:

mkdir -p ~/knowledge-rag
cat > ~/knowledge-rag/config.yaml <<'EOF'
server:
  host: "0.0.0.0"
  auth:
    bearer_token: "replace-with-a-long-random-token"
EOF
docker run -d --name knowledge-rag -p 127.0.0.1:8179:8179 \
  -v ~/knowledge-rag/config.yaml:/app/config.yaml:ro \
  -v ~/knowledge-rag/documents:/app/documents \
  -v ~/knowledge-rag/data:/app/data \
  -v knowledge-rag-models:/app/models_cache \
  ghcr.io/lyonzin/knowledge-rag:latest --transport streamable-http
curl http://127.0.0.1:8179/health
claude mcp add --transport http knowledge-rag http://127.0.0.1:8179/mcp \
  # if server.auth.bearer_token is set, add the token as a Bearer auth header

The image builds its model copies into FastEmbed's default cache, but the server reads /app/models_cache, so the models download again the first time they are needed. The knowledge-rag-models volume keeps them across containers.

Shared HTTP server for several clients

stdio fits one client that owns one process. For several clients on one index (Claude Code, Cursor, LM Studio, more terminals), run one Streamable HTTP server and point every client at it (#217, #223):

# ~/knowledge-rag/config.yaml
server:
  transport: "streamable-http"      # "sse" for clients that only speak SSE
  host: "127.0.0.1"
  port: 8179
  auth:
    bearer_token: "replace-with-a-long-random-token"
  rate_limit:
    enabled: true
    requests_per_minute: 60
    burst: 10
  metrics:
    enabled: true
    port: 9179                      # binds 0.0.0.0 with no auth: firewall it
  logging:
    format: "json"

KNOWLEDGE_RAG_DIR=~/knowledge-rag knowledge-rag     # run from the venv where you installed it
curl http://127.0.0.1:8179/health                  # {"status":"ok","version":"5.0.0",...}
claude mcp add --transport http knowledge-rag http://127.0.0.1:8179/mcp \
  # if server.auth.bearer_token is set, add the token as a Bearer auth header

HTTP transports switch on the single-instance lock (KNOWLEDGE_RAG_SINGLE_INSTANCE=1), so a second server cannot open the same data directory. The default host is 127.0.0.1. If you bind to 0.0.0.0 or serve a network, set server.auth.bearer_token and put TLS in front with a reverse proxy such as Caddy or Nginx; without a token, the server prints a warning that the endpoint is unauthenticated.

Other MCP clients

Any client that starts a stdio MCP server from a command, or connects to a Streamable HTTP endpoint, can use knowledge-rag. The installation guide has the exact JSON for each of these:

{
  "mcpServers": {
    "knowledge-rag": {
      "command": "npx",
      "args": ["-y", "knowledge-rag"],
      "env": { "KNOWLEDGE_RAG_DIR": "~/knowledge-rag" }
    }
  }
}

[!WARNING] One writer per data directory. Over stdio, each MCP client starts and owns its own knowledge-rag process. Do not run a second writer against the same data_dir while that client is open: on Windows, two writers corrupted the ChromaDB HNSW segment (#216). Add "KNOWLEDGE_RAG_SINGLE_INSTANCE": "1" to the client's env block as a safety net, or share one HTTP server. Running CLI commands safely: docs/single-instance.md.

[!WARNING] Keep data_dir out of cloud-sync folders. OneDrive (including OneDrive - <tenant>), iCloud Drive, Dropbox, Google Drive, Box, pCloud, MEGAsync, Nextcloud/ownCloud and Sync hold handles on the live index while they upload it. That races with ChromaDB's HNSW memory maps and SQLite WAL writes and can crash the process (0xC0000005 on Windows, SIGBUS elsewhere); startup auto-repair then moves the index to data/backups/auto-repair-* and you reindex from scratch. Since v4.11.0, startup stops with exit code 78 (EX_CONFIG) when data_dir resolves under one of these providers. Use a local path such as %LOCALAPPDATA%\knowledge-rag\data on Windows or ~/.local/share/knowledge-rag/data on Linux and macOS, or set paths.allow_cloud_sync: true to accept the risk.

All install methods, GPU setup and troubleshooting for each client: docs/INSTALLATION.md.

The 13 MCP tools

Each result carries source (the file path), chunk_index and, in snippet mode, match_offset (a character offset into the content), so citations point to a file and chunk, not a line number. score is normalized per query, so the top result is always 1.0 and min_score filters relative to the other results; compare reranker_score, the raw cross-encoder score, across queries. Parameters, return shapes and examples: docs/API.md.

Agent skills

The 13 tools tell an agent what it can do; they do not tell it when to search. The skills/ folder adds ten MIT-licensed skills for Claude Code that make the agent check your index first, cite what it used and keep the index current. Other agents can use the same instructions as rules or system prompts (skills/README.md).

Each skill installs as ~/.claude/skills/<skill-name>/SKILL.md (or .claude/skills/<skill-name>/SKILL.md for one project):

# Node.js 22.20+: choose the skills and agents interactively; -g installs for all projects
npx skills add lyonzin/knowledge-rag -g

# Without Node (Linux, macOS, WSL, Git Bash): all ten skills into ~/.claude/skills
curl -fsSL https://raw.githubusercontent.com/lyonzin/knowledge-rag/master/skills/install.sh | bash

install.sh also takes --project, --target <path>, --only rag-check-first,rag-cite-sources, --dry-run, --list and --help. Restart Claude Code after installing. Skill chains and per-client setup: skills/CATALOG.md, skills/README.md.

How knowledge-rag compares to other RAG projects

As of 2026-11-06. 🗸 default: built in and on by default. 🗸 opt-in: built in, off until you enable it. 🗸: available, default state not compared. component: a library piece you assemble yourself. ✗: not built in, meaning absent from the code, needing an external provider (local embeddings column), or not found in code or docs; ✗ manual: you start it yourself. Words in parentheses name the mechanism.

Notes: Serviette was renamed from pathwaycom/llm-app and inherited its stars. The LlamaIndex README says its focus moved to LlamaParse. Quivr was repurposed as a content-stream search engine. The GraphRAG README says it is in maintenance mode. PrivateGPT's changelog lists a SentenceTransformer reranker (#1810) that the current code no longer contains.

Where knowledge-rag differs. It is the only row with hybrid search and cross-encoder reranking both on by default, and the only MCP server in the table that answers without Docker, a database server or a model key. Auto re-index is on by default only in knowledge-rag and Serviette, and Serviette ships no MCP server; everywhere else it takes a connector, a client, a setting, assembly or a manual run. It is code-first: 23 of its 33 default file types are code or infrastructure files. The license is plain MIT.

Where others are stronger. Multi-user scale: Postgres, OpenSearch and Elasticsearch stacks against one writer on an embedded ChromaDB. Web UIs and visual builders: knowledge-rag has no UI and does not generate answers; the MCP client's LLM does. Connectors, OCR, images and audio, and graph RAG. Evidence and reach: knowledge-rag's published quality check is a 12-question smoke test (Recall@5 1.0) and it has 301 stars. This table compares features, not retrieval quality.

Built for production

Each capability says whether it is on by default and links to the test, document or release note that backs it. Opt-in features do nothing until you set their key in config.yaml or the environment variable shown.

Security

Observability

Scale and performance

Reliability

Compliance: the project holds no LGPD, GDPR or HIPAA certification. Local indexing can help with data-locality requirements, but the MCP client sends the results it retrieves to its own model provider as part of the conversation.

Configuration in 30 seconds

knowledge-rag init                    # copies config.example.yaml, the presets and an empty documents/
cp presets/general.yaml config.yaml   # or cybersecurity, developer, research, multilingual
# drop your files into documents/, restart, ask the agent

Settings live in config.yaml inside KNOWLEDGE_RAG_DIR. Every key is optional; these are the defaults:

paths:
  documents_dir: "./documents"
  data_dir: "./data"                 # local disk only, see the cloud-sync warning
  models_cache_dir: "./models_cache"
documents:
  chunking:
    chunk_size: 1000
    chunk_overlap: 200
models:
  embedding:
    profile: "custom"                # "compact" | "quality" | "multilingual" | "custom"
    model: "BAAI/bge-small-en-v1.5"  # used when profile is "custom"
    dimensions: 384
    gpu: "auto"                      # "auto" | "true" | "false" | "directml"
  reranker:
    enabled: true
    model: "Xenova/ms-marco-MiniLM-L-6-v2"
search:
  default_results: 5
  max_results: 100
  lexical_fast_path:
    enabled: false                   # SQLite FTS5 for identifier-like queries
server:
  transport: "stdio"                 # "stdio" | "streamable-http" | "sse"
  host: "127.0.0.1"
  port: 8179

Changing the embedding model or its prefixes needs a full reindex. GPU: auto tries NVIDIA CUDA on the first embedding and falls back to CPU; the [gpu] extra targets CUDA 12. directml is the experimental Windows AMD mode and needs models.embedding.device_id (GPU setup).

Presets are complete config.yaml files (knowledge-rag init copies them):

The presets list their own documents.supported_formats, so they index fewer file types than the defaults: general and multilingual index only .md, .txt, .pdf and .docx. Every field, every default and tuning recipes: docs/CONFIGURATION.md.

35 file formats

documents:
  supported_formats: [".md", ".pdf", ".mq4", ".mqh"]

FAQ

Documentation

Recent releases

• v5.0.0 (2026-11-05): breaking list_documents now returns 100 documents per page by default (up to 200 with limit); call it again with offset until you reach total. Also: an empty or missing documents_dir no longer clears the index, search snippets start at the match, add_from_url stops overwriting other pages, and failing tests now block merges.
• v4.11.1 (2026-10-25): fresh installs start again, Windows state writes retry through brief file locks, and the Docker image adds linux/arm64.
• v4.11.0 (2026-10-04): startup refuses a data_dir inside cloud-sync folders.
• v4.10.0 (2026-10-04): indexing reliability, bounded memory, include_folders/exclude_folders search scopes and opt-in Windows DirectML.
• v4.9.0 to v4.9.2 (2026-09): 15 more file types, watcher and index resilience, and a fix for the reindex loop on Windows stdio clients.

Full history, including the v4.8 releases: CHANGELOG.md.

Community and support

• Report a bug: open an issue
• Ask a question: GitHub Discussions
• Report a vulnerability: private security advisory
• Contribute: CONTRIBUTING.md

Security reports are acknowledged within 48 hours (SECURITY.md).

License

MIT. You may use, modify and redistribute knowledge-rag, including in commercial products, under its terms. There is no paid edition.

Acknowledgments

Built on these open-source projects:

• Model Context Protocol: the protocol and its Python SDK
• ChromaDB: embedded vector database
• FastEmbed: ONNX embeddings and reranking without PyTorch
• Hugging Face: model hosting, including the Xenova/ms-marco-MiniLM-L-6-v2 cross-encoder
• BAAI: the bge-small-en-v1.5 embedding model

Community contributors: @Hohlas, @eeshsaxena, Sergey Khokhlov, and everyone who filed issues or pull requests (all contributors).

Built by Ailton Rocha (Lyon.) · Star the repository if it saves you time · Report an issue · Contribute

_{knowledge-rag: a local RAG server with hybrid search and reranking for Claude Code, Cursor and other MCP clients.}