semcode

Why semcode

Coding agents are good at reading the project you have open. They are bad at answering questions that span the rest of your organisation: "which service publishes OrderPlaced?", "how do other teams call the billing API?", "why did someone change the retry logic last month?"

How it works

flowchart LR
    GH[GitHub repos] -->|source files| TS[Tree-sitter<br/>symbol parsing]
    TS --> DE[Dense embeddings<br/>Jina · Voyage · OpenAI · Ollama]
    TS --> BM[BM25 sparse vectors<br/>code-aware tokenizer]
    GH -->|commits + diffs| CE[Commit embeddings]
    DE --> Q[(Qdrant)]
    BM --> Q
    CE --> Q
    Q -->|hybrid search · RRF| S[semcode<br/>MCP server]
    S <-->|MCP| A[Your AI agent]
    CI[CI on merge] -->|POST /reindex| S

semcode fetches files from GitHub, splits them into symbols (functions, classes, methods, components) with Tree-sitter, and stores two vectors per symbol in Qdrant: a dense semantic embedding and a BM25 vector over code-identifier subwords (getUserById → get, user, by, id). Queries run both and fuse the rankings with Reciprocal Rank Fusion, so natural language and exact symbol names work. Indexing is incremental — unchanged files are skipped by Git blob SHA, and deleted files are cleaned up.

→ Deep dive: ingestion · dense vectors · sparse vectors · retrieval with RRF · the blog post

🚀 Quick start

You need: Docker, and a GitHub token with Contents: read on the repos you want indexed.

1. Start semcode, Qdrant and a local embedding model (no API key needed):

git clone https://github.com/GoodbyePlanet/semcode && cd semcode
cp .env.example .env          # then set GITHUB_TOKEN in .env
make docker-up-jina

2. Index a repo:

curl -N -X POST http://localhost:8090/reindex \
  -H 'Content-Type: application/json' \
  -d '{"service": "my-service", "github_repo": "owner/repo"}'

Progress streams back as NDJSON and ends with {"type": "done", ...}. The repo stays registered, so later reindexes are incremental.

3. Connect your agent:

claude mcp add --transport http semcode http://localhost:8090/mcp

That's it. Prefer a hosted embedding provider (Voyage, OpenAI, Jina API) or Ollama? Set EMBEDDINGS_PROVIDER in .env and run make docker-up instead. Want a curated list of repos in a config.yaml, or to run without Docker? See Deployment.

💬 Using it

Ask your agent in plain English:

1. "Index acme/payments with semcode."
2. "Where do we publish order events?"
3. "Find everything that uses PlaceOrderRequest, across all services."
4. "Search the commit history: who changed the retry logic in payments, and why?"
5. Run the system_design_overview prompt for a full architecture write-up with Mermaid diagrams.

✨ Features

• Many repos, one index. List services in config.yaml, or let each repo register itself from CI with the drop-in GitHub Actions workflow.
• Hybrid retrieval. Dense embeddings plus BM25 with a camelCase/snake_case-aware tokenizer, fused with RRF.
• Commit history. Commit messages and changed files are searchable; full diffs are one tool call away.
• 25 languages and file formats, framework-aware. Go, Java, Python, TypeScript/JavaScript, Rust, C#, C, C++, Ruby, PHP, Kotlin, Scala, Swift, Dart, Bash, SQL, Lua, R, Dockerfile, Docker Compose, Markdown, JSON, HTML, CSS, XML. Parsers extract Spring stereotypes and HTTP routes, FastAPI/Pydantic, ASP.NET, Rails, Laravel/Symfony, React/SwiftUI/Flutter widgets and more.
• Pluggable embeddings. Jina (local TEI or hosted API), Voyage, OpenAI or Ollama.
• CI-friendly. Streaming NDJSON reindex endpoints for pipelines and schedulers.
• GitHub Enterprise Server support via GITHUB_API_URL.

🛠 MCP tools

Prompts: service_overview(service) walks the agent through an architectural overview of one service — HTTP entry points, domain types, framework conventions. system_design_overview does the whole system: service inventory, communication topology, shared data stores and cross-cutting concerns, with Mermaid diagrams.

🔌 Connecting other clients

The server speaks streamable HTTP at http://localhost:8090/mcp.

{
  "servers": {
    "semcode": { "type": "http", "url": "http://localhost:8090/mcp" }
  }
}

code --add-mcp '{"name":"semcode","type":"http","url":"http://localhost:8090/mcp"}'

[mcp_servers.semcode]
url = "http://localhost:8090/mcp"

{
  "mcpServers": {
    "semcode": { "transport": "http", "url": "http://localhost:8090/mcp" }
  }
}

claude mcp add semcode --transport stdio --env MCP_TRANSPORT=stdio \
  -- uv run --directory /path/to/semcode python -m server.main

{
  "mcpServers": {
    "semcode": {
      "command": "uv",
      "args": ["run", "--directory", "/path/to/semcode", "python", "-m", "server.main"],
      "env": { "MCP_TRANSPORT": "stdio" }
    }
  }
}

claude mcp add --transport sse semcode http://localhost:8090/sse

⚖️ How it compares

When semcode doesn't help: a single small repo you already have open — your agent can read it directly, and editing-oriented tools like Serena are a better fit there. semcode pays off when the answer lives in code you don't have checked out.

📚 Documentation

The docs site covers everything this page leaves out:

• Deployment — Docker services, embedding paths, config.yaml, transports
• Configuration — every environment variable and embedding provider setting
• HTTP API — /reindex, /reindex-history, NDJSON frame shapes
• Qdrant collections — vectors, HNSW settings, payload fields
• Ingestion and retrieval with RRF — how it works inside
• Changelog — release history, generated from commit messages

🤝 Contributing

Contributions are welcome! See CONTRIBUTING.md for dev setup, tests, adding a language parser, and the Conventional Commits rules that drive releases.

License

MIT