Local AI MCP Servers

Two self-hosted MCP servers: manage a local model machine (Ollama pull/switch, LoRA training) and bridge to local Ollama/vLLM for language tasks.

GitHubTry this MCPSponsored

Documentation

Local-AI MCP servers: Ollama & vLLM model management and local LLM access for Claude, ChatGPT and other MCP clients

Two self-hostable Model Context Protocol (MCP) servers for running and steering your own local language models: manage an Ollama/vLLM model machine, and query a local LLM (Ollama, vLLM or an OpenAI-compatible backend) for language work with schema-validated JSON output. Self-hosted and privacy-first: the models and their machine stay on your own hardware, and neither server process reaches out to a public address on its own. Version 1.0.0, single-operator tool, actively used by the author.

Both servers speak the Model Context Protocol (MCP), so an AI tool such as Claude or ChatGPT can use them. Each server's own traffic goes only to your own computer or your local network; a start-up guard rejects public addresses. Installing the two direct dependencies needs the internet once, at setup time. Fetching a model is a separate matter: in mcp-modelmanager, calling fetch_model is a deliberate, tool-triggered connection from your model machine to a public model registry, at whatever point you invoke it, not something limited to setup time. The default source="ollama" makes the machine's own Ollama daemon pull from registry.ollama.ai for plain model names, or from whatever other registry a prefixed name such as hf.co/user/repo points at; source="huggingface" instead reaches Hugging Face directly. switch_service never reaches either registry itself; it refuses and asks for fetch_model first if the model is not already on the machine. Each server stands on its own: pick one, or run both.

A caveat: the AI tool you drive these servers with (Claude, ChatGPT, etc.) is itself a hosted service in the usual case. The instructions you give it and the tool inputs and results it handles pass through that provider, exactly as any other MCP call would. "Local" here means the models, the model machine and its data; it is not a claim that nothing you type ever reaches a third party. Drive them with a local model if that matters to you.

The two servers

`mcp-modelmanager` -> mcp-modelmanager/

Manage your own model machine through an AI tool: read state (GPU memory, disk, running service, containers), pull and remove models, switch the vLLM inference service to a different model, build custom Ollama variants and run LoRA training. There is deliberately no tool for free remote control: every operation is a fixed template and every parameter first passes an allowlist, checked independently on both sides. This server ships both halves: the client/server code the AI tool talks to, and the machine-side scripts, wrapper and network-fence templates that run on and lock down the model machine.

`mcp-ollama-vllm` -> mcp-ollama-vllm/

Call a local model directly from an AI session for the language work: writing, summarizing, classifying, extracting structured data. Its distinctive feature is schema-validated JSON output, checked by its own dependency-free validator and retried automatically on a violation, the same way whether the configured backend is Ollama or vLLM. It is a model call, not an agent: no tools, no file access, no shell.

The two are complementary. mcp-ollama-vllm asks a model (read-only, no management rights); mcp-modelmanager manages the machine. Keeping them separate means a query call never accidentally holds management rights.

Repository layout

.
|-- README.md                 this file
|-- CHANGELOG.md              version history (Keep a Changelog style)
|-- CONTRIBUTING.md           how to contribute
|-- LICENSE                   MIT, covers the repo as a whole
|-- .gitignore
|-- .github/
|   `-- workflows/
|       `-- ci.yml            lockfile check, locked-tree tests, shell lint, advisory pip-audit
|-- mcp-modelmanager/         manage your own model machine (server + machine side)
|   |-- README.md             server and client guide (start here)
|   |-- SETUP.md              client-side step-by-step setup
|   |-- AGENT-SETUP.md        setup procedure written for an AI agent
|   |-- MACHINE.md            machine-side overview (start here for the machine)
|   |-- SETUP-MACHINE.md      gapless machine provisioning
|   |-- SECURITY.md           the three-layer security model
|   |-- LICENSE
|   |-- .gitignore            SSH-key and build patterns (kept from the server)
|   |-- requirements.txt
|   |-- pyproject.toml
|   |-- uv.lock               full resolved dependency tree, pinned
|   |-- src/mcp_modelmanager/  server package (server code and shared modules)
|   |-- setup/                machine-side setup scripts
|   |-- vm_side/              machine-side wrapper and job runners
|   `-- security/             network-fence templates (sshd, firewall, WireGuard)
`-- mcp-ollama-vllm/          call local models with schema-checked JSON
    |-- README.md             server guide (start here)
    |-- AGENT-SETUP.md        setup procedure written for an AI agent
    |-- SECURITY.md           what the bridge does and does not protect
    |-- LICENSE
    |-- requirements.txt
    |-- pyproject.toml
    |-- uv.lock               full resolved dependency tree, pinned
    `-- src/mcp_ollama_vllm/  server package (server code)

Quickstart

Each server has its own guide; start with the README in its folder.

mcp-modelmanager: read mcp-modelmanager/README.md and follow mcp-modelmanager/SETUP.md for the client side. For the model machine itself, start from mcp-modelmanager/MACHINE.md.
mcp-ollama-vllm: read mcp-ollama-vllm/README.md; it has no separate SETUP.md (the README covers setup, there is no machine side and no management rights to document separately), and its SECURITY.md states the boundaries of the bridge.

mcp-modelmanager needs Python 3.11 or newer, mcp-ollama-vllm 3.10 or newer; each has only two direct, pinned dependencies (mcp and httpx); mcp itself pulls in a number of further packages transitively (anyio, pydantic, starlette, uvicorn, jsonschema, pyjwt among them). The full resolved tree, direct and transitive, is pinned per server in its uv.lock; CI checks with uv lock --check that the lock still matches pyproject.toml and runs the tests against the exact locked tree, so a drifted lock fails CI. No account, no paid service, no cloud is required.

Installation / Usage

Install whichever server(s) you need from PyPI:

pip install mcp-modelmanager
pip install mcp-ollama-vllm

Each ships a console entry point (mcp-modelmanager / mcp-ollama-vllm) after install, so an MCP client can point command straight at it, for example:

{
  "mcpServers": {
    "modelmanager": {
      "command": "/path/to/.venv/bin/mcp-modelmanager",
      "env": {
        "MM_ACCESS": "direct",
        "MM_VM_HOST": "<your-machine-or-127.0.0.1>",
        "MM_VM_USER": "<your-ssh-user>",
        "MM_CONTAINER_ROOT": "/srv/models"
      }
    },
    "local-models": {
      "command": "/path/to/.venv/bin/mcp-ollama-vllm",
      "env": {
        "LOCAL_BACKEND": "ollama",
        "LOCAL_HOST": "http://localhost:11434"
      }
    }
  }
}

The required environment variables differ per server (mcp-modelmanager needs the three MM_* fields above to start at all; mcp-ollama-vllm runs with defaults for a local Ollama). This is only the shape; do not copy the values verbatim. For the full, current install/registration steps, including the checkout-based (non-PyPI) path and running from source, see each server's own README: mcp-modelmanager/README.md and mcp-ollama-vllm/README.md.

Security and scope

Local and self-hosted. Both server processes run on your own hardware and talk only to your own computer or your local/private network. Public addresses are rejected at start-up, and names are resolved so the guard cannot be bypassed via DNS. This governs the servers' own traffic; the AI tool that drives them is a separate service (see the caveat above), and so is a model download you explicitly trigger through mcp-modelmanager (fetch_model): with the default source="ollama" it makes the machine's own Ollama daemon pull from registry.ollama.ai (or from the registry a prefixed model name such as hf.co/user/repo points at), and with source="huggingface" it reaches Hugging Face directly, both from the model machine at the time you ask for it. switch_service does not reach either registry; it refuses a model that is not already on the machine and points you at fetch_model instead.
No credentials in this repository. No keys, tokens or passwords are stored here; SSH keys for mcp-modelmanager live outside the repo in your ~/.ssh, and the .gitignore additionally excludes key and .env patterns.
Placeholders, not real values. Every machine-specific value in the docs and templates is a <placeholder> or an environment variable you fill in on your own machine at setup time.
Each server has its own SECURITY.md (mcp-modelmanager, mcp-ollama-vllm) that goes into the actual trust boundaries and layers, and states plainly what remains unaddressed.