GovData MCP

GovData MCP

A Model Context Protocol (MCP) server that lets AI agents search, explore, and fetch open datasets from official government data portals. One unified tool interface across the United States, United Kingdom, Canada, and Australia — with per-portal quirks (API migrations, auth walls, bilingual metadata, stale DataStore flags) absorbed behind a common adapter layer.

Built for the typical agent workflow: "Is there a government dataset for X?" → find candidates → inspect the files → preview rows or grab a download link and move on.

Key Features

• Four national portals, one interface. US (data.gov v4 / DCAT), UK (data.gov.uk / CKAN), Canada (open.canada.ca / CKAN), Australia (data.gov.au / CKAN + DataStore).
• Token-frugal by design. Descriptions are HTML-stripped and truncated, search results are compact summaries, and large files are never inlined by accident.
• Row-level preview. Uses CKAN DataStore where available (Australia), and transparently falls back to reading the head of CSV/TSV files everywhere else.
• Agent-friendly failure modes. Errors are returned as actionable text ("use get_download_link instead", "retry with dataset_id", "available portals: …") — never raw tracebacks.
• Extensible registry. Adding another CKAN-based portal is a single config entry; non-CKAN portals plug in via a small adapter class.
• Dual transport. stdio for local clients (Claude Desktop etc.), stateless streamable-http for serverless deployment (Google Cloud Run).

Supported Portals

Requirements

• Python 3.11+
• An api.data.gov API key for the US portal (free; other portals need no credentials)

Installation

git clone https://github.com/YOUR_ORG/Gov-Stat-MCP-Server.git
cd Gov-Stat-MCP-Server

# pip
pip install -e ".[dev]"

# or conda
conda env create -f environment.yml
conda activate opendata-mcp

Copy .env.example to .env and set your key:

DATAGOV_API_KEY=your-api-data-gov-key

Getting Started

Standard config for most MCP clients (stdio transport):

{
  "mcpServers": {
    "govdata": {
      "command": "govdata-mcp",
      "env": {
        "DATAGOV_API_KEY": "your-api-data-gov-key"
      }
    }
  }
}

claude mcp add govdata -e DATAGOV_API_KEY=your-key -- govdata-mcp

MCP_TRANSPORT=http govdata-mcp

{
  "mcpServers": {
    "govdata": {
      "url": "http://localhost:8080/mcp"
    }
  }
}

Tools

The intended agent flow: list_portals → search_datasets → get_dataset → preview_resource / fetch_resource / get_download_link.

Configuration

All settings are environment variables (or a local .env file):

Docker

make docker-build          # build image
make docker-run            # run on :8080, exactly as Cloud Run would

The image runs the HTTP transport in stateless mode and is ready for serverless platforms (Google Cloud Run deployment script included under deploy/).

Development

make run          # stdio mode
make run-http     # HTTP mode on :8080
make inspect      # MCP Inspector interactive UI (requires Node.js)
make test         # offline unit tests
make test-live    # live end-to-end tests against all four real portals
make lint         # ruff check + format check

The live suite (pytest -m live) verifies the full search → explore → preview → fetch chain per country plus portal-specific behavior (Canada's bilingual normalization, UK's dataset-scoped resource lookup, Australia's DataStore paths). Run it before deploying — government portals change without notice.

Architecture

src/govdata_mcp/
├── server.py            # FastMCP instance + tool registration
├── config.py            # env-driven settings
├── ckan/                # generic async CKAN client, models, typed errors
├── portals/
│   ├── registry.py      # portal registry — single source of truth
│   ├── base.py          # Portal config + default (vanilla CKAN) adapter
│   ├── us.py            # data.gov v4 DCAT adapter (auth, cursor paging)
│   ├── uk.py            # dataset-scoped resource lookup
│   └── ca.py            # bilingual metadata normalization
├── tools/               # MCP tools: search, explore, fetch
└── utils/               # formatting (token economy), CSV-head preview

Adding a portal: for a standard CKAN portal, add one Portal(...) entry to portals/registry.py — done. For portals with quirks, subclass PortalAdapter and override the narrow hooks (normalize_dataset, build_search_params) or, for non-CKAN APIs, the operation methods themselves (see us.py for a full custom adapter).

TODO / Roadmap

• 🇨🇳 China portal support — no unified national open-data API exists; planned as a custom adapter targeting National Bureau of Statistics data (via an existing stats package) with graceful degradation.
• 🇯🇵 Japan portal support — e-Gov Data Portal (data.e-gov.go.jp) is CKAN-compatible and should slot into the existing registry; the richer e-Stat statistics API (app-ID auth, Japanese-language metadata) planned as a dedicated adapter.
• Offline unit test suite with recorded portal fixtures (respx) for CI
• Google Cloud Run deployment walkthrough (service is container-ready; script in deploy/)
• US v4 publisher/format search filters (pending API documentation)
• More CKAN portals (New Zealand data.govt.nz is a near-free addition)
• Content-type sniffing to rescue mislabeled resource formats

Security Notes

• This server is read-only toward the portals — no tool can create, modify, or delete anything.
• fetch_resource and preview_resource download from URLs contained in portal metadata; content is size-capped and binary-checked before being returned to the model, but treat fetched content as untrusted input.
• The server itself has no authentication. For remote deployment, put it behind an authenticating proxy or platform-level auth (e.g. Cloud Run with --no-allow-unauthenticated).
• Keep DATAGOV_API_KEY in .env / secret manager — never commit it.

License

MIT