claimcite

Zero-dependency citation audit CLI & stdio MCP server verifying DOIs, arXiv IDs, titles, authors, years, and retractions against Crossref, OpenAlex, and arXiv.

Documentation

claimcite

English · 中文

Check that every citation in a document is real, points to the paper the text says it does, and actually contains the words quoted from it.

AI-written articles, health advice and even papers increasingly cite sources that don't exist, or attach a real DOI to the wrong paper. Readers can't check dozens of references by hand. claimcite does it in one command.

$ claimcite check examples/sample.md
# Citation check

1 ok · 2 warnings · 2 errors · 0 not checked

- **Error** · line 7 · `doi:10.1016/s0140-6736(10)61388-8`
  - Title does not match this identifier. Registry title: Worldwide burden of disease from
    exposure to second-hand smoke: a retrospective analysis of data from 192 countries.
  - Year differs: document 2021, registry 2011
- **Error** · line 9 · `doi:10.1001/jama.2022.99999`
  - Identifier does not exist
- **Warning** · line 11 · `pmid:33237064`
  - Study type (as tagged by PubMed): Systematic review / meta-analysis
  - Quoted text not found in the abstract (it may be in the full text): “calcium and vitamin D
    supplementation cured osteoporosis in all women”
...

What it checks today

CheckHow
The DOI / PMID / arXiv ID existsdoi.org handle API, PubMed, arXiv Atom API (including 10.48550/arXiv.*)
The paper has not been retractedPubMed Retracted Publication / RetractionIn, Crossref updated-by & relation.is-retracted-by
The title written in the text matches the identifierPubMed, Crossref, or arXiv; fuzzy match (ignoring publisher RETRACTED: prefix)
The year matches (±1 for online-first vs print)same (published & updated years for arXiv)
Text in quotes ("…", “…”, 「…」) appears in the abstractPubMed / Crossref / arXiv abstract
Study type: meta-analysis, RCT, observational, animal…PubMed publication types and MeSH
A web link is alive and contains the quoted textfetch the page (respecting robots.txt), strip HTML, normalise whitespace
A reference with no DOI or PMID existssearch the title in Crossref, then PubMed; then check it like any other paper

It reads Markdown or plain text and understands the common Author (Year). Title. Journal: "quote". https://doi.org/… pattern, plus doi:…, bare DOIs, PMID: … and PubMed links. Numbered citations work too: a quote in the text followed by [1], [^1], [1, 2] or [1-3] is checked against the matching entry in the reference list ([1] …, 1. … or [^1]: …).

Most invented citations have no DOI: Chen, L. (2021). A plausible title. Journal of Something, 14(2), 88–97. Entries like that under a reference-list heading (## References, 参考文献, Sources …), and [1] … or [^1]: … entries anywhere, are searched by title in Crossref and then PubMed. APA, Harvard, MLA, Vancouver and GB/T 7714 styles are understood. A paper that is found is checked like one with a DOI. One that neither registry has is a warning, not an error: it may not exist, or the title may be worded differently. Books (most are in neither registry) and Chinese-language references are left not checked. Searches run one a second (Crossref's limit), so 30 such references take about half a minute the first time. --no-search turns this off. Code blocks and inline code are skipped. A quote in Chinese is only compared with Chinese text: next to an English abstract or page it is noted as not comparable word for word, not reported as missing. Other web links are fetched (respecting robots.txt) and checked for availability and for the quoted text. A dead link or a missing quote is a warning, never an error — pages change and some sites block bots. When a page cannot be read the link stays not checked.

What it does not do (yet): check paraphrases. Only text in quotation marks is compared with the source, word for word; a sentence that restates a source without quotes is not extracted or checked. It also cannot decide whether a source supports a claim. That needs a language model and a labelled test set. See the roadmap. claimcite only confirms that sources exist and match what the text says about them. It is not medical advice.

Install and run

Requires Python 3.10+. No API keys needed.

pipx install claimcite                   # or: uv tool install claimcite
claimcite check article.md               # Markdown report
claimcite check article.md --lang zh     # 中文报告
claimcite check article.md --format json # for scripts and CI
cat article.md | claimcite check -

The exit code is 1 when any citation has an error, so it works as a CI gate (--fail-on warning|never to tune). Set NCBI_API_KEY to raise PubMed's rate limit.

Lookup cache

Successful registry responses are cached on disk for 30 days (default ~/.cache/claimcite/v1/, or $XDG_CACHE_HOME/claimcite/v1/), so re-running after a one-line edit does not repeat every request. Lookup failures are never cached. Use --no-cache to bypass the cache for one run.

Web pages are fetched at most once a second per site, so the first run over a document with hundreds of links takes minutes. Use --no-web to check only DOIs and PMIDs. Quotes in PDFs are not checked yet.

In CI: GitHub Action

# .github/workflows/citations.yml
on: [push, pull_request]
jobs:
  claimcite:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: wasabihu/claimcite@v0.3.0
        with:
          files: "docs/**/*.md"  # default: every Markdown file
          fail-on: error         # or: warning, never
          web: "true"            # "false" checks DOIs and PMIDs only
          search: "true"         # "false" skips title search for references without a DOI

The report also appears in the job summary. Runs on Linux and macOS runners.

Before each commit: pre-commit

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/wasabihu/claimcite
    rev: v0.3.0
    hooks:
      - id: claimcite
        # args: [--no-web]   # faster: skip ordinary web links

Every changed Markdown file is checked; the commit stops when a citation has an error. You can also pass several files on the command line: claimcite check docs/*.md, or a whole folder: claimcite check docs/.

MCP server & Agent Skill

claimcite ships with a zero-dependency stdio Model Context Protocol server (claimcite mcp) exposing check_citations and check_files:

{
  "mcpServers": {
    "claimcite": {
      "command": "uvx",
      "args": ["claimcite", "mcp"]
    }
  }
}

A ready-to-use Agent Skill guide lives in skills/claimcite/SKILL.md.

Offline benchmark

Run the 20-case fixture-backed benchmark locally (zero network calls):

uv run python benchmarks/run.py

See benchmarks/README.md for what each category covers.

Roadmap

Each item is a self-contained piece of work. Pick one and open an issue to say you're on it.

  • Web links: check that the page is alive and contains the quoted text
  • References without identifiers: find them by title (Crossref, PubMed)
  • Retractions: flag retracted papers via Crossref (updated-by / relation) and PubMed (Retracted Publication)
  • More registries: OpenAlex, Semantic Scholar, CNKI/Wanfang metadata (arXiv and DataCite 10.48550/arXiv.* are supported)
  • Full text: look for quotes in open-access full text (PubMed Central, Europe PMC), not only abstracts
  • Support check: does the source support the sentence? Pluggable LLM judge, local models first
  • Benchmark: offline 20-case fixture benchmark in benchmarks/ (more public corpora welcome)
  • Numbered citations: [1], [1-3] and footnotes [^1] linked to the reference list
  • More input formats: BibTeX, Word / PDF input
  • Integrations: GitHub Action, pre-commit hook, stdio MCP server (claimcite mcp), Agent Skill
  • Browser extension
  • Caching between runs (30-day on-disk cache, --no-cache to bypass)
  • Faster batch lookups

Related project

Looking to build a minimal, audited AI agent runtime in Python? Check out our sister project: cairnkit (includes a citation-verifying tool example wired cleanly with claimcite).

Contributing

Contributions of every size are welcome, including AI-assisted ones. See CONTRIBUTING.md.

License

MIT