DiffCI

Tells AI coding agents which tests and checks their change affects, explains why, and verifies the working tree before push. Falls back to the full suite when evidence is incomplete.

Hosted MCP Server

npx add-mcp 'https://diffci.com/mcp/v1'

Installs into Claude Code, Codex, Cursor and more

Documentation

DiffCI Core

Core CI OpenSSF Scorecard OpenSSF Best Practices npm provenance License: MIT

npm GitHub Marketplace GitLab CI/CD Catalog Sponsor

Verification for AI coding agents. DiffCI tells Claude Code, Codex, Cursor (or you) which tests and checks a change affects, runs them before the push instead of letting them fail at the full CI gate afterwards, and leaves a receipt of what ran and what was skipped. MIT licensed, runs locally, never uploads your code or reports unless you configure it (it does count runs by default: usage counting), and falls back to the full suite whenever the evidence is incomplete.

npx "@diffci.com/diffci@latest" check

Run it in any Git checkout with Node.js 22.5+. What the output means.

If DiffCI saves you CI time, a ⭐ helps others find it.

On every pull request, with no workflow file: install the DiffCI GitHub App. On each pull request of a public repository it posts one "DiffCI change impact" check listing the affected tests and why. The check is always neutral, so it never blocks a merge. The App reads code and Actions runs and its only write permission is that check run; it never skips tests or changes required checks. Data handling.

Or in your own workflow (including private repositories), add one job beside your existing CI. Its job summary shows which tests the change affects and why, or that the full suite is needed and why. It runs no tests and skips or changes nothing:

- uses: actions/checkout@v7
- uses: DiffCI/core@v0.3.60

Add with: comment: "true" (and pull-requests: write on the job) and the same answer appears on the pull request itself, updated in place on each push:

DiffCI pull request comment: 18 of 624 test files are affected by 1 changed file, with a table of each test and why it was selected

Full workflow · GitLab CI/CD component

It's just your CI

DiffCI doesn't replace your CI, your test runner or your required checks. It reads them.

  • Remove it and nothing changes. The App and the observe job run no tests, skip nothing and change no required check. Uninstall the App or delete the job, and your pipeline is exactly what it was.
  • When unsure, it runs everything. When DiffCI can't tell what a change affects (a config or dependency change it can't narrow, an import it can't follow), verify runs the full suite and checks instead of guessing, and says why. Your CI keeps running everything unless you choose to run DiffCI's selection there yourself.
  • Nothing to sign up for. No account, config file or hosted service is needed. Plans and receipts are plain JSON in an open format (change plan).
  • The agent setup is files in your repository. To undo diffci init, delete what it wrote: its section of AGENTS.md and CLAUDE.md, the diffci entries in the MCP configs, the diffci-verify skill folders, the stop hooks and the pre-push hook (what init writes).

DiffCI demo video

▶ Watch the demo on YouTube

Why

Coding agents pass the narrow check they picked and miss wider breakage, then rerun the whole suite when they are unsure. We measured this on our own agent-built production app (about 1,100 commits in two months, most of them co-authored by Claude):

  • 104 pushes failed the full gates after push. Each passed whatever the agent ran locally.
  • Agents chose their own test files 928 times and ran the full suite 282 times, but asked a tool which tests were affected only 24 times.
  • When DiffCI could narrow the run, it selected 2.2% of test files, cutting test time by 93%. None of those commits had a failing test, so this shows speed, not regression catching.
  • DiffCI still fell back to the full suite on 58% of commits, mostly because a config file or package.json changed. Narrowing those safely is the main open problem, and a good place to contribute.

Check your own repository. How many of its red pull request builds failed on checks the author could have run before pushing? This reads GitHub Actions history only, runs nothing, and needs a GitHub token (method):

npx "@diffci.com/diffci@latest" red-builds owner/repo

MCP server

This repository contains DiffCI's MCP server. It is implemented in packages/diffci-cli/src/client/mcp.ts and published as the diffci-mcp binary of @diffci.com/diffci: a local stdio server speaking JSON-RPC 2.0 (initialize, tools/list, tools/call) that runs against the checkout it is pointed at. It needs no account and sends no code or reports; like the CLI, it counts runs by default (usage counting).

npx -y -p @diffci.com/diffci@latest diffci-mcp
{
  "mcpServers": {
    "diffci": { "command": "npx", "args": ["-y", "-p", "@diffci.com/diffci@latest", "diffci-mcp"] }
  }
}
ToolWhat it does
diffci_changed_filesLists staged, unstaged and untracked changes; runs no tests
diffci_select_testsReturns the affected-test plan for the current changes, with fallback reasons; runs no tests
diffci_explain_selectionThe same plan, for explaining why each test is in it
diffci_run_affected_testsVerifies the current snapshot with the affected tests, or the full suite as a fallback
diffci_verify_changedVerifies the exact working-tree snapshot; fails closed
diffci_verifyVerifies a clean commit range and returns a receipt bound to HEAD
diffci_checkCompares full and selected test runs and reports paired runtime
diffci_initSeeds agent instructions, MCP configs, the verify skill and a pre-push hook
diffci_verify_workflowChecks that a DiffCI GitHub Actions job cannot affect the rest of CI

A hosted, read-only guidance server runs at https://diffci.com/mcp/v1. It is listed in the Official MCP Registry as io.github.DiffCI/diffci, and on Glama, Agent Tools, Wellknown, Agenstry and SealKeeper.

Use it from your coding agent

The public integrations/agent-plugin package packages the MCP server above as a checkout-aware local plugin for agent marketplaces: marketplace metadata, skills, and a launcher pinned to the published diffci-mcp binary. It does not include product services or private source. The cross-agent installation guide covers Codex, Claude Code, Cursor, Grok, Meta Muse Code, and generic MCP hosts.

# Codex
codex plugin marketplace add DiffCI/core
codex plugin add diffci@diffci-local

# Claude Code
claude plugin marketplace add DiffCI/core
claude plugin install diffci@diffci-marketplace

Cursor Marketplace reads .cursor-plugin/marketplace.json from this repository. Meta Muse Code reads the native root marketplace.json. Before a public listing is approved, use the local MCP configuration in the agent-plugin package or visit https://diffci.com/mcp-server.

Installing the plugin does not make an agent use it. Paste the short agent-instructions snippet into CLAUDE.md or AGENTS.md so it runs DiffCI before pushing.

GitHub Action

The public action.yml ("DiffCI change impact") shows, in the job summary of every pull request, which tests the change affects and why, or that the full suite is needed and why. It is observation only: it runs the pinned published DiffCI CLI outside the observed checkout, retains existing CI jobs, skips nothing, and uploads the report as an artifact. See the observer Action guide and workflow example.

Set comment: "true" (with pull-requests: write) to also keep one comment on the pull request up to date with the answer, where reviewers see it. Outputs such as full-suite-required and affected-tests let a workflow act on the answer; see outputs.

Set verify: "true" for a fast check: the Action also runs the affected tests and lint/typecheck, likeliest failures first, and fails within minutes when one breaks. It adds a job and skips nothing, so your full test job stays as it is (example).

Add check-run: "true" for a merge gate: a "DiffCI merge gate" check run that branch protection can require. It passes when the affected tests passed, fails when they failed, stays neutral when DiffCI is unsure, and names the AI coding agent that wrote the change.

The v0.3.10 corresponding source remains publicly available at tagged Core commit 4c76be14 (cli-v0.3.10-source) while distribution migrates to the Core-native CLI package.

diffci check compares full and selected runtime when safe to measure, writes reports outside the checkout, and uploads no report by default; it does send a usage count (usage counting). Test commands can generate files; keep required CI authoritative.

Measured example: one controlled Cal.com sandbox replay measured 44.2% net reduction in install + pretest + test runtime including analysis overhead. This is a job-equivalent comparison, not production savings or a forecast for your repository. Method and limitations.

Recorded Cal.com sandbox comparison

Case studies

Each study is a replay on real repositories, with its method, raw data and limitations on the linked page. None is a forecast for your repository.

StudyResult
Agent breakage replayEvery red CI commit in two agent-built repositories: 32 of 38 introduced breakages caught before push; about 70% of red commits only inherited a broken main
LobeHub vs a Vitest planner40 commits against the maintainers' own affected-test planner: same share of tests, no planner-picked test missed, 15 of 18 introduced failures caught before push
Paperclip red builds74 red PR builds classified job by job: 12 failed only on the PR's own pre-push checks (11 agent-written), about 15% of red runs
Cal.comSix real merges, executed and timed: 86–91.6% test-stage reduction, 44.2% at job level
deepseek-harnessFive merges on a dirty baseline: 79.5–89.5% job-level reduction; one result withheld because the selection was not honored
DiffCI's own CICost per job on ephemeral containers, and the bug our own green CI was hiding

Star Core · Volunteer a pilot repository · Find a first contribution. Watch → Custom → Releases and Discussions for benchmark results and support updates. Public roadmap. Quick questions and CI/CD talk: DiffCI Discord.

This is the standalone Core engine. It analyzes local JavaScript/TypeScript repositories, supported Vue components, root Go modules, conventional Maven reactors, Python projects tested with pytest and, experimentally, Cargo workspaces (Rust support is new and has not yet narrowed a real project's history, so expect full runs there for now), traces dependency impact, infers GitHub Actions structure and proposes test selections with evidence and full-run fallbacks. It works without a DiffCI account, hosted service or API key. See the release boundary for support limits.

Want to try DiffCI in an existing repository? Use the published CLI: npx @diffci.com/diffci@latest check. See the CLI quickstart for a no-execution analysis option, prerequisites, and result interpretation. To get help setting it up and have the cases that matter to your repository fixed first, join as a design partner. Tried it already? Share your result, including fallbacks, refusals and misses, and choose whether we may publish it. The commands below build the standalone engine from source.

Run locally

Requires Git and Node.js 22.5 or later.

git clone https://github.com/DiffCI/core.git
cd core
npm ci
npm run build
npm test
node dist/cli.js plan --repo /path/to/clean-checkout --base HEAD~1 --head HEAD

The target must be a clean Git repository root checked out at --head. Fetch the base commit beforehand if using a shallow clone. JSON goes to stdout; analysis errors exit nonzero and recommend full CI. Core does not execute repository scripts, contact a DiffCI backend or send telemetry. Installation requires the npm registry.

The diffci CLI, the GitHub Action and the MCP server count runs with the repository name (host/owner/repo), never its content. diffci telemetry --mode count-only drops the name; DO_NOT_TRACK=1 stops counting. Details: usage counting.

To evaluate the same command on pull requests without changing required jobs, copy the advisory GitHub Actions example.

Plans are advisory. This release does not execute, skip or cancel CI jobs. SKIP_CANDIDATE is a candidate, not permission to bypass a check. Keep full CI authoritative while evaluating Core. Missing command synthesis, unsupported configuration and incomplete evidence must not be treated as an empty test suite. Static analysis is not proof that a test can safely be omitted.

For Maven jobs whose CI runs a lifecycle goal or profiles other than test, declare them in diffci.json (or the diffci key in package.json):

{ "maven": { "goal": "verify", "profiles": ["run-its"] } }

This changes the proposed command, such as mvn -pl tools -am verify -P run-its; Core still does not run it. Confirm the command against the repository's CI workflow before using it for execution.

What is included

Declared pnpm workspaces with a single Vitest test phase can receive package-local command plans. Each selected path must belong to a discovered test in a declared package. Runtime selections are batched to keep commands below Windows limits; packages declaring --typecheck retain a separate full type-test phase. Packages with compound scripts, lifecycle hooks, overlapping ownership or discovery blind spots refuse selective execution. Commands remain advisory and must be executed separately; CLI automatic timing of multiple commands requires additional integration.

AreaImplementation
Changes and dependenciesGit deltas, repository inventory, JS/TS dependency graph, impact traversal
Safety and planningRisk signals, conservative fallback, always-run policy, test selection and command synthesis
CI analysisGitHub Actions evidence, job/reference graphs, conditions and matrix inference
Reproducible evaluationSynthetic benchmark and portable regression tests
MeasurementCurrent-process wall time, CPU time and ending RSS; explicit-input cost/energy/carbon estimates

The library entry point exports analyzeCheckout, buildDependencyGraph, ImpactAnalyzer, DefaultCIPlanner, collectEvidence, inferPipeline, measure and estimateImpact. Lower-level modules are available as package subpaths. This repository is the engine source, and the published @diffci.com/diffci CLI is released from it. The corresponding source for the previously published v0.3.10 package is retained in the source archive at tagged commit 4c76be14.

npm run typecheck
npm run audit:boundary
npm run benchmark

The benchmark constructs one temporary synthetic repository containing 100 chained modules and 100 importing test files. In one Node.js process it builds that same dependency graph three times, sequentially, and removes the fixture afterward. Install dependencies first with npm ci, then run npm run benchmark under Node.js 22.5 or later. Record the emitted node and platform alongside the machine CPU, memory, operating system, and competing workload when comparing runs. The three builds share process, operating-system, and compiler warmup effects; they are not independent repetitions with controlled cold caches.

Each entry in runs has these fields:

FieldInterpretation
basis: "measured"The process counters were observed rather than estimated.
scope: "current-process"Only the benchmark's Node.js process is covered, not the whole host, container, or CI job.
wallSecondsElapsed wall-clock time around one buildDependencyGraph call.
cpuSecondsUser plus system CPU time consumed by the current process during that call; it can differ from wall time.
rssBytesAtEndResident-set size sampled after the call. It is neither peak memory nor the memory allocated by that call alone.
nodes, edgesGraph size. The current fixture should produce 200 nodes and 199 dependency edges.
confidenceDependency-graph confidence for this synthetic fixture. COMPLETE does not prove that a real repository can safely skip tests.

For example, wallSeconds: 0.79, cpuSeconds: 0.84, and rssBytesAtEnd: 327868416 mean that one graph build elapsed for about 0.79 seconds, consumed about 0.84 current-process CPU seconds, and ended with roughly 328 MB RSS. They do not mean that CI became 0.79 seconds faster or saved 328 MB. The script does not execute tests, installation, builds, or a full-versus-selected comparison, so its timings must not be reported as CI savings.

The separate estimateImpact API can calculate estimatedUsd, estimatedKwh, and estimatedKgCo2e only from caller-supplied computeSeconds, price, power, and grid-intensity assumptions. Those values have basis: "estimated"; they are modeled scenarios, not benchmark output, physical measurements, billing records, or verified avoided emissions.

Sponsors

DiffCI Core is supported by its GitHub Sponsors. Sponsorship funds maintenance, broader language and CI support, and reproducible benchmarks.

Partners

No Partner sponsors yet. Become the first.

Team Sponsors

No Team Sponsors yet. Team Sponsor logos also appear on diffci.com/sponsors.

Backers

No Backers yet. Backers at $25/month or above are listed here.

Sponsors' bug reports are labelled sponsor and triaged first.

Core and Cloud

ComponentAvailabilityLicense
This Core repositoryPublic sourceMIT
DiffCI Cloud: hosting, billing, organization management, dashboard, managed servicesPrivate sourceProprietary; not included here
Optional enterprise modulesNot released hereA separate source-available license would apply if offered

Only the explicitly extracted Core files and standalone tooling are in this repository. There is no private monorepo history, customer data, deployment configuration or Cloud implementation here. Core is usable independently; it is not a promise that the private product has already migrated to consume this package.

DiffCI Cloud's team features are free for public open-source repositories: an organization whose connected repositories are all public on GitHub gets them at no cost.

What stays in Core, and never moves to Cloud, is set out in STEWARDSHIP.md. The plan DiffCI writes for each change is an open format other tools can produce or consume: the change plan. Report vulnerabilities privately as described in SECURITY.md.

Contributing

Most open work is teaching DiffCI to narrow changes it currently treats as "run everything": config files, package.json edits and lockfiles. Each case is small and comes with a synthetic fixture, so you don't need to learn the whole engine. Django and PHP support are next and split into one-PR steps: see the Django and PHP tracking issues, or the Go, Maven and Rust ones for fewer full-suite fallbacks. Pick a good first issue and see CONTRIBUTING.md; a dev container is included.

Contributors

Thank you to everyone who has improved DiffCI:

  • @KingEmma7 Emmanuel Mawulolo Tagbor, Regular contributor: diffci check-plan for comparing a saved plan with full-suite JUnit results (#74), Yarn Classic lockfile narrowing (#63), and follow-ups (#67, #70); narrow supported Yarn Berry lockfile changes (#245).
  • @FredApps @FredApps, Area maintainer: Rust: the Rust (Cargo) adapter, from Cargo package and test-target mapping (#91) through crate-dependency selection (#92), conservative cargo test commands (#93), registration inside the release boundary (#94) and a pinned public replay on hosted CI (#95), and five safety fixes for runtime module dependencies, Maven module resources, runner suites, deleted assets and Go runtime inputs (#32–#36); match the verify usage line that lists --staged and --paths (#203); list npm run agent:check with the local checks in CONTRIBUTING (#289); say Rust is supported experimentally in the roadmap (#291); retry removing Cargo-built fixtures on Windows (#235); assert the interpreter the analysis recorded, not a second probe (#310); build dist-client once instead of in every test file (#202); find the npm-cli.js npm runs, not one under its global prefix (#240); drop the dead --share-usage flag from the help (#312); expand matrix include the way GitHub does (#165); compare if: strings ignoring case, as GitHub does (#170); an empty or comment-only workflow file no longer crashes diffci plan (#171); parse numstat -z renames so renamed files keep isBinary (#176); compile globs character by character so regex metacharacters stay literal (#179); split run lines into argv the way sh would (#181); read GitHub Actions path filters from the workflow's on: triggers (#182); plan a purpose from a job that runs its step (#185); treat Azure Pipelines and other CI providers' pipeline files as CI config (#186); only an ancestor script makes a cycle (#188); match task input patterns as path globs (#174); read template text after a substitution in script checks (#219); treat a change to an extended tsconfig base as global (#237); lint changed files of types the ESLint config adds (#242); check double-quoted expansion in linear time (#302); resolve imports against a test package's basedir (#205); follow pytest plugins that addopts loads with -p (#207); expand wildcards in pytest testpaths (#209); recognise Failsafe's IT* and *ITCase test names (#211); ignore XML comments when reading a POM (#213); ignore go.mod fixtures under testdata, _ and . directories (#215); keep +, $ and ^ literal in globs, and match everything with ** (#249); resolve a leading in testPathIgnorePatterns (#253); install with --mode=skip-build on Yarn 2 and later (#200); coverage map records files a test reads by relative path (#201); turn the shared graph cache off when its secret is too short (#264); answer ping with an empty result (#295); list both paths of a renamed file in changed_files (#299); replace a local action-v tag left by a rejected push on rerun (#280); do not persist write-scoped checkout tokens in publish and merge-gate (#282); count observe runs as the GitHub Action surface (#285); say usage counting is on where the docs said DiffCI sends nothing (#287); count Azure DevOps task runs as surface azure-devops-task (#307); resolve Vue paths aliases against the declaring tsconfig without baseUrl (#268); link the implementation next to a resolved declaration file (#270); name the adapter runner when a repository has no test files (#217); fall back when an npm lockfile bump meets undeclared imports (#246); translate wildcards inside import.meta.glob brace alternatives (#274); treat modelled Python files as code when resolving readers (#257); let declared inputs and alwaysRunTests select pytest and Maven tests (#272); cover a program the test script runs directly (#306); keep a GitLab subgroup project's full path in the report (#266); name a member NAME@VERSION when a dependency shares its name (#232); keep required-feature targets' packages whole instead of blocking the repository (#248); confine helper scripts and native sources to the crate that owns them (#250); add covering crates instead of refusing when features would differ (#254); pinned anstyle replay narrows real revisions with the same failures (#258); read byte and C raw strings as raw in the input scan (#228); treat quoted keys and inline tables in Cargo config as build settings (#229); scan doctest code in doc comments for inputs (#230); resolve literal paths from the workspace root too (#231); list the Rust planner's no-command reasons and check them (#233); restore the non-Windows branch of the .mcp.json MCP launcher (#167); infer go test or pytest when package.json has no test script (#293); cover every commit of a branch's first push (#309); --tail-bytes 0 keeps no output, and tails are cut by bytes (#239); fail verify when the time limit leaves gate checks unrun (#260); plan verify --redact-paths with real paths, redact the receipt (#276); read nested package-manager calls like top-level ones (#198); run a Bun repository's test script with bun run test (#199); accept literal #[path] modules inside their own crate (#234); accept rustup chains, deny-warnings flags and whole-workspace cargo test as CI evidence (#252); do not run gate steps once the time limit is used up (#262); Do not report a changed worktree when there was no checkout to compare (#283); install dependencies in the generated verification workflow (#297); model local path packages inside the repository instead of blocking it (#256).
  • @shounakjoshi88-a11y Shounak Joshi: Go package-owned testdata selection, so a golden-file edit runs the owning package's tests instead of the full suite (#85).
  • @bfs2021 @bfs2021: bun.lock and npm-shrinkwrap.json recognition in the lockfile signal (#64).
  • @sarankumar1325 @sarankumar1325: lockfile and config-file signals. One shared lockfile list replaced every hand-written copy in src/ (#97): reference resolution, impact analysis and the lockfile-free dependency check (#118–#120), config-file discovery and impact classification, where npm-shrinkwrap.json is now a config change like every other lockfile (#151, #152), the Vue analysis cache (#107), CI inference (#105, #146) and both path baselines, which now also fall back on npm-shrinkwrap.json (#147, #148). A full run when only babel.config.* or .babelrc* changes (#108) or a package's Jest, Vitest or Babel config changes (#121), where these files used to select no tests. Regression tests for vite.config-only changes in Vitest projects (#109) and for a root Vitest config that turns isolation off (#149), and removal of dead config-file patterns with a test for every entry (#106). Go replace targets inside the repository modelled under root-module scope instead of forcing the full suite (#110). The no-runnable-command reasons, documented with a test that keeps the table complete (#150), and "\0" escapes in place of raw NUL bytes, so git and ripgrep can search those source files again (#145); document verify-workflow, pilot-packet, validate-specs and telemetry (#153); pytest.ini wins even when empty, and read .pytest.ini (#314); read pyproject pytest lists as TOML strings (#315); pytest settings from tox.ini and setup.cfg (#316); show that declared inputs select pytest tests, and document it (#319); find tests under a relative (#317); read pytest.toml and the native [tool.pytest] table (#318); fall back for Java sources outside the standard roots (#328).
  • @robertwmccarty-caelum @robertwmccarty-caelum: the Vitest reporter/timeout config fixture and its narrowing conditions (#53, landed in #65).
  • @varshith84 Varshith Sai Pasupuleti: tests for transitive Vue selection and full fallback (#16).
  • @aniKet0753 @aniKet0753: a diffci help command list and USAGE text that match the commands the CLI runs, with a test that fails when a command is missing (#130).
  • @tyza66 @tyza66: name the tool configuration behind a fallback instead of "Unknown changed file" (#144); name a tsconfig variant as TypeScript configuration instead of unknown (#155); fall back for Mocha configuration, and say so (#156); name tool-version files instead of reporting them as unknown (#326); name mise.local.toml with the other tool-version files (#332).

Every merged contribution is credited in the release notes, here, on diffci.com/contributors, and next to the feature in the docs. The README and website entries are added automatically when a pull request merges. A contributor tier above Contributor is written after the name, as in Area maintainer: Rust, and diffci.com copies it from here.

License

See LICENSE, NOTICE and CONTRIBUTING.md. Releases before the MIT switch were published under AGPL-3.0-only and are now also available under MIT; see LICENSE-HISTORY.md. Grant eligibility depends on the particular grant agreement. No eligibility or legal separation is guaranteed by this repository layout.

See the release boundary for scope and current limitations.

Useful? Star the engine and share a reproducible pilot result.