docs-corpus-audit

docs-corpus-audit -- whole-corpus regrounding pass

The docs corpus drifts silently between releases. docs-sync catches drift introduced by individual PRs at PR-open time. This skill catches the accumulated drift that slips past per-PR review -- stale flag names, dead nav links from past IA reshuffles, deprecation banners that outlived their version targets, factual claims whose source-side truth has moved.

The pattern is A1 PANEL + WAVE EXECUTION + S7 DETERMINISTIC TOOL BRIDGE + A8 ALIGNMENT LOOP + A9 SUPERVISED EXECUTION. The corpus is split into disjoint page scopes; one verifier subagent owns each scope; agents extract factual claims, S7-verify against source, apply surgical fixes inline. The orchestrator then runs an alignment-loop pass to re-verify that applied edits actually ground out true.

This skill is ADVISORY but ACTIONABLE: agents apply edits inline on a working branch. The orchestrator is the sole writer to git -- stages, commits, pushes. Maintainer reviews the resulting PR.

Sibling contract with docs-sync

These two skills share substrate. Be explicit:

Trigger boundary (avoid DISPATCH COLLISION):

• docs-sync triggers on a PR event ("PR opened/synchronized", source-diff-driven).
• docs-corpus-audit triggers on a maintainer ask for a WHOLE-CORPUS pass ("audit the corpus", "reground", "pre-release sweep") -- no PR required, no diff required, the whole corpus is the input.

If a maintainer asks "review this PR's doc impact", route to docs-sync. If they ask "audit all our docs" or "the docs feel stale everywhere", route here.

Architecture invariants

• Wave-batched, not flat. Pages are partitioned into 6-8 disjoint scopes; each scope is one verifier subagent. Cost scales with wave size, not corpus size. A wave of 6 agents on ~10 pages each is the canonical shape.
• Disjoint page ownership. Each subagent has EDIT AUTHORITY on its scope only. No two agents touch the same file -- guarantees no merge conflicts during fan-in.
• S7 verification is mandatory. Every factual claim is verified against deterministic source: uv run apm <verb> --help for CLI, grep -n src/apm_cli/ for symbols, python -c "import ..." for module shape, file-existence checks for nav links. Never assert from LLM recall.
• Surgical edits only. 1-3 line patches per drift, preserving voice. Restructuring is deferred to the orchestrator post-pass, never auto-applied by per-scope agents.
• Single-writer interlock for git. Subagents NEVER run git commit, git push, or gh pr <write>. Orchestrator commits per wave; pushes once per session.
• Alignment loop (A8). After waves return, orchestrator re-greps the corpus for the patterns the agents claimed to fix. Any residue triggers a targeted re-dispatch (max 2 redrafts) or is escalated to maintainer.

Roster (composition, not invention)

Reuse docs-sync's personas. Do NOT invent a one-off "grounding- verifier" role; that's R3 EXTRACT in reverse.

The per-scope subagent prompt that composes python-architect + doc-writer is in assets/subagent-prompt-template.md -- the orchestrator substitutes scope + working dir + branch and dispatches via the task tool.

Process

1. PROBE (A9 SUPERVISED EXECUTION)
   - Check working tree: docs/src/content/docs/ exists?
   - Check working tree: packages/apm-guide/.apm/skills/apm-usage/
     exists? (Rule-4 backfill target. If missing, the audit cannot
     close Rule 4; ask maintainer before continuing.)
   - Check `.apm/docs-index.yml` reachable.
   - Verify on a working branch (not main).

2. RISK-TRIAGE (orchestrator, ~1 LLM call)
   - Read .apm/docs-index.yml only (NOT the corpus body).
   - Bucket pages by drift risk: HIGH (CLI ref, schemas, consumer
     flows), MEDIUM (producer, enterprise policy), LOW (concepts,
     contributing, troubleshooting, integrations).
   - Decide wave order: HIGH first, MEDIUM next, LOW last.

3. WAVE-PLANNER (orchestrator, deterministic)
   - Partition pages into 6-8 disjoint scopes per wave.
   - Each agent gets ~9 pages, mixed surface types.

4. WAVE EXECUTION (parallel, one subagent per scope)
   - Orchestrator dispatches one task per scope using the prompt
     template in assets/subagent-prompt-template.md.
   - Subagents read pages, extract claims, S7-verify, apply
     surgical edits, return JSON per the docs-sync panelist
     schema (mirrored at assets/panelist-return-schema.json).
   - Validate every return against the schema; reject malformed
     JSON.

5. CROSS-CORPUS POST-PASS (orchestrator, deterministic)
   - Run scripts/scan-cross-corpus-drift.sh to grep for patterns
     a per-scope agent cannot see (IA-reshuffle dead links, stale
     deprecation version targets, phantom flag references).
   - Patch residue inline.

6. ALIGNMENT LOOP (orchestrator, deterministic)
   - Re-run scripts/scan-cross-corpus-drift.sh.
   - Re-grep for claims the agents marked DRIFTED-FIXED.
   - If residue: targeted re-dispatch to the owning agent
     (bounded: max 2 redrafts per wave).

7. COMMIT + PUSH (orchestrator, single writer)
   - One commit per wave; structured message naming closed items.
   - Push to working branch.

8. PR + SUMMARY COMMENT (orchestrator)
   - If no PR exists: open one with the [pr-description-skill]
     (../pr-description-skill/SKILL.md).
   - Post per-wave summary comment: pages audited, drift caught,
     fixes applied, items deferred, alignment-loop residue.

Bundled assets

• assets/subagent-prompt-template.md -- the per-scope prompt the orchestrator substitutes and dispatches. Composes python-architect (S7) + doc-writer (surgical edit). Loaded once per scope.
• assets/panelist-return-schema.json -- subagent return schema, mirrored from docs-sync. Loaded once at wave start; validated against every return.
• scripts/scan-cross-corpus-drift.sh -- deterministic grep sweep for cross-corpus patterns (IA dead links, stale deprecation targets, phantom flags). Non-interactive; emits structured matches on stdout, diagnostics on stderr. Run --help for pattern list. Update this script after each major IA reshuffle.

Cost model

Compared to docs-sync (15-call flat ceiling), this skill scales as O(waves), not O(claims), because per-agent work fits in one context window. S7 verification dominates wall-time, not LLM cost.

Boundary (what this skill does NOT do)

• Per-PR doc-impact review -- use docs-sync.
• Single-page typo or copy edit -- direct edit is faster.
• Writing docs for a brand-new feature -- use docs-impact-architect and doc-writer directly.
• Auto-merging or pushing without maintainer review.
• Reviewing code quality, security, or test coverage (out of scope).

Evals

See evals/:

• evals/content-evals.json -- 3 corpus snapshots with seeded drift (stale CLI flag, dead nav link, expired deprecation target); expected behavior is that the skill catches all three and applies surgical fixes that ground out true on re-verification.
• evals/trigger-evals.json -- 10 should-trigger + 10 should-NOT- trigger queries, 60/40 train/val. The val split is the ship gate (>=0.5 should-trigger AND <0.5 should-not-trigger).
• evals/README.md -- how to run.

Provenance

This skill was extracted from a real session that audited the microsoft/apm corpus across 3 waves (PR #1511, 2026-05-27): 112/112 pages audited, 49 surgical fixes, ~25 LLM dispatches, ~30 min wall-time. The session design artifact (genesis hand-off packet) lives in session state, not in this bundle (maintainer- scope, not runtime-loaded).