fake-driven-testing

Fake-Driven Testing Architecture for Python

Use this skill when: Writing tests, fixing bugs, adding features, or modifying gateway layers in Python projects.

No prerequisites. This skill is self-contained. It focuses on testing architecture, not language-specific style.

Overview

This skill provides a defense-in-depth testing strategy with five layers for Python applications:

┌─────────────────────────────────────────────────┐
│  Layer 5 "smoke": Business Logic Integration Tests (5%)  │  ← Smoke tests over real system
├─────────────────────────────────────────────────┤
│  Layer 4 "logic": Business Logic Tests (70%)             │  ← Tests over fakes (MOST TESTS)
├─────────────────────────────────────────────────┤
│  Layer 3 "pure": Pure Unit Tests (10%)                   │  ← Zero dependencies, isolated testing
├─────────────────────────────────────────────────┤
│  Layer 2 "real-sanity": Integration Sanity Tests (10%)   │  ← Fast validation with mocking
├─────────────────────────────────────────────────┤
│  Layer 1 "fake-check": Fake Infrastructure Tests (5%)    │  ← Verify test doubles work
└─────────────────────────────────────────────────┘

Philosophy: Test business logic extensively over fast in-memory fakes. Use real implementations sparingly for integration validation.

Terminology note: The "gateway layer" (also called adapters/providers) refers to thin wrappers around heavyweight external APIs (databases, filesystems, HTTP APIs, message queues, etc.). The pattern matters more than the name.

Quick Decision: What Should I Read?

Adding a feature or fixing a bug? → Read quick-reference.md first, then workflows.md#adding-a-new-feature

Need to understand where to put a test? → Read testing-strategy.md

Working with Python-specific patterns? → Read python-specific.md

Adding/changing a gateway interface? → Read gateway-architecture.md, then workflows.md#adding-a-gateway-method

Wondering where the DI boundary is? → Read gateway-architecture.md#the-di-boundary-only-fake-gateways — only gateways get fakes

Need to understand non-ideal states vs exceptions? → Read non-ideal-states.md

Found tests using unittest.mock that should use fakes? → Read mock-to-fake-conversion.md

Need to implement a specific pattern (CliRunner, builders, etc.)? → Read patterns.md

Want to extend the gateway system (e.g., dry-run preview)? → Read advanced-extensions.md

Not sure if I'm doing it right? → Read anti-patterns.md

Just need a quick lookup? → Read quick-reference.md

When to Read Each Reference Document

📖 gateway-architecture.md

Read when:

• Adding or changing gateway/ABC interfaces
• Understanding the ABC/Real/Fake pattern
• Need examples of gateway implementations
• Want to understand what gateways are (and why they're thin)
• Creating a backend (higher-level abstraction that composes gateways)

Contents:

• What are gateway classes? (naming: gateways/adapters/providers)
• The three core implementations (ABC, Real, Fake)
• Code examples for each
• When to add/change gateway methods
• Design principles (keep gateways thin)
• Common gateway types (Database, API, FileSystem, MessageQueue)
• The DI boundary — only gateways get fakes

📖 non-ideal-states.md

Read when:

• Designing return types for gateway operations that can fail
• Deciding between exceptions and discriminated unions
• Implementing error injection in fakes
• Understanding error boundaries (where try/except belongs)

Contents:

• Non-ideal states vs exceptions (the core distinction)
• Decision framework for choosing between them
• How this shapes gateway signatures and fake design
• Error boundaries (try/except only in Real implementations)
• Three test categories per discriminated union operation
• The tracking-on-error decision
• isinstance() for type narrowing (never truthiness)

📖 mock-to-fake-conversion.md

Read when:

• Tests use unittest.mock.patch or @patch decorators
• An agent wrote tests with mocks instead of gateway fakes
• Converting existing mock-based tests to the gateway pattern

Contents:

• Step-by-step conversion workflow (audit, find/create gateway, inject, rewrite)
• "subprocess.run is never the right gateway boundary"
• Monkeypatch decision tree (when it's still OK)
• Common pitfalls (wrong abstraction level, subprocess-level gateways)

📖 testing-strategy.md

Read when:

• Deciding where to put a test
• Understanding the five testing layers
• Need test distribution guidance (5/70/10/10/5 rule)
• Want to know which layer tests what

Contents:

• Layer 1 "fake-check": Unit tests of fakes (verify test infrastructure)
• Layer 2 "real-sanity": Integration sanity tests with mocking (quick validation)
• Layer 3 "pure": Pure unit tests (zero dependencies, isolated testing)
• Layer 4 "logic": Business logic over fakes (majority of tests)
• Layer 5 "smoke": Business logic integration tests (smoke tests over real systems)
• Decision tree: where should my test go?
• Test distribution examples

📖 python-specific.md

Read when:

• Working with pytest fixtures
• Need Python mocking patterns
• Testing Flask/FastAPI/Django applications
• Understanding Python testing tools
• Need Python-specific commands

Contents:

• pytest fixtures and parametrization
• Mocking with unittest.mock and pytest-mock
• Testing web frameworks (Flask, FastAPI, Django)
• Python testing commands
• Type hints in tests
• Python packaging for test utilities

📖 workflows.md

Read when:

• Adding a new feature (step-by-step)
• Fixing a bug (step-by-step)
• Adding a gateway method (complete checklist)
• Changing an interface (what to update)
• Managing dry-run features

Contents:

• Adding a new feature (TDD workflow)
• Fixing a bug (reproduce → fix → regression test)
• Adding a gateway method (8-step checklist with examples)
• Changing an interface (update all layers)
• Managing dry-run features (wrapping pattern)
• Testing with builder patterns

📖 patterns.md

Read when:

• Implementing constructor injection for fakes
• Adding mutation tracking to fakes
• Using CliRunner for CLI tests
• Building complex test scenarios with builders
• Testing dry-run behavior
• Need code examples of specific patterns

Contents:

• Constructor injection (how and why)
• Mutation tracking properties (read-only access)
• Using CliRunner (not subprocess)
• Builder patterns for complex scenarios
• Simulated environment pattern
• Error injection pattern
• Dry-run testing pattern

📖 anti-patterns.md

Read when:

• Unsure if your approach is correct
• Want to avoid common mistakes
• Reviewing code for bad patterns
• Debugging why tests are slow/brittle

Contents:

• ❌ Testing speculative features
• ❌ Hardcoded paths in tests (catastrophic)
• ❌ Not updating all layers
• ❌ Using subprocess in unit tests
• ❌ Complex logic in gateway classes
• ❌ Fakes with I/O operations
• ❌ Testing implementation details
• ❌ Incomplete test coverage for gateways

📖 quick-reference.md

Read when:

• Quick lookup for file locations
• Finding example tests to reference
• Looking up common fixtures
• Need command reference
• Want test distribution guidelines

Contents:

• Decision tree (where to add test)
• File location map (source + tests)
• Common fixtures (tmp_path, CliRunner, etc.)
• Common test patterns (code snippets)
• Example tests to reference
• Useful commands (pytest, ty, etc.)
• Quick checklist for adding gateway methods

Quick Navigation by Task

I'm adding a new feature

1. Quick start: quick-reference.md → Decision tree
2. Step-by-step: workflows.md#adding-a-new-feature
3. Patterns: patterns.md (CliRunner, builders)
4. Avoid: anti-patterns.md (speculative tests, hardcoded paths)

I'm fixing a bug

1. Step-by-step: workflows.md#fixing-a-bug
2. Patterns: patterns.md#constructor-injection-for-fakes
3. Examples: quick-reference.md#example-tests-to-reference

I'm adding/changing a gateway method

1. Understanding: gateway-architecture.md
2. Step-by-step: workflows.md#adding-a-gateway-method
3. Checklist: quick-reference.md#quick-checklist-adding-a-new-gateway-method
4. Avoid: anti-patterns.md#not-updating-all-layers

I don't know where my test should go

1. Decision tree: quick-reference.md#decision-tree
2. Detailed guide: testing-strategy.md
3. Examples: quick-reference.md#example-tests-to-reference

I need to implement a pattern

1. All patterns: patterns.md
2. Examples: quick-reference.md#common-test-patterns

I think I'm doing something wrong

1. Anti-patterns: anti-patterns.md
2. Correct approach: workflows.md

Visual Layer Guide

┌──────────────────────────────────────────────────────────────┐
│ Layer 5 "smoke": Business Logic Integration Tests (5%)       │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Real database, filesystem, APIs, actual subprocess        │ │
│ │ Purpose: Smoke tests, catch integration issues           │ │
│ │ When: Sparingly, for critical workflows                  │ │
│ │ Speed: Seconds per test                                   │ │
│ │ Location: tests/e2e/ or tests/integration/               │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 4 "logic": Business Logic Tests (70%) ← MOST TESTS    │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ FakeDatabase, FakeApiClient, FakeFileSystem              │ │
│ │ Purpose: Test features and business logic extensively    │ │
│ │ When: For EVERY feature and bug fix                      │ │
│ │ Speed: Milliseconds per test                              │ │
│ │ Location: tests/unit/, tests/services/, tests/commands/  │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 3 "pure": Pure Unit Tests (10%)                        │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Zero dependencies, no fakes, no mocks                    │ │
│ │ Purpose: Test isolated utilities and helpers             │ │
│ │ When: For pure functions, data structures, parsers       │ │
│ │ Speed: Milliseconds per test                              │ │
│ │ Location: tests/unit/                                     │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 2 "real-sanity": Integration Sanity Tests (10%)        │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ RealDatabase with mocked connections                     │ │
│ │ Purpose: Quick validation, catch syntax errors           │ │
│ │ When: When adding/changing real implementation           │ │
│ │ Speed: Fast (mocked)                                      │ │
│ │ Location: tests/integration/test_real_*.py               │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

┌──────────────────────────────────────────────────────────────┐
│ Layer 1 "fake-check": Fake Infrastructure Tests (5%)         │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Test FakeDatabase itself                                 │ │
│ │ Purpose: Verify test infrastructure is reliable          │ │
│ │ When: When adding/changing fake implementation           │ │
│ │ Speed: Milliseconds per test                              │ │
│ │ Location: tests/unit/fakes/test_fake_*.py               │ │
│ └──────────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────────┘

Key Principles

1. Thin gateway layer: Wrap external state, push complexity to business logic
2. Fast tests over fakes: 70% of tests should use in-memory fakes
3. Defense in depth: Fakes → sanity tests → pure unit → business logic → integration
4. Test what you're building: No speculative tests, only active work
5. Update all layers: When changing interfaces, update ABC/Real/Fake
6. The DI boundary: Only gateways get fakes; everything above them is tested with real logic and fake gateways

Layer Selection Guide

Distinguishing Layer 3 "pure" from Layer 4 "logic":

• Layer 3 "pure" (Pure Unit Tests): ZERO dependencies - no fakes, no mocks, no external state

  • Testing string utilities: sanitize_branch_name("feat/FOO") → "feat-foo"
  • Testing parsers: parse_git_status("## main") → {"branch": "main"}
  • Testing data structures: LinkedList.append() without any external dependencies

• Layer 4 "logic" (Business Logic Tests): Uses fakes for external dependencies

  • Testing commands: create_worktree(fake_git, name="feature")
  • Testing workflows: submit_pr(fake_gh, fake_git, ...)
  • Testing business logic that coordinates multiple integrations

If your test imports a Fake*, it belongs in Layer 4 "logic", not Layer 3 "pure".

Default Testing Strategy

When in doubt:

• Write test over fakes (Layer 4 "logic") for business logic
• Write pure unit test (Layer 3 "pure") for utilities/helpers with no dependencies
• Use pytest with fixtures
• Use tmp_path fixture (not hardcoded paths)
• Follow examples in quick-reference.md

Summary

For quick tasks: Start with quick-reference.md

For understanding: Start with testing-strategy.md or gateway-architecture.md

For step-by-step guidance: Use workflows.md

For implementation details: Use patterns.md

For error handling design: Check non-ideal-states.md

For converting mocks to fakes: Check mock-to-fake-conversion.md

For validation: Check anti-patterns.md

For Python specifics: Check python-specific.md