qa-testing-blueprints
Enterprise QA-11X v3.0 skill pack & MCP server (@sinfallas/mcp-qa-11x) for 27+ AI coding agents: 10 modular SKILL.md skills, 18 MCP tools, and 12 stack blueprints enforcing a 13-layer QA pyramid, ephemeral non-root Docker execution (Rule Zero), >=95% coverage, Screenplay E2E, AST blast radius, AI evals, and DISA STIG V6R5 / OWASP v4.2 security.
Документация
🌐 Language: English | Español
QA Testing Blueprints (QA-11X v3.0)
Zero-Infra Agentic QA, AI Evals & DevSecOps Governance Blueprint
QA-11X v3.0 turns both AI Coding Agents (Google Antigravity, Claude Code, Cursor, Windsurf, Codex, Copilot, Hermes Agent, and 20+ more) and human engineering teams into deterministic Staff-level QA & DevSecOps operators. Every check runs inside ephemeral, non-root Docker containers (Rule Zero) with zero host contamination, enforcing >= 95% coverage, real database/contract parity, dual-mode AI evaluations, and DISA ASD STIG V6R5 + OWASP v4.2 security controls from day one.
⚡ 30-Second Zero-Friction Quickstart
Bootstrap any repository in 30 seconds without cloning this monorepo:
# Auto-detect stacks & vibe-coded markers, configure .qa-11x.json, AGENTS.md, ACTIVE_TASK.md & PR workflow:
npx -y @sinfallas/mcp-qa-11x@latest quickstart
# Explicit tier, output language policy, agent selection, or dry-run preview:
npx -y @sinfallas/mcp-qa-11x@latest quickstart --tier standard --lang en --agents auto
npx -y @sinfallas/mcp-qa-11x@latest quickstart --tier enterprise --lang es --dry-run
Add Proportional PR Protection to GitHub Actions
Use the hardened composite action .github/actions/qa-11x-pr-check/action.yml in your pull-request workflow:
name: QA-11X PR Check
on: [pull_request]
permissions:
contents: read
jobs:
qa-11x:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683
with:
fetch-depth: 0
- uses: tecno-consultores/qa-testing-blueprints/.github/actions/qa-11x-pr-check@v2
with:
tier: standard
lang: en
🧭 Differential Architecture (QA-11X v3.0)
flowchart TD
UserReq["Developer or PR Trigger"] --> Quickstart["npx @sinfallas/mcp-qa-11x quickstart"]
Quickstart --> Router{"3-Path Execution & Tiers"}
Router -->|"DOCS_ONLY"| FastDocs["make sync-governance + make lint-docs"]
Router -->|"MICRO_CHANGE / standard"| BlastAST["AST Blast Radius & Tiered CI"]
Router -->|"FULL_11X / enterprise"| FullPyramid["13-Layer QA Pyramid + FINOS CALM"]
BlastAST --> RuleZero["Rule Zero Non-Root Docker sinfallas/*"]
FullPyramid --> RuleZero
RuleZero --> Shields["AI Evals Dual-Mode + DISA/OWASP + Release Gate"]
FastDocs --> LocalCommit["Certified Local Commit QA-GIT-GATE-01"]
Shields --> LocalCommit
Progressive Adoption Tiers
| Tier | Target Adoption | Included Gates & Guarantees |
|---|---|---|
starter | Day-1 Adoption (30s) | Zero-plain-text secret scanning, Rule Zero container hygiene, .qa-11x.json, AGENTS.md connector across 27+ AI coding agents, and DOCS_ONLY fast-path |
standard | Daily PR Workflows | Everything in starter + >= 95% unit coverage, strict types (mypy --strict, tsc, PHPStan max), AST Blast-Radius CI partitioning (40%–60% compute savings), and UI anti-pattern linting (QA-UI-ANTI-001..005) |
enterprise | Regulated Production | Full 13-layer FULL_11X pyramid, blocking FINOS CALM architecture validation, PostgreSQL 17 RLS + WORM seq_num audit chains, DISA STIG V6R5 / OWASP v4.2, 3-phase clock assertions (t < starts_at, starts_at <= t <= ends_at, t > ends_at), Pact + k6 load, and multi-signal Release Readiness Gate (GO / NO_GO / SKIPPED) |
Zero-Infra Ephemeral Docker Matrix (sinfallas/*)
All runtime, lint, mutation, contract, and BDD suites execute inside non-root containers (-u "$(id -u):$(id -g)" -e HOME=/tmp) with Trivy HIGH/CRITICAL = 0:
| Ephemeral Image | Role in QA-11X v3.0 | Certified Tags & Pre-Baked Tooling |
|---|---|---|
| sinfallas/base-python-uv | Python, FastAPI, AI Evals & Vibe-Rescue | 3.11, 3.12, 3.13, 3.14, 3.15, latest (uv, ruff, mypy, pytest, bandit, radon, mutmut, deepeval, pact-python, tesseract-ocr) |
| sinfallas/base-node-ionic | MCP Server, Node, Vue 3, React & CALM | 22, 24, 26, latest (pnpm, promptfoo, @finos/calm-cli, playwright, @axe-core/playwright, @pact-foundation/pact) |
| sinfallas/base-mobile-appium | Native & Hybrid Mobile W3C Bridge (Android/iOS) | 26, latest (Node.js 26, pnpm, OpenJDK 21, Android SDK/ADB, Appium 3, uiautomator2, xcuitest, WebdriverIO v9) |
| sinfallas/basephpcomposer | PHP 8.2–8.5 APIs & Services | 8.2, 8.3, 8.4, 8.5, latest (pcov, phpstan, psalm, pest, phpunit, infection) |
| sinfallas/karatelabs | API & HTTP/Edge/OSHP/XML BDD Contracts | 1.5.1 / 11x-v2-local (Java + Maven + Karate JUnit5; proportional to stateful multi-service FULL_11X flows) |
| sinfallas/base-bash-qa | DevOps & POSIX Shell Scripts | latest (ShellCheck, shfmt, BATS-core, bats-mock, kcov) |
| sinfallas/remote-graphify | AST Context Optimization & Blast Radius | latest (graphify Tree-sitter AST graph & cluster extraction) |
| Infra & Load Engines | Live DB, Cache & Smoke Load | postgres:17-alpine, eqalpha/keydb:alpine, grafana/k6:2.4.0 |
🗂️ Full-Stack, Mobile, AI Evals & Vibe-Rescue Blueprints
| Directory | Domain | Key v2.0 Capabilities |
|---|---|---|
fastapi/ | Python Async APIs | Ruff, Mypy strict, PostgreSQL 17 RLS + Alembic parity, Pact V4 consumer (test_pact_consumer.py), k6 smoke load (smoke.k6.js), Karate BDD |
python/ | Python Libraries & CLIs | UV, Ruff (CC <= 5), Mypy strict, Pytest >= 95%, Hypothesis, Mutmut (MSI >= 80%), Bandit |
node-backend/ | Node.js / TS Microservices | ESLint --max-warnings 0, Vitest >= 95%, Pact V4 provider verifier (provider.pact.test.ts), Stryker, ZAP |
vue3/ | Vue 3 + Vite Frontends | Screenplay Actor.ts, Self-Healing DOM semanticLocator.ts, Vitest >= 95%, @axe-core, Lighthouse CI |
react-frontend/ | React + Vite Frontends | Screenplay Actor.ts, Self-Healing DOM semanticLocator.ts, RTL, Playwright, @axe-core |
php/ | PHP 8.2–8.5 Services | PHP-CS-Fixer, PHPStan Level Max, Psalm Taint Analysis, Pest/PHPUnit + PCOV >= 95%, Infection |
bash-scripts/ | POSIX / Bash Automation | ShellCheck, shfmt, BATS-core, bats-mock, kcov >= 95% |
mobile-appium/ | Native & Hybrid Mobile | Appium 3 / W3C + WebdriverIO (wdio.conf.ts), MobileActor.ts (Android, iOS, Ionic, RN) |
ai-evals/ | Dual-Mode AI & Multimodal QA | RAG faithfulness, Promptfoo OWASP LLM Top 10, WebRTC voice latency (test_voice_websocket_latency.py), PDF/OCR Decimal (test_pdf_table_fidelity.py), golden_from_traces.py |
vibe-rescue/ | Vibe-to-Enterprise Hardening | audit_vibe_prototype.py (VIBE-SEC-001..QA-005), PostgreSQL 17 RLS/WORM template (rls_hardening.sql) |
architecture/ | Architecture-as-Code | Blocking FINOS CALM topology (qa-11x.calm.json) & 7 mandatory 11X controls (make validate-calm) |
mcp-server/ | @sinfallas/mcp-qa-11x | Node.js MCP server (stdio), quickstart, install-skill, init-agents, reconcile-agents, tms-sync & A2A agent card |
🛠 Testing Tools Matrix
Below is the corporate validation stack summarized per environment and test type. N/A indicates test categories that do not apply due to the intrinsic nature of the environment (e.g. a standalone Python CLI, Bash script, or declarative architecture topology does not expose an HTTP server, so DAST and Chaos Engineering do not apply).
| Environment | Linting / Formatting | Unit Tests / Mocking | Coverage | Dependency Audit | Static Security (SAST) | Mutation | BDD / Black-Box | Concurrency / Stress | Profiling / Complexity | DAST (Dynamic) | Chaos Engineering | Observability (Metrics) | IaC / Docker Audit | Local Git Hooks |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Python (python/) | Ruff + Mypy (Strict) | Pytest + Hypothesis + pytest-mock / respx | Pytest-Cov (>= 95%) | pip-audit + CycloneDX SBOM | Bandit | Mutmut (>= 80%) | N/A | N/A | py-spy / memray + Radon (CC <= A) | N/A | N/A | Pushgateway + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
FastAPI (fastapi/) | Ruff + Mypy (Strict) | Pytest + pytest-asyncio + respx + PostgreSQL 17 + KeyDB + Alembic | Pytest-Cov (>= 95%) | pip-audit + CycloneDX SBOM | Bandit | Mutmut (>= 80%) | Karate Labs + Schemathesis + Pact V4 | Locust (Headless) + k6 | py-spy / memray + Radon (CC <= A) | OWASP ZAP Baseline | Pumba | prometheus-fastapi-instrumentator + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
Node Backend (node-backend/) | ESLint (--max-warnings 0) + Prettier + tsc | Vitest + Supertest + Nock | @vitest/coverage-v8 (>= 95%) | pnpm audit + .npmrc (ignore-scripts=true) | eslint-plugin-security | Stryker (>= 80%) | Karate Labs + Pact V4 | Artillery + k6 | 0x + Clinic.js + SonarJS (<= 15) | OWASP ZAP Baseline | Pumba | express-prom-bundle + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
Vue 3 (vue3/) | ESLint (--max-warnings 0) + Prettier + vue-tsc | Vitest + Vue Test Utils + Screenplay + Playwright + @axe-core | @vitest/coverage-v8 (>= 95%) | pnpm audit + .npmrc (ignore-scripts=true) | eslint-plugin-security | Stryker | Karate Labs (HTTP/OSHP) + Playwright UI | N/A | Lighthouse CI + SonarJS (<= 15) | OWASP ZAP Baseline | N/A | nginx-prometheus-exporter + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
React (react-frontend/) | ESLint (--max-warnings 0) + Prettier + tsc | Vitest + React Testing Library + Screenplay + Playwright + @axe-core | @vitest/coverage-v8 (>= 95%) | pnpm audit + .npmrc (ignore-scripts=true) | eslint-plugin-security + react-hooks | Stryker | Karate Labs (HTTP/OSHP) + Playwright UI | N/A | Lighthouse CI + SonarJS (<= 15) | OWASP ZAP Baseline | N/A | nginx-prometheus-exporter + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
PHP 8.2–8.5 (php/) | PHP-CS-Fixer (@PER-CS2.0) + PHPStan (Level Max) | Pest / PHPUnit 11.5+ + MockHandler + PostgreSQL 17 + KeyDB | PCOV (>= 95%) | composer audit + roave/security-advisories + CycloneDX SBOM | Psalm (--taint-analysis) | Infection (MSI >= 80%) | Karate Labs + Swagger-PHP / Schemathesis | Locust / k6 | tomasvotruba/cognitive-complexity (<= 9) | OWASP ZAP Baseline | Pumba | promphp/prometheus_client_php + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
Bash (bash-scripts/) | ShellCheck + shfmt | BATS-core + bats-support + bats-assert + bats-file + bats-mock | kcov (>= 95%) | N/A | ShellCheck | N/A | N/A | N/A | hyperfine + time -v | N/A | N/A | Pushgateway + Prometheus | Trivy + Hadolint + Gitleaks | Lefthook |
Mobile Appium (mobile-appium/) | tsc --strict + Prettier | WebdriverIO v9 + MobileActor Screenplay (W3C Bridge, Android/iOS/Ionic/RN) | node:test / C8 (>= 95%) | pnpm audit + .npmrc + CycloneDX SBOM | eslint-plugin-security + Deep-Link / Biometry guards | N/A | Appium 3 (uiautomator2/xcuitest) + Karate BFF | N/A | CC <= 5 + Fail-Verbose /tmp Traces | N/A (delegated to BFF OWASP ZAP) | N/A (Offline Sync & Biometry toggle) | Allure JSON Fail-Verbose (/tmp) | Trivy + Hadolint + Gitleaks (sinfallas/base-mobile-appium:26) | Lefthook |
AI Evals (ai-evals/) | Ruff (CC <= 5) + Mypy (Strict) | Pytest asyncio + DeepEval + WebRTC Voice + PDF/OCR Decimal | Pytest-Cov (>= 95%) | pip-audit + CycloneDX SBOM | Bandit + Promptfoo OWASP LLM Top 10 | N/A | DeepEval + Promptfoo (Offline + Live) | WebRTC TTFB <= 500ms | Radon (CC <= A) + golden_from_traces.py | Promptfoo LLM Red-Team | N/A | JSON Eval Trajectory Trace | Trivy + Gitleaks | Lefthook |
Vibe Rescue (vibe-rescue/) | Ruff (CC <= 5) + Mypy (Strict) | Pytest + audit_vibe_prototype.py (VIBE-SEC-001..QA-005) | Pytest-Cov (>= 95%) | pip-audit + rls_hardening.sql | Bandit + RLS / AES-256-GCM / WORM | N/A | QA-11X 10/10 Rescue Gate | N/A | Radon (CC <= 5) | OWASP ZAP (post-rescue) | N/A | Finding State Tracker JSON | Trivy + Hadolint + Gitleaks | Lefthook |
Architecture (architecture/) | AJV CLI / CALM (validate_calm_architecture.sh) | CALM Schema & 11-Control Verification (qa-11x.calm.json) | N/A | CALM Control Integrity | 11 Blocking Architecture Controls (RLS/WORM/mTLS) | N/A | FINOS CALM Architecture Gate | N/A | Graphify AST Coupling / Layer Check | N/A | N/A | CALM Validation Report | Trivy + Gitleaks | Lefthook |
MCP Server (mcp-server/) | ESLint (--max-warnings 0) + Prettier + tsc | Vitest + run-example-e2e.mjs (Clean-Room stdio) | @vitest/coverage-v8 (>= 95%) | pnpm audit + .npmrc (ignore-scripts=true) | eslint-plugin-security + promptGuard + audit_security_stig_owasp | Stryker | Karate Labs (mcp_cli_test.feature) | N/A | SonarJS (<= 15) | N/A | N/A | Pass-Quiet Telemetry & PR_VERDICT SHA-256 | Trivy + Gitleaks | Lefthook |
🛡️ Dual-Mode AI Evals, Local LLMs & Security Shield
- Mode 1 — Zero-Cost Deterministic Offline Contracts (
make test-ai-offline):- Runs hermetic RAG, agent-trajectory, WebRTC voice (
TTFB p95 <= 500ms), and OCRDecimaltable checks with stubbed judges (>= 95%coverage,CC <= 5,mypy --strict) at$0API cost. - Protects MCP tool execution fail-closed via
promptGuard.ts(NFKCcanonicalization, zero-width/Trojan Source sanitization, prompt-injection &QA-GIT-GATE-01bypass prevention, PII/secret redaction using<EXAMPLE_REDACTED>placeholders). - Compresses long transcripts deterministically with
.agents/scripts/compact_context_11x.pyand supports air-gapped Ollama / vLLM profiles plus A2A Agent Card discovery (agentCard.ts,docs/specs/14_LOCAL_LLM_AND_A2A_PROFILES.md).
- Runs hermetic RAG, agent-trajectory, WebRTC voice (
- Mode 2 — On-Demand Probabilistic & Red-Team Audits (
make audit-llm):- Executes DeepEval (
GEval,TaskCompletionMetric, faithfulness) and Promptfoo OWASP LLM Top 10 red-teaming against configured cloud providers or local Ollama/vLLM endpoints (SKIPPED_NO_LLM_SECRETS_CONFIGUREDwhen credentials are absent in CI).
- Executes DeepEval (
🤖 10 Modular Skills (< 20 KB), MCP Server & 27+ AI Agents
- Progressive Skill Routing (
< 20 KBbudget): Led by.agents/skills/qa-11x-architect/SKILL.md(< 12 KB), loading 9 specialized sub-skills on demand (qa-11x-devsecops-hardener,qa-11x-systematic-debugging,qa-11x-screenplay-builder,qa-11x-llm-evaluator,qa-11x-vibe-rescue,qa-11x-docker-certifier,docker-trivy-keydb-hardening,graphify-ast-navigator,refactoring-ui). - Public MCP Surface (
@sinfallas/mcp-qa-11xv3.0.0): Exposes 18 Tools and 12 Resources (blueprint://*) overstdio— combining the 12 core v1 tools + 6 v2.0 evaluators (FINOS CALM, Release Gate, Agent Trajectory Evals, OWASP LLM Red Team, Vibe-Rescue, Enterprise TMS Sync) protected end-to-end bypromptGuard&TaskStateGuard. - Human-Confirmed Git Closure (
QA-GIT-GATE-01): Enforced by.agents/rules/git_closure_gate.md— agents certify locally viamake certifyand stop at a localgit commit; automaticgit pushornpm publishis strictly forbidden. - Setup & Catalog Guides: See
docs/agent-setup.en.md(English),docs/agent-setup.md(Spanish),RESOURCES.md,llms.txt(GEO index),catalogs/(QASkills, SkillsLLM),docs/01_ARQUITECTURA_PIRAMIDE_11X.md,TESTING.md,SECURITY.md, andROADMAP.md.
# Install global skills, operational scripts, and thin AGENTS.md connector across detected IDEs:
npx -y @sinfallas/mcp-qa-11x@latest install-skill --init-agents --include-scripts --reconcile
📄 Author, Copyright & License
- Author: Jesus Palencia (
<sinfallas@gmail.com>) - Copyright:
Copyright (C) 2026 Jesus Palencia <sinfallas@gmail.com> (Creado en Tecno Consultores) - License: Distributed under the MIT License.