AlphaAssay

Independent statistical validation for trading signals and backtests: deflated Sharpe, out-of-sample and leakage forensics, signed replayable verdicts; a methodology audit, not investment advice.

GitHub

Documentation

AlphaAssay — MCP server

Signal validation & overfitting forensics for AI agents. Most trading signals are noise. AlphaAssay puts them on trial — deflated Sharpe, out-of-sample, leakage forensics — and returns signed pass/fail verdicts anyone can verify. Methodology audits, not investment advice.

What this repo is. The public documentation and the registry manifest (server.json) for the AlphaAssay MCP server. The validation engine is a hosted, remote MCP — this repo is docs and metadata, not the engine source. The endpoint is live at https://mcp.alphaassay.com/mcp — the calls below are runnable.

What it does

Use this when you need to validate a trading signal, strategy or backtest before capital touches it: statistical signal validation, overfitting detection, deflated Sharpe ratio (multiple-testing aware), walk-forward and out-of-sample testing, look-ahead and data-leakage forensics, placebo tests. It returns a deterministic verdict — pass, conditional, fail or insufficient_evidence — with machine-readable failure gradients and an Ed25519-signed, replayable audit record anyone can verify.

It does not generate strategies and does not give buy/sell advice. It is a methodology audit. Verdicts are demote-only: evidence can lower a grade, never inflate one. Asset-agnostic — crypto, stocks, futures, FX.

Connect (MCP)

The server is a remote, streamable-HTTP MCP endpoint:

https://mcp.alphaassay.com/mcp

Point any MCP-capable client at it. Example client config:

{
  "mcpServers": {
    "alphaassay": {
      "type": "streamable-http",
      "url": "https://mcp.alphaassay.com/mcp"
    }
  }
}

No account is needed for the free tools (assay_demo, assay_graveyard, assay_calibration, assay_certificate_verify, assay_provider_protocol, assay_preflight). Paid tools are billed per call via x402 or credits.

Run it locally (stdio clients, Docker)

This repo ships a real local stdio server (server/index.mjs, dependency-free, Node >= 18): it defines all 19 tools statically (server/tools.json, versioned here) and executes each call against the hosted engine — the engine itself is server-side by design, so a local install can never be a locally editable examiner. Free tools run without credentials; paid tools take an optional ALPHAASSAY_API_KEY environment variable.

node server/index.mjs            # directly
docker build -t alphaassay-mcp . && docker run -i --rm alphaassay-mcp

Stdio client config:

{
  "mcpServers": {
    "alphaassay": { "command": "node", "args": ["/path/to/mcp/server/index.mjs"] }
  }
}

Try it over plain HTTP

Every capability is also a plain HTTPS API — the same request/response shape the MCP wraps. Send a golden specimen (a test signal whose correct verdict is known in advance):

curl -s https://api.alphaassay.com/v1/assay/demo \
  -H "Content-Type: application/json" \
  -d @golden_lookahead.json

Expected response (this specimen is a whipsaw with no net-of-cost edge, so it dies at the first gate — abridged; the verbatim body is in examples/golden_lookahead.expected.json):

{
  "schema": "gauntlet.v1",
  "verdict": "fail",
  "died_at": "net_edge",
  "failure_codes": ["no_net_edge"],
  "family_id": "fam_004b8867ffc7a556",
  "stages": [
    {
      "stage": "net_edge",
      "verdict": "fail",
      "evidence": { "net_sharpe_annualized": null, "net_return_total_pct": 0.0, "trades": 0, "bars": 199 },
      "detail": "No positive net-of-cost edge -- dies before any deeper test."
    }
  ],
  "budget": { "cumulative_n": null, "n_trials_effective": null, "note": "..." },
  "_meta": { "engine_version": "024739a", "api_version": "v1", "as_of": "<varies per call>" },
  "disclaimer": "Consolidated research reality-check. NOT financial advice, no order path. Demote-only."
}

stages also carries funding_edge, family_deflation, power_honesty, significance, cpcv, walk_forward, concentration, placebo, capacity and graveyard_prior. Field-by-field explanation: https://alphaassay.com/docs/verdicts. Runnable scripts for all four specimens: examples/.

Golden specimens (free known-answer tests)

Four prepared signals with a planted property and a known correct verdict — the standing offer to catch AlphaAssay being wrong before you pay it.

specimen	planted property	expected verdict
`golden_clean`	a genuine, persistent edge	`pass` · `died_at: null`
`golden_lookahead`	a whipsaw with no net-of-cost edge	`fail` · `net_edge` · `no_net_edge`
`golden_cherry`	best-of-many parameter cherry-pick	`fail` · `family_deflation` · `deflated_out_at_n=50` + `BACKTEST_TOO_SHORT_FOR_N=50`
`golden_thin`	too little data to judge	`insufficient_evidence`

The verdict is bit-identical on every run — assert on verdict, died_at and failure_codes in CI; only the wall-clock timestamps vary (_meta.as_of, and generated_at in validator provenance). Catalog: https://alphaassay.com/docs/specimens.

Tools

tool	what it does	tier
`assay_demo`	free demo assay + the four golden specimens with known verdicts	free
`assay_signal`	full statistical battery on a signal → verdict + failure gradients + signed record	paid
`assay_backtest`	overfitting forensics on backtest results (deflated Sharpe, PBO/CPCV)	paid
`assay_preflight`	lint a payload's form (DSL schema, OHLCV sanity, trade rows) before spending a paid check	free
`assay_batch`	up to 25 sweep variants in one call, each a family-ledger trial under cumulative deflation — counts, never a ranking	paid
`assay_reproduce`	independently recompute a claimed track record from trades + candles; grades each headline claim against disclosed tolerances	paid
`assay_survivors`	Romano-Wolf stepwise FWER over a T×N sweep matrix — an error-budget disclosure, never a ranking	paid
`assay_pbo`	Probability of Backtest Overfitting over your full parameter-sweep trial matrix (CSCV, purged) — grades the selection process itself	paid
`assay_register`	pre-register a hypothesis with a tamper-proof timestamp, then forward-evaluate on post-cutoff data	paid
`assay_verdict`	fetch / replay / export an existing verdict; offline Ed25519 signature check	replay free · certificate paid
`assay_provider_protocol`	the falsification protocol as machine-readable rules — seven tests for judging ANY signal provider, AlphaAssay included	free
`assay_calibration`	the public signed calibration ledger — is AlphaAssay itself trustworthy?	free

Full tool set (19 assay_* tools) and reference: https://alphaassay.com/docs.

Pricing

Paid calls are flat-priced per completed call — pass and fail cost the same, because you are buying the trial, not a flattering outcome. The free tier (demo, graveyard, calibration, verification) never costs anything. Current numbers, always live: alphaassay.com/pricing or GET https://api.alphaassay.com/v1/meta/pricing.

Verify a verdict yourself

Every verdict is bit-identically replayable, and any paid verdict can be exported as an Ed25519-signed certificate — the free demo verdict is unsigned. Check a certificate in three seconds at https://alphaassay.com/verify, or fully offline with the published public key — https://alphaassay.com/docs/verify-offline.

Docs

Quickstart — https://alphaassay.com/docs/quickstart
Rules for your trading agent (copy-paste) — https://alphaassay.com/docs/ai-rules
Golden specimens — https://alphaassay.com/docs/specimens
Verdicts & the gradient — https://alphaassay.com/docs/verdicts
llms.txt — https://alphaassay.com/llms.txt

Publishing

Published in the official MCP registry as com.alphaassay/mcp; also listed on Glama, Smithery and the public MCP directories. Registry releases track this repository.

Licence

Apache-2.0.

Disclaimer. AlphaAssay is a methodology audit, not investment advice. No order path, no custody, no buy/sell signals. A pass means a signal survived a statistical trial most signals fail — never a promise of any outcome. Verdicts are demote-only.