SmartFetch

Zero-trust isolated web retrieval, PDF/Markdown extraction, headless Chromium screenshots, site intelligence (tech stack/contacts/JSON-LD), website URL mapping, and parallel batch fetch with Base + Solana x402 payments.

Hosted MCP Server

npx add-mcp 'https://smartfetch-production-ea53.up.railway.app/mcp'

Installs into Claude Code, Codex, Cursor and more

Documentation

SmartFetch V1.15.0 — Production Isolated Browser & x402 Payments

SmartFetch provides reliable, high-performance public webpage retrieval and rendering for AI agents and developer applications with zero-trust browser isolation and x402 micropayment protection.


Live Interactive Playground

Explore SmartFetch directly from your browser without installing anything:

Features:

  • Free Live 402 Inspection: Decode and inspect the live dual-chain Base Mainnet (eip155:8453) and Solana Mainnet (solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp) x402 challenge header, pricing ($0.005 / $0.01 / $0.015 USDC), seller treasury addresses, and service health without a wallet.
  • Dual-Chain Web3 Wallet Payments (Base + Solana): Sign EIP-712 TransferWithAuthorization authorizations via MetaMask, Coinbase Wallet, or Rabby on Base Mainnet (0x2105 / 8453) OR sign SPL TransferChecked transactions via Solana Wallet Standard (Phantom, Solflare, Backpack, MetaMask Solana) on Solana Mainnet with facilitator-sponsored gas.
  • HTML DOM + Local PDF Support: Interactive previews featuring sandboxed HTML rendering (sandbox="allow-scripts"), formatted Markdown, extracted plain text, and local PDF extraction.
  • Client-Side SSRF Protection: Real-time validation preventing requests to internal IPs, localhost, or cloud metadata endpoints (169.254.169.254).
  • Interactive Code Generator: Ready-to-copy integration code in cURL, Python (x402), TypeScript (viem / @solana/web3.js), and MCP configuration JSON.

Apify Store (Cards, PayPal, Standby MCP)

Prefer traditional cloud billing (credit/debit cards, PayPal, or prepaid Apify credits) instead of on-chain crypto wallets? SmartFetch is available as a self-contained Pay-Per-Event (PPE) Actor and Standby MCP Server on the Apify Store:

  • Public Apify Store Actor: https://apify.com/smartfetch.dev/smartfetch
  • Standby MCP Endpoint: https://smartfetch-dev--smartfetch.apify.actor/mcp (pass Authorization: Bearer <APIFY_API_TOKEN>)
  • Bulk B2B Lead & Tech-Stack Enrichment (bulk_inspect): Supply up to 50 bare domains or URLs (["stripe.com", "linear.app", ...]) to concurrently extract flat, CSV/Excel-ready rows containing domain, url, title, description, emails, phones, github, linkedin, x_twitter, tech_stack, https, has_hsts, and has_csp.
  • Site URL Discovery (map / map_website): Discover up to 500 internal URLs on any website via robots.txt (Sitemap:), /sitemap.xml (including nested <sitemapindex>), and internal <a href> links with optional keyword filtering.
  • Pay-Per-Event Pricing:
    • actor-start: $0.00 (no startup fee; $0.00001 platform minimum)
    • page-fetched: $0.01 per successfully retrieved webpage, PDF, or site map (fetch, batch_fetch, & map)
    • site-inspected: $0.01 per successful structured site intelligence or B2B lead enrichment (inspect & bulk_inspect)
    • screenshot-captured: $0.02 per successful headless Chromium screenshot (screenshot)
    • Zero charge on failed requests.

Architecture Overview

SmartFetch separates the public API/MCP interface from headless browser execution across two isolated tiers:

  1. Railway Public Gateway:

    • Hosts the streamable Model Context Protocol (MCP) server (/mcp) and REST API (/fetch).
    • Serves discovery documents: /.well-known/x402, /openapi.json, /llms.txt, /meta, /docs.
    • Strictly enforces x402 micropayment challenges across all retrieval operations ($0.005 USDC on Base Mainnet).
    • Verifies and settles EIP-712 payment authorizations via Coinbase Developer Platform (CDP) facilitators.
  2. DigitalOcean Isolated Browser Worker (VPS):

    • Dedicated headless browser rendering host communicating with Railway via mutual TLS (mTLS) and HMAC-SHA256 signature verification.
    • Replay protection with 16-byte nonce validation and sliding timestamp window.
    • Renders web pages inside ephemeral Docker containers (--network none, non-root UID 10001:10001).
    • All network traffic is mediated by an isolated Unix IPC policy broker enforcing strict SSRF protections against private IP ranges and cloud metadata endpoints (169.254.169.254).

Production Parameters

  • Service Version: 1.15.0
  • Operating Mode: isolated_browser
  • Supported Networks:
    • Base Mainnet (eip155:8453) — Native USDC (0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913)
    • Solana Mainnet (solana:5eykt4UsFv8P8NJdTREpY1vzqKqZKvdp) — Native SPL USDC (EPjFWdd5AufqSSqeM2qN1xzybapC8G4wEGGkZwyTDt1v)
  • Price: $0.005 USDC (5000 atomic units) for webpage retrieval, site inspection, and site URL mapping (POST /fetch, POST /inspect, POST /map, retrieval tools, inspect_webpage, and map_website); $0.01 USDC (10000 atomic units) for screenshot capture (POST /screenshot and screenshot_webpage); $0.015 USDC (15000 atomic units) for parallel batch retrieval (POST /batch-fetch and batch_fetch_webpages)
  • Seller Treasury (payTo):
    • Base (EVM): 0xE29Fcb6ef1534d5F138E48a8A71370BE7770A786
    • Solana (SVM): 88hE4EB4yZwb4F2arPXcG7efwQyU5FDN26DuVpVKVk2W
  • MCP Tools:
    • fetch_webpage ($0.005 USDC)
    • webpage_to_markdown ($0.005 USDC)
    • extract_webpage_text ($0.005 USDC)
    • render_webpage ($0.005 USDC)
    • screenshot_webpage ($0.01 USDC)
    • inspect_webpage ($0.005 USDC)
    • batch_fetch_webpages ($0.015 USDC)
    • map_website ($0.005 USDC)
  • Opt-In Endpoints (enabled when provider keys are configured on server):
    • POST /search / search_web ($0.010 USDC; requires SERPER_API_KEY or EXA_API_KEY)
    • POST /search-and-fetch / search_and_fetch ($0.020 USDC; requires SERPER_API_KEY or EXA_API_KEY)
    • POST /leads / search_leads ($0.030 USDC; requires SERPER_API_KEY; also available as search_leads on Apify)
    • POST /x-search / search_x_posts ($0.008 USDC; requires TWITTERAPI_KEY)
    • map_website ($0.005 USDC)
  • REST Endpoints:
    • POST /fetch ($0.005 USDC)
    • POST /screenshot ($0.01 USDC)
    • POST /inspect ($0.005 USDC)
    • POST /batch-fetch ($0.015 USDC)
    • POST /map ($0.005 USDC)

Documentation & Validation


Agent Quickstart & x402 Micropayments

SmartFetch enables autonomous AI agents to pay for web retrievals using off-chain EIP-712 payment authorizations settled on Base Mainnet.

5-Step Agent Workflow

  1. Discover: Query /.well-known/x402 or /.well-known/agent-card.json to verify pricing ($0.005 USDC on Base Mainnet).
  2. Challenge: Submit an unauthenticated POST /fetch request; SmartFetch replies with HTTP 402 Payment Required.
  3. Sign: Agent inspects the PAYMENT-REQUIRED header, checks its spending limit, and signs an EIP-712 TransferWithAuthorization.
  4. Execute: Resubmit POST /fetch with the signature in the PAYMENT-SIGNATURE header.
  5. Retrieve: SmartFetch verifies the authorization, renders the page inside the DigitalOcean isolated container, settles the payment on Base Mainnet, and returns HTTP 200 with the transaction receipt in PAYMENT-RESPONSE.

Discovery Manifests

EndpointStandardDescription
GET /.well-known/x402x402 Manifest v2Machine-readable pricing ($0.005), USDC asset address, and payment scheme
GET /.well-known/agent-card.jsonA2A ProtocolAgent-to-Agent discovery manifest (capabilities, MCP endpoints, contact)
GET /.well-known/owners.jsonVerifyMCP StandardVerified domain and MCP identity proof
GET /mcpModel Context ProtocolStreamable HTTP transport for AI agent tool invocation
GET /openapi.jsonOpenAPI 3.1REST API schema for automated code generation
GET /llms.txtLLMs.txtLLM-readable summary and prompt engineering reference