Spider Cloud MCP Server

Spider MCP Server

The fastest web crawling, scraping, and browser automation server for AI agents. Gives Claude direct access to the web through 22 tools — crawl sites at 100K+ pages/sec, extract structured data with AI, and control remote browsers with built-in anti-bot bypass.

Why Spider

Speed — Crawl 100K+ pages per second. Smart request routing picks HTTP or headless Chrome automatically. Streaming responses start delivering data immediately.
Cost — Pay-per-use credits with no subscription required for core tools. Check your balance anytime with spider_get_credits. AI tools available with an AI subscription.
Reliability — Anti-bot bypass with fingerprinting and proxy rotation. Browser fleet with automatic fallback across Chrome, Firefox, and more. Built-in retry and stealth escalation.

Quick Start

Claude Code

claude mcp add spider -- npx -y spider-cloud-mcp

Set your API key:

export SPIDER_API_KEY="your-api-key"

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Cursor

Add to .cursor/mcp.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Windsurf

Add to ~/.codeium/windsurf/mcp_config.json:

{
  "mcpServers": {
    "spider": {
      "command": "npx",
      "args": ["-y", "spider-cloud-mcp"],
      "env": {
        "SPIDER_API_KEY": "your-api-key"
      }
    }
  }
}

Configuration

Variable	Required	Description
`SPIDER_API_KEY`	Yes	Your Spider API key. Get one at spider.cloud/api-keys

Tools

Core Tools (8)

These work on pay-per-use credits with no subscription required.

Tool	Description
`spider_crawl`	Crawl a website and extract content from multiple pages. Follows links up to a depth/limit.
`spider_scrape`	Scrape a single page. Faster and cheaper than crawling when you need one URL.
`spider_search`	Search the web. Optionally fetch full page content from results.
`spider_links`	Extract all links from a page without fetching content.
`spider_screenshot`	Capture a page screenshot as base64 PNG.
`spider_unblocker`	Access bot-protected content with advanced anti-bot bypass.
`spider_transform`	Convert HTML to markdown or text without making web requests.
`spider_get_credits`	Check your API credit balance.

AI Tools (5)

Natural language web interaction. Describe what you want in plain English.

Requires an AI subscription.

Tool	Description
`spider_ai_crawl`	AI-guided crawling — describe what content to find.
`spider_ai_scrape`	Extract structured data with plain English — no CSS selectors.
`spider_ai_search`	AI-enhanced search with intent understanding.
`spider_ai_browser`	Automate browser actions with natural language.
`spider_ai_links`	Find and filter links by description.

Browser Automation Tools (9)

Direct browser control via spider-browser. Browsers run in Spider's cloud with smart retry, browser switching, anti-bot protection, proxy rotation, and automatic stealth escalation. Supports CDP and BiDi protocols.

Tool	Description
`spider_browser_open`	Open a remote browser session. Returns a session_id.
`spider_browser_navigate`	Navigate to a URL and wait for load.
`spider_browser_click`	Click an element by CSS selector.
`spider_browser_fill`	Fill a form field with text.
`spider_browser_screenshot`	Take a screenshot of the current page. Returns base64 PNG.
`spider_browser_content`	Get page HTML or visible text.
`spider_browser_evaluate`	Execute JavaScript in the page context.
`spider_browser_wait_for`	Wait for an element, navigation, or network idle.
`spider_browser_close`	Close the session and stop billing.

Browser sessions auto-close after 5 minutes of inactivity. Always call spider_browser_close when done.

Examples

Research and RAG

"Crawl the React documentation and summarize the hooks API"

Uses spider_crawl to fetch 50+ pages in seconds and return clean markdown ready for context.

spider_crawl: {
  url: "https://react.dev/reference/react",
  limit: 50,
  return_format: "markdown",
  filter_output_main_only: true
}

Structured Data Extraction

"Get all product names and prices from this e-commerce page"

Uses spider_ai_scrape to extract structured JSON with zero CSS selectors.

spider_ai_scrape: {
  url: "https://example-store.com/products",
  prompt: "Extract every product name, price, and availability status as JSON"
}

Multi-Step Browser Automation

"Log into the dashboard, go to reports, and screenshot the monthly summary"

Uses spider_browser_* tools to drive a remote browser with full anti-bot protection.

1. spider_browser_open: { browser: "auto" }
2. spider_browser_navigate: { url: "https://app.example.com/login" }
3. spider_browser_fill: { selector: "input[name='email']", value: "[email protected]" }
4. spider_browser_fill: { selector: "input[name='password']", value: "..." }
5. spider_browser_click: { selector: "button[type='submit']" }
6. spider_browser_wait_for: { navigation: true }
7. spider_browser_navigate: { url: "https://app.example.com/reports/monthly" }
8. spider_browser_screenshot: {}
9. spider_browser_close: {}

Competitive Intelligence

"Search for recent AI startup funding rounds and get the details"

Uses spider_search with time filtering, then spider_scrape for details.

spider_search: {
  search: "AI startup Series A funding 2025",
  num: 10,
  fetch_page_content: true,
  return_format: "markdown",
  tbs: "qdr:m"
}

API Reference

All tools map directly to the Spider API. Core tools accept the same parameters as their API counterparts:

Crawl/Scrape: url, return_format, request, readability, root_selector, proxy_enabled, cache, and many more
Search: search, num, fetch_page_content, country, language, tbs
AI tools: Add a prompt parameter describing what you want in natural language
Browser tools: Use session_id from spider_browser_open for all operations

Full parameter reference: spider.cloud/docs/api

License

MIT

Documentation