Web Scraping MCP API

Updated April 2026

Web Scraping MCP API for Claude, Cursor and AI Agents

Connect Scrappa as a remote web scraping MCP server and give Claude Desktop, Cursor, Codex, OpenCode, Gemini CLI, VS Code, or Windsurf direct access to 80+ scraping APIs. Search Google, scrape Google Maps, monitor reviews, fetch YouTube transcripts, and qualify leads from your AI client.

MCP server URL

MCP web scraping API endpoint using streamable-HTTP transport. Authenticate with the X-API-KEY header from your Scrappa dashboard.

https://scrappa.co/mcp

MCP scraping hub

Start with the integration guide, then move into client-specific setup and endpoint walkthroughs for Claude Desktop, Cursor, Google Maps, Google Search, YouTube, and agent use cases.

Maps

Google Maps scraper

simple-search

Find places, enrich business profiles, and pull review data for lead generation and local market research agents.

Google Search scraper

search

Give Claude, Cursor, and other AI agents structured SERP results for rank tracking, research, and competitor monitoring.

Social

YouTube scraper

youtube-transcript

Search videos, fetch metadata, collect comments, and retrieve transcripts for content research workflows.

Client setup

Claude Desktop setup

mcp-remote

Copy-paste the mcp-remote configuration and connect Claude Desktop to Scrappa in under a minute.

Client setup

Cursor MCP setup

cursor

Add Scrappa to Cursor so agent mode can call web scraping tools while you work in the IDE.

Use cases

Agent use cases

agent-workflows

Build lead generation, review monitoring, real estate research, and SERP tracking agents with live web data.

What is MCP?

The Model Context Protocol (MCP) is an open standard that allows AI assistants to connect to external tools and data sources. Scrappa's web scraping MCP server is a single managed remote endpoint — point any MCP-compatible client at the Scrappa MCP URL above, pass your API key, and your AI can:

Search Google, YouTube, and other platforms

Translate text between languages

Get business reviews from Trustpilot, Kununu, and Google

Fetch YouTube video data, comments, and channel info

Search products on Amazon and Vinted

Supported Clients

C

Claude Desktop

Anthropic's official desktop app

Claude Code

Terminal-based AI coding assistant

Codex

OpenAI coding agent in the CLI and IDE

OpenCode

Open-source terminal AI coding agent

Gemini CLI

Google's command-line AI assistant

Cu

Cursor

AI-powered code editor

Cu

VS Code

GitHub Copilot Agent mode

Cu

Windsurf

Prerequisites

1

Get Your API Key

Sign up at scrappa.co and get your API key from the dashboard.

2

Install Node.js for Bridge-Based Clients

Node.js 18+ is only needed for clients that use mcp-remote: Claude Desktop, Cursor, VS Code, and Windsurf. Codex, OpenCode, and Gemini CLI connect through native remote MCP configuration.

Required before install

Get the API key you will paste into MCP

Create a free Scrappa account, copy the dashboard key, and use it anywhere this guide shows YOUR_SCRAPPA_API_KEY.

Get API key

Optional sanity check

Test a live request before wiring your client

Open the public playground, run a Google Search or Maps request, and confirm the JSON shape your AI agent will receive through MCP.

Open playground

Quick Install

Using Claude Code or Gemini CLI? Run the matching command:

Claude Code

Copy Copied to clipboard

claude mcp add scrappa --transport http https://scrappa.co/mcp -H 'X-API-KEY: YOUR_SCRAPPA_API_KEY'

Gemini CLI

gemini mcp add --transport http --header 'X-API-KEY: YOUR_SCRAPPA_API_KEY' scrappa https://scrappa.co/mcp

Done! Restart your AI client and start using Scrappa tools.

Using a different tool? See the detailed instructions below.

Not ready to edit your MCP config yet?

Run the same Scrappa endpoints in the playground first, then paste your API key into Claude, Codex, Gemini CLI, Cursor, VS Code, or Windsurf.

Create API key Try playground

Detailed Installation

Choose your AI tool and follow the installation instructions.

Claude Desktop Claude Code Codex OpenCode Gemini CLI Cursor VS Code Windsurf

Add the following to your Claude Desktop configuration file:

macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Windows: %APPDATA%\Claude\claude_desktop_config.json

{
  "mcpServers": {
    "scrappa": {
      "command": "npx",
      "args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
    }
  }
}

Run this single command in your terminal:

That's it! Restart Claude Code and you're ready to go.

Or add it to your project's .mcp.json file:

Add to your Codex configuration file:

File: ~/.codex/config.toml

[mcp_servers.scrappa]
url = "https://scrappa.co/mcp"
http_headers = { "X-API-KEY" = "YOUR_SCRAPPA_API_KEY" }

Add to your OpenCode configuration file:

File: opencode.json in your project root, or ~/.config/opencode/opencode.json for global setup.

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "scrappa": {
      "type": "remote",
      "url": "https://scrappa.co/mcp",
      "enabled": true,
      "oauth": false,
      "headers": {
        "X-API-KEY": "YOUR_SCRAPPA_API_KEY"
      }
    }
  }
}

Run this command in your terminal:

Or add it to ~/.gemini/settings.json:

{
  "mcpServers": {
    "scrappa": {
      "httpUrl": "https://scrappa.co/mcp",
      "headers": {
        "X-API-KEY": "YOUR_SCRAPPA_API_KEY"
      }
    }
  }
}

Add to your Cursor MCP configuration:

File: ~/.cursor/mcp.json

Add to your VS Code settings (settings.json):

{
  "mcp": {
    "servers": {
      "scrappa": {
        "command": "npx",
        "args": ["-y", "mcp-remote", "https://scrappa.co/mcp", "--header", "X-API-KEY: YOUR_SCRAPPA_API_KEY"]
      }
    }
  }
}

Add to your Windsurf MCP configuration file:

File: ~/.codeium/windsurf/mcp_config.json

Available Tools

The Scrappa MCP server provides the following tools to your AI assistant:

G Google Search, Maps, Flights & Hotels

search - Google web SERPs

google-search-light - Lightweight web search

simple-search - Google Maps places (simple search)

google-reviews - Google Maps reviews

google-business-details - Maps business profile

google-flights-one-way - One-way flight search

google-flights-round-trip - Round-trip flights

google-hotels-search - Hotel listings & prices

google-jobs - Jobs SERPs

google-news - Google News results

google-translate-api - Translate text

google-images - Search images

google-videos - Search videos

★ Reviews & Reputation

trustpilot-company-reviews - Trustpilot ratings & reviews

trustpilot-company-details - Trustpilot company profile

kununu-reviews - Kununu employer reviews

trustedshops-reviews - TrustedShops reviews

jameda-reviews - Jameda doctor reviews

in Professional & Marketplace

linkedin-profile - LinkedIn profile data

linkedin-company - LinkedIn company pages

linkedin-post - LinkedIn post details

indeed-jobs - Indeed job listings

vinted-search - Vinted product search

brave-search - Brave search results

YT YouTube

youtube-search - Search videos

youtube-video - Get video details

youtube-channel - Channel profile

youtube-comments - Fetch comments

youtube-transcript - Transcript segments

youtube-related - Related videos

youtube-chapters - Video chapters

youtube-trending - Trending videos

youtube-suggestions - Search suggestions

youtube-channel-videos - Channel uploads

youtube-channel-playlists - Channel playlists

youtube-channel-community - Community posts

Usage Examples

Once Scrappa is connected, just ask your AI assistant naturally. These prompts are real web scraping workflows people run today through Claude, Codex, Cursor, and Gemini CLI:

"Search Google for "best web scraping API 2026" and summarise the top 10 results"

"Pull Google Maps reviews for the three top-rated coffee shops in Berlin Mitte"

"Get the Trustpilot rating, review count, and three latest reviews for stripe.com"

"Fetch the LinkedIn profile and current company for https://linkedin.com/in/satyanadella"

"Grab the YouTube transcript for https://youtube.com/watch?v=... and turn it into a blog outline"

"Find Indeed remote senior Laravel jobs posted in the last 7 days and group them by salary band"

"Compare Google Flights prices BER → JFK for next Friday and the Friday after"

"Translate this product description into German, French, and Spanish"

Code examples in TypeScript and Python

MCP is the fastest path for AI agents, but every tool is backed by the same REST API. Use direct calls when you need a scheduled job, backend enrichment pipeline, or test fixture outside the AI client.

TypeScript

const response = await fetch('https://api.scrappa.co/google-maps/simple-search?query=coffee%20shops%20Austin', {
  headers: { 'X-API-KEY': process.env.SCRAPPA_API_KEY! }
});

const places = await response.json();
console.log(places);

Python

import os
import requests

response = requests.get(
    'https://api.scrappa.co/google/search',
    params={'query': 'best mcp scraper'},
    headers={'X-API-KEY': os.environ['SCRAPPA_API_KEY']},
)

print(response.json())

Pricing for MCP usage

MCP calls use the same Scrappa credits as REST API calls. There is no separate MCP plan, no agent-seat minimum, and every account can start with 500 free monthly credits before moving to credit packs or subscriptions on the pricing page.

Scrappa MCP vs alternatives

Provider	MCP focus	Best fit
Scrappa	80+ structured scraping APIs exposed as MCP tools	Agents that need search, maps, reviews, jobs, social, and marketplace data
Firecrawl	Website crawl and page extraction MCP	Markdown extraction from known websites
Apify	Actor ecosystem via MCP	Teams already maintaining Apify actors
Bright Data	Enterprise data collection tooling	Large proxy-heavy operations with enterprise procurement

Get Your API Key

Sign up for a free Scrappa account to get your API key and start using the MCP integration with your AI assistant.

Get Started Free

Related resources

Pair the MCP setup guide with the main docs, pricing overview, server comparison, and full API reference while you evaluate production use.

Homepage

Return to the main product overview and marketing entry points.

Quick start docs

Start with account setup, first requests, and core documentation.

Pricing

Check credits, savings, and alternative-page comparisons before rollout.

Best MCP servers

Compare Scrappa with Firecrawl, Apify, Bright Data, and other web scraping MCP options.

API reference

Open the full reference and OpenAPI docs for endpoint-level detail.

Web scraping MCP server troubleshooting

If you are evaluating a web scraping MCP server for Claude or another AI client, the Scrappa MCP URL https://scrappa.co/mcp is a single managed endpoint that fronts every Scrappa scraping API. These are the issues that come up most often during setup.

The MCP server URL must be exact

Use https://scrappa.co/mcp with no trailing slash, no /v1, and no extra path segments. Native remote MCP clients reject mismatched URLs silently and fall back to "no tools".

X-API-KEY header formatting

The header name is X-API-KEY (uppercase). Pass the raw key value with no "Bearer" prefix and no surrounding quotes. Bridge-based clients pass it through --header "X-API-KEY: YOUR_KEY".

mcp-remote bridge requires Node.js 18+

Claude Desktop, Cursor, VS Code, and Windsurf launch the npx mcp-remote bridge against the Scrappa MCP URL. Make sure node --version reports 18 or higher and that npx is on PATH for the user the AI client runs as.

Restart the AI client after editing config

MCP servers are loaded once per client session. After updating claude_desktop_config.json, ~/.codex/config.toml, opencode.json, or ~/.cursor/mcp.json, fully quit and relaunch the client — reloading the window is not enough.

Verify the server is registered

Run "claude mcp list" or "gemini mcp list" to confirm scrappa shows up. Inside the chat, asking "list your tools" should return Scrappa endpoints such as search, trustpilot-company-reviews, linkedin-profile, and youtube-transcript.

Frequently Asked Questions

What is an MCP server and how does it work with AI clients?

MCP (Model Context Protocol) is an open standard that lets AI assistants connect to external tools and data sources. Scrappa's MCP server exposes 80+ web scraping APIs as tools that Claude, Codex, OpenCode, Gemini CLI, Cursor, VS Code, and Windsurf can call directly during conversations.

What is the Scrappa MCP server URL?

The Scrappa MCP server URL is https://scrappa.co/mcp. It is a remote streamable-HTTP MCP endpoint authenticated with the X-API-KEY header from your Scrappa dashboard. Native remote MCP clients (Claude Code, Codex, OpenCode, Gemini CLI) connect to this URL directly, while Claude Desktop, Cursor, VS Code, and Windsurf reach it through the mcp-remote bridge.

Which AI clients support Scrappa's MCP server?

Scrappa's MCP server works with Claude Desktop, Claude Code, Codex, OpenCode, Gemini CLI, Cursor, VS Code with GitHub Copilot, and Windsurf. Any client that supports the Model Context Protocol can connect to Scrappa.

Do I need to install anything to use Scrappa through MCP?

Claude Code and Gemini CLI can add Scrappa with one command. Claude Desktop, Cursor, VS Code, and Windsurf use JSON settings, Codex uses TOML, and OpenCode uses opencode.json. Node.js 18+ is required only for clients that use the mcp-remote bridge.

How many APIs are available through the MCP server?

Scrappa provides 80+ API endpoints across Google Maps, Google Search, YouTube, LinkedIn, Trustpilot, Google Flights, Google Hotels, Indeed, Brave Search, and more. All endpoints are automatically available as MCP tools, including the latest YouTube transcript endpoint added in April 2026.

Can I use Scrappa's MCP server for web scraping in Claude?

Yes. The Scrappa MCP server is purpose-built for web scraping inside Claude, Claude Code, Codex, Cursor, and other MCP clients. Once connected, Claude can search Google, fetch Google Maps reviews, pull Trustpilot or Kununu ratings, scrape LinkedIn profiles, retrieve YouTube transcripts, and translate text — all without leaving the chat window.

Can I use Scrappa's MCP server for free?

You can sign up for a free Scrappa account to get an API key. The MCP server uses the same API credits as the REST API, including the 500 free monthly credits on every account. Check the pricing page for paid plans and credit packs once you outgrow the free tier.

How do I troubleshoot an MCP server that will not connect?

Most issues come from header formatting or stale clients. Confirm the MCP server URL is exactly https://scrappa.co/mcp, that the X-API-KEY header is unquoted on a single line, and that your client has been fully restarted after editing the config. Run "claude mcp list" or "gemini mcp list" to verify the server is registered, and check that mcp-remote-using clients have Node.js 18+ on PATH.

Need Help?

Check out our authentication docs for more details on API keys, or visit the API playground to test endpoints directly.

MCP integration guide last updated April 2026.