Crawlora MCP

Hosted MCP exposing 300+ structured web-data tools — search, maps, commerce, social, and finance — that return clean JSON.

View on crawlora.net

Documentation

Developer guides

Crawlora MCP Integration Guide

Connect MCP-capable clients and agent-native workflows to Crawlora's hosted Streamable HTTP endpoint for structured public web data tools.

Hosted MCP endpointStreamable HTTPStructured toolsServer-side keys

Browse APIs Try Playground Explore Integrations

Hosted MCP endpoint

https://mcp.crawlora.net/mcp

Streamable HTTP

Connection

URL: https://mcp.crawlora.net/mcp
Transport: Streamable HTTP
Auth header: Authorization: Bearer $CRAWLORA_API_KEY
Server card: https://crawlora.net/.well-known/mcp/server-card.json

MCP URL

https://mcp.crawlora.net/mcp

Transport

Streamable HTTP

Auth header

Authorization: Bearer

Crawlora exposes a hosted MCP endpoint at https://mcp.crawlora.net/mcp. Configure MCP-capable clients with Streamable HTTP transport and pass the same server-side Crawlora API key as Authorization: Bearer. The MCP endpoint also accepts x-api-key for clients that support custom headers. ChatGPT and Claude's web custom-connector UI authenticate through OAuth 2.1 instead — no API key needed.

Developer workflow

What MCP does for web data workflows

MCP lets AI clients discover and call tools exposed by a server. Crawlora's agent-native APIs fit this model because supported endpoints have clear inputs and structured JSON outputs.

Developer workflow

When to use Crawlora with MCP

AI research assistant.
SERP research tool.
Local business and geocoding research agent.
Travel and hospitality research agent.
Property market research agent.
Review and reputation summarizer.
App review summarizer.
YouTube transcript agent.
TikTok trend research agent.
Music catalog research agent.
Podcast and audio research agent.
E-commerce monitoring agent.

Developer workflow

Connect to hosted MCP

Use the hosted endpoint directly when your client supports remote Streamable HTTP MCP servers.

Endpoint: https://mcp.crawlora.net/mcp.
Transport: Streamable HTTP.
Preferred authentication header: Authorization: Bearer.
Alternate authentication header: x-api-key.
ChatGPT and Claude custom connectors authenticate automatically through OAuth 2.1 instead of a header — see below.
Server card: https://crawlora.net/.well-known/mcp/server-card.json.
A missing or invalid API key returns 401.

Developer workflow

Connect via ChatGPT or Claude (OAuth)

ChatGPT's and Claude's custom connector UI for remote MCP servers only accepts a real OAuth 2.1 flow, not a pasted API key. Add the hosted endpoint as a custom connector and the client handles discovery, registration, and sign-in for you.

Add the connector

In ChatGPT or Claude's custom connector settings, add a new connector and paste https://mcp.crawlora.net/mcp as the server URL.

Client discovers OAuth automatically

The client fetches the endpoint's protected-resource metadata, follows it to Crawlora's authorization server at https://crawlora.net, and registers itself with Dynamic Client Registration — no manual client setup or API key entry.

Sign in and approve access

You're redirected to sign in to your Crawlora account (or use an existing session) and approve the connector on a consent screen. No API key is ever pasted into the client.

Start calling tools

The client stores and refreshes its own access token. Disconnect the connector in ChatGPT or Claude at any time to revoke its access.

Developer workflow

Connection values

Use these values in MCP-capable hosts that take a manual API key instead of OAuth. Keep the API key outside model-visible prompts and browser code.

Hosted MCP connection · text

Copy

MCP URL: https://mcp.crawlora.net/mcp Transport: Streamable HTTP Authentication: Authorization: Bearer $CRAWLORA_API_KEY Alternate auth: x-api-key: $CRAWLORA_API_KEY Server card: https://crawlora.net/.well-known/mcp/server-card.json

Codex direct HTTP MCP · bash

Copy

codex mcp add crawlora \ --url https://mcp.crawlora.net/mcp/agent \ --bearer-token-env-var CRAWLORA_API_KEY

Generic MCP client config · json

Copy

{ "mcpServers": { "crawlora": { "url": "https://mcp.crawlora.net/mcp", "transport": "streamable-http", "headers": { "Authorization": "Bearer $CRAWLORA_API_KEY" } } } }

Read server card · bash

Copy

curl https://crawlora.net/.well-known/mcp/server-card.json

Developer workflow

Discover and call tools

Add server

Authenticate

Pass the Crawlora API key as Authorization: Bearer from the MCP client or server-side host. Use x-api-key only when the host supports custom headers but not bearer token configuration.

Inspect tools

Let the MCP client list available tools and resources before choosing an endpoint.

Call narrowly

Send only documented inputs for the selected Crawlora tool and cap result sizes where possible.

Handle failures

Surface 401, plan, credit, rate-limit, and upstream errors clearly to the agent-native workflow.

Developer workflow

Integration options

Use the hosted Crawlora MCP endpoint for clients that support remote Streamable HTTP MCP.
Use Crawlora REST APIs directly when your environment does not support remote MCP.
Create your own lightweight MCP server only when you need custom tool filtering, tenancy, auditing, or local policy controls.

Developer workflow

Example MCP tool definition

Illustrative schema for a wrapper tool. Use exact generated tool names from endpoint docs where available.

Illustrative MCP tool · json

Copy

{ "name": "crawlora_bing_search", "description": "Search public Bing results through Crawlora and return normalized JSON.", "inputSchema": { "type": "object", "properties": { "q": { "type": "string" }, "country": { "type": "string" }, "count": { "type": "number" } }, "required": ["q"] }, "output": "Crawlora JSON envelope with result items, status, and message fields." }

Developer workflow

Optional TypeScript wrapper

Use this pattern only when you need to run a private MCP server that wraps Crawlora REST APIs behind your own policy layer.

Tool handler · typescript

Copy

async function callCrawloraTool(args: { q: string; country?: string; count?: number; }) { const url = new URL("https://api.crawlora.net/api/v1/bing/search"); url.searchParams.set("q", args.q); url.searchParams.set("country", args.country ?? "us"); url.searchParams.set("count", String(args.count ?? 10));

const response = await fetch(url, { headers: { "x-api-key": process.env.CRAWLORA_API_KEY ?? "", }, });

if (!response.ok) { throw new Error(`Crawlora request failed: ${response.status} ${await response.text()}`); }

return response.json(); }

// Register this handler with the MCP SDK version used by your server.

Developer workflow

Security checklist

Keep the Crawlora API key in the server environment.
Do not expose API keys to client-side apps.
Validate tool inputs.
Restrict available tools if needed.
Log tool calls carefully.
Avoid collecting unnecessary sensitive data.
Respect Crawlora terms and target platform rules.

Developer workflow

Agent UX tips

Expose narrow tools, not arbitrary URL collection.
Return structured JSON.
Include source platform and request context where available.
Make failure states visible to the agent.
Cap result counts to control cost.

Developer workflow

Choose a smaller tool set: agent and family endpoints

The default endpoint lists every Crawlora tool, which is thousands of definitions. Clients that load them all up front spend context on the list, and clients that search tools by keyword can rank a broad tool such as web_search below narrower ones: a keyword search for "search the web" surfaces other *_search tools first. Two narrower endpoints on the same host take the same authentication.

Agent endpoint: https://mcp.crawlora.net/mcp/agent exposes three tools. The model describes the task, search_tools returns matching tool names, get_tool returns the input schema, and call_tool runs the tool.
Family endpoint: https://mcp.crawlora.net/mcp/family/ exposes one platform's tools only, for example /mcp/family/web for web search, scrape, crawl and map, or /mcp/family/youtube for YouTube.
The family name is the first segment of the REST route, so /api/v1/youtube/search belongs to the youtube family. GET https://mcp.crawlora.net/mcp/family/ lists every valid family with its tool count.
Use the agent endpoint when one connection should reach every platform, and a family endpoint when the agent needs only one platform.
Codex reads only the first page of 1000 tools from the full endpoint, so in Codex use /mcp/agent or a family endpoint; Claude Code, Grok Build and Pi read the whole list.

Agent and family endpoints · text

Copy

Agent endpoint: https://mcp.crawlora.net/mcp/agent 3 tools: search_tools, get_tool, call_tool Family endpoint: https://mcp.crawlora.net/mcp/family/ e.g. https://mcp.crawlora.net/mcp/family/web, https://mcp.crawlora.net/mcp/family/youtube Authentication: same Authorization: Bearer $CRAWLORA_API_KEY header

Add the agent or one family · bash

Copy

# One connection that can reach every platform through search_tools / call_tool codex mcp add crawlora-agent \ --url https://mcp.crawlora.net/mcp/agent \ --bearer-token-env-var CRAWLORA_API_KEY

Only the web search / scrape / crawl tools, in Grok Build

grok mcp add --transport http crawlora-web https://mcp.crawlora.net/mcp/family/web \ --header 'Authorization: Bearer ${CRAWLORA_API_KEY}'

Responsible public web data workflows

Use Crawlora for structured public web data workflows. Customers are responsible for compliance with applicable laws, third-party rights, platform rules, and Crawlora terms. Keep API keys server-side, validate inputs, and avoid collecting or storing unnecessary sensitive data.

Read Crawlora terms

Developer workflow

FAQ

Common questions for this Crawlora developer integration path.

What is MCP?

Model Context Protocol is a pattern for exposing tools and resources to AI clients through a server interface.

Does Crawlora have an official MCP server?

Yes. Use the hosted Streamable HTTP endpoint at https://mcp.crawlora.net/mcp with Authorization: Bearer. The MCP endpoint also accepts x-api-key for clients that support custom headers.

Can I connect Crawlora MCP to ChatGPT or Claude as a custom connector?

Yes. Add https://mcp.crawlora.net/mcp as a custom connector in ChatGPT or Claude's web UI. The client handles OAuth 2.1 discovery, registration, and sign-in automatically, so you never paste an API key into the connector.

Can I wrap Crawlora APIs as MCP tools?

Yes. Wrap narrow Crawlora HTTP calls behind server-side MCP tools and keep the API key in the server environment.

Which Crawlora APIs are good MCP tools?

Search, Maps, Geocoding, Airbnb, TripAdvisor, Zillow, YouTube, TikTok, App Store, Google Play, Spotify, Spotify Podcasts, Apple Podcasts, Amazon, Product Hunt, Trustpilot, and review endpoints are useful candidates when they match your agent's task.

How should I secure API keys?

Store keys server-side, restrict tool access, validate inputs, and avoid logging secrets.

Can MCP tools consume credits?

Yes. Tool calls that invoke Crawlora API endpoints can consume credits according to endpoint billing rules.

How is this different from LangChain tools?

MCP focuses on a protocol for exposing tools to clients. LangChain tools are framework-level wrappers inside LangChain workflows.

Connect Crawlora MCP

Start with the hosted MCP URL, validate Authorization: Bearer or the x-api-key fallback, then call one narrow search, travel, property, review, app-review, podcast, or music catalog tool.

Browse APIs Explore Integrations

Structured Data

[{"@context":"https://schema.org","@type":"Organization","@id":"https://crawlora.net/#organization","name":"Crawlora","alternateName":["Crawlora API","Crawlora.net"],"url":"https://crawlora.net/","logo":{"@type":"ImageObject","url":"https://crawlora.net/icons/favicon-512.png","width":512,"height":512},"description":"Crawlora provides agent-native web scraping APIs and hosted MCP tools for search, maps, geocoding, streaming media, travel, real estate, marketplaces, finance, social video, app intelligence, and AI workflows.","knowsAbout":["Web scraping","Web scraping API","SERP API","Structured web data extraction","Anti-bot bypass","Proxy infrastructure","Model Context Protocol","Web data for AI agents"],"sameAs":["https://github.com/Crawlora-org","https://www.npmjs.com/package/@crawlora-org/sdk","https://pypi.org/project/crawlora/","https://rubygems.org/gems/crawlora","https://packagist.org/packages/crawlora/sdk","https://pkg.go.dev/github.com/Crawlora-org/crawlora-go-sdk","https://mcpservers.org/servers/crawlora-net-mcp","https://www.wikidata.org/wiki/Q140216339","https://www.linkedin.com/company/crawlora-net","https://x.com/crawlora_net","https://www.crunchbase.com/organization/crawlora"]},{"@context":"https://schema.org","@type":"WebSite","@id":"https://crawlora.net/#website","name":"Crawlora","alternateName":"Crawlora API","url":"https://crawlora.net/","description":"Crawlora provides agent-native web scraping APIs and hosted MCP tools for search, maps, geocoding, streaming media, travel, real estate, marketplaces, finance, social video, app intelligence, and AI workflows.","inLanguage":"en","publisher":{"@id":"https://crawlora.net/#organization"}}]

[{"@context":"https://schema.org","@type":"WebPage","name":"Crawlora MCP Integration Guide - Crawlora","description":"Connect Crawlora agent-native structured public web data APIs to AI clients through the hosted MCP endpoint.","url":"https://crawlora.net/mcp","isPartOf":{"@type":"WebSite","name":"Crawlora","url":"https://crawlora.net/"}},{"@context":"https://schema.org","@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://crawlora.net/"},{"@type":"ListItem","position":2,"name":"MCP","item":"https://crawlora.net/mcp"}]},{"@context":"https://schema.org","@type":"FAQPage","mainEntity":[{"@type":"Question","name":"What is MCP?","acceptedAnswer":{"@type":"Answer","text":"Model Context Protocol is a pattern for exposing tools and resources to AI clients through a server interface."}},{"@type":"Question","name":"Does Crawlora have an official MCP server?","acceptedAnswer":{"@type":"Answer","text":"Yes. Use the hosted Streamable HTTP endpoint at https://mcp.crawlora.net/mcp with Authorization: Bearer. The MCP endpoint also accepts x-api-key for clients that support custom headers."}},{"@type":"Question","name":"Can I connect Crawlora MCP to ChatGPT or Claude as a custom connector?","acceptedAnswer":{"@type":"Answer","text":"Yes. Add https://mcp.crawlora.net/mcp as a custom connector in ChatGPT or Claude's web UI. The client handles OAuth 2.1 discovery, registration, and sign-in automatically, so you never paste an API key into the connector."}},{"@type":"Question","name":"Can I wrap Crawlora APIs as MCP tools?","acceptedAnswer":{"@type":"Answer","text":"Yes. Wrap narrow Crawlora HTTP calls behind server-side MCP tools and keep the API key in the server environment."}},{"@type":"Question","name":"Which Crawlora APIs are good MCP tools?","acceptedAnswer":{"@type":"Answer","text":"Search, Maps, Geocoding, Airbnb, TripAdvisor, Zillow, YouTube, TikTok, App Store, Google Play, Spotify, Spotify Podcasts, Apple Podcasts, Amazon, Product Hunt, Trustpilot, and review endpoints are useful candidates when they match your agent's task."}},{"@type":"Question","name":"How should I secure API keys?","acceptedAnswer":{"@type":"Answer","text":"Store keys server-side, restrict tool access, validate inputs, and avoid logging secrets."}},{"@type":"Question","name":"Can MCP tools consume credits?","acceptedAnswer":{"@type":"Answer","text":"Yes. Tool calls that invoke Crawlora API endpoints can consume credits according to endpoint billing rules."}},{"@type":"Question","name":"How is this different from LangChain tools?","acceptedAnswer":{"@type":"Answer","text":"MCP focuses on a protocol for exposing tools to clients. LangChain tools are framework-level wrappers inside LangChain workflows."}}]}]

Crawlora MCP

Documentation

Crawlora MCP Integration Guide

What MCP does for web data workflows

When to use Crawlora with MCP

Connect to hosted MCP

Connect via ChatGPT or Claude (OAuth)

Add the connector

Client discovers OAuth automatically

Sign in and approve access

Start calling tools

Connection values

Hosted MCP connection · text

Codex direct HTTP MCP · bash

Generic MCP client config · json

Read server card · bash

Discover and call tools

Add server

Authenticate

Inspect tools

Call narrowly

Handle failures

Integration options

Example MCP tool definition

Illustrative MCP tool · json

Optional TypeScript wrapper

Tool handler · typescript

Security checklist

Agent UX tips

Choose a smaller tool set: agent and family endpoints

Agent and family endpoints · text

Add the agent or one family · bash

Only the web search / scrape / crawl tools, in Grok Build

Responsible public web data workflows

Related developer links

MCP server card

OpenAI Agents

AI agents

TypeScript guide

Docs

Terms

Agent Skills

FAQ

Connect Crawlora MCP

Structured Data