Scrape.ly

npx add-mcp 'https://mcp.scrape.ly'

Remote MCP server

Connect Claude, Cursor, VS Code or any MCP client to Scrape.ly's remote MCP server: streamable HTTP endpoint, OAuth or API-token auth, and scoped keys.

The Scrape.ly MCP server lets AI agents and MCP clients such as Claude, Cursor, and VS Code search the Store, run Actors, check runs, and read results from your account. Use the hosted endpoint for most clients. A local stdio server is available as a Python package for clients that cannot connect to a remote server.

The source code is on GitHub: scrapely-app/scrapely-mcp.

Hosted endpoint

Connect to https://mcp.scrape.ly. The endpoint uses the MCP streamable HTTP transport. Every request needs a Scrape.ly API token or an OAuth access token. Without one the server answers 401 with a WWW-Authenticate header that points MCP clients to the OAuth metadata.

Authentication

Use one of:

• OAuth. Clients that support MCP authorization discover the flow on their own, register themselves, and open a browser window where you sign in to the dashboard and approve access. You do not need to copy a token.
• Bearer token. Send Authorization: Bearer <token> with a token from Settings → API & Integrations in the dashboard.
• Query parameter. https://mcp.scrape.ly?token=<token> for clients that cannot set headers. URLs end up in logs and history, so prefer a header.

Agents without a Scrape.ly account can pay per call with USDC instead: see x402 payments.

The MCP server acts with your token's permissions: an agent can do anything the token can, including starting paid runs.

To limit what an agent can do, give it a scoped API key from Settings → API & Integrations → API keys, for example the Read only preset, or actors:run, runs:read, datasets:read, and keyValueStores:read to run Actors and read their results (get-actor-output also returns the run's OUTPUT record, which needs keyValueStores:read; without it you still get the items, with a note that the record was skipped). See API keys & permissions.

OAuth details

For client authors: the API implements OAuth 2.1 with PKCE and dynamic client registration.

GET  /.well-known/oauth-protected-resource
GET  /.well-known/oauth-authorization-server
POST /v2/oauth/register
GET  /v2/oauth/authorize
POST /v2/oauth/token
POST /v2/oauth/revoke

• Protected resource metadata (RFC 9728) is served at /.well-known/oauth-protected-resource on the host of https://mcp.scrape.ly, and also at the path form /.well-known/oauth-protected-resource/mcp. Authorization server metadata (RFC 8414) is served from https://api.scrape.ly.
• Registration (RFC 7591) needs redirect_uris. Each must be https, or http on localhost or 127.0.0.1. Clients are public: token_endpoint_auth_method is none.
• authorize requires PKCE with code_challenge_method=S256.
• token supports grant_type=authorization_code and grant_type=refresh_token. Authorization codes are valid for 10 minutes.
• Access tokens start with oat_ and expire after 24 hours. The client renews them with its refresh token, so you stay signed in.
• Refresh tokens start with ort_ and are valid for 365 days from their last use: every refresh returns a new refresh token with a new 365-day window (refresh_token_expires_in in the token response). A client used at least once a year never has to sign in again.
• A refresh token works only for the client it was issued to. Each one can be used once; a repeat within 30 seconds returns the same new tokens, and a later reuse revokes every token from that sign-in.
• Revoking the app, changing your password, a suspension, deleting the account or resetting your API key revokes all refresh tokens. The client then asks you to sign in again.
• Client registrations unused for 366 days are deleted, and a registration that never completed a sign-in is deleted after a day. The token endpoint then returns invalid_client (HTTP 401) and the client registers again.
• revoke accepts an access token or a refresh token in the token field. A refresh token revokes every token from that sign-in.

Connect a client

Replace YOUR_TOKEN with your API token, or drop the header to sign in with OAuth in clients that support it.

Claude Code

claude mcp add --transport http scrapely https://mcp.scrape.ly

Then run /mcp inside Claude Code and choose the server to sign in with OAuth. Scopes, API keys and permissions: Claude Code.

Claude Desktop

Add the hosted endpoint as a custom connector under Customize → Connectors with the URL https://mcp.scrape.ly. Claude signs you in with OAuth. Team and Enterprise setup: Claude Desktop.

To run the local server instead, add it to claude_desktop_config.json:

{
  "mcpServers": {
    "scrapely": {
      "command": "uvx",
      "args": ["--from", "scrapely-mcp@latest", "scrapely-mcp-server"],
      "env": {
        "SCRAPELY_TOKEN": "YOUR_TOKEN",
        "SCRAPELY_API_BASE_URL": "https://api.scrape.ly"
      }
    }
  }
}

Cursor

Add to Cursor in one click and sign in with OAuth, or add the server to mcp.json by hand: see Cursor.

VS Code

Install in VS Code in one click and sign in with OAuth, or add the server to .vscode/mcp.json by hand: see VS Code.

Other clients

Any client that supports streamable HTTP can use https://mcp.scrape.ly with a Bearer header or OAuth. Clients that only support stdio can use the local server.

Tools

The server enables the actors, docs, runs, storage, tasks, and automations tool categories by default.

actors

Tool	What it does
search-actors	Search the Store by keywords. Returns up to 10 Actors (default 5). Only lists Actors whose owners allow AI agents (see Store for AI agents).
fetch-actor-details	Details for one Actor by ID or username/actor-name: description, input schema, README, pricing, stats, rating, and output schema. The input schema and README come from the build you run: on someone else's Actor, the User Build. The output argument selects which parts to return.
call-actor	Run any Actor with a JSON input. Waits up to waitSecs (0–45, default 30), then returns the run status and storage IDs. Optional callOptions: memory (MB), timeout (seconds), build (your own Actors only), maxItems, maxTotalChargeUsd.

runs

Tool	What it does
get-actor-run	Status, storages, and stats of a run, with a suggested next step. Waits up to waitSecs (0–45, default 30) for the run to finish.
get-actor-output	The results of a run: a page of its dataset items plus its key-value store OUTPUT record. See Reading results. Also in the storage category.
get-actor-run-list	Your runs, filterable by status. Up to 10 per call.
get-actor-log	The last lines lines of a run's log (default 10, at most 50; 0 for the whole log).
abort-actor-run	Abort a running run, optionally gracefully.

storage

Tool	What it does
get-actor-output	Results of a run or dataset, paged and kept compact. See Reading results.
get-dataset-items	Rows from a dataset. Supports offset, limit (default 20, up to 1000), fields, omit, flatten, desc, and clean.
get-dataset	Dataset metadata: field list, item count, and size.
get-dataset-schema	JSON schema inferred from a sample of items.
get-dataset-list	Your datasets. Up to 20 per call; unnamed: true includes run datasets.
get-key-value-store	Key-value store metadata.
get-key-value-store-keys	Keys in a store. Up to 10 per call.
get-key-value-store-record	One record by key. JSON and text records come back inline; binary records over 256 KiB, and any record over 1 MiB, come back as a download link instead.
get-key-value-store-list	Your key-value stores. Up to 10 per call.

Datasets and stores accept an ID or username~name.

tasks

Tool	What it does
create-actor-task	Save an Actor's input and run options as a task. The build option only applies to your own Actors.
get-actor-task	Read a task. Secret input fields come back encrypted.
update-actor-task	Change a task's input, run options, or public page settings. The new input replaces the old one. The build option only applies to your own Actors.
publish-actor-task / unpublish-actor-task	Publish a task as a public page for its Actor, or take it down.

automations

Tool	What it does
list-saved-inputs	Your saved inputs. With actorId, one Actor's saved inputs, the default first, plus its example inputs; without it, all saved inputs in your account.
run-saved-input	Start a run of a saved input by savedInputId. input changes only the fields it names; the other fields come from the saved input. Returns right after the start, or waits up to waitSecs (at most 45).
list-automations	Your automations, with their trigger, steps, AI step, and deliveries. Filter with actorId or savedInputId.
run-automation	Run one of your automations now, whatever its trigger. Its runs are billed like any run, and an AI step like the AI chat. Returns the automation run with the started Actor runs.

Agents can't create or change automations over MCP; use the dashboard, the AI chat, or the Automations API. On a scoped key, list-automations needs automations:read, and run-automation needs automations:write with actors:run and the run permission actor-default (see API keys & permissions).

docs

Tool	What it does
search-scrapely-docs	Full-text search over Scrape.ly or Crawlee docs (docSource: scrapely, crawlee-js, or crawlee-py).
fetch-scrapely-docs	Fetch a docs page as Markdown by URL: Scrape.ly docs pages (https://scrape.ly/docs/…) or crawlee.dev.

Other tools

• report-problem sends a short problem report about a tool or Actor to the Scrape.ly team. It is in the dev category, which is off by default, and is never offered to Claude clients.
• Interactive MCP Apps variants of search-actors, fetch-actor-details, call-actor, and get-actor-run are turned off in the current release, so every client gets the plain tools.

Typical flow

1. search-actors finds candidate Actors for the task.
2. fetch-actor-details returns the input schema.
3. call-actor starts the run. Fast Actors finish within waitSecs; for long ones, the agent passes waitSecs: 0 and polls get-actor-run.
4. get-actor-output reads the results: the run's dataset items, page by page, and its OUTPUT record.

Results are not inlined into the call-actor response. The agent reads them with get-actor-output (or the storage tools), so it can page through large datasets and pick only the fields it needs.

Reading results

get-actor-output takes a runId (it uses the run's default dataset and key-value store) or a datasetId. It reads with your token, so you only see your own runs and storages.

Argument	Default	Description
runId / datasetId	—	One of them is required.
fields	all	Comma-separated fields to keep, with dot notation for nested values (address.city).
offset	0	Items to skip.
limit	100	Items per page, up to 1000.
includeOutputRecord	true with a runId or keyValueStoreId	Also return the key-value store OUTPUT record. When it can't be read, for example with a key without keyValueStores:read, the items still come back and outputRecordNote says why the record was skipped.
keyValueStoreId	the run's store	Read OUTPUT from this store instead.

To keep replies small for the model, text values over 1,000 characters are cut, a page holds about 50 KB (so it can have fewer items than limit), and the OUTPUT record is cut at about 20 KB. Binary records are described, not inlined. The reply says totalItemCount, hasMore, nextOffset, and whether anything was truncated; pass nextOffset as offset for the next page.

Local server

The Python package scrapely-mcp runs the same server over stdio on your machine. It requires Python 3.11 or newer and version 0.2.0 or later of the package; the automations tools come with version 0.3.0. The command is scrapely-mcp-server:

uvx --from scrapely-mcp@latest scrapely-mcp-server

or

pip install -U scrapely-mcp
scrapely-mcp-server

The CLI can add the local server to Claude Code, Claude Desktop, Cursor, VS Code, or Codex for you: scrapely mcp install <client>.

[!NOTE] Since 0.2.0 the server connects to https://api.scrape.ly by default. Releases before 0.2.0 defaulted to a non-production API URL; if you still run one, upgrade or set SCRAPELY_API_BASE_URL=https://api.scrape.ly.

Environment variables

Variable	Description
SCRAPELY_TOKEN	API token. If unset, the server reads token from ~/.scrapely/credentials.json or ~/.scrapely/auth.json, which the CLI writes when you log in.
SCRAPELY_API_BASE_URL	API origin. Defaults to https://api.scrape.ly.

A token is required unless every selected tool works without one (search-actors, fetch-actor-details, and the docs tools). If a token is required and none is found, the server exits with an error.

Options

Flag	Description
--tools	Comma-separated categories (actors, docs, runs, storage, tasks, automations, dev) or individual tool names. Replaces the default set.
--actors	Comma-separated Actors (username/actor-name) to expose as their own tools.
--ui	auto (default) enables MCP Apps widgets when the client supports them; apps forces them; default turns them off. Widgets are disabled in the current release, so every client gets the plain tools.
--telemetry-enabled	Send anonymous usage telemetry. Off by default.

# Only docs search, no token needed
scrapely-mcp-server --tools=docs

# Runs and storage only
scrapely-mcp-server --tools=runs,storage

Whenever call-actor, get-actor-run, or an Actor tool from --actors is enabled, the server also adds get-actor-run, get-actor-output, get-dataset-items, get-key-value-store-record, and abort-actor-run so the agent can follow up on runs.

Costs and limits

• Actor runs started through MCP are normal runs on your account. They use your plan's compute and storage, and paid Actors charge as usual. See Account.
• callOptions.memory accepts 128 MB up to your plan's maximum for one run (at most 32,768 MB). Your plan also caps total memory across concurrent runs; a run that would exceed it is rejected.
• call-actor and get-actor-run wait at most 45 seconds per call. Runs keep going after the tool returns.
• The hosted endpoint allows 1,000 requests per minute per IP address.
• On the hosted endpoint, ?tools= and ?actors= pick the tools the same way as --tools and --actors, for example https://mcp.scrape.ly?actors=janedoe/my-actor. One selection can name at most 50 Actors; more returns 400 with the error "Too many Actors selected".
• Each token can have 50 open sessions on the hosted endpoint, and each IP address 100 (an IPv6 /64 counts as one address, and one IPv6 /48 can have 400). Past that, your oldest idle session is closed, or the request gets 429.
• A tool result can be at most 4 MiB. Clients that run tools as protocol tasks get finished task results for up to 15 minutes after the task ends, or less when the task's ttl is shorter. A result is released a minute after it's fetched, and closing the session drops its tasks.

---

• Previous: Python client
• Next: x402 payments
• All docs: Docs index, llms.txt