Simplescraper MCP Server

Return to top

Using Simplescraper's MCP Server ​

Model Context Protocol (MCP) is a standardized way for AI applications like Claude, Cursor and ChatGPT to interact with web apps like Simplescraper.

By connecting to Simplescraper's MCP server, you can automatically scrape websites, manage recipes and analyze scraped data using any AI application or AI agents.

Simplescraper's MCP server URL:

https://mcp.simplescraper.io/mcp

---

Quick Setup ​

Claude Desktop ​

1. Open Claude Desktop settings
2. Navigate to Settings (bottom-left) → Integrations
3. Click "Add MCP Server"
4. Enter these details:
   • Name: Simplescraper
   • URL: https://mcp.simplescraper.io/mcp
5. Complete the authentication flow to connect your Simplescraper account
6. Start using natural language to control your scrapes!

Claude Integrations availability

Integrations are currently available on the Max plan, with broader availability coming soon.

Cursor IDE ​

Cursor supports MCP with OAuth authentication. When you add the server, Cursor will open a browser window for you to log in with your Simplescraper account.

Global Setup (All Projects):

1. Create ~/.cursor/mcp.json in your home directory
2. Add this configuration:

json

{
  "mcpServers": {
    "simplescraper": {
      "url": "https://mcp.simplescraper.io/mcp"
    }
  }
}

1. When you toggle the server on in Cursor, it will prompt you to authenticate with your Simplescraper account

Project-Specific Setup:

For tools specific to a project, create a .cursor/mcp.json file in your project directory:

1. Create .cursor/mcp.json in your project root
2. Use the same JSON configuration as above
3. The server will only be available in that project

Via Cursor Settings UI:

1. Open Cursor Settings (Ctrl/Cmd + ,)
2. Navigate to MCP in the sidebar
3. Click "Add Server"
4. Enter:
   • Name: Simplescraper
   • URL: https://mcp.simplescraper.io/mcp
5. When prompted, log in with your Simplescraper account

Do not add an Authorization header manually

Cursor handles authentication automatically through OAuth. Adding "headers": {"Authorization": "Bearer ..."} with your API key will prevent the login flow from working. Use the config above with just the URL.

Other AI Applications ​

Most AI applications that support MCP will have a similar setup process. Look for MCP or "Tools" settings and add our server URL: https://mcp.simplescraper.io/mcp

---

Available Capabilities ​

Simplescraper exposes 12 tools your LLM can call to scrape sites and manage your recipes.

Data Extraction ​

Tool	What it does	Example prompt
run_recipe	Run a saved recipe to extract fresh data from its configured URL. Consumes credits.	"Run my Twitter profile scraper now and show me the new data"
extract_urls	Discover all page URLs on a site by parsing its sitemap. Free - no credits deducted.	"Get me a list of all pages on stripe.com"
extract_markdown	Fetch one page and return its body as clean Markdown. Best for reading articles, blog posts, docs, or any page content.	"Read https://example.com/blog/post and summarize it"
smart_extract	Use AI to extract specific named fields from a page (price, title, author, etc) - no CSS selectors required.	"Extract product names, prices, and ratings from https://example.com/products"
get_latest_results	Get the most recent scraped data for a recipe (existing data, no new scrape).	"Show me the latest results from my Twitter profile scraper"
get_results	Fetch data from a specific scrape run by results_id.	"Get the data from results ID abc123"
get_results_history	List previous scrape runs for a recipe with timestamps and page counts.	"When did my Amazon scraper last run, and how many times this week?"

Recipe Management ​

Tool	What it does	Example prompt
list_recipes	Find recipes by domain, name keyword, or recent activity.	"List all my recipes that scrape from linkedin.com"
get_recipe	View full details of a specific recipe, including selectors.	"Show me the selectors in my Amazon price tracker recipe"
create_recipe	Build a new scrape recipe with CSS selectors.	"Create a recipe for nytimes.com that grabs headlines and timestamps"
update_recipe	Modify an existing recipe's name, URL, or selectors.	"The price selector on my Amazon scraper needs updating to .new-price-class"

Batch Operations ​

Tool	What it does	Example prompt
update_batch_urls	Add or replace the list of URLs for batch (crawler) scraping. Pair with extract_urls to discover URLs first, then run_recipe to execute.	"Add these 10 product URLs to my product details scraper"

Advanced Usage Examples ​

URL Discovery and Batch Scraping ​

"Find all product pages on example.com/shop, add them to my product scraper recipe, 
and run the scrape."

Claude will parse the sitemap, filter URLs to match your description (e.g. paths containing /product/), update the batch URL list, and execute the recipe.

Complex Workflows ​

"Check my Amazon price tracker recipe. If prices have changed by more than 10% 
since the last run, create a summary report of the changes."

Multi-Step Operations ​

"Find all recipes that haven't run in the last week, run them, 
and summarize any that return errors."

Data Transformation ​

"Get the latest results from my job listings scraper and format them 
as a markdown table sorted by salary."

---

Getting Help ​

• Support: Reach out via chat on the Simplescraper website
• Learn more about MCP: Read our complete guide to the MCP Protocol