Getting Started with Scrapfly MCP Server

View as markdown

The Model Context Protocol (MCP) is an open standard that allows AI models to interact with external tools and data sources. The Scrapfly MCP Server transforms your AI from being limited by training data to having real-time access to any website-giving your AI the power to scrape live web data on demand.

What This Unlocks

Instead of your AI answering "I don't have current information," it can now:

Scrape Live Data

Pull current prices, listings, news, or any webpage content in real-time

Bypass Anti-Bot Systems

Automatically handle CAPTCHAs, proxies, JavaScript rendering, and rate limits

Extract Structured Data

Parse complex websites into clean JSON using AI-powered extraction

Capture Screenshots

Take visual snapshots of pages or specific elements for analysis

Why Scrapfly MCP Server?

Built on battle-tested infrastructure used by thousands of developers:

Always Reliable

Your AI never gets blocked

Fast Setup

Running in under 60 seconds

Secure

OAuth2 + enterprise auth

The Power of MCP Connectors

MCP is a universal connector that turns any AI client into a web scraping powerhouse. Instead of building custom integrations for each AI platform, MCP provides a single standard interface.

Works With Your Favorite AI Tools:

• Claude Desktop, Cursor, Cline, Windsurf
• LangChain, LlamaIndex, CrewAI
• OpenAI function calling, custom agents
• n8n, Make, Zapier workflows

One Integration, Infinite Possibilities:

Configure once, use everywhere. The same MCP server works across all platforms-no need to rebuild integrations for each AI tool you adopt.

How to Use Scrapfly MCP

Getting started is simple-choose your AI client and follow the setup guide:

Setup for Your AI Client

15+ integrations: Claude Desktop, Cursor, Cline, Windsurf, LangChain, n8n, and more

Available Tools

5 powerful tools: web_get_page, web_scrape, screenshot, and more

Authentication Options

API key, OAuth2, or header-based auth-choose what fits your needs

Next Steps

See Real-World Examples - Job aggregation, price monitoring, data extraction, and more Client-Specific Setup Guides - Cursor, Cline, LangChain, n8n, and all supported platforms Tools & API Specification - Complete parameter documentation and usage guides Set Up OAuth2 Authentication - Secure, enterprise-grade authentication for production Read the FAQ - Common questions and troubleshooting

Verify Your Setup

After configuration, verify everything works correctly:

1. Check Tool Availability
   Look for the hammer icon (🔨) in your AI client or check tool listings to confirm Scrapfly tools are loaded.
2. Test a Simple Request
   Try asking: "What are the top posts on Hacker News right now?"
   Your AI should automatically use Scrapfly tools to fetch and display current results.
3. Check Account Usage
   Ask: "How many Scrapfly credits do I have left?"
   The AI will use info_account to show your current usage and quota.

Troubleshooting

If you encounter issues, here are common problems and solutions:

Tools Not Detected

• Verify JSON config - Ensure your configuration has no syntax errors (missing commas, brackets)
• Restart your AI client - Many clients require a full restart to load new MCP servers
• Check npx availability - Run npx --version in your terminal to verify it's installed
• Validate API key - Ensure your Scrapfly API key is correct and has available credits

Authentication Errors

• Invalid API key - Double-check your key at Dashboard → Project Settings
• OAuth2 issues - Ensure you've completed the OAuth flow and tokens are refreshed
• Expired credentials - Re-authenticate if your session has expired

Need More Help?

Check out our detailed troubleshooting resources:

• MCP FAQ - Common questions and detailed answers
• Discord Community - Get help from the community
• Contact Support - Reach out to our team directly