Oxylabs

Scrape websites with Oxylabs Web API, supporting dynamic rendering and parsing for structured data extraction.

smithery badge pypi package Licence Verified on MseeP Coverage badge


📖 Overview

The Oxylabs MCP server provides a bridge between AI models and the web. It enables them to scrape any URL, render JavaScript-heavy pages, extract and format content for AI use, bypass anti-scraping measures, and access geo-restricted web data from 195+ countries.

This implementation leverages the Model Context Protocol (MCP) to create a secure, standardized way for AI assistants to interact with web content.


Why Oxylabs MCP?  🕸️ ➜ 📦 ➜ 🤖

Imagine telling your LLM "Summarise the latest Hacker News discussion about GPT‑7" – and it simply answers.
MCP (Multi‑Client Proxy) makes that happen by doing the boring parts for you:

What Oxylabs MCP doesWhy it matters to you
Bypasses anti‑bot walls with the Oxylabs global proxy networkKeeps you unblocked and anonymous
Renders JavaScript in headless ChromeSingle‑page apps, sorted
Cleans HTML → JSONDrop straight into vector DBs or prompts
Optional structured parsers (Google, Amazon, etc.)One‑line access to popular targets

✨ Key Features

  • Extract data from any URL, including complex single-page applications
  • Fully render dynamic websites using headless browser support
  • Choose full JavaScript rendering, HTML-only, or none
  • Emulate Mobile and Desktop viewports for realistic rendering
  • Automatically clean and convert HTML to Markdown for improved readability
  • Use automated parsers for popular targets like Google, Amazon, and etc.
  • Bypass sophisticated bot protection systems with high success rate
  • Reliably scrape even the most complex websites
  • Get automatically rotating IPs from a proxy pool covering 195+ countries
  • Set rendering and parsing options if needed
  • Feed data directly into AI models or analytics tools
  • Works on macOS, Windows, and Linux
  • Comprehensive error handling and reporting
  • Smart rate limiting and request management

💡 Example Queries

When you've set up the MCP server with Claude, you can make requests like:

  • Could you scrape https://www.google.com/search?q=ai page?
  • Scrape https://www.amazon.de/-/en/Smartphone-Contract-Function-Manufacturer-Exclusive/dp/B0CNKD651V with parse enabled
  • Scrape https://www.amazon.de/-/en/gp/bestsellers/beauty/ref=zg_bs_nav_beauty_0 with parse and render enabled
  • Use web unblocker with render to scrape https://www.bestbuy.com/site/top-deals/all-electronics-on-sale/pcmcat1674241939957.c

✅ Prerequisites

Before you begin, make sure you have:

  • Oxylabs Account: Obtain your username and password from Oxylabs (1-week free trial available)

Basic Usage

Via Smithery CLI:

  • Node.js (v16+)
  • npx command-line tool

Via uv:

  • uv package manager – install it using this guide

Local/Dev Setup

  • Python 3.12+
  • uv package manager – install it using this guide

🧩 API Parameters

The Oxylabs MCP Universal Scraper accepts these parameters:

ParameterDescriptionValues
urlThe URL to scrapeAny valid URL
renderUse headless browser renderinghtml or None
geo_locationSets the proxy's geo location to retrieve data.Brasil, Canada, etc.
user_agent_typeDevice type and browserdesktop, tablet, etc.
output_formatThe format of the outputlinks, md, html

🔧 Configuration

{
  "mcpServers": {
    "oxylabs_scraper_uvx": {
      "command": "uvx",
      "args": ["oxylabs-mcp"],
      "env": {
        "OXYLABS_USERNAME": "YOUR_USERNAME",
        "OXYLABS_PASSWORD": "YOUR_PASSWORD"
      }
    }
  }
}
{
  "mcpServers": {
    "oxylabs-mcp": {
      "command": "npx",
      "args": [
        "-y",
        "@smithery/cli@latest",
        "run",
        "@oxylabs/oxylabs-mcp",
        "--config",
        "\"{\\\"oxylabsUsername\\\":\\\"OXYLABS_USERNAME\\\",\\\"oxylabsPassword\\\":\\\"OXYLABS_PASSWORD\\\"}\""
      ]
    }
  }
}
{
  "mcpServers": {
    "oxylabs_scraper": {
      "command": "uv",
      "args": [
        "--directory",
        "/<Absolute-path-to-folder>/oxylabs-mcp",
        "run",
        "oxylabs-mcp"
      ],
      "env": {
        "OXYLABS_USERNAME": "OXYLABS_USERNAME",
        "OXYLABS_PASSWORD": "OXYLABS_PASSWORD"
      }
    }
  }
}

Manual Setup with Claude Desktop

Navigate to Claude → Settings → Developer → Edit Config and add one of the configurations above to the claude_desktop_config.json file.

Manual Setup with Cursor AI

Navigate to Cursor → Settings → Cursor Settings → MCP. Click Add new global MCP server and add one of the configurations above.


🛠️ Technical Details

This server provides two main tools:

  1. universal_scraper: Uses Oxylabs Web Scraper API for general website scraping
  2. google_search_scraper: Uses Oxylabs Web Scraper API to extract results from Google Search
  3. amazon_search_scraper: Uses Oxylabs Web Scraper API to scrape Amazon search result pages
  4. amazon_product_scraper: Uses Oxylabs Web Scraper API to extract data from individual Amazon product pages

🛡️ License

Distributed under the MIT License – see LICENSE for details.


About Oxylabs

Established in 2015, Oxylabs is a market-leading web intelligence collection platform, driven by the highest business, ethics, and compliance standards, enabling companies worldwide to unlock data-driven insights.

image

Related Servers