Olostep MCP Server

A server for web scraping, Google searches, and website URL lookups using the Olostep API.

Olostep MCP Server

A Model Context Protocol (MCP) server implementation that integrates with Olostep for web scraping, content extraction, and search capabilities. To set up Olostep MCP Server, you need to have an API key. You can get the API key by signing up on the Olostep website.

Features

Scrape website content in HTML, Markdown, JSON or Plain Text (with optional parsers)
Parser-based web search with structured results
AI Answers with citations and optional JSON-shaped outputs
Batch scraping of up to 10k URLs
Autonomous site crawling from a start URL
Website URL discovery and mapping (with include/exclude filters)
Country-specific request routing for geo-targeted content
Configurable wait times for JavaScript-heavy websites
Comprehensive error handling and reporting
Simple API key configuration

Installation

🐳 Running with Docker (Recommended)

The easiest way to run the Olostep MCP server:

docker pull olostep/mcp-server

docker run -i --rm \
  -e OLOSTEP_API_KEY="your-api-key" \
  olostep/mcp-server

Local-only Docker build (no Docker Hub required)

If the Docker Hub image isn’t available from your environment, you can build and run the image locally from this repository:

cd olostep-mcp-server
npm install
npm run build
docker build -t olostep/mcp-server:local .

docker run -i --rm -e OLOSTEP_API_KEY="your-api-key" olostep/mcp-server:local

Local smoke test (initialize + tools/list)

This MCP server uses stdio transport. You can validate it starts and lists tools without needing a working API key:

On Windows (PowerShell):

cd .\olostep-mcp-server
powershell -ExecutionPolicy Bypass -File .\scripts\smoke-test.ps1

To actually call tools successfully, provide OLOSTEP_API_KEY when running the container.

Using Docker with Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "olostep": {
      "command": "docker",
      "args": [
        "run", "-i", "--rm",
        "-e", "OLOSTEP_API_KEY=YOUR_API_KEY_HERE",
        "olostep/mcp-server"
      ]
    }
  }
}

Using Docker with Cursor

Add an MCP server with:

Name: olostep
Type: command
Command: docker run -i --rm -e OLOSTEP_API_KEY=your-api-key olostep/mcp-server

Running with npx

env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp

On Windows (PowerShell):

$env:OLOSTEP_API_KEY = \"your-api-key\"; npx -y olostep-mcp

On Windows (CMD):

set OLOSTEP_API_KEY=your-api-key && npx -y olostep-mcp

Manual Installation

npm install -g olostep-mcp

Running on Claude Desktop

Add this to your claude_desktop_config.json:

{
  "mcpServers": {
    "mcp-server-olostep": {
      "command": "npx",
      "args": ["-y", "olostep-mcp"],
      "env": {
        "OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
      }
    }
  }
}

Or for a more straightforward way you can install via the Smithery CLI by running the following code in your device terminal

npx -y @smithery/cli install @olostep/olostep-mcp-server --client claude

Running on Windsurf

Add this to your ./codeium/windsurf/model_config.json:

{
  "mcpServers": {
    "mcp-server-olostep": {
      "command": "npx",
      "args": ["-y", "olostep-mcp"],
      "env": {
        "OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
      }
    }
  }
}

Running on Cursor

To configure Olostep MCP in Cursor:

Open Cursor Settings
Go to Features > MCP Servers
Click "+ Add New MCP Server"
Enter the following:
- Name: "olostep-mcp" (or your preferred name)
- Type: "command"
- Command: env OLOSTEP_API_KEY=your-api-key npx -y olostep-mcp

Replace your-api-key with your Olostep API key.

Running on Metorial

Option 1: One-Click Installation (Recommended)

Open Metorial dashboard
Navigate to MCP Servers directory
Search for "Olostep"
Click "Install" and enter your API key

Option 2: Manual Configuration

Add this to your Metorial MCP server configuration:

{
  "olostep": {
    "command": "npx",
    "args": ["-y", "olostep-mcp"],
    "env": {
      "OLOSTEP_API_KEY": "YOUR_API_KEY_HERE"
    }
  }
}

The Olostep tools will then be available in your Metorial AI chats.

Configuration

Environment Variables

OLOSTEP_API_KEY: Your Olostep API key (required)
ORBIT_KEY: An optional key for using Orbit to route requests.

Available Tools

1. Scrape Website (`scrape_website`)

Extract content from a single URL. Supports multiple formats and JavaScript rendering.

{
  "name": "scrape_website",
  "arguments": {
    "url_to_scrape": "https://example.com",
    "output_format": "markdown",
    "country": "US",
    "wait_before_scraping": 1000,
    "parser": "@olostep/amazon-product"
  }
}

Parameters:

url_to_scrape: The URL of the website you want to scrape (required)
output_format: Choose format (html, markdown, json, or text) - default: markdown
country: Optional country code (e.g., US, GB, CA) for location-specific scraping
wait_before_scraping: Wait time in milliseconds before scraping (0-10000)
parser: Optional parser ID for specialized extraction

Response (example):

{
  "content": [
    {
      "type": "text",
      "text": "{\n  \"id\": \"scrp_...\",\n  \"url\": \"https://example.com\",\n  \"markdown_content\": \"# ...\",\n  \"html_content\": null,\n  \"json_content\": null,\n  \"text_content\": null,\n  \"status\": \"succeeded\",\n  \"timestamp\": \"2025-11-14T12:34:56Z\",\n  \"screenshot_hosted_url\": null,\n  \"page_metadata\": { }\n}"
    }
  ]
}

2. Search the Web (`search_web`)

Search the Web for a given query and get structured results (non-AI, parser-based).

{
  "name": "search_web",
  "arguments": {
    "query": "your search query",
    "country": "US"
  }
}

Parameters:

query: Search query (required)
country: Optional country code for localized results (default: US)

Response:

Structured JSON (as text) representing parser-based results

3. Answers (AI) (`answers`)

Search the web and return AI-powered answers in the JSON structure you want, with sources and citations.

{
  "name": "answers",
  "arguments": {
    "task": "Who are the top 5 competitors to Acme Inc. in the EU?",
    "json": "Return a list of the top 5 competitors with name and homepage URL"
  }
}

Parameters:

task: Question or task to answer using web data (required)
json: Optional JSON schema/object or a short description of the desired output shape

Response includes:

answer_id, object, task, result (JSON if provided), sources, created

4. Batch Scrape URLs (`batch_scrape_urls`)

Scrape up to 10k URLs at the same time. Perfect for large-scale data extraction.

{
  "name": "batch_scrape_urls",
  "arguments": {
    "urls_to_scrape": [
      {"url": "https://example.com/a", "custom_id": "a"},
      {"url": "https://example.com/b", "custom_id": "b"}
    ],
    "output_format": "markdown",
    "country": "US",
    "wait_before_scraping": 500,
    "parser": "@olostep/amazon-product"
  }
}

Response includes:

batch_id, status, total_urls, created_at, formats, country, parser, urls

5. Create Crawl (`create_crawl`)

Autonomously discover and scrape entire websites by following links.

{
  "name": "create_crawl",
  "arguments": {
    "start_url": "https://example.com/docs",
    "max_pages": 25,
    "follow_links": true,
    "output_format": "markdown",
    "country": "US",
    "parser": "@olostep/doc-parser"
  }
}

Response includes:

crawl_id, object, status, start_url, max_pages, follow_links, created, formats, country, parser

6. Create Map (`create_map`)

Get all URLs on a website. Extract all URLs for discovery and analysis.

{
  "name": "create_map",
  "arguments": {
    "website_url": "https://example.com",
    "search_query": "blog",
    "top_n": 200,
    "include_url_patterns": ["/blog/**"],
    "exclude_url_patterns": ["/admin/**"]
  }
}

Response includes:

map_id, object, url, total_urls, urls, search_query, top_n

7. Get Webpage Content (`get_webpage_content`)

Retrieves webpage content in clean markdown format with support for JavaScript rendering.

{
  "name": "get_webpage_content",
  "arguments": {
    "url_to_scrape": "https://example.com",
    "wait_before_scraping": 1000,
    "country": "US"
  }
}

Parameters:

url_to_scrape: The URL of the webpage to scrape (required)
wait_before_scraping: Time to wait in milliseconds before starting the scrape (default: 0)
country: Residential country to load the request from (e.g., US, CA, GB) (optional)

Response:

{
  "content": [
    {
      "type": "text",
      "text": "# Example Website\n\nThis is the markdown content of the webpage..."
    }
  ]
}

8. Get Website URLs (`get_website_urls`)

Search and retrieve relevant URLs from a website, sorted by relevance to your query.

{
  "name": "get_website_urls",
  "arguments": {
    "url": "https://example.com",
    "search_query": "your search term"
  }
}

Parameters:

url: The URL of the website to map (required)
search_query: The search query to sort URLs by (required)

Response:

{
  "content": [
    {
      "type": "text",
      "text": "Found 42 URLs matching your query:\n\nhttps://example.com/page1\nhttps://example.com/page2\n..."
    }
  ]
}

9. Web Search (`google_search`)

Retrieve structured data from web search results.

{
  "name": "google_search",
  "arguments": {
    "query": "your search query",
    "country": "US"
  }
}

Parameters:

query: The search query to perform (required)
country: Country code for localized results (e.g., US, GB) (default: "US")

Response includes:

Organic search results with titles, links, and snippets
Knowledge graph data when available
Related questions (People Also Ask)
Related searches
Rich snippets and other structured data

Error Handling

The server provides robust error handling:

Detailed error messages for API issues
Network error reporting
Authentication failure handling
Rate limit information

Example error response:

{
  "isError": true,
  "content": [
    {
      "type": "text",
      "text": "Olostep API Error: 401 Unauthorized. Details: {\"error\":\"Invalid API key\"}"
    }
  ]
}

Distribution

Docker Images

The MCP server is available as a Docker image:

Docker Hub: olostep/mcp-server
Official Docker MCP Registry: mcp/olostep (coming soon - enhanced security with signatures & SBOMs)
GitHub Container Registry: ghcr.io/olostep/olostep-mcp-server

Docker Desktop MCP Toolkit

The Olostep MCP Server is being added to Docker Desktop's official MCP Toolkit, which means users will be able to:

Discover it in Docker Desktop's MCP Toolkit UI
Install it with one click
Configure it visually
Use it with any MCP-compatible client (Claude Desktop, Cursor, etc.)

Status: Submission in progress to Docker MCP Registry

Supported Platforms

linux/amd64
linux/arm64

Building Locally

# Clone the repository
git clone https://github.com/olostep/olostep-mcp-server.git
cd olostep-mcp-server

# Build the image
npm install
npm run build
docker build -t olostep/mcp-server .

# Run locally
docker run -i --rm -e OLOSTEP_API_KEY="your-key" olostep/mcp-server

License

ISC License

Related Servers

Bright Data

sponsor

Discover, extract, and interact with the web - one interface powering automated access across the public internet.

Feed

A server for fetching and parsing RSS, Atom, and JSON feeds.

MCP Chrome Integration

Control the Chrome browser for web automation using an AI model. Requires the MCP Chrome extension.

Scrapfly

Scrapfly MCP Server gives AI agents a simple, unified way to scrape live web data with built-in anti-bot handling.

Firecrawl

Extract web data with Firecrawl

Web Search

Performs web searches and extracts full page content from search results.

Claimify

Extracts factual claims from text using the Claimify methodology. Requires an OpenAI API key.

Browserbase

Automate browser interactions in the cloud (e.g. web navigation, data extraction, form filling, and more)

brosh

A browser screenshot tool to capture scrolling screenshots of webpages using Playwright, with support for intelligent section identification and multiple output formats.

MCP Browser Console Capture Service

A browser automation service for capturing console output, useful for tasks like public sentiment analysis.

Shufersal MCP Server

Automates shopping on the Shufersal website, enabling LLMs to search for products, create shopping lists, and manage the cart.

Olostep MCP Server

Olostep MCP Server

Features

Installation

🐳 Running with Docker (Recommended)

Local-only Docker build (no Docker Hub required)

Local smoke test (initialize + tools/list)

Using Docker with Claude Desktop

Using Docker with Cursor

Running with npx

Manual Installation

Running on Claude Desktop

Running on Windsurf

Running on Cursor

Running on Metorial

Configuration

Environment Variables

Available Tools

1. Scrape Website (scrape_website)

Parameters:

Response (example):

2. Search the Web (search_web)

Parameters:

Response:

3. Answers (AI) (answers)

Parameters:

Response includes:

4. Batch Scrape URLs (batch_scrape_urls)

Response includes:

5. Create Crawl (create_crawl)

Response includes:

6. Create Map (create_map)

Response includes:

7. Get Webpage Content (get_webpage_content)

Parameters:

Response:

8. Get Website URLs (get_website_urls)

Parameters:

Response:

9. Web Search (google_search)

Parameters:

Response includes:

Error Handling

Distribution

Docker Images

Docker Desktop MCP Toolkit

Supported Platforms

Building Locally

License

Related Servers

Bright Data

Feed

MCP Chrome Integration

Scrapfly

Firecrawl

Web Search

Claimify

Browserbase

brosh

MCP Browser Console Capture Service

Shufersal MCP Server

1. Scrape Website (`scrape_website`)

2. Search the Web (`search_web`)

3. Answers (AI) (`answers`)

4. Batch Scrape URLs (`batch_scrape_urls`)

5. Create Crawl (`create_crawl`)

6. Create Map (`create_map`)

7. Get Webpage Content (`get_webpage_content`)

8. Get Website URLs (`get_website_urls`)

9. Web Search (`google_search`)