searxNcrawl

An MCP server to search and crawl the web for your AI agents

Run the open-source MCP server for web search and Markdown crawling yourself, or join the waitlist for searxNcrawl Cloud.

searxNcrawl is an open-source MCP server that gives AI agents web search and clean Markdown crawling; a managed Cloud version is planned.

MCP server · Web search · Web crawling · Markdown for LLMs · Open source (MIT) · Developed in Germany

Open source: Run it yourself

Free, MIT licence.

• MCP server over STDIO or HTTP
• Search via SearXNG and/or You.com
• Crawl pages and whole sites to Markdown
• Crawl behind logins with a captured session

Cloud: Let us run it

Planned, waitlist open.

• Drop-in replacement for the Firecrawl API
• One hosted MCP endpoint with an API key
• Premium search and stealth crawling
• Batch crawls, monitoring and extraction

uvx searxncrawl

From query to Markdown in one call

searxNcrawl searches the web through your own SearXNG instance and, if you add your own API key, the You.com search API, queried in parallel and merged into one result list. It crawls pages into clean, deduplicated Markdown with Crawl4AI and Playwright. It runs on your own hardware, needs no API key with SearXNG and is published on GitHub and PyPI.

• Web search: Your own SearXNG instance and, opt-in with your own API key, You.com, queried in parallel. Results are merged, deduplicated and attributed to their provider, filtered by language and time range. With include_content=true, each result also carries the Markdown of its page.
• Page crawl: One or many URLs to Markdown or JSON. Repeated blocks are removed, links can be stripped.
• Site crawl: Crawls of whole sites with a depth limit and a page limit, tuned for documentation.
• Authenticated crawls: Capture a login session once with crawl-capture, from a login window or a running browser, then crawl behind the login.
• STDIO and HTTP: Plug into local MCP clients, or expose an HTTP endpoint with host and CORS guards.
• CLI and Python API: crawl and search in your shell, crawl_page_async and crawl_site in your code.

How to run searxNcrawl

Three steps from an empty machine to an agent that searches and crawls.

1. Pick a search provider: Run your own SearXNG instance with JSON output enabled, add a You.com API key, or both. Crawling works without either.
2. Register it with your agents: Run uvx searxncrawl setup. It finds Claude Code, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop, asks for your SearXNG URL and an optional You.com API key, and backs up every file before it changes it.
3. Search and crawl: Your agent calls search, crawl or crawl_site. Chromium installs itself on the first crawl. Other clients take the MCP configuration below.

uvx

# Register it with your coding agents
uvx searxncrawl setup

# or run the MCP server yourself (STDIO)
export SEARXNG_URL=http://localhost:8888
uvx searxncrawl

# or serve it over HTTP
uvx searxncrawl --transport http --port 9555

CLI

# Crawl a page
crawl https://docs.example.com

# Crawl a site with limits
crawl https://docs.example.com --site --max-depth 2 --max-pages 10 -o docs/

# Search the configured providers
search "python tutorials" --max-results 5

Python

from crawler import crawl_page_async, crawl_site

# One page
doc = await crawl_page_async("https://docs.example.com/intro")
print(doc.markdown)

# A whole site
result = crawl_site("https://docs.example.com", max_depth=2, max_pages=10)
for doc in result.documents:
    print(doc.status, doc.final_url)

Docker

# MCP server with Chromium included
cp .env.example .env   # set SEARXNG_URL, YDC_API_KEY or both
docker compose up --build

# MCP endpoint
http://localhost:9555/mcp

MCP configuration

One configuration block: add it to your MCP client and point it at your SearXNG instance.

{
  "mcpServers": {
    "crawler": {
      "command": "uvx",
      "args": ["searxncrawl"],
      "env": { "SEARXNG_URL": "http://your-searxng:8888" }
    }
  }
}

MCP clients

It runs where your agents run: STDIO for local clients, HTTP for remote ones. Claude Code, VS Code, Zed, opencode, Any STDIO or HTTP MCP client.

searxNcrawl Cloud (Planned)

The open-source tools as a managed service, plus the parts that are hard to run yourself: a Firecrawl-compatible API, premium search and crawling, job queues, webhooks, monitoring and extraction.

• Drop-in for the Firecrawl API: A Firecrawl-compatible REST API for scrape, crawl, map and search. Point an existing Firecrawl SDK or integration at searxNcrawl by changing the base URL. Planned for the open-source server as well.
• Hosted MCP endpoint: One HTTPS address and an API key. Works with every MCP client, with no SearXNG, browser or server to run.
• Search with page content: One call returns the search results together with the Markdown of each result page. Released in the open-source project with v0.33.
• Premium search: Several search sources, including commercial search APIs, queried in parallel, merged, deduplicated and labelled per source, with a fallback when one source fails. Builds on multi-provider search, released in the open-source project with v0.32.
• Premium crawling: A hardened browser with realistic fingerprints and rotating residential proxies, for sites that block crawlers from data centres.
• Batch crawls with webhooks: Queue many URLs or whole sites and receive a signed webhook when the results are ready.
• Crawls behind logins: Login sessions stored encrypted for your team, so agents crawl authenticated pages without session files on laptops. Builds on crawl-capture and session files.
• Change monitoring: Watch pages on a schedule, compare the Markdown and get notified when the content changes.
• Structured extraction: Send a JSON Schema with a URL and receive typed JSON instead of Markdown.

Planned feature set. Scope and timing can change before launch; people on the waitlist hear first. Firecrawl is a trademark of its owner; searxNcrawl is not affiliated with it.

Open source or Cloud

The same tools and the same output format. The difference is who runs the infrastructure.

Capability	Open source	Cloud
Firecrawl compatibility: Firecrawl-compatible REST API	Not included: Planned for open source too	Planned: Change the base URL
Getting started: Setup	Available: uvx searxncrawl setup and SearXNG or You.com	Planned: An API key, nothing to install
Getting started: MCP tools	Available: search, crawl, crawl_site	Planned: The same tools
Getting started: Who runs it	Available: You: server, SearXNG, browser	Planned: We do
Search: Search sources	Available: Your SearXNG and/or You.com, merged	Planned: Several sources, merged
Search: Fallback when a source fails	Partly: The other provider still answers	Planned: Automatic
Search: Commercial search APIs	Available: You.com, with your own API key	Planned: Included in premium search
Search: Search with page content	Available: One call: include_content=true	Planned: The same, hosted
Crawling: Pages and sites to Markdown	Available: Depth and page limits	Planned: The same output
Crawling: Crawls behind logins	Available: Local session file from crawl-capture	Planned: Encrypted sessions per team
Crawling: Batch crawls	Partly: Synchronous, on your machine	Planned: Queued, with signed webhooks
Crawling: Sites that block crawlers	Not included: Standard Chromium from your network	Planned: Stealth browser, residential proxies
Automation: Change monitoring	Not included: Not included	Planned: Scheduled, with notifications
Automation: Structured extraction	Not included: Not included	Planned: JSON Schema to typed JSON
Support: Help	Available: GitHub issues	Planned: E-mail support

Join the waitlist

Leave your e-mail address and we will notify you when early access to searxNcrawl Cloud opens.

Join the waitlist: https://www.searxncrawl.io/#waitlist

• One e-mail when access opens
• Revocable at any time
• No third parties

Frequently asked questions

What is searxNcrawl?

searxNcrawl is an open-source MCP server that gives AI agents web search and clean Markdown crawling; a managed Cloud version is planned. It searches through your own SearXNG instance and, opt-in, the You.com API, merges the results, turns web pages into Markdown without repeated blocks, is MIT licensed and is maintained by Das Digitale Momentum.

How do I install searxNcrawl?

Run uvx searxncrawl setup. It registers the MCP server with Claude Code, Codex, Gemini CLI, Cursor, VS Code and Claude Desktop, asks for your SearXNG URL and an optional You.com API key, shows each change before applying it and backs up every file it touches. To start the server by hand, run uvx searxncrawl with SEARXNG_URL and/or YDC_API_KEY set. Chromium is downloaded on the first crawl. Docker, pip and uv installs are described in the installation guide on GitHub.

Can searxNcrawl replace Firecrawl?

That is the plan: a Firecrawl-compatible REST API for scrape, crawl, map and search is planned for the open-source server and for searxNcrawl Cloud, so existing Firecrawl SDKs work by changing the base URL. Today searxNcrawl offers the MCP tools search, crawl and crawl_site, a CLI and a Python API. The Firecrawl comparison shows where each one fits.

Which MCP clients work with searxNcrawl?

Any MCP client. searxNcrawl speaks the STDIO transport for local clients such as Claude Code, VS Code, Zed and opencode, and the HTTP transport for remote and browser-based clients.

Do I need an API key?

Not with SearXNG. The open-source version needs no API key and has no usage fees when it searches through your own SearXNG instance with JSON output enabled. You.com search is opt-in: it needs your own You.com API key (YDC_API_KEY), and those queries go to You.com in the US.

What is searxNcrawl Cloud?

searxNcrawl Cloud is a planned managed version: a hosted MCP endpoint with search and page content in one call, search across several sources, batch crawls with webhooks, crawls behind logins, change monitoring and structured extraction. The waitlist is open.

Will the open-source version stay free?

Yes. The open-source searxNcrawl stays MIT licensed and free to run yourself. The Cloud version adds managed infrastructure and builds on the open-source project.

Built by DDM

searxNcrawl is developed and maintained by DDM, Das Digitale Momentum GmbH & Co. KG, a software company in Much near Cologne. It succeeds the earlier searxng-mcp project. Read who builds searxNcrawl or contact the team.

• searxNcrawl at Das Digitale Momentum
• All open-source projects of Das Digitale Momentum

searxNcrawl is an open-source project of Das Digitale Momentum GmbH & Co. KG, Much, Germany.

Updated: 2026-10-15

---

Source: https://www.searxncrawl.io/