OmniExtract MCP
Cookie-free MCP server & Apify Actor turning X Articles, watermark-free Douyin/TikTok media, YouTube transcripts, and anti-bot web pages into clean Markdown.
Hosted MCP Server
npx add-mcp 'https://mcp.apify.com/?actors=poweragy/omni-extract-mcp'Installs into Claude Code, Codex, Cursor and more
Documentation
OmniExtract MCP & Apify Actor (omni-extract-mcp)
Universal, Stateless, Cookie-Free Content & Transcript Extractor for AI Agents (Claude Desktop, Cursor, Windsurf, OpenClaw, n8n, Make, and Apify x402).
π Run Instantly in the Cloud (Zero Setup, Pay-Per-Result / x402 Ready): https://apify.com/poweragy/omni-extract-mcp
Why OmniExtract?
Standard WebFetch and browser tools fail with 403 Forbidden, login walls, or empty JavaScript shells when given social media or video URLs. OmniExtract converts complex links into clean, LLM-ready Markdown + Timestamped Transcripts + Watermark-Free Media URLs in one call:
- X (Twitter) Posts & Long-Form X Articles: Full Draft.js block rendering (
# / ## / ###, ordered/unordered lists, code blocks, blockquotes, quoted posts) and media extraction with zero personal cookies required. - Douyin & TikTok Watermark-Free Extraction: Pure-algorithm
x-secsdk-web-signaturesigning (ttwid+uifid+ MD5 salt) + 302 shortlink resolution + binary MIME magic sniffing (WebP/JPEG/PNG) β zero headless browser overhead. - YouTube & Bilibili 3-Layer Subtitle Engine: Extracts native & auto-generated VTT/SRT captions into deduplicated full-text transcripts and timestamped segments in ~2 seconds without downloading heavy video files.
- Anti-Bot Web & GitHub Markdown: Automatic GitHub raw README/blob fast-path, semantic HTML-to-Markdown conversion, bot-wall detection, and Jina Reader fallback.
Option 1: Hosted Cloud API & MCP via Apify Store (Recommended)
No local dependencies (ffmpeg / yt-dlp) or server maintenance required:
- Apify Store Page: https://apify.com/poweragy/omni-extract-mcp
- REST API Call:
curl -X POST "https://api.apify.com/v2/acts/poweragy~omni-extract-mcp/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "urls": [ "https://x.com/ai_chuhai/status/2037437653071835440", "https://v.douyin.com/0kH2cQVrf0o/" ], "include_transcript": true }'
Option 2: Local MCP Server (Claude Desktop / Cursor / Windsurf)
pip install git+https://github.com/solarfans/omni-extract-mcp.git
Add to your claude_desktop_config.json or .cursor/mcp.json:
{
"mcpServers": {
"omni-extract": {
"command": "python3",
"args": ["-m", "omni_extract.mcp_server", "--mode", "stdio"]
}
}
}
Exposed MCP Tools
omni_extract_url(url, include_transcript=True, output_format="markdown"): Extract a single URL into structured Markdown or JSON.omni_extract_batch(urls, include_transcript=True): Concurrently extract up to 10 URLs in parallel.omni_check_quota(api_key=""): Check remaining API key credits or daily quota.
Option 3: Self-Hosted HTTP / Remote MCP Gateway
python3 -m omni_extract.mcp_server --mode http --port 8080
GET /v1/health: Health & capability checkPOST /mcp: Streamable HTTP MCP JSON-RPC 2.0 endpointPOST /v1/extract: Direct REST single URL extractionPOST /v1/extract/batch: Direct REST concurrent batch extractionGET /v1/quota: Check remaining API key credits or daily anonymous free quota
Option 4: Quick Directory Integration (Smithery / mcpservers.org / Cursor / Claude Desktop)
Hosted Apify MCP Server (Zero Local Install)
Connect directly using Apify's hosted MCP endpoint (https://mcp.apify.com/?actors=poweragy/omni-extract-mcp):
{
"mcpServers": {
"omni-extract-cloud": {
"command": "npx",
"args": ["-y", "mcp-remote", "https://mcp.apify.com/?actors=poweragy/omni-extract-mcp"],
"env": {
"APIFY_TOKEN": "YOUR_APIFY_TOKEN"
}
}
}
}
Troubleshooting & Common Error Fixes (FAQ)
- Fix: X (Twitter) Articles Truncated / JS Shell: Standard HTTP scrapers stop at the X JS shell or truncated preview. Pass the full
x.com/<user>/status/<id>URL toomni_extract_urlβ OmniExtract resolves the underlying Draft.js blocks (article.content.blocks) and entity media automatically without cookies. - Fix: Douyin Watermark & Cookie Expired Errors: Pass
v.douyin.comordouyin.com/video/...links directly. OmniExtract resolves 302 redirects and computesx-secsdk-web-signaturenatively, returning clean Markdown + direct media URLs without headless browser overhead. - Fix: YouTube Transcript Blocked on Cloud IPs: Runs a 3-layer native +
yt-dlpcaption pipeline without downloading full MP4 streams. - Empty
{}health checks vs billed items: Empty input{}runs execute a read-only connectivity check and skip PPE item billing (apify_ppe_skipped: true), while real extraction runs emit transparentcharged/charge_failedcounters in the Actor summary. - Registry manifests included:
smithery.yamlandserver.json.