Read URL MCP
Extracts web content from a URL and converts it to clean Markdown format.
Read URL MCP
A Model Context Protocol (MCP) server that provides URL reading capabilities, extracting web content and converting it to clean Markdown format.
Features
- Fetch web content from HTTP/HTTPS URLs
- Extract main content using readability algorithm
- Convert HTML to clean Markdown format
- Configurable timeout and content size limits
Installation
uv install
MCP Setting
{
"mcpServers": {
"read_url_mcp": {
"command": "uv",
"args": [
"run",
"--directory",
"<directory>",
"read_url_mcp/mcp_server"
],
"env": {
"PYTHONPATH": "<directory>"
}
}
}
}
Usage
Run the MCP server:
uv run python read_url_mcp/mcp_server.py
Available Tools
readURLMarkdown(url: str)- Fetches URL content and returns it as Markdown
Development
Code Quality
uv run ruff check # Lint code
uv run ruff format # Format code
uv run ruff check --fix # Auto-fix issues
Dependencies
- mcp[cli] - MCP framework
- requests - HTTP client
- readability-lxml - Content extraction
- html2text - HTML to Markdown conversion
Configuration
Server settings in mcp_server.py:
- Timeout: 30 seconds
- Max content length: 1MB
- User agent: my-mcp-tools/1.0
Related Servers
Bright Data
sponsorDiscover, extract, and interact with the web - one interface powering automated access across the public internet.
Amazon MCP Server
Scrapes and searches for products on Amazon.
Stepstone
Fetches job listings from Stepstone.de based on keywords and location parameters.
Puppeteer
A server for browser automation using Puppeteer, enabling web scraping, screenshots, and JavaScript execution.
Humanizer PRO
Humanizer PRO turn AI content into Human written content undetectable and bypass all AI detectors.
Puppeteer Vision
Scrape webpages and convert them to markdown using Puppeteer. Features AI-driven interaction capabilities.
Nefino
Access the Nefino renewable energy news API.
Browser MCP
A fast, lightweight MCP server that empowers LLMs with browser automation via Puppeteer’s structured accessibility data, featuring optional vision mode for complex visual understanding and flexible, cross-platform configuration.
MCP Chrome Integration
Control the Chrome browser for web automation using an AI model. Requires the MCP Chrome extension.
Yanyue MCP
Fetches cigarette data and information from Yanyue.cn.
Document Extractor MCP Server
Extracts document content from Microsoft Learn and GitHub URLs and stores it in PocketBase for retrieval and search.