Read URL MCP
Extracts web content from a URL and converts it to clean Markdown format.
Read URL MCP
A Model Context Protocol (MCP) server that provides URL reading capabilities, extracting web content and converting it to clean Markdown format.
Features
- Fetch web content from HTTP/HTTPS URLs
- Extract main content using readability algorithm
- Convert HTML to clean Markdown format
- Configurable timeout and content size limits
Installation
uv install
MCP Setting
{
"mcpServers": {
"read_url_mcp": {
"command": "uv",
"args": [
"run",
"--directory",
"<directory>",
"read_url_mcp/mcp_server"
],
"env": {
"PYTHONPATH": "<directory>"
}
}
}
}
Usage
Run the MCP server:
uv run python read_url_mcp/mcp_server.py
Available Tools
readURLMarkdown(url: str)- Fetches URL content and returns it as Markdown
Development
Code Quality
uv run ruff check # Lint code
uv run ruff format # Format code
uv run ruff check --fix # Auto-fix issues
Dependencies
- mcp[cli] - MCP framework
- requests - HTTP client
- readability-lxml - Content extraction
- html2text - HTML to Markdown conversion
Configuration
Server settings in mcp_server.py:
- Timeout: 30 seconds
- Max content length: 1MB
- User agent: my-mcp-tools/1.0
Related Servers
Bright Data
sponsorDiscover, extract, and interact with the web - one interface powering automated access across the public internet.
Outscraper
Access Outscraper's data extraction services for business intelligence, location data, reviews, and contact information from various online platforms.
Yahoo Finance
Provides comprehensive financial data from Yahoo Finance, including historical prices, company info, financial statements, and market news.
Web Fetch
Fetches and transforms web content, including JavaScript-rendered pages and media files, into various formats.
WebSearch
A web search and content extraction tool using the Firecrawl API for advanced web scraping, searching, and content analysis.
YouTube Transcript
An MCP server for extracting and processing transcripts from YouTube videos.
MCP Web Research Server
A server for web research that brings real-time information into AI models like Claude.
SABIS MCP Server
Access academic grades from the Sakarya University SABIS system via automated web scraping.
Nefino
Access the Nefino renewable energy news API.
Crawl4AI RAG
Integrate web crawling and Retrieval-Augmented Generation (RAG) into AI agents and coding assistants.
MCP Browser Console Capture Service
A browser automation service for capturing console output, useful for tasks like public sentiment analysis.