Read URL MCP
Extracts web content from a URL and converts it to clean Markdown format.
Read URL MCP
A Model Context Protocol (MCP) server that provides URL reading capabilities, extracting web content and converting it to clean Markdown format.
Features
- Fetch web content from HTTP/HTTPS URLs
- Extract main content using readability algorithm
- Convert HTML to clean Markdown format
- Configurable timeout and content size limits
Installation
uv install
MCP Setting
{
"mcpServers": {
"read_url_mcp": {
"command": "uv",
"args": [
"run",
"--directory",
"<directory>",
"read_url_mcp/mcp_server"
],
"env": {
"PYTHONPATH": "<directory>"
}
}
}
}
Usage
Run the MCP server:
uv run python read_url_mcp/mcp_server.py
Available Tools
readURLMarkdown(url: str)- Fetches URL content and returns it as Markdown
Development
Code Quality
uv run ruff check # Lint code
uv run ruff format # Format code
uv run ruff check --fix # Auto-fix issues
Dependencies
- mcp[cli] - MCP framework
- requests - HTTP client
- readability-lxml - Content extraction
- html2text - HTML to Markdown conversion
Configuration
Server settings in mcp_server.py:
- Timeout: 30 seconds
- Max content length: 1MB
- User agent: my-mcp-tools/1.0
Server Terkait
Bright Data
sponsorDiscover, extract, and interact with the web - one interface powering automated access across the public internet.
MCP URL2SNAP
A lightweight MCP server that captures screenshots of any URL and returns the image URL. Requires an AbstractAPI key.
Oxylabs AI Studio
AI tools for web scraping, crawling, browser control, and web search via the Oxylabs AI Studio API.
SABIS MCP Server
Access academic grades from the Sakarya University SABIS system via automated web scraping.
YouTube Transcript MCP Server
A high-performance MCP server for fetching YouTube video transcripts, with support for caching, rate limiting, and proxy rotation.
Bilibili
Interact with the Bilibili video website, enabling actions like searching for videos, retrieving video information, and accessing user data.
AI Shopping Assistant
A conversational AI shopping assistant for web-based product discovery and decision-making.
Clawpage
Extract and structure any web page into clean JSON.
Sports MCP Server
Live sports scores and stats from NBA, NFL, and NHL
Crawl4AI RAG
Integrates web crawling and Retrieval-Augmented Generation (RAG) into AI agents and coding assistants.
Yahoo Finance
Fetch stock data, news, and financial information from Yahoo Finance.