Doormat CrawlerCheck

通过 MCP 检查 AI 爬虫访问权限,免费。工具 check_ai_crawler_access 会报告你的 robots.txt 允许或屏蔽了哪些具名 AI 机器人(GPTBot、ClaudeBot 等)。付费的 /report(0.03 USDC,通过 Base 上的 x402)会提供通俗易懂的建议和可直接粘贴的 robots.txt 代码片段。

文档

crawlercheck

Checks which of the ~20 named AI crawlers (GPTBot, ClaudeBot, PerplexityBot, CCBot, Google-Extended and more) your site's robots.txt currently allows or blocks.

Free, no signup. Paid (0.03 USDC on Base): /report?url= adds a plain-English recommendation and a ready-to-paste robots.txt snippet.

crawlercheck by Doormat Capital — free AI-crawler robots.txt checker. Submitted to Twelve Tools.

Our fortnightly notes on what we ship: doormat.sebastiaan-ba3.workers.dev