AI Crawler Access Check by Citant.ai

ChatGPT, Claude, Perplexity 및 기타 AI 크롤러(GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot)가 실제로 웹사이트에 접속할 수 있는지 확인합니다. 각 크롤러로 사이트를 가져와 robots.txt 판독기로는 놓치는 Cloudflare, Akamai 및 기타 방화벽/CDN 차단을 찾아냅니다. 0~100점의 AI Access Score, 크롤러별 판정 및 해결 방법을 제공합니다. 무료이며 읽기 전용이고 로그인할 필요가 없습니다.

citant.ai에서 보기

호스팅형 MCP 서버

npx add-mcp 'https://check.citant.ai/mcp?src=mcpservers'

Claude Code, Codex, Cursor 등에 설치됩니다

문서

AI Crawler Access Check MCP server

A free, remote MCP server that lets your AI app answer one question: can ChatGPT, Claude, Perplexity and the other AI crawlers actually reach a website? It runs the same 21 checks as the AI Crawler Access Report, fetching the site as each crawler, so it catches firewall and CDN blocks that a robots.txt reader misses. Read-only, no login, no API key.

Server URL: https://check.citant.ai/mcp
Transport: Streamable HTTP. Tool: check_ai_crawler_access (input: a domain or URL).

Add it to your AI app

Claude (web and desktop)

Open Settings, then Connectors, choose Add custom connector, give it a name and paste the URL. No authentication.

https://check.citant.ai/mcp?src=docs

Claude Code

Run once in a terminal:

claude mcp add --transport http citant-crawler-check "https://check.citant.ai/mcp?src=docs"

Cursor

Add to ~/.cursor/mcp.json (or.cursor/mcp.json in a project):

{
  "mcpServers": {
    "citant-crawler-check": {
      "url": "https://check.citant.ai/mcp?src=docs"
    }
  }
}

VS Code

Add to.vscode/mcp.json:

{
  "servers": {
    "citant-crawler-check": {
      "type": "http",
      "url": "https://check.citant.ai/mcp?src=docs"
    }
  }
}

ChatGPT (developer mode) and other MCP apps

Create a custom connector that points at this URL, with no authentication:

https://check.citant.ai/mcp?src=docs

What to ask

  • Can ChatGPT and Perplexity crawl example.com?
  • Is my site blocking GPTBot at the firewall, or only in robots.txt?
  • Check AI crawler access for our pricing page and tell me the top fix.

What it returns

  • An AI Access Score from 0 to 100 with three sub-scores (live answers, training data, search index)
  • A verdict for each crawler: what robots.txt says and what a real fetch as that crawler got
  • Up to 8 critical and warning findings, each with a fix
  • A link to the full report on check.citant.ai

A scan takes 15 to 60 seconds. Results for the same domain are cached for 24 hours. The tool reads public pages only and never changes anything. What it fetches and stores is documented on the CitantScanBot page.

Where it is listed

Prefer a browser?

Run the same check on check.citant.ai. If the scan shows you are readable but AI assistants still do not name you, that is a different problem, and it is what the AI Search Audit by Citant.ai measures.