AI Crawler Access Check by Citant.ai

Memeriksa apakah ChatGPT, Claude, Perplexity, dan perayap AI lainnya (GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot) benar-benar dapat mengakses situs web. Mengambil situs sebagai masing-masing perayap, sehingga dapat mendeteksi pemblokiran oleh Cloudflare, Akamai, dan firewall/CDN lainnya yang tidak terdeteksi oleh pembaca robots.txt. Menghasilkan Skor Akses AI 0-100, hasil untuk setiap perayap, dan solusi. Gratis, hanya-baca, tanpa login.

Lihat di citant.ai

Server MCP Terhosting

npx add-mcp 'https://check.citant.ai/mcp?src=mcpservers'

Terpasang ke Claude Code, Codex, Cursor, dan lainnya

Dokumentasi

AI Crawler Access Check MCP server

A free, remote MCP server that lets your AI app answer one question: can ChatGPT, Claude, Perplexity and the other AI crawlers actually reach a website? It runs the same 21 checks as the AI Crawler Access Report, fetching the site as each crawler, so it catches firewall and CDN blocks that a robots.txt reader misses. Read-only, no login, no API key.

Server URL: https://check.citant.ai/mcp
Transport: Streamable HTTP. Tool: check_ai_crawler_access (input: a domain or URL).

Add it to your AI app

Claude (web and desktop)

Open Settings, then Connectors, choose Add custom connector, give it a name and paste the URL. No authentication.

https://check.citant.ai/mcp?src=docs

Claude Code

Run once in a terminal:

claude mcp add --transport http citant-crawler-check "https://check.citant.ai/mcp?src=docs"

Cursor

Add to ~/.cursor/mcp.json (or.cursor/mcp.json in a project):

{
  "mcpServers": {
    "citant-crawler-check": {
      "url": "https://check.citant.ai/mcp?src=docs"
    }
  }
}

VS Code

Add to.vscode/mcp.json:

{
  "servers": {
    "citant-crawler-check": {
      "type": "http",
      "url": "https://check.citant.ai/mcp?src=docs"
    }
  }
}

ChatGPT (developer mode) and other MCP apps

Create a custom connector that points at this URL, with no authentication:

https://check.citant.ai/mcp?src=docs

What to ask

  • Can ChatGPT and Perplexity crawl example.com?
  • Is my site blocking GPTBot at the firewall, or only in robots.txt?
  • Check AI crawler access for our pricing page and tell me the top fix.

What it returns

  • An AI Access Score from 0 to 100 with three sub-scores (live answers, training data, search index)
  • A verdict for each crawler: what robots.txt says and what a real fetch as that crawler got
  • Up to 8 critical and warning findings, each with a fix
  • A link to the full report on check.citant.ai

A scan takes 15 to 60 seconds. Results for the same domain are cached for 24 hours. The tool reads public pages only and never changes anything. What it fetches and stores is documented on the CitantScanBot page.

Where it is listed

Prefer a browser?

Run the same check on check.citant.ai. If the scan shows you are readable but AI assistants still do not name you, that is a different problem, and it is what the AI Search Audit by Citant.ai measures.