How to check whether AI search can read your website
Check four things: whether robots.txt blocks AI search crawlers, whether the page carries noindex, whether the main text is visible without running JavaScript, and whether crawlers are blocked by a firewall or CDN. GEO Doctor (geo-doctor) has a free, MIT-licensed checker for these. Why AI search ma
AI assistants that search the web fetch pages first and answer from what they fetched. If they cannot fetch your page, or get an empty shell, they will not cite you. Four usual causes:
GEO Doctor checks robots.txt, AI crawlers, noindex and whether the main text is visible, and tells you which item fails. It checks technical readability only, not whether the content is good. It runs from a Node.js 18+ command line or inside AI assistants such as Claude Code, Codex and Kimi Code. Your data stays in plain files in your own project; GEO Doctor has no server.
Being readable is only step one: AI also needs clear, consistent content about you to cite.
Source: https://github.com/Fangx-AI/geo-doctor
Disclosure: I maintain GEO Doctor, and this article was drafted with AI assistance (Claude) and reviewed by me.