You can check whether AI can see your website in about five minutes by inspecting your robots.txt file for AI crawler restrictions, confirming your key content renders without JavaScript, and asking ChatGPT or Perplexity directly what they know about your business. If any of the three fail, AI systems may be forming recommendations without ever actually reading your site — and no amount of content quality fixes that until access is restored.
A dentist’s office had beautiful content: clear service pages, real patient testimonials, a genuinely helpful FAQ. None of it mattered. Their website platform’s default settings blocked every major AI crawler. ChatGPT had never read a single page. The content was invisible, not bad — and those are very different problems with very different fixes.
Step 1: Check your robots.txt file for AI crawler blocks
Your robots.txt file, at yoursite.com/robots.txt, may explicitly disallow the crawlers AI companies use to read the web — check it directly. Look for lines mentioning GPTBot (OpenAI), PerplexityBot (Perplexity), Google-Extended (Gemini’s training access), and ClaudeBot (Anthropic). If any of these appear next to Disallow: /, that crawler is blocked from your entire site.
This matters more than most business owners realize because some website platforms enable these blocks by default, without a clear notification. Squarespace, for instance, offers a one-click setting to block known AI scrapers, and on some plans it isn’t user-editable at all — meaning a business owner can be blocking AI without ever having chosen to.
Key takeaway: A blocked crawler is a closed door — check robots.txt before anything else.
Step 2: Confirm your content doesn’t require JavaScript to display
Even with an open robots.txt, your site can still be invisible to AI if your content only appears after JavaScript runs. Industry estimates suggest a majority of AI crawlers cannot execute JavaScript, meaning a site built with heavy client-side rendering can hand these bots a blank page regardless of what robots.txt allows.
The quick test: view your page’s source code (right-click, “View Page Source,” not “Inspect”) and search for your actual service descriptions and business details in the raw HTML. If they’re missing from the source but visible in the browser, JavaScript is rendering them after the fact — and many AI crawlers never see them.
Key takeaway: If your content only exists after JavaScript runs, most AI crawlers never see it.
Step 3: Ask ChatGPT and Perplexity what they know about your business
Open a new conversation and ask directly: “What can you tell me about [your business name] in [your city]?” Then ask a question a real customer would ask, without naming your business: “Who’s a good [your service] in [your area]?” The first question tells you whether the AI has any accurate information about you at all. The second tells you whether it would ever surface you unprompted — the test that actually matters for new customers.
If the first answer is vague, wrong, or nonexistent, and the second never mentions you, access or authority is broken somewhere upstream of content quality.
Key takeaway: The real test isn’t whether AI knows you exist — it’s whether it would recommend you unprompted.
What to do if you fail any of these checks
Fix access before touching anything else — content improvements on a blocked site accomplish nothing. If robots.txt is blocking crawlers, update it or ask your web platform how to unblock AI bots specifically. If your platform can’t be changed (some page builders lock this down), it may be a sign your platform itself is limiting your AI visibility — worth weighing against a rebuild. If access is fine but the AI still doesn’t recommend you, the next layer is structure and authority: schema markup, direct answers, and consistent business information across the web.
Our free AI visibility audit runs all three of these checks automatically, along with 40+ other signals, and tells you exactly where the block is.
Key takeaway: Fix the door before worrying about what’s inside the house.
Frequently asked questions
How do I know if my website blocks AI crawlers?
Visit yoursite.com/robots.txt and look for Disallow rules targeting GPTBot, PerplexityBot, Google-Extended, or ClaudeBot. If found next to a blanket disallow, that crawler can’t access your site.
Does blocking AI crawlers hurt my Google ranking?
Generally no — blocking GPTBot specifically has shown no measurable impact on Google Search rankings in publisher analyses, since Google’s own crawler is separate. But it does mean AI answer engines can’t cite you.
Why would my website block AI crawlers without me choosing to?
Some platforms, including certain Squarespace configurations, block major AI crawlers by default and don’t always allow the setting to be changed, especially on lower-tier plans.
Can ChatGPT read my website if it uses JavaScript?
Often not fully. A majority of AI crawlers can’t execute JavaScript, so content that only renders client-side may be invisible even with an open robots.txt.
What’s the fastest way to check all of this at once?
Run a free AI visibility audit — it checks crawler access, rendering, and structured data automatically in about a minute.
Skip the manual check
Run the free AI visibility audit and get your crawler access, structure, and authority score in one pass — no login, no card, about a minute.