# AI Crawlability

> A practical, source-grounded guide to making your website visible and citable to AI answer engines like ChatGPT, Perplexity, and Google AI Overviews.

AI crawlability is configuring a website so AI crawlers (GPTBot, ClaudeBot, PerplexityBot, Google-Extended and others) can reach, parse, and cite its content. Guidance here is source-grounded and avoids unverified citation guarantees.

## Tools

- [AI Crawlability Checker](https://aicrawlability.com/checker): Free per-bot check of robots.txt, headers, JavaScript rendering, llms.txt and sitemap, with specific fixes.
- [Agent policy](https://aicrawlability.com/AGENT-POLICY.md): Scope and permissions for automated agents using the checker.
- [Machine-readable capabilities](https://aicrawlability.com/agents.json): The checker capability and its supported API endpoint.
- [OpenAPI description](https://aicrawlability.com/openapi.json): Machine-readable contract for the public checker endpoint.

## Guide

- [AI Crawlability: Make Your Site Visible to AI](https://aicrawlability.com/ai-crawlability): Make your site accessible to AI answer engines with practical guidance on bots, robots.txt, llms.txt, JavaScript rendering and schema.

## Articles

- [llms.txt Guide: What It Is and What Helps](https://aicrawlability.com/llms-txt): Learn what llms.txt is, how to create it, how it differs from robots.txt and what evidence says about AI visibility.
- [robots.txt for AI Crawlers: GPTBot & More](https://aicrawlability.com/robots-txt-ai-crawlers): Set robots.txt rules for GPTBot, ClaudeBot, PerplexityBot and other AI crawlers, with examples, common mistakes and practical limits.
- [How to Structure Content for AI Extraction](https://aicrawlability.com/content-structure-ai-extraction): How to structure pages so AI answer engines can extract and cite them: direct answers, question-led headings, scannable formatting, and FAQ and Article schema.