What we check
Passes if /robots.txt could be fetched at all. Fails if no robots.txt file is present.
Why it matters
Without a robots.txt that loads, you have no place to tell any crawler, AI agents included, where your sitemap is or which parts of your site they may visit.
More checks in Access
Can agents reach and crawl the site at all?
Scan your site to see if you pass.
Run the free scan →Free score · no signup · about a minute
HelpAEO