HelpAEO
Log inScan my site
← All the checks
Access

robots.txt is fetchable

  • Our estimate: medium impact
  • Runs in the free scan

What we check

Passes if /robots.txt could be fetched at all. Fails if no robots.txt file is present.

Why it matters

Without a robots.txt that loads, you have no place to tell any crawler, AI agents included, where your sitemap is or which parts of your site they may visit.

More checks in Access

Can agents reach and crawl the site at all?

Scan your site to see if you pass.

Run the free scan →Free score · no signup · about a minute