TOOLKIT

Robots.txt Generator

Configure crawler rules and generate a robots.txt file instantly.

Allow all crawlers?

Block specific paths

Block specific bots

Sitemap URL

Crawl delay (seconds, optional)

Generated robots.txt
User-agent: *
Allow: /
Disallow: /admin