SEO ToolsPrivate by design
Robots.txt Generator
Build a robots.txt file by adding Allow/Disallow rules, an optional crawl-delay, and your sitemap URL — copy the result straight into your site's root.
How to use this tool
- 1Add Allow or Disallow rules for the paths you want to control.
- 2Enter your sitemap URL and an optional crawl-delay.
- 3Copy the generated robots.txt into your site's root directory.
Frequently asked questions
Where does robots.txt need to go?+
At the root of your domain, e.g. https://example.com/robots.txt — it won't be respected anywhere else.
Does robots.txt guarantee pages won't be indexed?+
No — it only asks well-behaved crawlers not to fetch a path. Pages can still be indexed if linked from elsewhere; use a noindex meta tag to actually prevent indexing.
Related tools
Keep the boring tools boring.
If this saved you five minutes of nonsense, you can help keep the free tools running.
Buy the toolbox a coffee