How to Use the Robots.txt Generator: Steps and a Worked Example
Generate a simple robots.txt file with optional disallow paths and sitemap URL. This guide shows what to enter, a worked example with real numbers, and the limits to keep in mind. The Robots.txt Generator tool is free, needs no account and runs in your browser.
What you need
- Sitemap URL (optional), for example
https://example.com/sitemap.xml - Disallow paths, one per line
Step by step
- Open the Robots.txt Generator tool.
- Fill in Sitemap URL (optional).
- Fill in Disallow paths, one per line.
- Read the result, which updates as soon as the inputs change.
Worked example
With these inputs:
- Sitemap URL (optional):
https://example.com/sitemap.xml
the tool returns:
User-agent: *
Allow: /
Disallow: /private/
Disallow: /admin/
Sitemap: https://example.com/sitemap.xml
How it works
Outputs robots.txt directives from user-supplied values. Everything runs in your browser, so nothing you type is uploaded.
Things to keep in mind
Robots.txt is a crawl directive, not access control. Never rely on it to protect private content.
Related tools
If this is not quite what you need, try SEO Meta Tag Generator, UTM Link Builder or XML Sitemap Generator.