How to Use the Robots.txt Generator: Steps and a Worked Example

By 1 min read

Generate a simple robots.txt file with optional disallow paths and sitemap URL. This guide shows what to enter, a worked example with real numbers, and the limits to keep in mind. The Robots.txt Generator tool is free, needs no account and runs in your browser.

What you need

  • Sitemap URL (optional), for example https://example.com/sitemap.xml
  • Disallow paths, one per line

Step by step

  1. Open the Robots.txt Generator tool.
  2. Fill in Sitemap URL (optional).
  3. Fill in Disallow paths, one per line.
  4. Read the result, which updates as soon as the inputs change.

Worked example

With these inputs:

  • Sitemap URL (optional): https://example.com/sitemap.xml

the tool returns:

User-agent: *
Allow: /
Disallow: /private/
Disallow: /admin/
Sitemap: https://example.com/sitemap.xml

How it works

Outputs robots.txt directives from user-supplied values. Everything runs in your browser, so nothing you type is uploaded.

Things to keep in mind

Robots.txt is a crawl directive, not access control. Never rely on it to protect private content.

If this is not quite what you need, try SEO Meta Tag Generator, UTM Link Builder or XML Sitemap Generator.

More from the blog