Robots.txt Generator
Choose which crawlers the rules apply to, list paths to allow or disallow and add your sitemap. Copy the result or download robots.txt.
Robots.txt GeneratorRuns in your browser
Note: robots.txt only gives crawling instructions to well-behaved bots. It is public and is not access security. Do not use it to hide private content.
Your input stays in your browser and is not sent to a server by this page.
About this tool
Place the file at the root of your site, for example https://example.com/robots.txt. A disallowed URL can still appear in search results if other pages link to it; use noindex or authentication when you need real protection.