Robots.txt Generator with Sitemap Line
Use this free robots.txt generator to create a clean robots.txt with the Sitemap line that points crawlers to your sitemap.xml. It runs entirely in your browser, builds a matching XML sitemap from the URLs you provide, and needs no sign-up.
Generate your sitemap
Paste your page URLs (or extract them from a page), set options, and download sitemap.xml, an HTML sitemap, and robots.txt.
Your sitemap.xml will appear here as you add URLs.
How to create robots.txt with a sitemap line
Add your URLs
Paste your page URLs one per line, or paste the HTML of a page so the tool can extract its links. Set lastmod, change frequency, and priority (or let it auto-set priority by URL depth).
Generate the files
The tool builds a valid XML sitemap, an HTML sitemap page, and a robots.txt snippet that already includes the Sitemap: line pointing to your sitemap.xml URL.
Copy, upload, and submit
Copy or download robots.txt to your site root and sitemap.xml alongside it, then submit the sitemap in Google Search Console and Bing Webmaster Tools.
Why the robots.txt sitemap line matters
Robots.txt lives at the root of your domain and tells crawlers which paths they may request. Adding a Sitemap: line gives search engines a direct pointer to your full URL list, so they can discover pages that internal links alone might miss. Unlike crawl rules, the Sitemap directive is independent of any User-agent block, so a single absolute line such as Sitemap: https://example.com/sitemap.xml is enough for Googlebot, Bingbot, and others.
A practical tip: keep the sitemap URL absolute and HTTPS, and make sure it is not blocked by a Disallow rule, or crawlers may skip it. This generator builds the robots.txt and the sitemap from the URL list you paste (or the links it extracts from pasted HTML); because browsers cannot fetch other domains due to CORS, it does not crawl your live site from a server, so you stay in control of exactly which URLs are included. You can still submit the sitemap directly in Google Search Console for indexing reports.
Frequently asked questions
Does this tool crawl my website to find URLs?
No. Browsers block cross-domain requests (CORS), so it cannot fetch other sites from a server. You paste your list of page URLs, or paste the HTML of a page and it extracts the links from that markup. It then builds the robots.txt and sitemap from exactly the URLs you provide.
What does the robots.txt sitemap line look like?
It is a single line, independent of any User-agent group, in the form Sitemap: https://example.com/sitemap.xml. Use the full absolute HTTPS URL. The generator inserts this line for you based on the sitemap location you specify, so you can copy it straight into your robots.txt file.
Where do I put robots.txt and sitemap.xml?
Place robots.txt at the root of your domain (https://example.com/robots.txt) so crawlers can find it. Upload sitemap.xml to a public URL too, commonly the root. Then the Sitemap line in robots.txt should point to that exact sitemap URL.
Can I still submit the sitemap to Search Console?
Yes. The robots.txt Sitemap line helps crawlers discover it automatically, but you can also submit sitemap.xml directly in Google Search Console and Bing Webmaster Tools to get indexing and coverage reports for the URLs you included.
Is the generated XML sitemap valid and free to use?
Yes. It follows the sitemaps.org XML protocol with loc, and optional lastmod, changefreq, and priority tags. The tool is completely free, requires no sign-up, and runs 100% in your browser, so your URL list is never uploaded to a server.