Allow crawlers except admin
Sample
Disallow /admin/ and /api/private/ in the UI checkboxes.
What you get
Download a robots.txt tuned to your paths and crawl-delay preference.
Robots.txt Generator for E-commerce Sites
A misconfigured robots.txt can tank your e-commerce SEO — blocking product pages or letting bots crawl cart and checkout URLs endlessly. This generator creates platform-aware robots.txt files with sensible defaults for Shopify, WooCommerce, Magento, and custom stores.
Robots.txt Generator for E-commerce Sites
How to Use This Tool
- Select your e-commerce platform or choose custom.
- Toggle rules for common paths (cart, checkout, admin, search filters).
- Add custom allow/disallow rules as needed.
- Copy or download the generated robots.txt file.
Features
- Platform presets for Shopify, WooCommerce, and Magento
- Blocks crawl-wasting paths (faceted navigation, internal search, cart)
- Sitemap URL inclusion
- Crawl-delay configuration per bot
- Syntax validation and linting before download
Example
Frequently Asked Questions
Related Tools
XML Sitemap Generator for Next.js Apps
Generate XML sitemaps and sitemap.ts code for Next.js applications.
JSON to Shopify Product Schema Converter
Convert JSON product data to Shopify-compatible schema format.
CSV to Airtable Schema Mapper
Map CSV data to Airtable fields with automatic type detection.
Markdown to Notion-Ready Table Converter
Convert Markdown tables into Notion-ready format instantly.
SQL Query to Python DataFrame Script
Generate Python pandas scripts from SQL queries.
CSV to SQL INSERT Converter
Generate SQL CREATE TABLE and INSERT statements from CSV data.
Regex Email Extractor
Extract and deduplicate email addresses from any text block using regex.
XML to JSON Converter
Convert XML documents to clean, structured JSON with one click.
HTML to Markdown Converter
Convert HTML content to clean, readable Markdown for docs and CMS.