robots.txt analyzer

A single robots.txt line can hide your whole site from search engines, while an unsupported directive has no effect at all. The tool splits the file into groups, finds faulty lines, CSS/JS blocks and rules that advertise secret paths, and evaluates a path for the chosen bot with the longest-match rule. It also stresses that robots.txt is not a security measure.

How to use it

1. Paste the robots.txt content; optionally choose a bot and a path to test.
2. Press the run button. Results are listed separately as definite problems, possible issues, unverifiable and passed; each finding shows what was found, what it means, the fix and the limits of the check.
3. Apply the suggested fix and run the tool again to confirm. Use “Copy result” to add the report to a support ticket.

Example

If a “Disallow: /” line left from development is still on the live site, the tool reports that the whole site is blocked as a definite problem. The “Example” button in the form fills in a scenario like this and shows the result immediately.

Scope and limits

The analysis runs in your browser on the pasted text; your site is not queried. Pattern matching is done without regular expressions.

Your inputs are not added to the address bar or passed to analytics, and result pages are not indexed by search engines.

Related guides

• Core Web Vitals guide
• WordPress launch checklist
• Web Hosting plans

More tools

• Redirect rule builder
• Sitemap checker
• Error log interpreter
• All tools →