One robots snapshot · five separate decisions

AI crawler policy checker

Test a public URL across search indexing, AI search, user-triggered retrieval, model training, and open-corpus archiving. Every result keeps the operator's purpose and control limitation attached.

The path matters. Rules for /private/ can differ from rules for /.