User-agent: * Allow: / # Block all parameter-based URLs (filters, sort, pagination) Disallow: /*? # AI assistants. Same policy as everyone else — these groups exist so the policy can be tuned per # agent without touching the wildcard group, and so it is obvious which agents we considered. # # ⚠️ robots.txt groups do NOT inherit. An agent named below obeys ONLY its own group, so every # group must repeat `Disallow: /*?` — dropping that line silently opens parameterised URLs to # exactly the agents we most care about. See docs/AI_TRAFFIC.md. # User-triggered fetchers — dispatched when a human asks an assistant about a page. Blocking these # makes the assistant tell the user it could not read our site. User-agent: ChatGPT-User Allow: / Disallow: /*? User-agent: Claude-User Allow: / Disallow: /*? User-agent: Perplexity-User Allow: / Disallow: /*? User-agent: Google-Agent Allow: / Disallow: /*? # Search indexers — build the citation index AI answers link from. Blocking these removes us from # AI answer citations while leaving model training unaffected. User-agent: OAI-SearchBot Allow: / Disallow: /*? User-agent: Claude-SearchBot Allow: / Disallow: /*? User-agent: PerplexityBot Allow: / Disallow: /*? # Training crawlers (GPTBot, ClaudeBot, Bytespider, Meta-ExternalAgent) are intentionally NOT # listed: they fall through to the wildcard group above and stay allowed. Blocking them costs no # live AI visibility, so it is a content-licensing decision, not a technical one — take it # deliberately rather than by omission. Sitemap: https://movingdoors.com/sitemap.xml