# https://www.robotstxt.org/robotstxt.html User-agent: * Allow: / Disallow: /api/ # Search and retrieval bots used to surface pages in AI search products. User-agent: OAI-SearchBot Allow: / Disallow: /api/ User-agent: ChatGPT-User Allow: / Disallow: /api/ User-agent: Claude-SearchBot Allow: / Disallow: /api/ User-agent: Claude-User Allow: / Disallow: /api/ User-agent: PerplexityBot Allow: / Disallow: /api/ User-agent: Google-Extended Allow: / Disallow: /api/ User-agent: CCBot Allow: / Disallow: /api/ Sitemap: https://getavora.ai/sitemap.xml