# robots.txt for https://devakey.com # # Search engines and AI assistants are welcome to crawl the public site. # AI crawlers are named explicitly so the policy is unambiguous; they share # the same rules as every other crawler (a bot with its own separate group # would ignore the "*" rules below). # # AI-readable summary of the site: https://devakey.com/llms.txt # Full text for AI systems: https://devakey.com/llms-full.txt # AI usage policy: https://devakey.com/ai.txt User-agent: * # Search engines User-agent: Googlebot User-agent: Bingbot User-agent: Applebot User-agent: DuckDuckBot User-agent: YandexBot # AI search and assistants User-agent: OAI-SearchBot User-agent: ChatGPT-User User-agent: GPTBot User-agent: Claude-SearchBot User-agent: Claude-User User-agent: ClaudeBot User-agent: PerplexityBot User-agent: Perplexity-User User-agent: Google-Extended User-agent: GoogleOther User-agent: Applebot-Extended User-agent: Amazonbot User-agent: Meta-ExternalAgent User-agent: DuckAssistBot User-agent: MistralAI-User User-agent: CCBot Allow: / # Admin dashboard - private Disallow: /admin_login Disallow: /admin_logout Disallow: /check_login Disallow: /forgot-password Disallow: /reset-password Disallow: /dashboard Disallow: /contact_list Disallow: /enquiry_list Disallow: /service_quate Disallow: /newsletter$ Disallow: /visitors Disallow: /add_blog Disallow: /save_blog Disallow: /edit-blog Disallow: /update_blog Disallow: /delete_blog Disallow: /view_blog Disallow: /delete_contact Disallow: /admin_assets/ # Form / AJAX endpoints - not pages Disallow: /submit_ Disallow: /Welcome/get_ipdata Disallow: /welcome/ # Internal tooling and debug URLs Disallow: /tools/ Disallow: /*?gadebug= # CSS, JS, fonts and images stay crawlable so Google can render pages. Allow: /assets/ Sitemap: https://devakey.com/sitemap.xml