We read your robots.txt and show which of 12 AI and search crawlers it lets in, and what each one is for. Free, nothing to sign up for.
A small file at yourwebsite/robots.txt that tells crawlers which pages they may read. Most website builders write one for you, and some block AI crawlers without saying so.
It depends what each one does. Search crawlers like OAI-SearchBot, Claude-SearchBot and PerplexityBot read your site to answer customers' questions, so blocking them can keep you out of those answers. Training crawlers like GPTBot and ClaudeBot collect pages for future models; blocking those is a reasonable choice and doesn't stop the search crawlers.
No. Google says AI Overviews and AI Mode use the normal Search index, which Googlebot builds. Google-Extended only controls whether Google may use your pages for Gemini. To be in AI Mode, Googlebot must be allowed.
Not on its own. Being readable is step one. AI also needs clear facts about you in places it trusts: your Google listing, your website and pages like your free ListedOn Profile. Our free check shows whether Google's AI names you today.
We ask Google’s AI 20 questions your customers ask and show you who it names. Free, no card.