# How the major AI discovery channels differ

> Search crawlers, user-requested visits, connected tools, and merchant feeds are different routes into an assistant.

Canonical: https://byagents.dev/humans/ai-discovery-platforms
Published: 2026-09-13
Reviewed: 2026-09-13
Publisher: ByAgents
Type: Guide
Level: Mixed
Outcome: Choose the right access controls and distribution route for the platforms your customers use.

Start with the route, not the assistant's logo. One application can search an index, fetch a supplied URL, invoke an installed tool, or navigate a website. Improving one route does not prove the others work. This comparison summarizes documented mechanisms, not a ranking of which assistant sends the most traffic.

## Search and training are separate decisions

OpenAI distinguishes OAI-SearchBot for search from GPTBot for model training. ChatGPT-User represents certain user-initiated visits; robots rules may not apply to those requests. A training opt-out is not inherently a search opt-out. Verify the current [crawler documentation](https://developers.openai.com/api/docs/bots) before editing access rules.

Anthropic separately identifies ClaudeBot, Claude-SearchBot, and Claude-User, and says its bots honor robots.txt. [Anthropic's documentation](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler).

Perplexity distinguishes PerplexityBot from Perplexity-User; its documentation says the latter generally ignores robots.txt for user-requested fetches. Do not apply another vendor's policy by analogy. [Perplexity's documentation](https://docs.perplexity.ai/docs/resources/perplexity-crawlers).

## A channel comparison you can act on

| Route | What the business can prepare | What this does not prove |
| --- | --- | --- |
| ChatGPT search | Permitted search access, clear public pages, current product facts | Selection, endorsement, or a purchase |
| Google AI Search experiences | Search eligibility, useful indexed content, appropriate property controls | Inclusion for every query or coverage in every Gemini feature |
| Claude or Perplexity web research | Retrievable source pages and vendor-specific access rules | Every conversation using the web |
| Bing and Copilot citations | Search-ready sources and inspection of available webmaster reports | Ranking or commercial value of each citation |
| Connected tools | A compatible integration, installation route, and usable workflow | Automatic installation across all hosts |
| Merchant catalogs | Accurate eligible product records and account onboarding | Enabled transactional checkout |

Google says AI Search eligibility does not require a special AI file or special schema. Its newer Search Console controls should be reviewed alongside these baseline requirements. [Google Search guidance](https://developers.google.com/search/docs/appearance/ai-features). Gemini is also a product family: do not treat a Search setting as a universal control over every Gemini experience.

## Verify the traffic before changing security rules

A request claiming a crawler name is not proof of identity. Use the provider's verification method where available and log the verification basis. Scope an exception to the intended public content; preserve authentication, request limits, and protections around private routes. Do not broadly disable the firewall to improve a visibility score.

A local test that sends a crawler user-agent string can reveal how your own rules respond to that string. It cannot reproduce the provider's IP, infrastructure, index, location, or eventual recommendation. Keep that limitation in the audit record.

## Pick one route to test first

Ask how the intended customer currently works. A developer using a connected coding environment may benefit from a tool integration and example repository. A shopper comparing products may encounter search and catalog data. A local business may need accurate service-area and booking information before a new API.

Record a concrete hypothesis: “This task can encounter our public comparison guide through search.” Record the source that was actually retrieved, whether the brand was named, and the next action. If the run never searches, its failure tells you little about indexing. If it finds the page but cannot obtain a quote, the problem has moved downstream.

## Use platform reports for the claims they support

Microsoft's AI Performance documentation describes citations and grounding-query insights, not an authority or ranking score. Google's dedicated Search report concerns impressions. Neither replaces first-party inquiry, activation, or order records. Combine the streams in a report while retaining their different units. See [measurement](https://byagents.dev/humans/measuring-agent-traffic) and the [crawl audit recipe](https://byagents.dev/agents/crawl-audit-recipe).

## Sources

- [OpenAI: Crawler roles and verification](https://developers.openai.com/api/docs/bots)
- [Anthropic: Crawlers and site-owner controls](https://support.claude.com/en/articles/8896518-does-anthropic-crawl-data-from-the-web-and-how-can-site-owners-block-the-crawler)
- [Perplexity: Crawlers and user requests](https://docs.perplexity.ai/docs/resources/perplexity-crawlers)
- [Google: AI features and your website](https://developers.google.com/search/docs/appearance/ai-features)
- [Microsoft: AI Performance in Bing Webmaster Tools](https://blogs.bing.com/webmaster/February-2026/Introducing-AI-Performance-in-Bing-Webmaster-Tools-Public-Preview)

## Related reading

- [Can an agent actually read your website?](https://byagents.dev/humans/crawl-access)
- [Recipe: audit public crawl and retrieval access](https://byagents.dev/agents/crawl-audit-recipe)
- [Agent distribution: what matters now](https://byagents.dev/humans/agent-distribution-now)