How crawlability, indexability, eligibility, and publisher controls determine whether a page can enter AI search at all.
Discoverability is necessary, not sufficient
Discoverability is the upstream gateway to retrieval-based visibility. A source that cannot be found cannot enter live retrieval, and a source that is never retrieved cannot participate in downstream citation or absorption through that path.
Every transition creates another filter: crawlable pages may be indexed; indexed pages may be retrieved; retrieved pages may enter context; in-context pages may be cited; cited pages may materially influence the answer.
Crawlability asks whether the system can fetch the page
Robots restrictions, authentication walls, inaccessible rendering, broken internal links, blocked user agents, server errors, and redirect loops can all prevent a crawler from accessing important content.
Google treats public, crawlable content as a foundation for generative Search visibility. OpenAI similarly advises publishers to allow OAI-SearchBot when they want pages to be discoverable, summarized, and linked in ChatGPT Search.
Indexability asks whether the page can enter searchable space
A crawlable page is not automatically indexed. Indexability describes whether a system is allowed and willing to store a searchable representation. Technical compliance creates an opportunity; it does not guarantee crawling, indexing, or serving.
Eligibility adds a policy and quality gate
An indexed page may still be ineligible for a particular search surface. Google states that supporting pages in its generative Search features must be indexed and eligible to appear with a snippet.
Technical access creates the possibility of competition. It does not create selection.
Publisher controls also require precision: OpenAI distinguishes blocking its search crawler from using a noindex signal when a publisher does not want a discovered URL surfaced.
Questions about this topic
What is the difference between crawlability and indexability?+
Crawlability asks whether a system can fetch a page. Indexability asks whether it can store and use a searchable representation of that page.
Does meeting technical requirements guarantee indexing?+
No. Technical requirements make crawling and indexing possible, but platforms do not guarantee that every eligible page will be indexed or served.
What does eligibility mean in generative search?+
It means a page satisfies the policy, technical, and quality conditions required to participate in a particular search surface.
Is blocking a crawler the same as using noindex?+
No. Blocking prevents access, while noindex is a directive the crawler must be able to read. Publisher guidance should be followed for the specific platform and outcome.
References
Sources are listed in APA 7 style. Preprints are identified as such and should not be treated as peer-reviewed findings unless separately published.
- Google Search Central. (2026). Optimizing your website for generative AI features on Google Search. https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- OpenAI. (2026). Publishers and developers—FAQ. https://help.openai.com/en/articles/12627856-publishers-and-developers-faq
- Martinez, O. (2026). Optimizing visibility in generative engines: A critical survey of generative engine optimization (2023–2026) [Preprint]. arXiv. https://doi.org/10.48550/arXiv.2607.14035