Check Access for AI Assistants

Before AI assistants can cite your pages, they must reach and read them. RankScan checks this on every crawlCrawling is the process by which a search engine bot fetches a URL and downloads its content. It is the first step before a page can be indexed.Open glossary entry → of your website. On purpose it shows no score, only checks with a proven effect.

The five checks #

Open “Website Health” in the menu and the “Access for AI assistants” card. Each check is “met” or “not met”:

• AI botsAI crawlers are automated bots that fetch web pages for AI providers in order to find content, make it available for search functions, retrieve it for answers, or, depending on the provider, use it for model training.Open glossary entry → may read the website robots.txtThe robots.txt is a publicly accessible text file in a site's root directory. It tells crawlers which URLs they may fetch and therefore controls crawling.Open glossary entry → blocks no search or answer bots.
• Pages are indexableIndexing is the step in which a search engine processes crawled content and decides whether a URL is added to the search index. Only indexed pages can appear in search results.Open glossary entry → Only a small share of pages is set to noindexnoindex is a robots directive telling search engines that a page should not appear in search results. The page must remain crawlable for it to work.Open glossary entry →.
• SitemapAn XML sitemap is a machine-readable file listing a website's important URLs. It helps search engines discover content more efficiently but does not replace internal linking or guarantee indexing.Open glossary entry → present There is an XML sitemapExtensible Markup Language (XML) is a flexible text format for storing and exchanging data in a structured way. In the SEO context, the XML sitemap is best known.Open glossary entry →, ideally with a current modification datelastmod is an element in the XML sitemap indicating when a URL's main content was last significantly changed. It is only useful when maintained reliably.Open glossary entry →.
• Content readable without JavaScript Important content is directly in the HTMLHyperText Markup Language (HTML) is the markup language for web pages. It defines the structure and content of a page, such as headings, paragraphs, links and images.Open glossary entry →.
• Organization schemaOrganization schema describes a company or organisation as an entity, with details such as name, URL, logo, address, contact and official profiles.Open glossary entry → The home page says in structured dataSchema markup is code that describes a page's content in a structured way, so search engines understand it not just as text but as clearly classified information.Open glossary entry → who is behind the website.

How to fix blockers #

Fix the checks that are “not met” first. Blocked bots and non-indexable key pages weigh most.

Then open the structure and AI findings in Website Health. Each finding leads to the matching measure.

If the website is poorly prepared for AI overall, the task “Low AI readinessThe AI Readiness Score is a maturity level that shows how well a website is prepared for AI search technically, structurally, in terms of content and trust. It is a scoring model, not an official industry standard.Open glossary entry →” appears under “Measures”. It names the weaker area and the steps.

RankScan does not count an llms.txtllms.txt is a voluntary Markdown file in a website's root directory that gives language models a compact overview of important content. It is a proposal or de facto format, not a binding official web standard.Open glossary entry →. No AI system is proven to read it.

Read Website Health and Fix Findings explains all other technical findings.