Indexability
Whether a search engine is allowed and able to add a page to its index.
The Definition
A page is indexable when it returns a 200 status, isn't blocked by a noindex directive, has no canonical or one that points at itself, and can be crawled (robots.txt controls crawling, noindex controls indexing). Even an indexable page may not be indexed if Google judges it duplicate or low value.
Why It Matters
A page that can't be indexed can't rank, however good it is. Indexability problems are often accidental: a staging noindex left on after launch, a canonical pointing at the wrong URL, or a robots.txt rule that blocks a whole folder.
Best Practices
Keep pages you want ranked at 200, self-canonical and free of noindex
Use noindex, not robots.txt, to keep a page out of the index
List only indexable URLs in your XML sitemap
Check the Search Console Page indexing report after every release
Mistakes to Avoid
- 1
Shipping a staging noindex or 'discourage search engines' setting to production
- 2
Blocking a page in robots.txt and expecting it to drop out of the index
- 3
Canonicals that point at redirected, noindexed or 404 URLs
- 4
Listing non-indexable URLs in sitemaps
Audit Checks
How Digispot AI identifies and fixes related issues
The page serves materially different content to search-engine crawlers than to real visitors (cloaking).
Impact: Google treats serving different content to crawlers as a manual-action offence — the page, or the whole site, can be removed from search results.
Compare the rendered HTML a crawler receives against what a browser shows. • Serve identical content to bots and users; use the same markup for both. • If the gap comes from JavaScript rendering, ensure the server-rendered HTML carries the primary content.
A meta robots tag is sitting in the page body instead of the <head>.
Impact: Crawlers only read robots directives in the <head> — one in the body is silently ignored, so your index/follow rule never applies.
Move the <meta name="robots"> tag inside the <head>. • Make sure no script injects it into the body after load. • Re-crawl to confirm the directive is now honored.
The meta robots tag uses the "none" directive, equivalent to noindex,nofollow.
Impact: A "none" directive removes the page from search results and stops link-following.
Remove the "none" directive unless the page is intentionally de-indexed.
The page tells Google not to index it (noindex) — likely by mistake.
Impact: A noindexed page can never appear in search results, so you lose all its potential traffic.
Confirm whether this page should be in search results. • If yes, remove the noindex directive. • Re-crawl and check it becomes indexable.
The page uses both noindex and nofollow together.
Impact: This combination can cut off link value and may not be what you intended.
Confirm you really want the page hidden and its links dropped. • Remove a directive if it was unintentional. • Keep the rules consistent with the page's purpose.
The meta robots tag has directives that contradict each other (e.g. index and noindex).
Impact: Conflicting rules make crawl/index behavior unpredictable — Google may pick the restrictive one.
Decide the intended rule for the page. • Keep one consistent set (e.g. index, follow). • Remove the contradicting directive.
Related Terms
Meta Robots Tag
An HTML meta tag that controls how search engines crawl and index a page.
Robots.txt
A text file that tells search engine crawlers which pages they can and cannot access.
Canonical Tag
An HTML element that tells search engines which URL is the preferred version of a page.
Crawl Budget
The number of pages search engines will crawl on your site within a given time period.