Technical SEO

Indexability

Whether a search engine is allowed and able to add a page to its index.

The Definition

A page is indexable when it returns a 200 status, isn't blocked by a noindex directive, has no canonical or one that points at itself, and can be crawled (robots.txt controls crawling, noindex controls indexing). Even an indexable page may not be indexed if Google judges it duplicate or low value.

Why It Matters

A page that can't be indexed can't rank, however good it is. Indexability problems are often accidental: a staging noindex left on after launch, a canonical pointing at the wrong URL, or a robots.txt rule that blocks a whole folder.

Best Practices

  • Keep pages you want ranked at 200, self-canonical and free of noindex

  • Use noindex, not robots.txt, to keep a page out of the index

  • List only indexable URLs in your XML sitemap

  • Check the Search Console Page indexing report after every release

Mistakes to Avoid

  • 1

    Shipping a staging noindex or 'discourage search engines' setting to production

  • 2

    Blocking a page in robots.txt and expecting it to drop out of the index

  • 3

    Canonicals that point at redirected, noindexed or 404 URLs

  • 4

    Listing non-indexable URLs in sitemaps

Audit Checks

How Digispot AI identifies and fixes related issues

View all indexability solutions
high

The page serves materially different content to search-engine crawlers than to real visitors (cloaking).

Impact: Google treats serving different content to crawlers as a manual-action offence — the page, or the whole site, can be removed from search results.

Compare the rendered HTML a crawler receives against what a browser shows. • Serve identical content to bots and users; use the same markup for both. • If the gap comes from JavaScript rendering, ensure the server-rendered HTML carries the primary content.

high

A meta robots tag is sitting in the page body instead of the <head>.

Impact: Crawlers only read robots directives in the <head> — one in the body is silently ignored, so your index/follow rule never applies.

Move the <meta name="robots"> tag inside the <head>. • Make sure no script injects it into the body after load. • Re-crawl to confirm the directive is now honored.

high

The meta robots tag uses the "none" directive, equivalent to noindex,nofollow.

Impact: A "none" directive removes the page from search results and stops link-following.

Remove the "none" directive unless the page is intentionally de-indexed.

high

The page tells Google not to index it (noindex) — likely by mistake.

Impact: A noindexed page can never appear in search results, so you lose all its potential traffic.

Confirm whether this page should be in search results. • If yes, remove the noindex directive. • Re-crawl and check it becomes indexable.

medium

The page uses both noindex and nofollow together.

Impact: This combination can cut off link value and may not be what you intended.

Confirm you really want the page hidden and its links dropped. • Remove a directive if it was unintentional. • Keep the rules consistent with the page's purpose.

medium

The meta robots tag has directives that contradict each other (e.g. index and noindex).

Impact: Conflicting rules make crawl/index behavior unpredictable — Google may pick the restrictive one.

Decide the intended rule for the page. • Keep one consistent set (e.g. index, follow). • Remove the contradicting directive.