XML Sitemap
A file that lists all important URLs on your site to help search engines discover and crawl them.
The Definition
An XML sitemap is a structured file (usually at /sitemap.xml) that lists the URLs of your website along with optional metadata like last modification date, change frequency, and priority. It helps search engines discover pages that might be difficult to find through normal crawling, especially on large or deep websites.
Why It Matters
Sitemaps ensure search engines find all your important pages, especially new content, deep pages, or pages with few internal links. For sites with thousands of pages, a well-maintained sitemap is essential for complete indexation.
Best Practices
Include only canonical, indexable URLs that return 200 status codes — no redirects, no noindex pages
Keep individual sitemaps under 50,000 URLs or 50MB uncompressed — use sitemap index files for larger sites
Update lastmod dates only when page content actually changes, not on every build or deployment
Submit your sitemap to Google Search Console and Bing Webmaster Tools for faster discovery
Use separate sitemaps for different content types (pages, images, videos, news) for better organization
Regenerate sitemaps automatically as part of your build or deployment process
Mistakes to Avoid
- 1
Including URLs blocked by robots.txt in the sitemap, creating contradictory signals
- 2
Never updating lastmod dates, causing search engines to ignore freshness signals entirely
- 3
Including non-canonical URLs or redirect URLs that waste crawl budget
- 4
Having a sitemap reference in robots.txt that points to a broken or missing sitemap file
Audit Checks
How Digispot AI identifies and fixes related issues
The sitemap is missing or has an incorrect XML namespace declaration.
Impact: Without the right namespace, crawlers may not recognize the file as a sitemap.
Add the correct xmlns to the root <urlset> or <sitemapindex> element. • Validate against the sitemap schema. • Re-submit the file.
Your sitemap file is malformed and doesn't follow the required XML format.
Impact: A broken sitemap can't be read, so search engines ignore it and may miss your pages.
Fix the XML formatting errors. • Validate it against the sitemap schema. • Re-submit it in Google Search Console.
The sitemap is not served as valid XML.
Impact: If the file isn't recognized as XML, search engines can't parse it.
Ensure the sitemap is well-formed XML. • Serve it with the correct content type (application/xml). • Re-validate the file.
We found pages on your site by following links, but those pages aren't listed in sitemap.xml.
Impact: Google can still find these pages through links, but a complete sitemap speeds up discovery and makes Search Console coverage reports accurate. Pages missing from the sitemap are also more likely to be missed if internal linking changes.
Open the list of affected URLs from the issue context. • Add the ones you want indexed to your XML sitemap. • Exclude any that are intentionally non-indexable (admin, search, filter pages). • Re-submit the updated sitemap in Google Search Console.
Your site has no XML sitemap — the file that lists your pages for search engines.
Impact: Without a sitemap, search engines may miss or be slow to find some of your pages.
Generate an XML sitemap of your important pages. • Publish it (e.g. at /sitemap.xml) and link it in robots.txt. • Submit it in Google Search Console.
The sitemap uses an encoding other than UTF-8.
Impact: Non-UTF-8 encoding can garble special characters and break parsing.
Save and serve the sitemap as UTF-8. • Confirm special characters display correctly. • Re-validate the file.