A sitemap is a machine-readable list of the pages on your site that you want search engines to find, with the date each page last changed. An AI-friendly sitemap lists every page worth finding, leaves out every page that is not, and keeps its change dates honest. Build it from the pages you want cited, not from every address your site can serve. A sitemap does not guarantee a page gets crawled or indexed. Google says submitting a sitemap is a hint, not a guarantee, and this guide does not claim otherwise.
A sitemap is a developer-led job with one marketer check. The developer owns the file, the format, and the submission. The marketer owns the decision about which pages belong in it, because that decision is about what the brand wants found. If you are both people, agree the page list first, then build the file, in that order, before you submit anything.
Where it goes wrong
Sitemaps fail quietly. The file loads, the submission says success, and the pages that matter are still missing or stale.
- The sitemap lists everything, including the pages that should never be cited. Cart, checkout, account, search results, and filtered faceted pages all end up in the file, and crawlers spend their attention on addresses no shopper should land on.
- The sitemap lists pages that no longer exist. A product removed from the catalog still sits in the file, and a crawler that follows it finds a dead end. Dead links in a sitemap teach the crawler to trust the file less.
- The change dates are fake. Every page carries the same new date, stamped when the file was generated, not when the page changed. Google says it uses the lastmod value only if it is consistently and verifiably accurate, and that a significant update means a change to the main content, structured data, or links on the page. A file that stamps every page with the same date teaches the crawler to ignore the signal on every page.
- The file is too big. A single sitemap holds up to 50,000 URLs and no more than 50MB uncompressed. A catalog past that limit in one file breaks the format, and the pages past the limit never get listed.
- The sitemap is never submitted. The file sits at a sensible address, but no search engine was told about it, and no robots.txt line points to it. Discovery is left to chance.
- The sitemap and the robots file disagree. The sitemap lists a page that robots.txt blocks, or robots.txt allows a section the sitemap never lists. A crawler given two answers follows the block, and the sitemap entry is wasted.
- The sitemap lists the wrong address. A product appears under an old address, a preview domain, or a non-canonical address, while the live page sits somewhere else. An AI system given the wrong address cites a page shoppers cannot use.
The steps
Step 1. Decide which pages belong in it. The marketer lists the pages worth finding: product pages, category pages, brand pages, guides, the about page, and the policy pages shoppers ask about. Leave out the cart, the checkout, the account pages, the site search results, and filtered faceted addresses that only rearrange a category. A sitemap says here is what matters. If it lists everything, it says nothing. Check the list against the live site. Every address in the file should load, show the right page, and match the canonical address for that page.
Step 2. Build the file in the right format. The developer builds the sitemap as an XML file following the sitemaps protocol, with one url entry per page, a loc child carrying the full absolute address, and a lastmod child carrying the date the page last changed in W3C Datetime format, such as YYYY-MM-DD. The file itself must be UTF-8 encoded, and all values must be entity-escaped, so an ampersand in an address is written as its escape code, not as a bare character. Do not include the priority or changefreq tags. Google says it does not use them. Leave them out rather than filling them with guesses. Keep the file to the pages from a single host, since the protocol requires all addresses in a sitemap to share one host.
Step 3. Split the file when the catalog grows. A single sitemap holds up to 50,000 URLs and no more than 50MB uncompressed, so a large catalog needs several sitemaps grouped under a sitemap index file. The developer splits the catalog by page type, such as one sitemap for products, one for categories, and one for guides, and lists each one in the index with its own lastmod value set to the most recent change inside it. A sitemap index file may list up to 50,000 sitemaps and must also stay under 50MB. Submit the index, not the individual files, and each child sitemap is discovered from it. If the catalog is small, one file is enough, and an index adds nothing.
Step 4. Keep the change dates honest. The developer sets the lastmod value from the real date the page last changed, with time and timezone where the system stores them, and changes it only when the main content, structured data, or links on the page change. Do not bump the date for a cosmetic edit, a footer change, or a new copyright line. Do not stamp every page with the generation date. A trustworthy lastmod is the highest-value field in the file, because it tells a crawler which pages are worth re-fetching. A file that lies about it gets ignored.
Step 5. Submit it and point to it. The developer submits the sitemap in Google Search Console using the Sitemaps report, and adds a sitemap line to the robots.txt file so crawlers can find it there. Check the Sitemaps report after submission and fix any errors it shows, such as addresses that fail to load or a file that fails to parse. Keep the sitemap at a stable address. Moving it breaks the submission and the robots.txt line at the same time. If the platform generates the sitemap for you, confirm the generated file lists the pages from step one before you submit it, and submit the address the platform serves.
Step 6. Recheck it when the catalog changes. The marketer and the developer recheck the sitemap when products are added, removed, or renamed. A removed product should leave the file the same day it leaves the catalog. A renamed product should appear under its new address with its new lastmod value. A new guide should appear in the file when it is published, not weeks later. A sitemap that lags the catalog teaches the model last season's store. Build the recheck into the same routine that publishes a product, so the file stays current without a separate project.
The machine-readable piece
This is a sitemap for a small store, following the sitemaps protocol. Replace the placeholders with your own addresses and the real dates each page last changed. The loc value must be the full absolute address, and the lastmod value uses a placeholder shape: replace YYYY-MM-DD with the real date.
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<url>
<loc>https://[yourdomain.com]/</loc>
<lastmod>YYYY-MM-DD</lastmod>
</url>
<url>
<loc>https://[yourdomain.com]/collections/dog-food</loc>
<lastmod>YYYY-MM-DD</lastmod>
</url>
<url>
<loc>https://[yourdomain.com]/products/example-dog-food</loc>
<lastmod>YYYY-MM-DD</lastmod>
</url>
<url>
<loc>https://[yourdomain.com]/pages/feeding-guide</loc>
<lastmod>YYYY-MM-DD</lastmod>
</url>
</urlset>
For a large catalog, split the file by page type and group the parts under a sitemap index. This index lists two child sitemaps, each with its own lastmod value set to the most recent change inside it. The format comes from the sitemaps protocol, and the limits come from the same source.
<?xml version="1.0" encoding="UTF-8"?>
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>https://[yourdomain.com]/sitemap-products.xml</loc>
<lastmod>YYYY-MM-DD</lastmod>
</sitemap>
<sitemap>
<loc>https://[yourdomain.com]/sitemap-guides.xml</loc>
<lastmod>YYYY-MM-DD</lastmod>
</sitemap>
</sitemapindex>
Check the file before you submit it. Open the sitemap address in a browser and confirm it loads as XML. Open a few listed addresses and confirm each one loads the right page. Submit the index in Search Console, and confirm the Sitemaps report shows it without errors. If the report shows an error, fix the file and resubmit it. Do not submit the same unchanged file repeatedly.
What it looks like when it works
On the page, nothing looks different to a human reader. A sitemap is not a page shoppers see. The change is under the surface: every product page, category page, and guide has one canonical address, the sitemap lists exactly those addresses with honest change dates, and the robots file points at the sitemap. A crawler that reads the file finds the pages worth finding, skips the pages that are not in it, and re-fetches the pages whose change dates moved. An assistant asked about the range finds the current catalog, not a mix of live pages and dead ends.
Say you run a pet supplies store with a large catalog, selling dog food, cat food, treats, toys, and health products across hundreds of products. The marketer lists the pages worth finding and cuts the cart, checkout, account, search results, and filtered faceted addresses from the list. The developer builds the sitemap by page type, with one file for products, one for categories, and one for guides, grouped under an index, because the catalog passes the size where one file is enough. Each entry carries the real lastmod date from the page itself. The developer submits the index in Search Console and adds the sitemap line to the robots file. When a product is discontinued, the marketer removes it from the catalog and the developer removes it from the sitemap the same day. A shopper asks an assistant for a grain-free dog food for a sensitive stomach under a set budget. The assistant can find the current category page, the current product pages, and the feeding guide, and it cites the store instead of a stale listing.
The fundamentals
These mechanics are shared across every engine and covered in full in their own guides:
- •The llms.txt format and a complete sample, see our guide to writing a good llms.txt file
- •The markup AI systems read to verify facts, see our product schema guide
- •The complete crawler rule reference, see our robots.txt guide for AI search
What this guide does not cover
This guide covers building a sitemap that helps AI systems and search engines find your pages. It does not cover the full technical audit that checks every crawler and every page, the transaction layer that lets an agent complete a purchase, or the per-engine playbooks for getting each assistant to find you. It does not cover faceted navigation design, image or video sitemaps, or international sitemaps for multi-language stores. Those are separate jobs, and this guide is the map, not the whole trip.
Check your own store now
Run the free scan. One click, no signup, and it shows whether AI crawlers can read your store today: aizii.ai/aeo-scan
Run the free scanSources
- Google Search Central, Build and submit a sitemap (specific): https://developers.google.com/search/docs/crawling-indexing/sitemaps/build-sitemap
- Sitemaps.org, Sitemaps XML format (specific): https://www.sitemaps.org/protocol.html
- Google Search Central, Optimizing your website for generative AI features on Google Search (specific): https://developers.google.com/search/docs/fundamentals/ai-optimization-guide
- Google Search Central, Product structured data (specific): https://developers.google.com/search/docs/appearance/structured-data/product
- Schema.org, Product type (specific): https://schema.org/Product
Note: links marked specific point to the page that supports the claim in this guide. Links marked general docs point to the vendor's general docs section for that topic, not to a page that proves every step. Check the vendor docs for the current path before relying on a menu name or click path in this guide.
