Aizii Learn · AI Search
← Back to Learn

How to get Grok to recommend your brand

Grok recommends brands it can read live on the open web. X discussion may inform real-time and sentiment questions, but it does not drive buying recommendations. xAI publishes no official crawler documentation, so there is no documented token, no IP ranges, and no webmaster console. And when Grok answers a question, its fetches often arrive as ordinary browsers that never request robots.txt. The playbook for Grok is therefore unlike the others: the allowlist is best-effort, clean readable HTML is the real lever, and posting on X does not get you recommended to buyers.

One warning first: there is no way to submit your brand to Grok. Anyone selling guaranteed placement is selling something they do not control.

Why Grok skips your brand

Grok's failure modes are its own:

  1. Answer-time fetches bypass robots.txt. Independent tests have watched Grok read robots.txt-blocked pages and arrive as spoofed browsers that never request the file. The declared rule is necessary, but it reaches only the crawl lane.
  2. JavaScript-only content may not be read. Grok's rendering behavior is undocumented, so assume it does not execute JavaScript unless xAI says otherwise. If your price and specs appear only after scripts run, there is nothing to cite.
  3. The "just post on X" consensus is wrong. X shapes real-time and sentiment questions, not buyer recommendations. A buying recommendation is built from what Grok can read on your site, not from your posts.
  4. You cannot verify inclusion through any official channel. No crawler docs, no IP list, no webmaster tool. The only check is a honeypot canary URL, and even a hit cannot be attributed to Grok with confidence, because the traffic arrives as ordinary browsers.

Step 1: Allow GrokBot in robots.txt, and understand what it does not do

First, check your current robots.txt. Open [yourdomain.com]/robots.txt in your browser, replacing yourdomain.com with your actual domain. If nothing loads, you do not have one yet. Create it: a plain text file named robots.txt at the root of your site.

Then add this allow rule near the top, before any broad disallow rules. GrokBot is a real crawler: it shows up in server logs in the wild, so this rule catches genuine Grok crawl traffic. One caveat: xAI publishes no official crawler documentation, the way OpenAI and Google do, so this is the observed name rather than a documented one. Add it anyway. It costs nothing.

User-agent: GrokBot
Allow: /

Reload the file and confirm the rule is there.

Now the part most guides get wrong. The robots.txt rule above is necessary, but for Grok it is not sufficient. When Grok answers a user's question, its fetch traffic often arrives disguised as an ordinary browser, sometimes through proxy networks, and never requests robots.txt at all. A robots.txt rule cannot reach traffic that never reads the file.

Independent tests have reported Grok reading robots.txt-blocked pages while never requesting robots.txt, with fetches arriving as ordinary browsers from proxy addresses. Those tests are not linked here, so treat them as reported behavior, not as verified fact. The practical rule stands either way: the pages themselves have to be legible as clean HTML, because Grok reads whatever it fetches however it arrives.

So do Step 1, but do not stop there: the pages themselves have to be legible as clean HTML (Step 2), because Grok reads whatever it fetches however it arrives.

One more check most guides skip: robots.txt is what you declare, your edge is what enforces. Cloudflare Bot Fight Mode, AWS WAF bot rules, and similar protections often block AI crawlers at the edge even when robots.txt allows them. If you run Cloudflare, check Security > Bots and make sure AI crawlers are not being challenged or blocked. You can test this by fetching your site as each crawler and comparing what comes back. A mismatch between what robots.txt says and what the edge does is the an urgent finding to check.

Check for the observed token in your server logs:

grep -i "grokbot" /var/log/nginx/access.log

Do not treat a clean result as proof Grok never visits: some of its retrieval traffic arrives with ordinary browser user agents, indistinguishable from human visitors. The log check confirms the crawler lane is open, not that every Grok visit is counted.

While you're in there: add the other major AI crawlers too. You are already editing the file.

User-agent: GPTBot
Allow: /

User-agent: OAI-SearchBot
Allow: /

User-agent: ClaudeBot
Allow: /

User-agent: Claude-SearchBot
Allow: /

User-agent: PerplexityBot
Allow: /

The complete reference for every AI crawler lives in our robots.txt guide for AI search.

Step 2: Make your pages legible as plain HTML

This is the real lever for Grok and the step the generic checklists skip. View source on a product page: if the price, description, and specs are in the HTML, you are fine. If they only appear after JavaScript runs, Grok's answer-time fetch reads an empty page, and robots.txt cannot help because the fetch never consulted it.

  • Price, description, and specs as text in the HTML, never trapped behind scripts or inside images.
  • Stable product URLs. Frequently changing URLs fragment whatever index presence you build.
  • One clear H1 per page, with headings that match the questions buyers ask.

You do not need to rebuild the site. Pre-rendered HTML is an option where your platform supports it without changing what shoppers see. This matters doubly for Grok's Deep Research mode: published logs have reported Grok arriving as an ordinary browser user agent, never touching robots.txt. The page Grok lands on has to carry the facts in the HTML itself, because that page is what gets read.

Step 3: Build a real X presence

This is the step every other Grok guide gets wrong. The consensus advice is "post on X and Grok will recommend you." Posting on X does not make Grok recommend your product to a buyer comparing options. That decision is made from what Grok can read on your site.

That does not mean X is worthless. It means X may help with questions about: real-time topics (what happened today in your category), sentiment-shaped questions (what do people think of your brand right now), and brand-specific questions where fresh mentions are the evidence. So do this:

  • Keep a real, active X account for your brand. Post about your category and your products, not just promotions.
  • Earn mentions from real accounts in your space. Replies, quotes, and genuine discussion carry more weight than your own posts.
  • Correct mistakes about your brand on X publicly. Grok reads those corrections, and wrong facts travel fast on the platform.
  • Keep your brand facts consistent between your site and your X profile: one canonical brand name, one location, one description. You cannot fix old press, but you can fix everything you own.
  • Do not buy fake engagement. Grok's whole value proposition is reading X directly, so gaming it is gaming the model's raw input, and it reads poorly.

Then set your expectations correctly: X presence helps Grok answer questions about what is happening now and what people think. It does not make Grok recommend your product to a buyer comparing options. That is decided by your site, your schema, and your reviews.

Step 4: Verify, knowing verification is weak

There is no official verification path for Grok. No webmaster tool, no IP list, no documented token. The only method anyone has is a honeypot canary URL: a page linked from nowhere, pasted into Grok chat, watched in your server logs.

Say the limits plainly. A fetch of the canary page proves someone fetched it, not that Grok did, because the fetches arrive as ordinary browser user agents from proxy IPs. The absence of a hit proves nothing at all. This is the best anyone can do with Grok, because xAI has published nothing to check against. Any vendor promising confirmed Grok inclusion is selling something they cannot verify.

The fundamentals

These mechanics are shared across every engine and covered in full in their own guides:

A hypothetical, to make it concrete

Say you run a sporting-goods store. A scan comes back showing GrokBot allowed in robots.txt but challenged at the edge by Cloudflare Bot Fight Mode, no llms.txt, and a product catalog with prices only in JavaScript. The owner had read a guide that said "just post on X," so they had been posting daily with no change in Grok's answers. The scan explains why: Grok could not read the site at all, so there was nothing for any recommendation to be built on, and posting on X had not changed what Grok recommended. The fix is Steps 1 and 2 first. X is the layer on top, not the foundation.

What this guide does not cover

This gets you readable to Grok. It does not keep you readable. Grok's fetch behavior shifts without any announcement, your catalog drifts, and xAI publishes nothing you can monitor against, so the monthly re-check is the real work. And getting recommended is not getting bought from: letting the agent complete the purchase is a separate job.

For the ChatGPT version of this playbook, see our guide to getting ChatGPT to recommend your brand. For Claude's, see our guide to getting Claude to recommend your brand.

Check your own brand now

Run the free scan. It takes seconds, no signup, and shows exactly where Grok stands with your site: aizii.ai/aeo-scan

Run the free scan

Sources

Note: links marked specific point to the page that supports the claim in this guide. Links marked general docs point to the vendor's general docs section for that topic, not to a page that proves every step. Check the vendor docs for the current path before relying on a menu name or click path in this guide.