LLM SEO: optimizing your site for large language models
The on site half of getting cited by AI: what the language model crawlers can read, how to make sure they can read you, and how to structure pages they can quote.
On this pageShow 10Hide
A lot of advice about getting cited by AI focuses on what other sites say about you. That matters, and our checklist for AI search optimization covers it. This guide is about the other half: your own site, and whether the large language model products can read it, understand it and quote it.
Call it LLM SEO or LLM optimization. The work is mostly technical, mostly cheap, and it is the part you fully control.
How language model products reach your pages
There are two routes, and they need different things from you.
- Training data. Some crawlers collect pages that may be used to train future models. What gets in shapes what a model knows by default, but you cannot see or quickly change it.
- Live retrieval. When an assistant searches the web to answer a question, it uses pages it finds at that moment and can cite them. ChatGPT can search the web and answer with links to sources, Perplexity searches in real time and cites every answer, and Google says AI Overviews and AI Mode may run several related searches to build a response. This is the route you can influence quickly.
The practical upshot: make sure the search crawlers can reach your pages, and make sure what they fetch contains the answer.
Check your AI visibility free
See which buying questions ChatGPT, Perplexity, Gemini and Google's AI answers name you for, and who they name instead. No call, no card.
Crawler access
Each AI company publishes its crawlers and what they are for. Check these pages yourself, because names and rules change:
- OpenAI documents separate user agents for search (OAI-SearchBot), for model training (GPTBot) and for visits a user asks for (ChatGPT-User). It says sites that block OAI-SearchBot will not be shown in ChatGPT search answers, and that each setting is independent. See its crawler overview.
- Perplexity documents PerplexityBot, which it uses to surface and link sites in its search results and not for model training, and Perplexity-User, for visits made on a user's behalf, which it says generally ignores robots.txt, in its crawler docs.
- Google uses Googlebot for Search, including AI Overviews and AI Mode. Its separate Google-Extended token controls whether content may be used to train Gemini models and to ground Gemini Apps answers, and does not affect inclusion in Google Search, according to its crawler documentation.
What to check:
- robots.txt allows the search crawlers you want to appear in.
- Your CDN or firewall does not block them anyway. Bot protection rules can stop crawlers robots.txt allows.
- Your server logs show them arriving and getting 200 responses on your key pages.
Server rendering
If your page builds its main content in the browser with JavaScript, some crawlers may see an empty shell. A December 2024 study by Vercel found that none of the major AI crawlers it measured rendered JavaScript, while Gemini benefited from Googlebot's rendering. Vercel's advice was to server render critical content.
What to check:
- Fetch your key pages with JavaScript turned off, or view the raw HTML. The headline, the main answer, prices and product details should all be there.
- Do not hide important content behind tabs or accordions that only load on click.
- Avoid putting key facts only in images or PDFs.
Page structure that is easy to quote
Google says no special formatting or markup is needed for its AI features, so write for the reader. Clear pages are easier to quote accurately, by people and by models.
- Lead with the answer. The first paragraph of a page should state what it is about and the core answer, in plain words.
- One topic per page. A page that tries to rank for everything gives a model nothing specific to quote.
- Descriptive headings. Phrase them the way people ask, so the section under each heading answers it.
- Short, factual sentences for key facts. Prices, specs, locations, who it is for. A sentence like "Plans start at $129 a month" is easier to repeat accurately than a pricing grid drawn in images.
- Lists and steps where they fit. Numbered steps and short lists are easy to scan and hard to misread.
Structured data
Google says structured data is not required for generative AI search and that there is no special markup to add. It still helps machines tie facts to entities and helps you qualify for rich results, and it costs little:
- Organization: name, logo, URL, profiles.
- Product or Service, with Offer: prices that match the visible page.
- Article: on guides, with author and dates.
- FAQPage: where you genuinely answer questions on the page.
Keep markup honest. If the visible page does not say it, the markup should not either. Validate with Google's Rich Results Test or the Schema.org validator.
Entity consistency on your own site
Models build a picture of a brand from many pages. Make your own pages agree:
- One version of your company name, category description and pricing across the site.
- An about page that states plainly what you do, for whom, and since when.
- Old product pages updated or redirected, so a model does not read two versions of your offer.
What about llms.txt?
llms.txt is a proposed convention for a plain text file that points language models to your most useful pages. Google is direct about it: its AI optimization guide says Google Search does not use llms.txt or other AI text files, and that having one neither helps nor harms. We found no statement from OpenAI or Perplexity that their search products read it. It is harmless to add, but it should not come before crawler access, rendering and clear pages.
How to check it worked
On site fixes show up in answers only after the engines read your pages again (OpenAI and Perplexity both say robots.txt changes take up to about a day to apply), and the answers themselves vary from run to run because the models are not deterministic. Measure on a fixed prompt set over a few weeks rather than refreshing one answer.
The fastest first reading is our free checker: check how LLMs see your brand across Google AI Overviews, ChatGPT, Gemini and Perplexity, with every answer and the pages each one cited. Then use the method in our LLM visibility guide to track it.
LLM SEO checklist
- Allow the AI search crawlers you want, and confirm in logs that they get through.
- Server render the main content of every key page.
- Put the answer in the first paragraph.
- Use descriptive headings, short factual sentences and lists.
- Add honest, validated structured data.
- Make your brand facts agree across your own site.
- Measure on a fixed prompt set over weeks, not one answer.
Questions
- What is LLM SEO?
- LLM SEO, also called LLM optimization, is the work of making a website easy for large language model products to find, read and cite: crawler access, server rendered content, clear page structure, accurate schema and consistent facts.
- Do AI crawlers run JavaScript?
- A December 2024 study by Vercel found that none of the major AI crawlers it measured rendered JavaScript, while Gemini used Googlebot's rendering. Behavior can change, so the safe choice is to put important content in the server rendered HTML.
- Does llms.txt help?
- Not for Google: Google says Search does not use llms.txt and that having one neither helps nor harms. We found no statement from OpenAI or Perplexity that they read it. Crawler access and readable HTML matter more.
- Is LLM SEO different from GEO?
- They overlap. LLM SEO usually means the technical and on site work. GEO covers that plus off site work, such as getting named on the pages AI engines cite, and measuring the answers.
Check your AI visibility free
See which buying questions ChatGPT, Perplexity, Gemini and Google's AI answers name you for, and who they name instead. No call, no card.