Agent experience

Agent experience: make your site easy for AI agents to read and use.

More and more, an AI agent reads your page before your buyer does. xSeek shows you what that agent finds: which crawlers your robots.txt lets in, whether your text is there without JavaScript, your llms.txt, and the files agents look for.

Book a demo Start for free

Free agent-ready scan, no signup

Knows the crawlers of

Runs from

The agentic web

Agent experience is UX for the software that reads for your buyers.

ChatGPT, Claude and Perplexity now fetch pages, read them and act on them for a person. Agent experience is how easily they can do that on your site. It comes down to three questions.

01
Can they get in?
Your robots.txt decides which AI crawlers may read your site. Two lines, User-agent: * and Disallow: /, shut every one of them out.
robots.txt checker
02
Can they read it?
AI crawlers read the HTML your server sends. If your text is drawn later by JavaScript, they get an empty page. An llms.txt file gives them a map.
AI-readable check llms.txt generator
03
Can they use it?
Agents look for standard files before they act: a sitemap, Link headers, an MCP server card, an API catalog, OAuth metadata.
Agent-ready scan

01 · Agent-ready scan

Every file an agent looks for, checked in one scan

Paste a URL. The scan checks the files and headers AI agents probe before they use a site, gives you a readiness level, and lists what passes and what fails.

Discoverability: robots.txt, sitemap and Link response headers
Content and bot access: Markdown content negotiation, AI bot rules, Content Signals, Web Bot Auth
Agent discovery: MCP server card, API catalog, OAuth metadata, Agent Skills, WebMCP
Agent commerce: x402, MPP, UCP and ACP
The checks to add for the next level, and a link to share the result

We hold xseek.io to the same checks: it publishes an MCP server card, an API catalog, Content Signals in robots.txt and WebMCP tools.

Run the agent-ready scan

Bot access control3 checks

PassAI bot rules in robots.txtWildcard rules apply to AI bots too
PassContent SignalsContent Signals found in robots.txt
InfoWeb Bot AuthDirectory not found (informational)

Agent discovery5 checks

PassMCP Server CardFound at /.well-known/mcp/server-card.json
PassAPI catalog (RFC 9727)API Catalog found with 1 API listed
PassOAuth 2.0 discoveryRFC 8414 metadata found
PassWebMCPFound 4 WebMCP tools
FailAgent SkillsAgent Skills index not found

02 · AI-readable check

What a crawler gets when it does not run JavaScript

The check fetches your page the way GPTBot or ClaudeBot does, with no JavaScript, removes the code and counts the text that is left.

A verdict: readable, readable but thin, or invisible to AI
Spots the empty app shell that client-only React builds send
Flags a missing title, meta description or H1
Shows the first lines a crawler actually reads

Test a page

Readable by AI1,057 words readable without JavaScript

Your content is in the HTML. A crawler reads it without running any code.

Server-rendered content1,057 words readable without running JavaScript. An AI crawler sees your page.

What the crawler actually reads

Stop guessing. Get cited by AI. xSeek shows you what to create and optimize to get cited by ChatGPT, Claude, Perplexity and Gemini...

title ✓meta description ✓h1: 1Next.js

03 · llms.txt generator

A map of your site, written for AI

llms.txt is a short markdown file at the root of a site: who you are, then links to the pages that matter. Paste your URL and the llms.txt generator drafts one for you.

A markdown draft to copy, or to download as llms.txt
Edit it, then host it at yoursite.com/llms.txt
Your assistant can then check the live file through the xSeek MCP server: valid, invalid or missing
It costs nothing to host. Treat it as a bet on where agents are going, not a ranking factor

Generate my llms.txt

llms.txtGeneratedCopyDownload .txt

# xSeek> xSeek measures whether AI assistants mention a brand> when someone asks them a buying question.## Product- [AI Visibility](https://www.xseek.io/en/product/ai-visibility)- [Action Plan](https://www.xseek.io/en/product/action-plan)## Docs- [MCP server](https://www.xseek.io/docs/ai-assistants)

04 · robots.txt checker

Which AI crawlers your robots.txt lets in

The checker reads your robots.txt and works out, bot by bot, whether each AI crawler may read your site, from the rules for its name or for every bot (*).

Rules checked for the 48 AI user agents on xSeek's list, from 26 companies
Allowed or blocked, for GPTBot, OAI-SearchBot, ClaudeBot, PerplexityBot, Google-Extended and more
A score out of 100: answer bots weigh most, training bots next
Your raw robots.txt, with each rule highlighted

Check my robots.txt

After the scan

A scan shows what agents could read. xSeek then shows what they do.

See the AI bots that actually come

The xSeek tracker runs on your server or CDN and records each AI bot visit with the page it read. It recognizes more than 40 AI crawlers and counts assistant fetches apart from training crawls.

Installs on WordPress, Next.js, Cloudflare or CloudFront
On paid plans, the 50 pages AI bots visit most are re-checked daily. A broken one becomes an urgent action

See AI Traffic

Run every check from your assistant

Connect the xSeek MCP server to Claude, ChatGPT or Cursor and ask in plain words: is my homepage readable by AI? Is my llms.txt live? The assistant runs the same checks and reads you the result.

agent_ready_scan
ai_readable_scan
check_llms_txt
llms_txt_scan

See the MCP server

How it works, in three steps.

01
Scan for free
Run the three checks on your homepage and draft your llms.txt. No account needed.
02
Fix what fails
Each check says what it found and what is missing. Hand that to whoever edits your site, then scan again to confirm the fix is live.
03
Track the real visits
Install the xSeek tracker to see which AI bots read which pages, and fix the broken pages they hit first.

FAQ

Questions

What is agent experience?

Agent experience (AX) is how easily an AI agent can find, read and use your website. Good AX means AI crawlers are allowed in, your pages carry their text in the HTML, and the standard files agents look for are in place. xSeek checks all three with free scans.

What does agent ready mean?

An agent-ready site exposes what AI agents probe for before they use it: a valid robots.txt and sitemap, Link headers, and discovery files such as an MCP server card, an API catalog or OAuth metadata. The agent-ready scan checks each one and gives you a readiness level.

What is llms.txt, and does it help?

llms.txt is a markdown file at the root of a site: an H1 with your name, a short summary, then sections of links to the pages that matter. It costs nothing to host. Treat it as a bet on where agents are going, not a ranking factor. The xSeek llms.txt generator drafts one from your URL.

What is an agents.md file? Does xSeek make one?

AGENTS.md is a README for coding agents: a markdown file in a code repository that tells tools like Codex or Cursor how to build and test the project. It lives in your repository, not on your public site, so xSeek does not read it and does not write one. For your public site, the closest files are llms.txt and the discovery files the agent-ready scan checks.

Should I block AI crawlers in robots.txt?

Only if you do not want AI engines to read your pages. A crawler you block cannot read the page it would have quoted. The robots.txt checker shows which AI bots your file blocks today, so the choice is yours on purpose, not by accident.

What is agentic SEO?

Agentic SEO is search optimization for the agentic web: making sure the assistants that research for your buyers can reach, read and cite your pages. It starts with the checks on this page. Then you track what AI engines say about you, which is what xSeek AI Visibility does.