Methodology

How we test your site

Page scan

Your page plus up to 3 more, 21 checks

Agent test

An AI agent in a real browser

Comparison

Same tasks, browser vs connector

Scoring version

agent-readiness-0.6

Method

Three test types, one evidence standard

Your pageAny page for a scan. A site you’ve verified for agent tests.

Page scanFetch the HTMLYour page and a few more, no JavaScript, free

Agent testAI agent in a real browserYour tasks, 1, 3 or 5 runs each

Connector testSame tasks through your connectorIts tools only, no browser

Independent checks

21 published scan rules
A success check the agent never sees

Evidence you can review

Findings with fixes
Every step, replayed
Result per run, with uncertainty

Page scan

How we read your page

What a scan requests

GET/robots.txtrules read

GET/your-pageHTML, up to 750 KB

GET/sitemap.xmldates and pages read

GETUp to 3 more pagesfrom the sitemap or navigation

GET/llms.txtlooked for

—JavaScriptnot run

—Formsnever submitted

Timeout

8 s, then one retry of up to 45 s

Redirects

Up to 3, crawler rules rechecked at each

Access restrictions

Respected, never bypassed

What we check

Access

Can AI tools reach it?

Page accessHigh impact
AI search and assistant accessHigh impact
Search indexingHigh impact
Sitemap freshnessLow impact
Response timeMedium impact

Content

Can they read it?

Page textHigh impact
Content without JavaScriptHigh impact
Other pages on your siteLow impact
Title and headingMedium impact
Image descriptionsLow impact
Expected phraseHigh impact

Structure

Can they understand it?

HeadingsLow impact
Structured dataMedium impact
Structured data matches the pageMedium impact
Language and descriptionLow impact
Business identityLow impact
Heading orderLow impact

Actions

Can they use it?

Links in the pageMedium impact
Links and buttons agents can useMedium impact
Form labelsLow impact
Link to findMedium impact

Scoring

Points weighted by impact

How a score adds up

Worked example

Passed
Passed
Passed
Failed
Passed
Passed
Failed
Passed
Passed
Not assessed

15earned÷20available=75score

Not assessed

Excluded from the score

Score shown when

At least 6 checks have results and the page is readable

Never scored

Training-crawler rules, llms.txt, MCP server card, protection findings

Agent standards

Agent protocols, scored on their own

What we check

Discovery

Can agents find your files?

robots.txt
Sitemap
Link headers point to agent files

Content

Do you serve Markdown?

Markdown for agents

Bot access

Are your AI rules clear?

AI bot rules
Content Signals

Protocols

Can agents use your API?

API catalog
OAuth discovery
MCP server card
Agent skills
A2A agent card
WebMCP

Score

Share of the checks that apply and passed; doesn’t change the readiness score

Doesn’t apply

OAuth discovery without a signed-in API, the A2A card without an agent

Never scored

WebMCP

Also helps search

About half the readiness checks (text without JavaScript, headings, canonical URL, link previews, sitemap) help search engines too.

Agent tests

AI agents run your tasks in a real browser

Every step is recorded

Example

Find when the Bosch dishwasher warranty ends

010:04Opens the home page
020:11Clicks “My products”
030:23Clicks “Bosch dishwasher”
040:38Reads “Warranty ends 14 Mar 2027”
050:41Answers “14 March 2027”

The agent never sees the check

Example

Check

Answer matches

Expected

14 Mar 2027

Agent answered

14 March 2027

Result

Passed

NumbersMust match exactly
Different wordingCan go to an AI grader
Other checksA page to reach, or text to find

Each run gets one of five results

Passed

Met the success check

Failed

Did not meet the success check

Inconclusive

The check couldn’t decide

Self-reported

No success check; based on the agent’s answer

Not run

Stopped before starting

What a test may do

Signed out

Reads only

With a test account you provide

Makes the changes you allow

Never

Pays, closes accounts or messages other people

Comparisons

Same tasks and model, browser vs connector

Browser vs connector

Add a reminder 30 days before the Dyson V15 warranty ends

Browser2:07

Connector0:26

Held constant

AI model, tasks, checks and limits

Trials

3 to 10, order alternates

Only matched passes count

Example

TrialBrowserConnector

1 Browser firstPassedPassed
2 Connector firstPassedPassed
3 Browser firstFailedPassed
4 Connector firstPassedPassed
5 Browser firstPassedInconclusive
Both passed3 of 5 trials

Speed-up reporting

A speed-up is shown only after three matched passes

3 of 5 trials passed on both sides

Speed-up shown

2 of 5 trials passed on both sides

No speed-up shown

What we report

Median times and ranges from matched passes only

Limitations

What a scan can’t tell you

Whether AI services will recommend you

The score reflects our rules, not any service’s ranking.

Whether a firewall blocks a particular AI service

We check your published rules and never impersonate other crawlers.

How the page renders after JavaScript runs

The scan reads raw HTML only; agent tests use a real browser.

The rest of your website

Each scan reads your page and a sample of up to three others, not every page.

Details

Full method

All scan checks

Check	What it asks	Impact
Page accessAccess	Did the server return the page’s HTML, or an error or verification screen?	High (3 pts)
AI search and assistant accessAccess	Do your robots.txt rules allow the crawlers behind Google and Bing search, ChatGPT, Perplexity, Siri and other AI assistants? Blocking AI training crawlers is reported separately and doesn’t count.	High (3 pts)
Search indexingAccess	Does the page ask search engines not to index it (noindex)?	High (3 pts)
Sitemap freshnessAccess	Is your sitemap’s newest date within the last year, and does it list the page you scanned and the pages your navigation links to? A missing sitemap, or one without dates, is advice and doesn’t count against you; dates are what the sitemap says.	Low (1 pt)
Response timeAccess	Did your server answer within 5 seconds? If the first request gets no answer within 8 seconds, we try once more, waiting up to 45 seconds, and report how long it took. A slow response can have several causes, including a server waking from idle.	Medium (2 pts)
Page textContent	Can we extract main text from the HTML?	High (3 pts)
Content without JavaScriptContent	Is the main content in the HTML, or does the page look like an empty shell filled in by JavaScript? A “Loading…” placeholder, or content hidden until a script reveals it, counts as depending on JavaScript. This is an inference; we don’t run the page in a browser.	High (3 pts)
Other pages on your siteContent	Do up to three other pages (a listing and a recent detail page from your sitemap, or navigation links, plus the home page) have readable content in their HTML? Pages we can’t fetch don’t count.	Low (1 pt)
Title and headingContent	Are a page title and main heading present?	Medium (2 pts)
Image descriptionsContent	Do images have alt text or a decorative marker? Only scored when the page has images.	Low (1 pt)
Expected phraseContent	Does a phrase you supply appear in the text? Only scored when you supply one.	High (3 pts)
HeadingsStructure	Does the main content have headings?	Low (1 pt)
Structured dataStructure	Can we parse the page’s JSON-LD? Having none isn’t a failure.	Medium (2 pts)
Structured data matches the pageStructure	Do prices and names in the structured data also appear on the visible page?	Medium (2 pts)
Language and descriptionStructure	Does the page declare its language and include a meta description?	Low (1 pt)
Business identityStructure	Does the page say which business or person it belongs to, through Organization structured data or a site name?	Low (1 pt)
Heading orderStructure	Is there exactly one main (H1) heading, with no skipped levels such as H2 to H4?	Low (1 pt)
Links in the pageActions	Does the main content contain web links?	Medium (2 pts)
Links and buttons agents can useActions	Are clickable elements real links and buttons, or click handlers that only work with JavaScript?	Medium (2 pts)
Form labelsActions	Does every form field have a label? Only scored when the page has a form.	Low (1 pt)
Link to findActions	Does a link you supply appear in the main content? Only scored when you supply one.	Medium (2 pts)

What happens during a scan

We read your robots.txt, request the page and pull out its main text and links. We read your sitemap (the one robots.txt names, or /sitemap.xml, and at most two parts of a sitemap index) and up to three other pages from it or from your navigation, in parallel with a 5-second wait each. We also look for an optional llms.txt file and an optional AI catalog or MCP server card. We limit request time, response size and redirects, and we never bypass access restrictions or submit forms.

How the score is calculated

Each check that passes earns points by impact: high is worth 3, medium 2 and low 1. The score is the share of available points earned, among the checks we could assess. A check we couldn’t assess doesn’t count for or against you. We only show a score when at least six checks have results and we could read the page. If the page’s main text is missing, or depends on JavaScript, the score is capped at 59: agents that can’t read the content can’t use anything else on the page.

Reported but not scored

We also report AI training crawler rules, TransicaBot’s crawler access, a missing or undated sitemap, llms.txt, an AI catalog or MCP server card, the page’s preferred address, link-preview tags and publication dates. These don’t change the score. Blocking training crawlers is a legitimate choice, and optional files like llms.txt or an MCP server card aren’t required by the major AI services.

Exposure findings

Some controls are hidden from people but still available to agents. We classify what we find by its purpose and behaviour: helpful shortcuts, spam traps, controls worth confirming, or hidden forms that send data. The free scan reads HTML and inline styles; it doesn’t use these controls. Exposure findings never change your AI readiness score.

How agent tests are graded

On a site you’ve verified, an AI agent tries your tasks in a real browser, several fresh runs each. Each task has a success check: an expected answer, a page to reach or text to find. The agent doesn’t see that check. Numbers must match exactly; different wording can go to an AI grader. Without a check, the result is labelled self-reported.

Signed out, a test only reads. With a test account you provide, it can make changes you allow. It never pays, closes accounts or sends messages to other people.

How speed-ups are calculated

A comparison runs the same tasks in the browser and through your connector, with the same AI and limits, alternating which goes first. We show a speed-up only when at least three runs passed their check on both sides, and we report medians with their ranges.

See the checks on a sample report

Scoring version agent-readiness-0.6

View sample report