Most crawlers identify themselves with a user-agent string. That is useful for classification, but it is not sufficient for verification because any client can send a fake user-agent.

Leadstore treats the user-agent as the first clue. Verification needs additional evidence from the request path.

Use request-level evidence

■

Client IP or privacy-preserving IP hash.

■

Reverse DNS lookup and forward confirmation where available.

■

ASN and network ownership context.

■

Cloudflare bot fields, bot score, bot tags, and verified bot categories where available.

■

Known bot signatures and expected crawler names.

■

Page URL, referrer, request time, country, transport, and event type.

Separate crawler classes

Class

Human

Known bot

Verified bot

Suspected spoof

Meaning

Normal browser visitor

User-agent matches a known crawler

Identity is supported by stronger evidence

Claims a bot identity but evidence does not match

Dashboard treatment

Count in human traffic and content performance

Show as bot activity with limited confidence

Use for crawler proof and AEO activity

Warn users and keep out of verified counts

Why Leadstore Tag helps

Leadstore Tag collects browser events and feeds the same analytics model that server-side bot checks use. That means the dashboard can compare what humans visit, what assistants refer, and what crawlers request.

When users also connect Cloudflare or server logs, Leadstore can raise confidence for request-level crawler proof.

How this fits into the three Leadstore stages

■

Stage 1, account + site added: users get the audit reward before any tag or key - four core channel scores, top fixes, readiness, generated llms.txt and llm-data.json, and the Leadstore Tag snippet to install.

■

Stage 2, tag installed: users get the live telemetry reward with no key required - AI-crawler activity by engine, verified-bot counts, human traffic, top sources, top pages, devices and geography.

■

Stage 3, key connected: users add answer tracking on top of the audit and telemetry layers - Visibility Score, SAIV, Brand Monitor, share of voice, citations, competitors and hallucination checks.

Next step

Install Leadstore Tag and connect log sources when available to move crawler reporting from observed to verified.

Install Leadstore Tag →

Read about Content Autopilot →

Sources

Google Analytics Data API dimensions and metrics

Google Analytics IP address usage

Google Search Console Search Analytics API

Cloudflare HTTP request log fields

Cloudflare verified bots

PostHog autocapture and default properties