AI crawler traffic, in full
const url = 'https://api.nsin.ir/analytics/ai-crawlers?domain=example.com&period=3h&crawler=gptbot';const options = {method: 'GET', headers: {Authorization: 'Bearer <token>'}};
try { const response = await fetch(url, options); const data = await response.json(); console.log(data);} catch (error) { console.error(error);}curl --request GET \ --url 'https://api.nsin.ir/analytics/ai-crawlers?domain=example.com&period=3h&crawler=gptbot' \ --header 'Authorization: Bearer <token>'Everything the AI Crawl Control pages are built from, in one response: period counters, a timeseries, a per-crawler table, and the paths crawlers read or were refused.
summary, series, top_paths and blocked_paths cover AI crawlers
only — the kinds listed in ai_kinds, or the single kind named by
crawler. Classic search and SEO crawlers (googlebot, bingbot,
yandexbot, ahrefsbot, semrushbot, mj12bot, generic-bot) are
deliberately left out of those, so they cannot drown the AI numbers.
crawlers is the exception: it lists every bot kind actually seen
on the domain, so nothing is invisible.
Human traffic never reaches any of these numbers.
The Markdown counters in summary describe the Markdown-for-Agents
feature: markdown_answered is what the edge actually rewrote to
Markdown, and markdown_missed is the rest of what could plausibly
have been Markdown (markdown_eligible — responses below 300).
Redirects, 404s and images are not counted against the feature.
Authorizations
Section titled “Authorizations ”Parameters
Section titled “ Parameters ”Query Parameters
Section titled “Query Parameters ”The domain name (for example example.com). These endpoints take the
domain as a query parameter rather than a path segment.
Example
example.comTime window, ending now. Buckets are hourly up to 24h and daily for
7d and 30d. An unrecognised value falls back to 24h.
Narrow every panel except crawlers to one bot kind, from the
canonical catalog — the values in ai_kinds, plus googlebot,
bingbot, duckduckbot, yandexbot, ahrefsbot, semrushbot,
mj12bot and generic-bot. Omit for all AI crawlers.
Example
gptbotNarrow to one subdomain. Matches the exact host, any subdomain of it, or
a bare label — so example.com matches api.example.com, and api
matches api.example.com, but exam matches neither.
Narrow to a URL path prefix.
Responses
Section titled “ Responses ”AI crawler activity for the period.
object
object
Requests answered with a status below 400.
Requests answered with 4xx or 5xx.
Bytes served to crawlers.
Responses the edge rewrote to Markdown.
Eligible responses that were not rewritten.
Responses that could plausibly have been Markdown — status below 300.
Every bot kind seen, busiest first — not limited to AI crawlers.
object
Bot kind, e.g. gptbot or claudebot.
Responses served to this crawler as Markdown.
Most recent request from this crawler in the period.
object
Top 10 paths AI crawlers read successfully.
object
URL path
Top 10 paths AI crawlers asked for and did not get (4xx
or 5xx) — the content agents want but cannot cite.
object
URL path
The bot kinds counted as AI crawler traffic.
Example generated
{ "summary": { "requests": 1, "allowed": 1, "unsuccessful": 1, "bytes": 1, "markdown_answered": 1, "markdown_missed": 1, "markdown_eligible": 1 }, "crawlers": [ { "kind": "example", "requests": 1, "allowed": 1, "unsuccessful": 1, "bytes": 1, "markdown": 1, "last_seen": "2026-04-15T12:00:00Z" } ], "series": [ { "timestamp": "2026-04-15T12:00:00Z", "requests": 1, "allowed": 1, "unsuccessful": 1, "bytes": 1, "s2xx": 1, "s3xx": 1, "s4xx": 1, "s5xx": 1 } ], "top_paths": [ { "path": "example", "hostname": "example", "requests": 1 } ], "blocked_paths": [ { "path": "example", "hostname": "example", "requests": 1 } ], "ai_kinds": [ "example" ]}The domain query parameter is missing, or crawler is not a known bot kind.
The single error shape used by every endpoint.
object
Human-readable description of what went wrong.
Examples
{ "error": "unknown crawler"}Missing, malformed, revoked or expired API key — or the owning account is inactive.
The single error shape used by every endpoint.
object
Human-readable description of what went wrong.
Examples
{ "error": "invalid API key"}The domain’s plan does not include the feature this endpoint needs
(monitoring for most sections, logs for raw and top-N request data).
The single error shape used by every endpoint.
object
Human-readable description of what went wrong.
Example
{ "error": "read-only API key"}No such domain, or it is not visible to this account. Domains you cannot access are reported as not found rather than forbidden.
The single error shape used by every endpoint.
object
Human-readable description of what went wrong.
Example
{ "error": "read-only API key"}The key exceeded its request budget (300 requests per minute by default).
The single error shape used by every endpoint.
object
Human-readable description of what went wrong.
Examples
{ "error": "rate limit exceeded"}The analytics backend is temporarily unreachable. Retry; no data is lost.
The single error shape used by every endpoint.
object
Human-readable description of what went wrong.
Examples
{ "error": "analytics unavailable"}