AI Crawl ControlDecide which AI crawlers may read your site.
GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more are already fetching your pages. nsin names each one, shows what it took, and lets you block the ones that only train on you while keeping the ones that send readers back.
Every crawler, named and sorted
Crawlers are grouped by what they do: bulk training scrapers, AI search engines that cite you, assistants fetching a page on request, classic search engines and SEO tools.
- 21 crawlers classified by operator and purpose
- AI traffic by company over 24 hours, 7 or 30 days
- Requests and allowed share per crawler
One-click policies
Four ready-made templates cover the common choices. Applying one creates an ordinary bot rule you can edit, scope to specific hostnames and run in dry run first.
- Block AI training crawlers
- Allow AI search, block trainers
- Monitor only, to see what a policy would stop
Enforced at the edge, on verified identity
robots.txt is a request; this is a rule. Block with a status you choose, serve a licensing notice, or send the crawler to another server. Verified-only matching ignores copycats that fake a user-agent.
- Block, serve different content, reroute or tag
- Identity confirmed by forward-confirmed reverse DNS
- Googlebot and Bingbot untouched unless you name them
Frequently asked questions
Will blocking AI crawlers hurt my Google ranking?
No. AI crawlers and search crawlers are separate. You can block GPTBot or CCBot and leave Googlebot and Bingbot alone.
Is this the same as robots.txt?
No. robots.txt asks crawlers to stay away and depends on them obeying. AI Crawl Control acts on the request at the edge, whatever the crawler chooses to do.
Which AI crawlers does nsin recognise?
Trainers such as GPTBot, ClaudeBot, CCBot, Bytespider, Google-Extended and Amazonbot, answer engines such as OAI-SearchBot and PerplexityBot, and assistants such as ChatGPT-User and Claude-User.
Can I see what a policy would do before it blocks anything?
Yes. The Monitor only template blocks nothing and logs each match as if it had been blocked.