Bots & Challenges
nsin gives you three tools for automated traffic:
- Bot rules — named bots: search engines, AI crawlers, SEO tools.
- Challenges (CAPTCHA) — unsure traffic: prove it’s a person.
- Fingerprint rules — traffic that lies about who it is.
Bot rules
Section titled “Bot rules”A bot rule matches bots the edge recognises.
- Go to Rules → Add Rule → Bots.
- Pick the bots: search engines, AI crawlers, SEO tools, or any bot.
- Choose the action.
- Click Save.
| Action | What it does |
|---|---|
| Block | Refuse the request. |
| Serve different content | Return your own response, skip your server. |
| Send to a different server | Proxy to another origin. |
| Tag only | Let it through, marked in analytics. |
Verified bots only
Section titled “Verified bots only”Anyone can claim to be Googlebot — it’s just text in a request. Verified only matches bots confirmed by DNS or IP.
Challenges
Section titled “Challenges”A challenge rule shows an interactive check on chosen paths. Pass it and browsing continues; people aren’t asked twice.
Use one when blocking is too blunt: a login page under credential-stuffing, a scraped search endpoint, a spammy signup.
Rate limiting can also end in a challenge instead of a block — see Rate Limiting.
Fingerprint rules
Section titled “Fingerprint rules”Some traffic uses a normal user-agent but behaves nothing like a browser. Fingerprint rules match the TLS/HTTP fingerprint, and can drop, challenge, or tag it.
Advanced: start in tag mode, then decide.
Order matters
Section titled “Order matters”Rules run in order. Put allow-style rules above block-style ones. See Rules Overview.
Common questions
Section titled “Common questions”Will blocking AI crawlers hurt my ranking? No — AI and search crawlers are separate. Block AI, leave Googlebot alone.
Google stopped indexing my site. A bot or block rule may be catching it. Switch to tag or dry run to check.
Real visitors keep getting challenged.
Narrow the rule to the paths that need it (/login, /search), not the site.