Rate limiting & botsRate limiting and bot management at the edge.
Cap how many requests one visitor can make, ask unsure traffic to prove it is human, and decide which crawlers are welcome. All of it runs at the edge, before a request costs your server anything.
Rate limits scoped to what matters
Limit a path such as /login to a number of requests per time window, count only the HTTP methods you choose, and count per visitor or per visitor and path.
- Protect login and sign-up from credential stuffing
- Stop catalogue scraping and expensive API calls
- Burst allowance so a normal page load is not caught
Challenge instead of block
When blocking is too blunt, show an interactive check. A visitor who passes carries on and is not asked twice. A rate limit can end in a challenge too.
- CAPTCHA challenge on chosen paths
- Block or challenge on a rate-limit breach
- Real people continue, scripts stop
Bots by name, verified
Bot rules match search engines, AI crawlers and SEO tools. Verified-only matching confirms a bot by DNS or IP, so the real Googlebot gets in and a copycat does not. Fingerprint rules catch clients that only pretend to be a browser.
- Block, serve other content, reroute or tag
- Verified Googlebot, not anyone using its name
- TLS and HTTP fingerprint rules
Frequently asked questions
Will rate limiting block real visitors?
Not with a sensible limit scoped to one endpoint. If you are unsure, use challenge as the breach action so a real person can continue.
How do I protect a login form?
Add a rate limit on the login path, count only POST, and allow a handful of requests per minute per visitor with challenge on breach.
Could a bot rule stop Google indexing my site?
Only if a rule names search engines. Use verified-only matching to let the real Googlebot through, and test new rules in tag mode or dry run.
What is a fingerprint rule?
A rule that matches the TLS and HTTP fingerprint of a client, to catch traffic that sends a normal browser user-agent but does not behave like one.