Cloudflare CAPTCHA on at least one ampersand
The author was using Cloudflare's CAPTCHA to prevent crawlers from aggressively spidering their faceted search engine. However, they found that even simple searches were triggering the challenge, which led to frustration. The author discovered a custom rule that allows the CAPTCHA to only kick in for search URLs containing at least one ampersand.
- ▪Cloudflare's CAPTCHA was being used to prevent crawlers from spidering a faceted search engine.
- ▪The CAPTCHA was triggering for even simple searches, causing frustration for the author.
- ▪A custom rule was discovered to only trigger the CAPTCHA for search URLs with at least one ampersand.
Opening excerpt (first ~120 words) tap to expand
I'm using Cloudflare's CAPTCHA (they call it a "Web Application Firewall > Custom rules > Managed Challenge" these days) to prevent crawlers from aggresively spidering my faceted search engine on this site, but I got fed up of even simple ?q=term searches triggering the challenge. After some mucking around with Claude Code it turns out you can register the following rule instead, so the CAPTCHA only kicks in for search URLs containing at least one ampersand: (http.request.uri.path wildcard r"/search/*" and http.request.uri.query contains "&") And now /search/?q=lemur works without triggering a CAPTCHA!
Excerpt limited to ~120 words for fair-use compliance. The full article is at Simon Willison's Weblog.
Discussion
0 commentsMore from Simon Willison's Weblog
