Cloudflare CAPTCHA on at least one ampersand
The author was using Cloudflare's CAPTCHA to prevent crawlers from aggressively spidering their faceted search engine. However, they found that even simple searches were triggering the challenge, which led to frustration. The author discovered a custom rule that allows the CAPTCHA to only kick in for search URLs containing at least one ampersand.
- ▪Cloudflare's CAPTCHA was being used to prevent crawlers from spidering a faceted search engine.
- ▪The CAPTCHA was triggering for even simple searches, causing frustration for the author.
- ▪A custom rule was discovered to only trigger the CAPTCHA for search URLs with at least one ampersand.
Simon Willison files mainly under blogs. We currently carry 53 of its stories.
Story provenance
Source · retrieval · rights · ranking — open for full record
inspect →
Story provenance
Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.
Record
| Original publisher | Simon Willison's Weblog |
| Canonical URL | https://simonwillison.net/2026/Jun/16/captcha-on-at-least-one-ampersand/#atom-everything |
| Publication time | 2026-06-16T00:21:36+00:00 |
| Retrieval time | 2026-06-16T00:42:43.081Z |
| Last seen | 2026-06-16T01:03:36.106Z |
| Headline source | Publisher (no WeSearch rewrite) |
| Excerpt source | publisher body |
| Excerpt method | First ~120 words (~800 chars) of extracted publisher body, fair-use limited. |
| Summary | WeSearch · cerebras-chat (WeSearch summarizer) |
| Summary source text | contentText |
| Citation coverage | Summary is a WeSearch-generated derivative; primary citation is the original publisher URL. |
| Cluster | 3g2HAipTMUZT |
| Cluster logic | Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison. |
| Ranking reason | Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking. |
| Publisher visit | Yes — open original |
| Substitutes article? | No — link-out required for full text |
Rights status (four layers)
WeSearch handling by dimension
| Indexing | May the item be indexed (stored, ranked, made findable)? | Allowed |
| Snippet | May a short excerpt of the publisher's text be shown? | Allowed |
| AI summary | May WeSearch generate its own short summary of the article? | Limited |
| Retrieval / RAG | May the content be exposed for third-party retrieval-augmented generation? | Not asserted |
| Model training | May the content be used to train AI models? | Not asserted |
| Commercial reuse | May the content be reused commercially? | Not permitted |
Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.
Opening excerpt (first ~120 words) tap to expand
I'm using Cloudflare's CAPTCHA (they call it a "Web Application Firewall > Custom rules > Managed Challenge" these days) to prevent crawlers from aggresively spidering my faceted search engine on this site, but I got fed up of even simple ?q=term searches triggering the challenge. After some mucking around with Claude Code it turns out you can register the following rule instead, so the CAPTCHA only kicks in for search URLs containing at least one ampersand: (http.request.uri.path wildcard r"/search/*" and http.request.uri.query contains "&") And now /search/?q=lemur works without triggering a CAPTCHA!
Excerpt limited to ~120 words for fair-use compliance. The full article is at Simon Willison's Weblog.
Discussion
0 commentsMore from Simon Willison's Weblog
