AI scraperbots keep hammering us from residential proxies

Jul 27, 2026 · 5:28 PM UTC · 0 reactions · 0 comments · 3 views

via

Fosstodon

AI scraperbots keep hammering us. We blocked most datacenters we could identify, but residential proxies just keep going and are really hard to block. We are seeing 7000-8000 concurrent connections. We have enough resources and network bandwidth to handle the requests/data, but given how aggressive these bots new connection setup is really slow or even times out. Tomorrow we will try to put a nginx tls terminator in front of httpd which hopefully helps handling more concurrent connections.

About this source

Hacker News (AI / LLM) files mainly under ai. We currently carry 2,488 of its stories.

All Hacker News (AI / LLM) coverage →

Original article

Fosstodon

Read full at Fosstodon →

Story provenance

Source · retrieval · rights · ranking — open for full record

inspect →

Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.

Record

Original publisher	Fosstodon
Canonical URL	https://fosstodon.org/@sourceware/116982983262820236
Publication time	Mon, 27 Jul 2026 17:28:19 +0000
Retrieval time	2026-07-27T18:12:49.255Z
Last seen	2026-07-27T18:12:49.255Z
Headline source	Publisher (no WeSearch rewrite)
Excerpt source	publisher description
Excerpt method	Publisher-supplied description / RSS summary field.
Summary	None yet
Summary source text	description
Citation coverage	No WeSearch summary has been generated for this story yet.
Cluster	woWqaIHODGoI · 1 stories
Cluster logic	Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison.
Ranking reason	Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking.
Publisher visit	Yes — open original
Substitutes article?	No — link-out required for full text

Rights status (four layers)

Publisher-declared

No publisher-confirmed rights record for this source yet.

Machine-readable

No source-specific machine-readable restriction detected beyond the public feed.

WeSearch interpretation

WeSearch declared handling (basis: Derived from the published RSS/Atom feed). This is WeSearch policy, not a legal grant on the publisher's behalf.

Unknown

Retrieval and training permissions are not asserted unless the publisher confirms them.

WeSearch handling by dimension

Indexing	May the item be indexed (stored, ranked, made findable)?	Allowed
Snippet	May a short excerpt of the publisher's text be shown?	Allowed
AI summary	May WeSearch generate its own short summary of the article?	Limited
Retrieval / RAG	May the content be exposed for third-party retrieval-augmented generation?	Not asserted
Model training	May the content be used to train AI models?	Not asserted
Commercial reuse	May the content be reused commercially?	Not permitted

Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.

Open original publisher → Source rights JSON → Machine-readable provenance → Rights framework → WESEARCH-STORY-PROVENANCE-V1

Anonymous · no account needed

Discussion

0 comments