Show HN: Reproducing three AI text detectors, and why filtering hurt my model
A new open-source package allows researchers to reproduce an experiment comparing three AI text detectors for identifying AI-written movie reviews. The repository includes scripts to evaluate detector performance and assess how filtering affects downstream sentiment classification tasks. It also provides tools to audit a 2019 Mendeley dataset to verify the human origin of the source reviews used in the study.
- ▪The package requires Python 3.11 or newer and approximately 2 GB of disk space to download pretrained models like GPT-2 and all-MiniLM-L6-v2.
- ▪The experiment uses 600 training reviews and 200 reserved test reviews to score detectors based on perplexity, near-duplicate similarity, and embedding density.
- ▪The Mendeley Data collection used for the audit was published in 2019, prior to the public launch of ChatGPT, and is treated as human-written.
- ▪The repository includes exact generated examples from the original run to ensure reproducibility without requiring new model API calls.
- ▪Users can compare their local results against captured reference outputs, though minor numerical differences may occur due to hardware or library variations.
Hacker News (AI / LLM) files mainly under ai. We currently carry 8,030 of its stories.
Story provenance
Source · retrieval · rights · ranking — open for full record
inspect →
Story provenance
Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.
Record
| Original publisher | GitHub |
| Canonical URL | https://github.com/abduldattijo/ai-slop-detection-reproduction |
| Publication time | Mon, 12 Oct 2026 10:46:45 +0000 |
| Retrieval time | 2026-10-12T10:50:55.446Z |
| Last seen | 2026-10-12T10:50:55.446Z |
| Headline source | Publisher (no WeSearch rewrite) |
| Excerpt source | publisher body |
| Excerpt method | First ~120 words (~800 chars) of extracted publisher body, fair-use limited. |
| Summary | WeSearch · cerebras-chat (WeSearch summarizer) |
| Summary source text | contentText |
| Citation coverage | Summary is a WeSearch-generated derivative; primary citation is the original publisher URL. |
| Cluster | PlD0pE7uagJt |
| Cluster logic | Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison. |
| Ranking reason | Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking. |
| Publisher visit | Yes — open original |
| Substitutes article? | No — link-out required for full text |
Rights status (four layers)
WeSearch handling by dimension
| Indexing | May the item be indexed (stored, ranked, made findable)? | Allowed |
| Snippet | May a short excerpt of the publisher's text be shown? | Allowed |
| AI summary | May WeSearch generate its own short summary of the article? | Limited |
| Retrieval / RAG | May the content be exposed for third-party retrieval-augmented generation? | Not asserted |
| Model training | May the content be used to train AI models? | Not asserted |
| Commercial reuse | May the content be reused commercially? | Not permitted |
Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.
Opening excerpt (first ~120 words) tap to expand
AI-written review detection experiment This package contains the exact review files and scripts used for the experiment described in the accompanying article. It lets you rerun the detector comparison, the sentiment-classifier evaluation, and (if you download the source archive) the Mendeley collection audit on your own computer. Requirements Python 3.11 or newer The packages in requirements.txt About 2 GB of free disk space for the first run, when the pretrained models are downloaded The first run downloads GPT-2 and the all-MiniLM-L6-v2 sentence-embedding model from Hugging Face. The detector and classifier then run locally. The included review files preserve the exact generated examples used in the reported run, so they can be used directly in the analysis.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.