LLMs Are the Key to Mutation Testing and Better Compliance
Meta has developed an LLM-based tool called Automated Compliance Hardening (ACH) that enhances mutation testing to improve software compliance and security. By generating relevant code mutations and corresponding tests, ACH helps developers proactively identify compliance-related bugs and reduce manual effort. The tool simplifies risk assessment and supports scalable, continuous compliance in complex software systems.
- ▪Meta's Automated Compliance Hardening (ACH) tool uses large language models to automate mutation testing for compliance.
- ▪ACH generates both code mutants and tests to catch them, improving the effectiveness of software testing at scale.
- ▪The tool enables proactive identification of compliance risks, such as privacy violations, before code reaches production.
- ▪Meta has presented ACH at FSE 2025 and EuroSTAR 2025, highlighting LLMs' role in advancing automated software testing.
- ▪Mutation testing with ACH goes beyond traditional coverage methods by evaluating whether tests actually detect faulty code behavior.
Hacker News (Newest) files mainly under programming. We currently carry 5,306 of its stories.
Story provenance
Source · retrieval · rights · ranking — open for full record
inspect →
Attribution is not the same as permission. This drawer separates discovery metadata, excerpts, WeSearch-generated summaries, reuse status, and whether the publisher receives the visit. Nothing here claims a legal grant the publisher has not made.
Record
| Original publisher | Engineering at Meta |
| Canonical URL | https://engineering.fb.com/2025/09/30/security/llms-are-the-key-to-mutation-testing-and-better-compliance/ |
| Publication time | Sun, 17 May 2026 09:24:45 +0000 |
| Retrieval time | 2026-05-17T09:52:13.040Z |
| Last seen | 2026-05-17T09:52:13.040Z |
| Headline source | Publisher (no WeSearch rewrite) |
| Excerpt source | publisher body |
| Excerpt method | First ~120 words (~800 chars) of extracted publisher body, fair-use limited. |
| Summary | WeSearch · cerebras-chat (WeSearch summarizer) |
| Summary source text | contentText |
| Citation coverage | Summary is a WeSearch-generated derivative; primary citation is the original publisher URL. |
| Cluster | zSB9OgkqKyJy |
| Cluster logic | Grouped by semantic title/content similarity across sources within a rolling window. Same-publisher template collisions are excluded from coverage comparison. |
| Ranking reason | Story pages are not engagement-ranked. Hub feeds use recency, with optional source-diversified chronological ordering (cap consecutive stories per source). No personalized ranking. |
| Publisher visit | Yes — open original |
| Substitutes article? | No — link-out required for full text |
Rights status (four layers)
WeSearch handling by dimension
| Indexing | May the item be indexed (stored, ranked, made findable)? | Allowed |
| Snippet | May a short excerpt of the publisher's text be shown? | Allowed |
| AI summary | May WeSearch generate its own short summary of the article? | Limited |
| Retrieval / RAG | May the content be exposed for third-party retrieval-augmented generation? | Not asserted |
| Model training | May the content be used to train AI models? | Not asserted |
| Commercial reuse | May the content be reused commercially? | Not permitted |
Basis: Derived from the published RSS/Atom feed. Contact: [email protected]. Reviewed: 2026-07-24.
Opening excerpt (first ~120 words) tap to expand
POSTED ON SEPTEMBER 30, 2025 TO AI Research, ML Applications, Security & Privacy LLMs Are the Key to Mutation Testing and Better Compliance By Mark Harman Following our keynote presentations at FSE 2025 and Eurostar 2025, we’re delving further into the development of Meta’s Automated Compliance Hardening (ACH) tool, an LLM-based tool for software testing that is automating aspects of compliance adherence at Meta, while accelerating developer and product velocity. By leveraging LLMs we’ve been able to overcome the barriers that have prevented mutation testing from being efficiently deployed at scale. This allows us to greatly simplify risk assessments, reduce cognitive load for developers, and, ultimately, create a safer online ecosystem by enabling continuous compliance.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at Engineering at Meta.
Discussion
0 commentsMore from Engineering at Meta
