Show HN: An interactive way to exploit an LLM without going to jail
LLM Red Team Lab A hands-on kit for educational, authorized red teaming of any locally-run LLM. Injecting the agent — indirect prompt injection that steers an agent into a forbidden tool call (exfil, destructive, out-of-scope). Canary-safe by design: the model is told to protect a fake secret and the agent's tools are mocks, so the techniques are genuine but nothing harmful is ever produced.
- ▪LLM Red Team Lab A hands-on kit for educational, authorized red teaming of any locally-run LLM.
- ▪Injecting the agent — indirect prompt injection that steers an agent into a forbidden tool call (exfil, destructive, out-of-scope).
- ▪Canary-safe by design: the model is told to protect a fake secret and the agent's tools are mocks, so the techniques are genuine but nothing harmful is ever produced.
Hacker News (AI / LLM) files mainly under ai. We currently carry 2,358 of its stories.
Opening excerpt (first ~120 words) tap to expand
LLM Red Team Lab A hands-on kit for educational, authorized red teaming of any locally-run LLM. It works with any OpenAI-compatible model — Llama, Mistral, Qwen, Gemma, DeepSeek R1, and more — and covers the two ways an LLM system gets exploited: Jailbreaking the model — real techniques (instruction override, roleplay/DAN, hypothetical framing, encoding, payload splitting, and a reasoning-trace attack aimed at reasoning models like DeepSeek R1) that try to talk the model past its own guardrail. Injecting the agent — indirect prompt injection that steers an agent into a forbidden tool call (exfil, destructive, out-of-scope). Canary-safe by design: the model is told to protect a fake secret and the agent's tools are mocks, so the techniques are genuine but nothing harmful is ever produced.
…
Excerpt limited to ~120 words for fair-use compliance. The full article is at GitHub.