AI security & interpretability
AI reading list
An agent can read a webpage, access private information and act through tools. Which part lets an attacker redirect it, and which boundaries limit the result? Start with an attack report or a map of data access. For a different question—why the model produced a particular response—open the interpretability explanations and interventions. Each route follows evidence from a specific system or experiment.
People to read
Open the first material, then continue with the author on X. People appear once, grouped by their main reading use and ordered by handle within each group.
2 of 7 people
No people match that search. Try another name, subject or reading task.
Attacks on agents & models
Follow attacker-controlled content into an unwanted response or tool action. Compare the attacker’s access and the system that was tested.
-
Greshake explains attacks that arrive through retrieved content rather than a direct user prompt. Begin with his page-to-application example, then the coauthored paper’s threat taxonomy. His defense essay compares trust tracking and action restrictions, presenting design tradeoffs rather than a proven universal fix.
Start with this Web material · kai-greshake.de
How We Broke LLMs: Indirect Prompt Injection
Read the introduction and the explanation of indirect injection: attacker-authored material is retrieved during a normal task and treated as instructions. Continue to the linked coauthored paper for the threat taxonomy. The product demonstrations are historical.
This work in the author’s profile
Published . Material checked .
-
Weng supplies a map for reading adversarial-attack research. Her survey defines what the attacker can access, then compares jailbreak prompts, gradient-based attacks and human or model red-teaming. Use that distinction to ask which system and threat a reported result actually covers before comparing defenses.
Start with this Web material · lilianweng.github.io
Start with Threat Model and the table of attack types, then Jailbreak Prompting. This is her survey of inference-time attacks; it does not cover training-data extraction or poisoning and does not attribute the surveyed experiments to her.
This work in the author’s profile
Published . Material checked .
-
Rehberger follows attacks from the content an agent sees to the action it takes. His AI ClickFix report shows a computer-use agent following webpage instructions into terminal execution. Compare it with his MCP report, where a code comment redirects private-message access into a public write, then inspect which permissions enabled each step.
Start with this Web material · embracethered.com
AI ClickFix: Hijacking Computer-Use Agents Using ClickFix
Follow the webpage-to-terminal sequence in his 2025 proof of concept, then read Mitigations. The report tested a particular Claude computer-use setup; it helps identify browser, host and network boundaries to inspect.
This work in the author’s profile
Published . Material checked .
Data boundaries & tool permissions
Map what an agent can read, change or send out. Separately, study when a model can reproduce sensitive text from its training data.
-
Biderman addresses a different data boundary: text a model may reproduce from training. Her coauthored study asks whether small models or early checkpoints can predict which strings a finished model will memorize. Start with the extraction definition and threat model, then the corrected results and limits before treating a preliminary test as privacy assurance.
Start with this Web material · arxiv.org
Emergent and Predictable Memorization in Large Language Models
In the coauthored paper, read sections 2.1–2.2 for what extraction means and the model-builder threat. Then read sections 6–7: the revised analysis and Pythia setting limit what smaller-model tests can establish about sensitive training strings.
This work in the author’s profile
Published . Material checked .
-
Willison gives developers a clear model of agent data leaks: combine private data, attacker-controlled content and a route to send information out. Begin with his lethal trifecta, then map the three capabilities to your tools. His later Rule of Two commentary adds state-changing actions and explains why reducing risk is different from proving safety.
Start with this Web material · simonwillison.net
The lethal trifecta for AI agents: private data, untrusted content, and external communication
Use the three-part diagram to identify private inputs, external content and outbound communication in an agent workflow. Continue to his November commentary for writes and state changes; the original trifecta focuses on data theft.
This work in the author’s profile
Published . Material checked .
Understanding model behavior
Inspect features and circuits, then see how interventions test an explanation of a particular model response.
-
Olah makes internal model behavior visible through features, circuits and interventions. Start with his coauthored visual introduction using a vision network, then the Claude 3.5 Haiku case studies of refusals and hallucinations. The method explains parts of selected responses; its limits are essential to judging what the explanation establishes.
Start with this Web material · distill.pub
Zoom In: An Introduction to Circuits
Begin with curve detectors and how their connections form a circuit. This coauthored introduction uses a vision model to explain the vocabulary. Continue to On the Biology of a Large Language Model for language-model interventions and methodological limits.
This work in the author’s profile
Published . Material checked .
-
Nanda helps you turn interpretability vocabulary into a testable question. His explainer distinguishes features, circuits and interventions. Continue to attribution patching to compare a changed prompt with a clean one and investigate which activations affect the answer. He treats the fast approximation as a way to generate hypotheses for stronger tests.
Start with this Web material · neelnanda.io
A Comprehensive Mechanistic Interpretability Explainer & Glossary
Read the General section’s definitions of feature, circuit and intervention before the technical entries. Then his attribution-patching article shows how changing activations tests a hypothesis. Python, tensors and transformer basics help with the experiments.
This work in the author’s profile
Published . Material checked .
A few bearings for your reading
These guides explain the people, language and questions you may meet along the way.
Prompt injection in AI agents: follow the tool calls
How this list is selected
We selected people through their own attack reports, explanations and coauthored research. Product reports keep their documented date and setup; model studies keep the tested model and method. Each card opens a chosen material, with publication and checking dates kept separate.