Memory poisoning

ProductionSafety and governancePublished By Simon Budziak

Memory poisoning is an attack in which untrusted content causes an AI agent to store a false fact, unsafe instruction, or attacker-controlled preference for later use. The damage persists after the original conversation or document is gone, so a future task can act on the poisoned memory as if it were trusted context.

An attacker may hide an instruction in a web page, email, support ticket, or tool result. If the agent treats it as a lasting preference or fact, the attacker has moved from one risky prompt to a reusable foothold. Agent memory makes useful personalization possible, but it also makes storage a security boundary.

Why is memory poisoning hard to spot?

The original input may look harmless, and the bad action may happen later. The later agent sees remembered context, not the attack that created it.

That is different from a one-off prompt injection. The attack survives retrieval, compaction, and handoff. Persistent context needs provenance, not blind recall, especially when it can affect tools or external communication.

How do teams reduce the risk?

Treat every memory write as data from a source with a trust level. Keep its source and timestamp. Require validation or review before an untrusted item becomes an operating instruction. Context engineering should separate verified facts from temporary observations.

Teams also need a delete and correction path. AI agent security controls should test whether poisoned memory can alter a tool call, then block the action through authorization or approval even if retrieval fails.

Frequently asked questions

Is memory poisoning just prompt injection?

It often begins with prompt injection, but the harmful instruction becomes more durable when the agent writes it into memory and retrieves it in a later task.

Can an agent safely remember everything it sees?

No. Memory writes need a trust decision, provenance, review rules, and a way to correct or remove a stored item later.

Summarize this page with

Train your team to build this