Traditional incident response still provides the foundation, but agents introduce model prompts, retrieved data, memory, tool calls, approvals, and external side effects as evidence. The NIST AI Risk Management Framework organizes risk work around govern, map, measure, and manage functions that can anchor an agent response plan.
What counts as an AI agent incident?
An incident is not limited to a security breach. It can include unauthorized access, disclosure of sensitive data, harmful messages, incorrect transactions, uncontrolled cost, persistent policy failure, or a workflow that continues after its approved boundary. Severity should follow the real external effect, not how unusual the model output looked.
Define reportable events before launch. Otherwise teams may dismiss repeated tool errors as quality problems even when customers, data, or money are affected.
How should teams detect and contain agent incidents?
LLM observability should connect model decisions to tool requests, results, identities, approvals, and business outcomes. Alerts need enough context to identify the affected agent version and resources. Responders then activate an agent kill switch, revoke credentials, isolate queues, or disable a tool route. Containment must stop external actions while preserving evidence.
Do not erase memory, logs, or pending work before investigators capture what happened. If customer harm may continue, containment takes priority, but every stop action should still be recorded.
What evidence is needed for investigation?
Preserve prompts, model and tool responses, policy decisions, workflow state, configuration versions, identities, approval records, and timestamps. The agent audit trail should make it possible to reconstruct which promise led to which effect. A transcript alone cannot prove what the external system changed.
Investigators should compare intended behavior, observed state, and downstream consequences. They also need to separate the initiating cause from control failures that allowed the event to spread.
How should recovery and learning work?
Agent recovery begins only after the team understands the affected state and has a safe restoration plan. Repair data, reconcile transactions, notify affected parties where required, rotate credentials, and validate the fix against a representative regression set. Reopening requires evidence that the harmful path is blocked.
Update AI agent security controls, policies, evaluations, and operating procedures based on the root cause. Run tabletop exercises with named owners so authority is clear before a real event. The first incident is too late to decide who may stop the system or speak to customers.