AI Agent Incident Response: What to Do When an Agent Misbehaves

When an AI agent misbehaves, the first priority is to limit impact and preserve evidence. A calm incident process is more useful than trying to make the agent explain itself while it is still able to call tools or reach users.

Contain the active path

Pause the risky workflow, revoke or narrow affected credentials, disable a tool, reduce audience exposure, or route requests to a human. Choose the smallest safe control that stops further harm. If the issue involves personal data, external messages, money, or irreversible actions, use the incident severity rule already agreed with the service owner.

Preserve a useful record

Record the time window, users or workflows affected, model and prompt versions, retrieved sources, tool calls, approvals, outputs, configuration changes, and actions taken. Protect the record because it may contain sensitive content. Do not overwrite the evidence while attempting a quick fix.

Recover and learn

Confirm that the immediate risk is closed, replay representative cases in a safe environment, and compare the result with the last known-good version. Classify the cause: source change, prompt injection, permission failure, tool error, model change, routing bug, monitoring gap, or human process failure. Add a regression case and an owner before restoring the workflow.

• Publish a clear human fallback for affected users.
• Keep rollback and credential-revocation steps tested.
• Separate customer communication from technical speculation.
• Review whether the incident reveals a missing approval or test.

The agent safety boundaries guide helps prevent severe actions, while the workflow testing guide helps turn the incident into a durable check.