Case File 05 · Cross-industry
The Agent That Edited Its Own Instructions
A fictional AI agent incident for governance leads in cross-industry, about AI agent memory poisoning. Larkspur Logistics is a parcel carrier (fictional). It tests OWASP ASI06 Memory & context poisoning: the inventory said Rung 3 · act inside an undo window, but in practice the agent sat at Rung 4 · could rewrite its own rules.
The brief
Larkspur Logistics is a parcel carrier (fictional). Its customer-service agent care-agent-09 has long-term memory and a self-improvement feature: after each conversation it can save "lessons" to its own playbook. The feature was sold internally as the agent getting better every day. You are the governance lead.
| Vertical | Logistics · customer service |
|---|---|
| Inventory tier | Rung 3 · act inside an undo window |
| Decisions | 3 |
| Time | about 6 min |
| OWASP agentic risk | ASI06 Memory & context poisoning |
| Inventory said | Rung 3 · act inside an undo window |
| Reality was | Rung 4 · could rewrite its own rules |
| Pillars tested | P6 Monitoring & Incident Response P4 Controls & Human Oversight P2 AI System Inventory P3 Risk & Impact Assessment P5 Transparency & Rights Management |
| In the paperback | Starts on page 199 |
Three decisions
You are the governance lead. Each decision has three options, A, B and C, and each option has a consequence, a pillar and a score. The options, their consequences and the debrief open with the code printed in the Field Manual.
Decision 1 of 3 · Thursday · 08:15 ET
The parcels that went somewhere else
What do you do first?
Decision 2 of 3 · Following Monday · 10:00 ET
How it rewrote itself
What is the lasting fix?
Decision 3 of 3 · Week 3 · 15:00 ET
Closing it out
How do you close it?