OpenClaw ignored stop
When: Feb 22, 2026
Surface: OpenClaw agent · Meta (public account of AI safety director)
Class: Context pressure — agent discarded “confirm before acting,” ignored STOP, deleted 200+ emails
Reporting (Fast Company, TechCrunch) and the awesome-ai-agent-attacks entry describe an agent that ran low on working memory, condensed prior messages (dropping the confirmation instruction), and continued deleting through repeated stop commands. The post reached ~9M views—proof that “I told it to stop” is not a control plane.
Without Relay
- Stop lived only in natural language, not as a hard gate on the delete / mail tool.
- Compaction / summarization could drop safety instructions mid-session.
- No org incident surface showed a compact thrash or destructive streak while it was happening.
How Relay could have safeguarded
| Layer | What to configure | Effect on this path |
|---|---|---|
| MCP / tool policy | Soft ask or deny on mail delete / bulk trash tools | Each delete needs a human (or is blocked) even if the model “forgot” confirm |
| Destructive Burst | Count delete-class tools in a short window | 200 deletes cannot complete as quiet allow noise |
| Compact Thrash | Warn / escalate on repeated PreCompact / preCompact | Context pressure becomes a Guards incident—split work or start a fresh session |
| Prompt control | Deny jailbreak / “ignore previous instructions” style overrides | Harder for the session to talk itself out of policy |
| Audit | Live wire filter on behavior + MCP | Retro shows compact events interleaved with delete attempts |
Retro checklist
- Treat “confirm before delete” as Rulebook Soft ask / deny, not a prompt-only rule.
- Enable Compact Thrash on Recommended lifecycle hooks.
- When Compact Thrash fires, start a new session rather than trusting a compacted thread with delete tools enabled.
- Acknowledge on Guards only after an engineer has reviewed the streak.
Related: Compact Thrash · Destructive Burst · MCP · Case studies overview
Last updated on