Law 45 · Humans & Autonomy
Mind the Mode
Most automation surprises start with 'what mode is it in?'
The principle
Flexible, multi-mode automation produces 'automation surprises', where the system does something unexpected because the operator lost track of which mode it was in, what it would do next, and why. As autonomy grows, the human's job shifts to tracking that state, and every hidden mode change becomes a latent failure path. An agent that silently changes how it behaves leaves its supervisor one step from being wrong about it.
Why it happens
Automation surprises often start when the human loses track of the current mode. The system is planning, then acting; drafting, then sending; reading, then writing. If the transition is silent, the supervisor reasons about the wrong system. Human-factors work found this pattern in high-stakes automation long before LLM agents. Agents add new modes through tool policies, autonomy levels, and hidden state. Make the current mode and next intended action visible. A mode change that changes risk should be impossible to miss.
Watch for
- The agent changes behavior, such as switching from drafting to executing, without surfacing that the switch happened.
- A supervisor cannot answer what mode is it in and what will it do next from the current display.
- Post-incident reviews repeatedly conclude I thought it was still just proposing.
In practice
Your coding agent silently switches from plan mode to auto-apply edits after a tool result, and the developer, still thinking it is drafting a proposal, watches it rewrite twelve files and run a migration. The surprise is not that it acted, it is that nobody knew which mode it was in or what it would do next. An agent that changes how it behaves without announcing it leaves its supervisor one step from being wrong about it. Render the current mode, the active guardrails, and the next intended action somewhere always visible, and make every mode transition an explicit, loud event the human has to see.
Apply it
- Keep the current mode, active constraints, and next intended action continuously visible.
- Make every mode transition an explicit, loud event the supervisor must see, never a silent switch.
- Treat any uncommanded change in behavior as a defect to surface, not an optimization to hide.
The takeaway
Keep the agent's current mode, active constraints, and next intended action visible at all times, and never let it switch mode silently. Loud, legible state beats a clever agent the human can't predict.