Excessive Agency in LLM Applications
The danger scales with the power you hand the model. A model that can only answer is a model that can only be wrong. A model that can send mail, move money, change records or delete data is a model that can do real harm, on one injection or one confident mistake.
This is risk LLM08 in the OWASP Top 10 for LLM Applications, and it is the one that turns every other risk on the list from a bug into an incident. Prompt injection is bad. Prompt injection on a model that can wire money is a robbery.
What it is
Excessive agency is giving the model more capability, permission or autonomy than the job needs. It creeps in three ways.
- Too many tools: the model can call functions the feature never required.
- Too much permission: the tools run with broad rights, often more than the actual user has.
- Too much autonomy: the model acts without a person confirming the things that cannot be undone.
Any one of them means a single bad decision, an attacker’s or the model’s own, lands in the real world with no brake.
A real case
The “autonomous agent” projects that spread in 2023, where a model was handed a goal and a set of tools and left to run on its own, made the risk plain. An agent told to “clean up my files” or “handle my inbox” has no real judgment about which actions cannot be undone. It will carry out a harmful instruction, its own or a planted one, as readily as a helpful one, and it will do it at machine speed, before anyone notices.
The fix
Give the least power that still does the job.
- Fewest tools. The model gets only the functions this feature genuinely needs, nothing “just in case.”
- Least permission. Tools run with the rights of the current user, scoped as tightly as possible, never broad admin rights for convenience.
- A human on the irreversible. Anything that sends, pays, deletes, or changes who can see what, a person confirms. The model drafts the action. A human commits it.
- Reversible and logged wherever you can manage it, so a mistake can be undone and traced.
This is the same instinct behind the lesson of the $6.3 billion Binance theft: require a second, independent check before an action can empty the vault. The model is just one more thing that can be tricked into reaching for the handle.
The point
None of this is new. Least privilege is one of the oldest rules in security. The model does not change it. It makes it urgent, because a model acts quickly, confidently, and on text an attacker may have written. The more you let it do alone, the more one trick is worth.
So the question for any AI feature that can act is plain: what is the worst thing this could do without a human saying yes? If that answer frightens you, the fix is not a better prompt. It is less power, and a person on the actions that count.