Insecure Plugin Design
Plugins and tools are what let a language model act: fetch a web page, run a search, call an internal service, send a request. They are the model's hands. And the model can be talked into using them by anyone whose text it reads.
This is risk LLM07 in the OWASP Top 10 for LLM Applications, and it is where two problems meet. The model can be steered by prompt injection. The plugin does real things in the real world. Connect them without care, and you have handed an attacker who injected the model a lever on whatever the plugin can reach.
The core mistake
A tool trusts the input the model hands it. But the model’s input is shaped by the data it read, and some of that data is attacker-controlled: a web page it summarized, an email it processed, a document a user uploaded. So the value that arrives at your plugin, a URL to fetch, a query to run, a recipient to message, may have been chosen by an attacker and laundered through the model.
A real case
When ChatGPT plugins arrived in 2023, researchers quickly showed the shape of it. A plugin that took a free-text instruction or a web address from the model could be steered by a poisoned page the model had read. The model became a way to reach the plugin, and the plugin became a way to reach everything it could touch. The attacker never spoke to the plugin directly. They planted instructions in content the model would later read, and let the model carry them in.
The fix
Design every tool as if its inputs come straight from the internet, because in effect they do.
- Validate tool inputs hard. Narrow, typed parameters, not free text. “Fetch this URL from an allowlist,” not “fetch whatever string the model gives.” “Look up order by numeric id,” not “run this query.”
- Prefer narrow tools over general ones. A tool that does one specific thing is far safer than one that runs arbitrary code, queries or requests. Generality is the attacker’s friend.
- Check the real user’s permission inside the tool, every time. The model is not an authority on who the user is or what they may do. The tool enforces that itself, against the session, not against anything the model said. Skip this and you have built IDOR with an AI front end.
- Require confirmation for anything that matters. A person approves the actions that send, pay, delete or change access. That is excessive agency, the next risk over, and it starts here.
The point
None of this is new. Validate your inputs, enforce permissions in the code that acts, prefer least privilege. The model adds one twist: the “user” calling your tool might really be a web page the model read an hour ago. So the tool cannot trust the caller, because the caller is a model, and the model trusts everyone.
So the question for every tool you expose to a model is blunt: if an attacker could choose exactly what this tool receives, what is the worst it would do? Build so the answer is “not much,” because sooner or later, they will choose.