The OWASP Top 10 for LLM Applications, Explained
OWASP named the ten biggest risks in software built on language models. Here is what each one means, with a real case and the main defense.
For twenty years, the OWASP Top 10 has been the starting point for web application security. It is a plain list of the ten most serious risks, kept by a volunteer group, used by teams all over the world. In 2023, as companies rushed to build products on top of ChatGPT and similar models, OWASP published a new list for this new kind of software: the Top 10 for Large Language Model Applications. The first version came out on August 1, 2023, and a revision followed in October.
A language model does not remove the old risks. It adds a new part to your system that can be talked into doing the wrong thing. The list below is the plain-English version of each risk, with a real example from the past year or so, and the single defense that matters most. The companion to this piece, Securing an LLM Application: A Practical Checklist, turns these into checks you can run before you ship. Each risk below also links to a full piece on that one, if you want to go deeper.
LLM01: Prompt injection
The model cannot tell your instructions apart from the text it is reading. If that text contains instructions of its own, the model may follow them. The direct form is a user who types “ignore your rules.” The dangerous form is indirect: instructions hidden in a web page, an email or a document that the model reads later.
A real case. In February 2023, a Stanford student got Microsoft’s new Bing Chat to reveal its hidden rules and its internal name, “Sydney,” simply by asking it to ignore its previous instructions. The same month, researchers showed that instructions planted in a web page could hijack a model that later read that page, with the user never seeing them.
The main defense. Treat everything the model reads as untrusted, and everything it returns as untrusted. There is no complete fix, so limit what a successful injection can reach. I cover this in full in Prompt Injection: When the Data Gives the Orders.
LLM02: Insecure output handling
Teams check what goes into the model and forget what comes out. But the model’s output is just more untrusted text. If it flows into a web page, a database, a shell or a system that runs code, it can carry an attack, either one the attacker injected or one the model produced by accident.
A real case. Through 2023, researchers showed that a model could be made to answer with a Markdown image whose web address carried the user’s private conversation inside it. The moment the page displayed the image, the browser sent that data to the attacker’s server. The attack rode out on ordinary-looking output.
The main defense. Show model output as plain text, not as HTML. Never build SQL, shell commands or file paths by pasting the model’s words into them. Encode output for wherever it is going, exactly as you would for any user input.
Deep dive: Insecure Output Handling.
LLM03: Training data poisoning
If you train or fine-tune a model, whoever controls part of the training data controls part of the model’s behavior. A small number of poisoned records can plant a hidden flaw or a bias that only shows up under certain conditions.
A real case. In July 2023, a company called Mithril Security showed a proof of concept they named PoisonGPT. They took an open model, edited it to spread one specific piece of false information, and uploaded it to a popular model hub under a name close to a trusted one. A developer who downloaded it would have shipped the lie inside their product without knowing.
The main defense. Know where every training source comes from and keep that record. Test a tuned model against a fixed set of questions before and after each training run, and review what changed.
Deep dive: Training Data Poisoning.
LLM04: Model denial of service
Language models are slow and are priced by the amount of text they process. A small number of large or cleverly shaped requests can use up your capacity, your budget, or both. An attacker does not need to break anything. They just make you do expensive work.
A real case. This is less a single headline than a steady drain seen across 2023: automated clients sending very long inputs, or triggering chains of calls that feed the model’s output back into itself again and again, running up the bill with every loop.
The main defense. Cap the size of every input before it reaches the model, cap the length of every reply, limit how often one user can call, and set a daily spending limit with an alert. Limit how many times any automated chain may repeat.
Deep dive: Model Denial of Service.
LLM05: Supply chain vulnerabilities
Models, datasets and the libraries around them are dependencies, like any other code you pull in, but with far less mature tooling to check them. A poisoned model, a tampered dataset or a malicious package can enter your product through the front door of a normal install.
A real case. In December 2022, attackers used a trick called dependency confusion against the PyTorch machine learning library. They published a malicious package named torchtriton to the public index, and installs of the nightly build pulled the attacker’s package instead of the real one, running code that stole system information.
The main defense. Pin the exact versions you tested. Download models only from sources you trust and check their fingerprints. Prefer the safetensors format for model files, because the older format can run code the moment it is loaded.
Deep dive: The AI Supply Chain.
LLM06: Sensitive information disclosure
A model can reveal what is in its prompt, what is in its current context, or, sometimes, fragments of what it was trained on. The data your own staff paste into a public tool is part of this risk too.
A real case. In the spring of 2023, engineers at Samsung pasted source code and an internal meeting transcript into ChatGPT to get help. That text left the company and sat on an outside provider’s servers. Samsung restricted the use of generative AI tools on company devices soon after.
The main defense. Never put secrets or internal details in a prompt, and assume prompts will leak. Fetch only the data the current user is allowed to see, and enforce that in your own code, never by asking the model to keep a secret. I go deeper in Before You Paste Company Data into an AI Chatbot.
LLM07: Insecure plugin design
Plugins and tools are the model’s hands. They let it fetch a web page, run a search, or call an internal service. The inputs to those tools come from the model, which means they can come from an attacker who has injected the model.
A real case. When ChatGPT plugins arrived in 2023, researchers quickly showed that a plugin which took a free-text instruction or a web address from the model could be steered by a poisoned page the model had read. The model became a way to reach the plugin, and the plugin became a way to reach everything it could touch.
The main defense. Validate every value a tool receives as if it came straight from the internet. Prefer narrow tools with fixed, typed inputs over general ones such as “run this query” or “fetch this address.” Check the real user’s permission inside the tool, every time.
Deep dive: Insecure Plugin Design.
LLM08: Excessive agency
This is the risk of giving the model too much power. When a model can act on its own, by sending email, changing records, moving money or deleting data, a single successful injection or a single confident mistake turns into real damage in the world.
A real case. The “autonomous agent” projects that spread in 2023, where a model was handed a goal and a set of tools and left to run, made this plain. An agent told to “clean up my files” or “handle my inbox” has no real judgment about which actions cannot be undone, and will carry out a harmful instruction as readily as a helpful one.
The main defense. Give the model the least power that still does the job, and only the permissions of the current user. Require a person to confirm anything that sends, pays, deletes or changes who can see what.
Deep dive: Excessive Agency.
LLM09: Overreliance
The model produces fluent, confident answers that are sometimes wrong. The risk is not only technical. It is people and processes that stop checking because the output looks finished.
A real case. In June 2023, a federal judge fined two New York lawyers after they filed a brief citing six court decisions that did not exist. ChatGPT had produced them, and when asked whether they were real, it said yes. Earlier that year, the technology site CNET had to correct a series of articles written with an AI tool after readers found factual errors in them.
The main defense. Show the source for every factual claim and make it easy to open. Require human review wherever an error has real cost. I cover the checking routine in Why Language Models Make Things Up.
LLM10: Model theft
If you trained or tuned your own model, the model file is a valuable asset, and so is the behavior visible through your API. Attackers may steal the file outright, or copy the model’s behavior by querying it at scale and training a cheaper copy on the answers.
A real case. In March 2023, the weights of Meta’s LLaMA model, which had been shared only with approved researchers, leaked publicly and spread across the internet within days. A model meant for a controlled group was suddenly in everyone’s hands, which is exactly the outcome a company with a private model wants to avoid.
The main defense. Protect model files like source code and keys, with access control and logs. Rate limit and watch your API for patterns of mass querying, and do not return more detail in each answer than the product actually needs.
Deep dive: Model Theft.
How to use this list
Do not read this as ten separate problems. Read it as one message in ten parts: a language model is a new component that can be influenced by the very data it works on, so you cannot trust it the way you trust your own code. Everything else follows from that. You validate its output because it can be turned against you. You limit its power because it can be tricked. You check its facts because it does not know when it is wrong.
None of the defenses are new. Least privilege, input validation, output encoding, rate limiting and knowing your supply chain have protected software for decades. The model does not replace them. It makes them matter more. When you build your own list of checks, start from the practical checklist, and keep the full OWASP project open next to it.