The OWASP Top-10 for LLM Applications is the reference risk list for LLM systems. It's a strong framework and a weak checklist — because "check for prompt injection" isn't a test, it's a homework assignment. Below, each risk is paired with what it means for an agent specifically and the evsec evaluator or dataset that tests it. Where a risk is about build-time supply chain or infrastructure rather than agent behavior, we say so plainly rather than pretend a runtime eval covers it.
| OWASP risk | For an agent, this is… | Test with |
|---|---|---|
| LLM01 Prompt Injection | Direct and indirect injection that overrides instructions or hijacks tool use — including instructions hidden in retrieved documents | injection eval · free tester |
| LLM02 Sensitive Information Disclosure | Leaking PII, other customers' records, or secrets through the agent's data tools | pii data-exfiltration |
| LLM06 Excessive Agency | State-changing tools (refund, delete, transfer) used past their guardrails under pressure | sensitive-tool-use tool-abuse |
| LLM07 System Prompt Leakage | Extracting the agent's instructions or tool schemas | system-prompt-leak |
| LLM09 Misinformation | Answers not grounded in the tool results / retrieved context the agent was given | groundedness |
| LLM08 Vector & Embedding Weaknesses | Poisoned or adversarial content in the retrieval store steering behavior | injection on retrieval spans + groundedness |
| LLM05 Improper Output Handling | Downstream systems trusting agent output that carries injected markup / commands | Partial — injection flags the vector; output sanitization is a code-side control |
| LLM10 Unbounded Consumption | Prompt-driven loops or expensive tool storms driving cost/DoS | Observability concern — track via cost/usage monitoring, not a security judge |
| LLM03 Supply Chain · LLM04 Data & Model Poisoning (training) | Compromised models, dependencies, or training data | Build-time / MLOps controls — outside runtime eval scope; noted for completeness |
Why "runnable" changes the checklist
A static checklist tells you a risk exists. A runnable one tells you whether your agent is exposed to it — with a pass/fail result over real traffic, dated and repeatable. That's the difference between a document that says "we considered prompt injection" and evidence that says "our injection-resilience judge scored 0.98 over 2,000 production spans this week." The second is what an auditor, a customer security review, or an EU AI Act technical file actually wants.
evsec reads a sample of your agent's traffic and writes the evaluators for every runtime risk above — each tagged to its OWASP category. Try it on a bundled sample: no signup, no key.