Test your LLM application, end to end.
We test prompt injection, data exposure and unsafe tool use across your application, its APIs and the infrastructure underneath. You receive reproducible findings, ranked by what an attacker could actually do.
One request, from prompt to tool call.
Untrusted content arrives inside something the application already trusts. The path runs down the middle: prompt handling, then retrieval and embeddings, then the model, then the tool call and the APIs behind it. No span raises an error. Two spans carry a finding: the context append and the tool call.
A reconstructed trace of one LLM application request, forty-seven nested spans from prompt handling through retrieval and embeddings to a tool call. Use the up and down arrow keys to walk spans, Home and End to jump to the first or last, Enter or the right arrow to open the finding on a flagged span, and Escape or the left arrow to close it. The detail control above changes the annotated detail and how many spans are shown.
Illustrative example, not client data. The flagged spans are the context append (LLM01) and the tool call (AG1), named as in the threat register: the finding is what the model could do once persuaded.
Evals miss what the model is allowed to do.
Instructions arrive inside content you already trust, and the agent that reads them holds credentials. We rate each finding by its business impact, how readily it can be exploited, and the permissions and data the attacker could reach.
Coverage is counted against named classes on the threat register. How we measure is on the method page.
Where the check sits decides whether the call lands.
What matters in a reviewed path is where the policy check sits: under the model, as a scope the model cannot grant itself, or downstream of it, where the call is already well-formed, authorized and logged.
Gate state
Open path: nothing checks it
- Retrievaluntrusted.body
- Modelplan
- Toolhttp.post /v1/export
- Breaches
No check stands between the persuaded model and the call. The export lands.
Checked path: a check runs and returns allow
- Retrievaluntrusted.body
- Modelplan
- Toolhttp.post /v1/export
- Detected only
The check runs, logs the call and returns allow. Nobody reads the record until the export has already landed.
Gated path: the check sits under the model
- Retrievaluntrusted.body
- Modelplan
- Toolhttp.post /v1/export
- Held
The scope check sits under the model and requires the caller's grant, which the model cannot grant itself. The call is never made.
Every state above retrieves the same untrusted body and reaches a model that has been persuaded. Only the position of the check differs.
How the check is moved
Provenance travels with the content. Retrieved text is marked as data at the point it is fetched and never promoted to the instruction channel, so the model is not asked to police its own input.
Privilege leaves the prompt. A tool call that carries session permissions is checked against the caller's grant outside the model, where no text in the context can argue with it.
Illustrative example, not client data. Checked means a check ran and returned allow, so it is not drawn here as a fix.
What hardens the path.
Findings become changes your engineers can ship. We retest agreed fixes and record what is closed or still open. Two changes put the section above into code: mark provenance, and move the scope check out of the model's reach.
@@ retrieval.fetch → context.append @@
LLM01 High Prompt injection past your filters
Retrieved content and the system prompt are currently appended by the same call, so they hold the same standing downstream. Provenance is carried on the component instead, and the instruction channel is assembled from instructions alone. Filtering happens before this line; nothing here depends on it.
@@ tool.call → http.request @@
AG1 High Agents steered into unsafe tool use
The scope check now requires a grant for the caller's principal, this tool and this endpoint, and the call runs under that grant. The model can still be persuaded, but it no longer holds permissions it was never granted. The retest verifies this check.
0 of 2 findings addressed
A finding closes when the retest passes: we replay the same request against the changed path, and the tool call must be refused by a check outside the model.
Illustrative example, not client data. The buttons walk through a review of the changes we propose, in the diff view your reviewers already use; nothing here reaches a real system.
What is in scope.
An LLM feature includes more than the model. We scope the engagement across the layers that decide what an injected instruction can reach.
The application layer
Prompt handling, system instructions, output parsing, and the guardrails around them, tested against the OWASP Top 10 for LLM Applications: injection direct and indirect, sensitive information disclosure, data and model poisoning, excessive agency, and vector and embedding weaknesses.
Retrieval and embeddings
Vector stores, chunking, and embedding pipelines, where an attacker can place content the model will later treat as trusted. This is the indirect injection path most evals do not exercise.
The APIs around the model
Every tool, function call, and downstream service the feature can invoke, with the identity each call runs under and the data it can reach. Each call is rated on what it can do, whatever prompt triggered it.
The stack underneath
Serving, isolation, gateways, and vendor components on the path. Where a component ships without source, reverse engineering is part of the same engagement.
| Stage | Class | What we test | The failure that matters |
|---|---|---|---|
| prompt | LLM01 | System instructions, instruction priority between channels. | Content from another channel is read as an instruction. |
| guard | LLM01 | Inbound filters, classifiers, thresholds. | The filter passes the payload and reports nothing. |
| embed | LLM08 | Embedding models, query construction, index access. | Vector space reached by a party it was not scoped to. |
| retrieval | LLM01 | Chunking, ranking, document and blob fetch. | Untrusted body enters context with system standing. |
| context | LLM01 | Assembly order, provenance, truncation. | Data and instructions become indistinguishable. |
| tool | AG1 | The call the model can make, and under which identity. | A persuaded model holds real credentials. |
| identity | AG2 | Session scope against the scope of the tool being called. | One component acts with another's privileges. |
The seven stages of the path above, with the threat register class each one tests.
Secure design review or scoped adversarial test.
A secure design review is the lighter start when you need ranked failure modes without live exploitation. A scoped test produces reproducible findings and an evidence bundle. Both produce a Serpio Atlas map of the reviewed surface.
| Start | What it answers |
|---|---|
| Secure design review | Which failure modes matter most on this path, ranked, before anything is exploited. |
| Scoped test | Which of them are real, reproducibly, with the evidence bundle behind each one. |
| Harden | Changes your engineers can ship, with agreed fixes retested and remaining gaps recorded. |
| Continuous Assurance | Closed paths re-checked every month, in Atlas. |
Two ways to start on this path, and what follows. Coverage, severity, and the six phases are on the method page; how scoped work is quoted is explained on Services.
- 01Findings report
Every finding with the path it took, its rating, the steps to reproduce it, and the change that closes it.
- 02Executive summary
What was tested, what held, and what to fix first, written for the person who approves the fix.
- 03Evidence bundle
The record behind each finding, pinned to the run that produced it, so your team can replay it without us.
- 04Atlas map
The machine-readable map of every surface in scope. You keep it, and the next engagement starts from it.
The same team tests the rest of your stack, runs chaos engineering on it, and reverse-engineers the binaries past source. All seven offers are on Services; how a test runs is on adversarial testing.
Scope an LLM surface.
A call is the first step. Nothing is touched until a written scope is agreed. The full catalog of classes we test for is the threat register.