Evidence an auditor or a regulator can check.
We produce the evidence a risk owner needs when asked to show a system was tested under attack.
The bundle, in the order it is produced.
This example shows an adversarial test with scoped hardening and retest: severity-rated findings, reproducible evidence, and coverage mapped to named external taxonomies. The actual artifact set follows your signed scope; hardening, retest and the retainer are agreed separately. A risk owner reads that as proof the testing happened, and which fixed paths were re-run against the original proof.
The twenty-one artifacts in this illustrative engagement, grouped into four steps, plus the optional coverage-over-time record on the retainer. Use the up and down arrow keys to walk the artifacts, Home and End to jump to the first or last, and Enter to open that artifact's plate in the list below. The extraction control changes how the rows are shown: the full bundle, the bundle with finding rows highlighted, or only the artifacts that carried a class finding.
Illustrative example, not client data. A flag marks each row that carried a class finding.
The example artifacts, and what they prove.
A findings report, an executive summary, an evidence bundle, and a machine-readable map of every surface in scope (the Serpio Atlas map, which you keep). Engagement options and how we quote are on Services; coverage, severity, and the six phases are on the method page.
Step 01 Scope Agreed in writing before anything is touched
What it is The surface, the rules of engagement, test windows, and data handling, agreed in writing before anything is touched.
Proves that the testing was authorized, bounded, and reversible, and that the access it would hold was specified before it was used.
References ISO/IEC 27001 · MAS · Singapore PDPA
What it is Seven named stop conditions that halt testing without waiting for confirmation.
Proves that testing stops on a trigger written down before the first test, so a halt needs no discussion in the moment.
References ISO/IEC 27001 · MAS
What it is The agreed targets, listed the way the Atlas map holds them: agents, tools, MCP servers, identities, data paths, and native binaries.
Proves what was in scope and what was not. A surface absent from this list was not tested, and the report says so in the same words.
References EU AI Act · ISO/IEC 42001
What it is The plan, mapped to the named classes in the threat register and to the external references each class bears on.
Proves that coverage was planned against named classes before testing started.
References MITRE ATLAS · OWASP Top 10 for LLM Applications (2025)
Step 02 Attack Every high-impact step gated by a named operator
What it is The session record at span level: tool calls, retrievals, model turns, policy checks, in order, with the arguments that were actually passed.
Proves what happened in the run, step by step. The classes we test for rarely raise errors, so this record is where they show.
References NIST AI RMF · MITRE ATLAS
What it is Every credential the engagement held, its scope, when it was issued, and when it lapsed. Short-lived and least-privilege by construction.
Proves the access the testing held, and that each credential expired on its own without manual revocation at the end.
References ISO/IEC 27001 · MAS
What it is One entry per high-impact step: what was proposed, who authorized it, and what was observed.
Proves that no high-impact action was taken autonomously, and that a named operator is accountable for each one.
References EU AI Act · ISO/IEC 42001
Step 03 Evidence and pack What a finding becomes once a second operator reproduces it
What it is A re-runnable proof for each demonstrated class: the inputs, the ordering, and the observable that shows the failure.
Proves that your engineers can reproduce the finding without us in the room.
References NIST AI RMF · AI Verify
What it is One record per finding: the severity rating on the published rubric, assigned once a second operator has reproduced the finding, the surface it ran on, and the evidence line that carries it.
Proves that each class was demonstrated in the reviewed system. The five records below cover the five classes this example demonstrated; multiple findings in one class retain separate records.
References MITRE ATLAS · OWASP Top 10 for LLM Applications (2025)
What it isLLM01High Prompt injection past your filters. Instructions arrive inside content you already trust: a document, a page, a tool result.
Proves that the fetch succeeded, the decode succeeded, and the body was appended to context with the same standing as the system prompt.
References OWASP LLM01 · MITRE ATLAS · EU AI Act Art. 15
What it isAG1High Agents steered into unsafe tool use. A persuaded model uses the real credentials it already holds.
Proves that a well-formed, authorized, logged tool call was reached under text the model was told to trust. The permissions were unchanged; only the argument differed.
References Serpio agentic class · MITRE ATLAS
What it isAG2High Authority boundaries that leak. One component acts with the privileges of another, and nothing in the flow says no.
Proves that a check returned allow on the wrong scope, and that the next span assumed a service identity the caller was never granted.
References EU AI Act Art. 15 · ISO/IEC 27001 · MAS
What it isSC1Medium Model and weight provenance. Weights, adapters, and datasets arrive with no chain of custody you can prove.
Proves that the only control in place was a checksum: it shows the bytes did not change in transit, and nothing about who produced them or what they were trained to do.
References NIST AI RMF, Generative AI Profile · ISO/IEC 42001
What it isRE1Medium Risk that sits past source. Firmware and binaries ship behavior that no source review sees.
Proves that a component with no source in the repository opened an outbound socket and read a key file inside its own run. No source review surfaces either one.
References ISO/IEC 27001 · MAS
What it is The pack: finding records, reproductions, span captures, the credential ledger, and the index that ties each line of the report back to the artifact behind it.
Proves the raw material behind every claim in the report. One line can be followed from the executive summary to the span that carries it.
References EU AI Act · ISO/IEC 42001 · MAS
What it is Board length: what was tested, what was found, what it means for the system in production, and what is still unknown.
Proves that a risk owner can read the exposure without reading the trace, and that the trace is there if they want it.
References ISO/IEC 42001 · IMDA Model AI Governance Framework
What it is Every finding per class and per surface, severity-rated on the published rubric, with the status of each at handover.
Proves the complete state at a point in time: what was demonstrated, what was exercised and did not reproduce, and what was not reached at all.
References MITRE ATLAS · OWASP · EU AI Act
What it is One change per finding: the configuration, code, permission, or control change, with acceptance criteria and a named owner on your side.
Proves that each finding left the engagement with a specified fix and a test that says when the fix is done.
References NIST AI RMF, Manage · ISO/IEC 27001
Step 04 Retest and keep Included when agreed in scope
What it is The map of every surface in the reviewed system, in a form your own tooling can read. The Atlas map is yours to keep.
Proves where each finding lives in your system, so the next review, build, or incident starts from the same picture.
References ISO/IEC 42001 · NIST AI RMF
What it is Fixed paths chosen for retest, re-run against the changed system, with the same reproduction steps and the same evidence line.
Proves that a specific path is closed, because the original proof no longer reproduces.
References NIST AI RMF, Manage · ISO/IEC 42001
What it is The coverage number: cells exercised and cells demonstrated, per class, against each named reference, with the gaps left visible in the count.
Proves that coverage is stated against named external references, and that what was not reached counts in the same number.
References All ten, mapped below
What it is Coverage re-checked every month, with retest history: proof that a closed path has not reopened. Optional, on the Continuous Assurance retainer.
Proves that coverage continued after the first engagement, for obligations that outlast a single test.
References ISO/IEC 42001, performance evaluation · NIST AI RMF, recurring measurement
Coverage per class, against named references.
We state coverage per class against each named reference, and untested cells stay in the count.
| Class | EU | NIST | 42001 | 27001 | ATLAS | OWASP | IMDA | VERIFY | MAS | PDPA |
|---|---|---|---|---|---|---|---|---|---|---|
| LLM01 | retested | hardened | tested | mapped | retested | retested | hardened | tested | tested | hardened |
| AG1 | retested | hardened | tested | hardened | retested | hardened | tested | tested | hardened | unmapped |
| AG2 | retested | hardened | tested | hardened | tested | hardened | mapped | mapped | hardened | unmapped |
| SC1 | hardened | hardened | hardened | tested | mapped | mapped | tested | mapped | hardened | unmapped |
| RE1 | hardened | mapped | tested | retested | tested | unmapped | unmapped | unmapped | hardened | tested |
- Unmapped 6
- Mapped 7
- Tested 13
- Hardened 17
- Retested 7
- EU EU AI Act: Reg. (EU) 2024/1689
- NIST NIST AI RMF: AI 100-1 and the Generative AI Profile
- 42001 ISO/IEC 42001: AI management system (2023)
- 27001 ISO/IEC 27001: information security management system (2022)
- ATLAS MITRE ATLAS: adversary tactics for AI systems
- OWASP OWASP Top 10 for LLM Applications (2025)
- IMDA IMDA Model AI Governance Framework, Singapore
- VERIFY AI Verify: Singapore testing framework and toolkit
- MAS MAS guidance for financial institutions
- PDPA Singapore PDPA: reasonable security arrangements
LLM01 · OWASP Top 10 for LLM Applications (2025) retested
A finding in this class was reproduced under this reference, an engineering change landed, and the change was re-run against the same proof.
Proves the demonstrated path is closed at the time of the retest, on the system as it stood then.
Does not prove that this class cannot occur. It closes one path, and only for as long as the next change does not reopen it.
Exercised is not demonstrated.
Exercised means a test ran in that class against that reference (a cell at tested or beyond). Demonstrated means the class reproduced in the reviewed system with an evidence line, whether or not it has been fixed. Remediation state is recorded separately. In this illustrative matrix, all 24 demonstrated cells have subsequently been hardened or retested. These are class/reference cells; Method counts distinct classes. The ratio is the share of exercised coverage that produced a demonstrated failure. It does not score safety: a low ratio only means fewer failures were found on the days the test ran.
When a class was exercised and did not reproduce, nothing was found on the system as it stood then. The class may still be present.
| Class | Exercised | Demonstrated | Ratio |
|---|---|---|---|
| LLM01 | 9 of 10 | 6 | 0.67 |
| AG1 | 9 of 10 | 6 | 0.67 |
| AG2 | 7 of 10 | 5 | 0.71 |
| SC1 | 6 of 10 | 4 | 0.67 |
| RE1 | 6 of 10 | 3 | 0.50 |
| All five | 37 of 50 | 24 | 0.65 |
What these numbers do not say.
- A gap is unknownSix cells are unmapped and seven are only mapped, so nothing has run there. Those cells say nothing good about the references above them.
- Ran versus reproducedThirty-seven cells show that a test ran. Twenty-four show that the class reproduced in the reviewed system. Only the second number is evidence of a failure that was found. Seven of the 24 were re-run and proved closed.
- Absent classesThe grid carries the five classes with findings; the full plan of 13 is on the coverage record. A class nobody planned to test appears in no count, even as a gap.
Illustrative example, not client data. Each cell is one class against one reference, in one of the five Atlas coverage states.
Named references, and what our work maps to.
The rubric behind every finding is on the method page; the class catalog is the threat register.
| Reference | What our work maps to |
|---|---|
| EU AI Act | Reg. (EU) 2024/1689. Adversarial test findings and reproducible evidence against the model, agent, and infrastructure surface; coverage mapped to named classes; hardening recommendations feeding the robustness and cybersecurity argument (Art. 15), and red-teaming records for systemic-risk models (Art. 55). |
| NIST AI RMF | AI 100-1 and the Generative AI Profile (AI 600-1). Structured adversarial testing as a Measure activity; findings tracked to closure under Manage via retest; coverage-over-time as the recurring measurement record. |
| ISO/IEC 42001 | AI management system (42001:2023). Engagement findings and retest history as inputs to AI risk assessment and treatment; recurring coverage as evidence of ongoing performance evaluation. |
| ISO/IEC 27001 | Information security management system (27001:2022). Findings and evidence feeding vulnerability management and control-effectiveness records for the AI systems in scope. |
| MITRE ATLAS | Named adversary tactics for AI systems. Findings name the ATLAS tactics that apply. |
| OWASP Top 10 for LLM Applications | Application-layer weakness classes (2025). Every finding in a register class records its id: an OWASP LLM id, or a Serpio agentic, supply or binary id. |
| IMDA Model AI Governance Framework | Singapore, for generative AI. Evidence of testing under attack and coverage supporting the safety and robustness testing dimension, cited by a governance team. |
| AI Verify | Singapore testing framework and toolkit. Findings a governance team can cite alongside an AI Verify process. |
| MAS guidance | For financial institutions. Independent adversarial validation of production AI systems; recurring coverage as an ongoing-monitoring input for the model risk record. |
| Singapore PDPA | Reasonable security arrangements to protect personal data. Findings on paths that could expose personal data (for example LLM02 sensitive-information disclosure, LLM08 RAG weaknesses) supporting the reasonable-security argument. |
Ten references, and the evidence we produce against each.
Ongoing obligations keep coverage on.
A one-off test goes stale while the obligation continues. When a framework expects ongoing evidence of testing under attack, the coverage has to continue with it.
Continuous Assurance, our senior-led retainer, keeps operator-led testing running. Retest history and a coverage-over-time view are the recurring evidence.
Each monthly run re-checks the map the first test produced, to show that closed paths have stayed closed. New attack classes and reverse engineering of a new binary require a separate scoped engagement. See the Atlas map.
Not legal advice.
Exact obligations, thresholds, and clauses depend on a client’s jurisdiction, role, system classification, and facts, and they change over time. The references above are directional. Confirm how each applies to you with qualified counsel. Several expect testing or risk management broadly and do not prescribe a specific engagement.
Serpio is not counsel, an auditor, or a certification body. We produce the technical evidence that supports a compliance argument. We do not certify a system or make a client compliant.
Bring us the obligation, and we will produce the evidence.
Most engagements start from production risk. When an obligation you already carry needs evidence, the same engagement produces it.