Coverage

What an agent can get wrong, and what we catch

116 ways an agentic request can fail, built from the architecture of the request rather than from our own feature list. 105 of them are executed against the running product every night; the rest say plainly that they were assessed by reading the code.

Where the request can fail

Each layer is a stage the request actually travels through. The score counts a partial control as half, because most of the honest answers are partial.

What we do not catch

Published for the same reason the rest is: a coverage page that only listed successes would not be evidence of anything. 3 scenarios have no control at all, and 15 have a partial one.

• Invalid logical inferenceL0 model-intrinsic
• Context stuffing to push out the system promptL1 input and prompt
• An instruction in an image, or a scanned PDFL2 retrieval and context

What it costs, and what happens when it fails

Latency and false positives are the most-cited reasons guardrails get switched off. Every control here runs inside a declared budget and records its own degradation per decision, so "the check was down" is a fact in the record rather than a thing nobody noticed.

Four controls may not be configured to fail open

Not a setting with a safe default — a refusal. Their failure mode is a disclosure rather than an outage, and "we allowed it because the check was down" is not a defensible answer.

Tenant isolation

One customer's data reaching another is not an outage.

Entitlement filter

Same.

Data access scope

A query across every customer's rows reads as ordinary.

Audit chain

A gap in the record is the thing the record exists to prevent.

Generated by scripts/probe/run.py --json from the taxonomy the nightly harness executes. A row marked verified has fired at least once against the real product; a row that is not executable says so rather than borrowing the others' credibility. The full table, with the control behind each row, is in docs/coverage-map.md.