Incident record

Canned verdicts, 21-25 August 2026

A four-day window in which the verification service signed and returned receipts for evaluations it never performed.

Between 2026-08-21 and 2026-08-25, an outage of AgentOracle's /evaluate verification service caused it to sign and return receipts for evaluations it had never performed. It was discovered 2026-08-25 from the receipts themselves; the signed layer had correctly halted every affected request, and the human-readable body did not disclose that.

Five days undetected is the shape of this incident. Discovery from the receipts themselves — not from a customer complaint, not from a monitoring alert — is what made that window recoverable at all. The signed layer was doing what a signed layer is supposed to do; the presentation next to it was not.

What happened

The population

17,784 verdicts were issued during the affected window. 17,703 carried a byte-identical canned reason string — 99.54% — every one of them a valid Ed25519 signature over a receipt that correctly derived verdict: halt. The remaining 81 evaluations completed before the balance drained fully; those receipts are unaffected.

Why the receipts were still right

The cryptographic layer was right the whole time. The presentation layer understated it. For the full duration of a total upstream verification outage, the signed evidence artifact carried v_verdict: unverifiable and v_gate: halt. The v0.3 receipt derivation reaches act only through an evaluated, adversarially-resistant claim set. A response with no evaluated claims cannot express act in the signed layer, and none did.

If a customer had integrated on receipt verification, we would have nothing to disclose to them beyond the body text they saw next to a correct receipt.

What we changed

CommitWhen (UTC)Change
4a8b371772026-08-27 01:52ZHandler early-returns HTTP 503 not_evaluated when no source produced a parseable claim set. No confidence, no verdict, no receipt, no cache write.
4a8b371772026-08-27 01:52ZCache admission gated on evaluation completeness, across both the full-response and per-claim namespaces.
98b3b7e782026-08-27 04:41ZReverted a co-shipped change that collapsed un_probed_not_cleared into the generic unverifiable. Both halt, but the two states are distinct at the receipt layer.
c28c5dd6f2026-08-28 02:13ZRecommendation text derives from the caller's threshold instead of a hard-coded 0.80. Present since min_confidence became caller-tunable; unrelated to the outage but the same disclosure-asymmetry class.
49261a37b2026-08-28 03:50ZThe adversarial-flag confidence penalty re-checks the applied threshold, so a flagged response can no longer report act in the body when the receipt-side gate would halt. Same disclosure-asymmetry family as the row above.
a8ce2a18d2026-08-28 18:02ZMapping binding derived at boot from the published mapping document; /v1/compose, /v1/v_gate, /v1/sign, and /v1/sign/batch return 503 until their verdict path or auth is completed. See the externally-reported defect below.

The pre-registration for the benchmark run whose collection window this outage sat inside carries an append-only amendment (2351e29f6) withdrawing the pre-registered headline for the same reason. This note and that amendment describe the same withdrawal.

A defect found by someone else, while this note was being written

Externally reported

On 2026-08-28, @giskard09 independently ran the published recompute procedure against /v1/conformance/sample and reported two things (x402-foundation/x402#3234). Both are correct.

First, the sample's v_gate.mapping_hash was a hand-typed placeholder rather than a digest of any mapping document. The envelope's signatures were genuine — both issuers verified, canonical bytes recomputed — so the cryptographic layer faithfully committed to a mapping binding that had never been computed. A relying party following our published procedure would resolve that hash, fail to match it, and correctly halt. This is the same failure class as the first defect above: a genuine signature over content the service never produced.

Second, we found the same hard-coded constant was used by the live POST /v1/compose and POST /v1/v_gate endpoints, both reachable without authentication. It was not confined to a fixture. In the same audit we discovered POST /v1/sign and POST /v1/sign/batch signed caller-supplied bytes under the production issuer key with no authentication — a forgery oracle for this receipt format while it was open.

The same string was reported five weeks earlier. On 2026-07-28 Michael Msebenzi (headlessoracle) reported that the mapping hash carried in our published fixtures was a placeholder that had never resolved. We repaired the fixtures within twenty hours and computed the correct mapping hash on 2026-07-29 — the value the conformance sample carries today. The identical placeholder string remained in the production constant for five more weeks, and the defect above is that same string. The July fix was scoped to the artifact that had been reported rather than to the class, so the correct value existed, published, in one location while the placeholder kept being stamped from another.

Fixed in a8ce2a18d, 2026-08-28 18:02Z UTC. The mapping binding is now derived at boot from the published mapping document, or the process refuses to start. All four endpoints return 503 not_issuing; /v1/sign* require an authorization header, and the composed endpoints will resume issuing only when their verdict path is completed. The private issuer key was never exposed — an oracle that signs what you hand it is not the same as a stolen key — and closing the routes ended the capability.

Bound on the exposure window. Requests to these routes were not counted; the request-tracking helper is called zero times in the composing module. Log retention is roughly one hour. We cannot enumerate what was signed, or by whom, between 2026-06-23 and 2026-08-28. Absence of known forgery is absence of evidence, not evidence of absence, and it must be said that way rather than leaned on.

Where the recompute leg stands

Alongside the sample defect, @giskard09 also ran the full verification protocol against a genuine /evaluate receipt: cryptographic signature check, mapping-hash match against a SHA-256 he computed himself of the live mapping document, and the Section 4.3 recompute of candidate_recommendation and candidate_gate. All three held. In his own words:

"The recompute requirement stands as demonstrated — we ran the full leg independently (signature, mapping_hash, §4.3 steps 3-6) and it held. The signature-implies-issuance step is a separate claim from that, and it shouldn't carry the same weight until kid separation lands on your side. That's not a knock on the recompute work, it's just a different property being verified."

That split is right, and it names the property still open. Same-kid conflation. The kid ao-composed-2026-06-ed25519-c3abfce3 currently signs both receipts where AgentOracle evaluated the claim (/evaluate) and receipts where the service only signed bytes a caller handed it (/v1/sign*). A relying party cannot distinguish those provenances from a receipt alone.

This is a property of the composed format, not of one operator’s implementation. It is present on both issuers we have checked. @poteshniy independently identified the same conflation on Base-side composed issuance during this review window, and the two of us have converged on a format-level fix rather than two separate operational patches.

The fix is an issuance-path assertion: a role parameter in the JWS protected header naming what the signature attests, checked against a role member on the signing key in the issuer’s published JWKS, so a verifier resolves what a signature attests from the same fetch that already resolves the key. Semantics are additive — a receipt carrying no role resolves to unknown, not to malformed, so every receipt issued before role annotation keeps verifying. unknown is not a pass: it records that the issuance path was not established, and a relying party whose policy requires an established path treats it as its own fail.

Kid separation remains worth doing operationally on both sides as defence in depth, but it is no longer the durable fix — the assertion is, because it survives key rotation and does not require any verifier to carry a mapping table it has to keep current. No date is set here. The old kid stays in JWKS so nothing already issued is invalidated.

Evidence limitations

What did not happen

What is still open

Records cannot be backdated. An examiner asking in 2028 about behaviour in 2026 can only be answered by records that existed in 2026. That is the whole argument for this format, and it is why the incident belongs here, in the record, rather than nowhere.

Contact and references

joe@agentoracle.co. Commits 4a8b37177, 98b3b7e78, c28c5dd6f, 49261a37b, a8ce2a18d. External review thread: x402-foundation/x402#3234. Benchmark amendment: agentoracle-benchmark-a-b@2351e29f6.

Corrections

Corrections to this note are added here, dated. The body above is not edited after publication.

Added 2026-09-30 — a second signing path, found by our own audit.

This record covered the signing routes on agentoracle.co. A read-only audit of every published key on 24 September 2026 found a second path, on a separate service. The API gateway (agentoracle-gateway-main-39fa17e.zuplo.app) exposed its own POST /v1/compose, which signed a composed envelope over a verdict supplied by the caller. The caller set the verdict, confidence, threshold, adversarial result and recommendation, and the route copied them into the signed payload unchanged. The gateway evaluated nothing. From 27 July 2026 until 24 September 2026 it signed under ao-composed-2026-07-ed25519-3d44ba27. From 22 to 27 July it signed under a test fixture key whose private half is public and which is not in our published key set, so those signatures show nothing about issuance.

Our docs described this route as "decision in, signed receipt out", so it did what it said. The problem is what its output looked like. Its envelopes carried issuer: "agentoracle" and the same format as receipts from /evaluate, and the receipt carried no assertion of which route produced it. Its key ID and issuer value differed from an /evaluate receipt's, but nothing in the receipt said what either meant, so the receipt alone could not show that no evaluation happened. Access required an API key, sold from 2 August through the $99/month self-serve tier and limited to 100 requests an hour per key. The route was never open without a key.

What the records can and cannot show. The gateway wrote a log line for every signature, but the hosting plan it ran on keeps no request history, so the number of envelopes signed through this route, and which keys requested them, cannot be established. What the records do show: the only live-mode purchase of the self-serve tier was our own, on 3 August, cancelled on 22 September. On 24 September the production gateway held six API keys: four created by test-mode checkouts, which take no payment, one we created for a benchmark, and one we created for the check described below; the preview and development environments held none. At least one envelope reached the public: the receipt in Virtual Protocol's public showcase repository, committed on 3 August 2026, which was the demo receipt on tanilo.io until 2026-09-30, was signed through this route. We know of no others, and we cannot rule them out.

Withdrawn 24 September 2026. On 24 September we deleted the private key from every environment of the gateway and redeployed production, preview and development. Signing through this route ended then. On 28 September at 21:26:56 UTC an authenticated call to the route returned HTTP 500 signing_key_unavailable: the old code could no longer sign. We then deployed a change that removes the signing code from the route itself (commit 49dd88d, serving from 21:27:13 UTC), so no configuration change can make it sign again. The route returns HTTP 410 withdrawn since 28 September 2026: the same call at 21:31:11 UTC returned 410, with no signature. An unauthenticated request is still refused with 401 by the gateway's API-key check before it reaches the route; with no keys left, that is what any caller now sees. The key used for those two calls was then deleted, and the gateway now has no API keys in any environment.

ao-composed-2026-07-ed25519-3d44ba27 is retired. It stays published in the JWKS so existing signatures still verify. A valid signature under it proves the bytes haven't changed since signing. It does not show that we evaluated the claim: the gateway evaluated nothing it signed under that key.

Two routes named /v1/compose. This record's "Evidence limitations" list names /v1/compose among the routes that were not instrumented. That refers to the stub on agentoracle.co, which signed under ao-composed-2026-06-ed25519-c3abfce3 and stopped issuing on 28 August (a8ce2a18d). The gateway route in this note is a different service with a different key. The benchmarks page also mentions AgentTrust's /v1/compose, which belongs to AgentTrust.