Benchmarks

The benchmark is brutal. AgentOracle is roughly 2× the academic state of the art.

AVeriTeC is multi-evidence fact-checking with a 4-label space. Published academic systems land near 30%. AgentOracle's pipeline scores roughly double the strongest paper-provided baseline on the same dev set. The score is high because the benchmark is hard, not because the bar is low.

Headline

Held-out matches calibration → verdict mapping is not overfit. Selected via inspection on the calibration half; held-out is untouched during selection.

Per-category accuracy (held-out half)

Why the lower numbers on Conflicting / NEE are a safety property, not a model failure: when evidence genuinely conflicts, AgentOracle leans skeptical — most misclassified Conflicting claims received Refuted verdicts, not falsely-confident Supported ones. That is the right failure mode for regulated content. A verifier that fails toward caution is safer than a verifier that fails toward confident-wrong. We publish the absolute numbers as-is so a buyer can audit the failure shape themselves.

POST /v1/sign — single Ed25519 signature

Takes one canonical_bytes_b64u, returns one JWS signature entry. The primitive every composed envelope leg uses.

n=50 sequential calls. Host: https://agentoracle.co (Vercel edge). Dominated by network round-trip and TLS, not crypto — the Ed25519 sign itself is sub-millisecond.

curl -w "\n%{time_total}s\n" -X POST https://agentoracle.co/v1/sign \
  -H "Content-Type: application/json" \
  -d '{"canonical_bytes_b64u":"eyJ0ZXN0IjoidHJ1ZSJ9"}'

POST /v1/compose — full 2-signer composed envelope

Single HTTP call. Orchestrates AT /v1/compose → v_gate_skill, AO computes v_gate, builds canonical bytes once, AT /v1/sign for AT's signature, AO signs locally, assembles JWS general serialization. Returns full 2-signer envelope verifiable against both published JWKS.

n=20 sequential calls. Includes one round-trip to AT /v1/compose + one to AT /v1/sign over the public internet from the AO origin region to the AT origin region. Co-location would cut this further; published as-is.

curl -w "\n%{time_total}s\n" -X POST https://agentoracle.co/v1/compose \
  -H "Content-Type: application/json" \
  -d '{"claim_hash":"sha256-yourclaim","mcp_content":{"tool":"web.search"}}'

POST /v1/sign/batch — N canonical bytes, N signatures

Bulk signing primitive for high-frequency agent loops. Max 100 items per request. Returns one signature entry per input; all entries share the same kid (issuer is constant). Useful when an agent needs to anchor many envelopes in one network round-trip.

curl -X POST https://agentoracle.co/v1/sign/batch \
  -H "Content-Type: application/json" \
  -d '{"canonical_bytes_b64u":["...","...","..."]}'

Latency is dominated by HTTP round-trip, not crypto. Batched cost-per-item drops linearly with N for any N ≥ 2.