Testing & evidence
How quality is measured, and where the evidence behind every claim lives.
Testing & software quality
699/701 Rust workspace tests and 173 Swift shell tests pass against real aws-lc-rs cryptography (2 existing Rust tests ignored), run 2026-08-10. Organised below by assurance purpose, not framework.
Domain rules, guard functions, transitions, encoding/decoding, boundary conditions — every protocol guard maps 1:1 to a named abort reason and has a test.
transitions.rs (per-machine transition tables)crates/oid4vp · oid4vci · payment · qes · w2w · iso18013-5rejects_non_canonical / rejects_duplicate_map_keys / rejects_trailing_bytescrates/cosebinding_is_specific_to_receiver_and_credentialcrates/w2wThe six core Lean models emit transition traces; six Rust suites replay them against the production machines, byte-for-byte. The seventh model (app-shell navigation) is bound to the shipped Swift by an exhaustive (state × event) conformance test. Android has hosted shell tests but still needs its own formal shell model and exhaustive Kotlin conformance suite. CI regenerates traces and fails on staleness — no platform may claim formal parity without that binding.
conformance.rs × 6crates/{oid4vp,payment,iso18013-5,oid4vci,qes,w2w}/testsFull flows through the sans-IO core with real aws-lc-rs cryptography: issue → hold → present over live TCP sockets; payment SCA; QES; wallet-to-wallet; status blocking; export integrity; FFI JSON contract.
e2e_live_presentation.rs / e2e_live_lifecycle.rs (real sockets, RP verifies)crates/shell-io/testse2e_flow / e2e_issuance / e2e_payment / e2e_status / qes_flow / w2w_flow / add_credentialcrates/wallet-core/tests177 Swift tests: effects, QR, native journeys, identity transports, recovery, Lean conformanceios/Tests/WalletShellTests51named cases
Every important failure mode becomes a test: replay, wrong nonce/audience, untrusted issuer, revoked/expired credentials, tampered transactions, forged codes, malformed input, misdirected transfers, alg:none.
Distinct test functions whose names encode a rejection/attack case (rejects_*, *replay*, *tamper*, *forged*, *mismatch*, *untrusted*, *revoked*, *invalid*, *malformed*), counted by grep — a subset of the workspace tests.
rts_art4_code_cannot_be_forgedcrates/crypto-backend/tests/regulatory_sca.rsrejects_a_misdirected_transfer_in_corecrates/wallet-core/tests/w2w_flow.rsexport_round_trips_and_detects_tamperingcrates/wallet-core/tests/export.rsrejects_alg_nonecrates/sdjwt4fuzz targets
Coverage-guided fuzzing of the four codec attack surfaces; Kani bounded model checking on the mdoc codec. Both run in CI (bounded time per run).
cargo-fuzz targets in fuzz/fuzz_targets/.
cose_cbor · mdoc_cbor · sdjwt_parse · x509_parsefuzz/fuzz_targets/Kani bounded proofs (cargo kani -p mdoc).github/workflows/ci.yml (tier1-fuzz-kani)clippy -D warnings across all targets/features (PASS, 2026-08-10); rustfmt gate; cargo-deny (licenses/bans/sources/advisories) with a written dependency budget; cargo-audit; and a PUBLISHED CycloneDX SBOM (21 crates; wallet-core = 111 runtime components).
clippy PASS — verification reportdocs/certification-evidence/verification-report.mddeny.toml (2 documented build-time-only ignores)deny.tomlPublished SBOM (CycloneDX 1.3)docs/certification-evidence/sbom/73viable mutants caught
cargo-mutants mutates the security-critical OpenID4VP presentation crate and checks the tests catch each change — measuring whether tests constrain behaviour, not just execute it. The initial run caught 52/73 viable mutants (71%); the 21 misses were real test gaps, closed by adding crate-local tests. Re-run: 73/73 viable caught, 0 missed.
Viable mutants caught / total viable, cargo-mutants v27.1.0 on crates/oid4vp with the crate-local test suite (2026-07-19). 8 unviable (non-compiling) mutants excluded per standard practice. Scope: oid4vp only — a test-adequacy measure, not a correctness proof.
cargo mutants -p oid4vpdocs/certification-evidence/mutation-testing.mdDependency-free micro-benchmarks of the hot paths against the real aws-lc-rs backend (ES256 sign ~28.7µs / verify ~83.6µs, SHA-256 ~58ns, SD-JWT parse ~430ns; release, Apple Silicon, 2026-07-19). A flow is dominated by one or two ES256 ops plus one TLS round trip.
cargo run -p benches --releasedocs/certification-evidence/perf-benchmarks.mdA reproducible probe of the EUDI reference environment over platform TLS: the reference issuer's live OpenID4VCI metadata (HTTP 200, 27 configurations — 11 in the SD-JWT VC format this wallet implements) and reference-verifier reachability. Reachability + wire-shape, NOT a conformance pass.
tools/interop/probe.shdocs/certification-evidence/interop.mdTry it yourself — the presentation routes
PartialThe wallet presents over three channels: OpenID4VP by QR / deep link, in-person over Bluetooth (ISO 18013-5), and — the primary route for most users — the W3C Digital Credentials API, where a website calls navigator.credentials.get({ digital: … }) and the OS routes the request to the wallet. The DC-API core is implemented and tested (the byte-exact OpenID4VPDCAPIHandover binds the response to the browser-verified Origin, and an end-to-end test drives request → consent → a minimised mdoc vp_token). The iOS provider that surfaces this to Safari is being wired next; until it lands you can still see the OS route a request to the wallet.
The PID is issued and held in BOTH mdoc and SD-JWT VC form (per the ARF), so a verifier can ask for either format over any of these channels.
Coverage, honestly Code coverage (line/branch/function) has not been measured yet — the number is deliberately absent rather than implied. Coverage also would not prove correctness: the complementary assurance here is machine-checked invariants (Lean), symbolic protocol analysis (Tamarin), oracle-replay conformance binding model to code, adversarial tests, and fuzzing.
Requires an external party (not closeable by our code)
Planned (codeable) — not yet done, no result implied
Future interoperability role
Conditional future capability: every applicable local gate is green, the tested build is version-pinned, and official OpenID/EUDI conformance results for the exercised profiles are published.
At that point this wallet becomes a known-good protocol counterpart for teams building the other side in parallel. They can test against a stable, independently evidenced participant instead of waiting for every issuer, verifier and peer wallet to mature at the same time.
Drive a conformant wallet through metadata discovery, PAR/PKCE, proof, credential response, portrait/profile validation and status handling; attribute a failure to the issuer-facing boundary.
Send signed OpenID4VP/DCQL requests to the wallet and verify its SD-JWT or mdoc response, including encrypted direct_post.jwt, nonce, certificate binding and minimised disclosure.
Exercise wallet-to-wallet and proximity handshakes against pinned happy-path and hostile traces, including replay, substitution, interruption and recovery cases.