Skip to main content

​
The THRINDEX Certified badge

A backend that earns the THRINDEX Certified badge has passed the conformance harness against CONFORMANCE_ENVELOPE_V0 on the ratified fixture set. This means:
  • Its outputs agree with the reference simulator’s outputs within precisely stated bounds
  • The agreement was measured on a statistically valid fixture set (≥ 100 samples)
  • The measurement is reproducible from the frozen fixture set (CRC32 e2ebd845)
The badge is per-backend, per-model-class, and per-envelope-version. “THRINDEX Certified [v0]” means the V0 envelope on the default model class (feedforward LIF, SHD-class).

​
The reference simulator

The reference implementation is thrindex-sim: a pure software behavioral simulator that runs models in f32. It defines correct behavior. Every hardware backend is certified relative to the reference — the badge means the backend agrees with the simulator within the envelope, not that the backend is equivalent to the simulator.

​
The conformance harness

The harness (thrindex-conformance) runs both the reference simulator and the backend-under-test on each fixture sample, computes the spike-equivalence metric, and aggregates into T_mean, T_max, and P_pred. It then checks all three against the envelope thresholds.
Exit code: 0 (PASS), 1 (FAIL), 2 (error).

​
Why three metrics?

A single metric is not enough because different failure modes of a hardware backend look different:
  • T_mean catches systematic drift — a backend that is consistently wrong by a small amount on every sample
  • T_max (p99) catches tail failures — a backend that is mostly correct but occasionally very wrong
  • P_pred catches decision-level failures — backends that pass the first two but produce wrong classifications in edge cases
A backend must pass all three simultaneously. Failing any one is a FAIL.

​
Envelope versioning

The current envelope is CONFORMANCE_ENVELOPE_V0. Future envelopes (V1, V2, …) may add model classes (recurrent networks, Conv2d-heavy models) or tighten thresholds as hardware matures. A backend certified under V0 retains the V0 certification indefinitely. It must be re-certified under V1 to carry the V1 badge.

​
The fixture set

The conformance fixture set is a frozen set of 120 SHD test samples in sparse-event JSON format, CRC32-fingerprinted at e2ebd845. It lives at conformance/fixtures/ in the monorepo. The fixture set is:
  • Frozen: no sample is ever added, removed, or modified after ratification
  • Reproducible: generated by freeze_shd_fixtures.py with seed 0
  • Stratified: 6 samples per class × 20 classes
See Conformance: Envelope V0 for the full derivation.