heibench

Open adjudications

Real conflicts between shipped members, recorded rather than resolved.

Resolving one of these changes released behaviour, and that is a decision with an owner. They are written down so a consumer knows what it is reading, not so someone quietly harmonises them.

Do not "fix" a member to match this page without an issue and a decision entry. The vocabulary follows the tools; the tools do not silently follow the vocabulary. Decisions live in each repo's docs/DECISIONS.md, numbered, with the reasoning that produced them — do not relitigate a numbered decision; if it is wrong, add a superseding entry.

Shared vocabulary

Statuses, per check

Common core, in every member that adjudicates: pass · fail · unsupported · skipped. Only pass is green.

approximate is a domain-gated extension, for when a measured error interval straddles the threshold. partspec has it; netspec deliberately does not, because connectivity is discrete and importing interval epistemics into an exact domain adds concepts without adding truth (D9, guarded by test_report_carries_no_tolerance). Absence is a decision to record, not an omission to fix.

Exit codes

Agreed across partspec and netspec today:

0    satisfied
1    violated
4    environment fault / error — not a verdict on the design

The adjudications

A1 — exit code 2 means two different things open

verified against source 2026-09-05

partspec maps Verdict.INCOMPLETE to exit 2 (src/partspec/status.py:107) and puts usage at 64 (EX_USAGE). netspec uses EXIT_USAGE = 2 (src/kicad_netspec/cli.py:32), with no code for "could not evaluate". A consumer branching on 2 across both tools is reading two different facts.

Recommendation: adopt partspec's split.

A2 — netspec cannot report "could not tell" at the verdict level open

read from source 2026-09-05, not reproduced

Verdict = Literal["pass", "fail"] (src/kicad_netspec/check.py:39), green only when every rule is green — so an unsupported rule collapses into fail, saying the board is wrong when the truth is the tool could not check it. D10 also documents a verdict: "error" the type does not contain; the environment fault rides on the exit code, not on a report field as D10 requires.

Also case 2 in the record of the core defect.

A3 — gerberdiff has no third state open

reproduced 2026-09-05, v0.29.1 · gerberdiff#17

has_changes: boolean (docs/schema.md), layer status matched | added | removed. For a differ, too, silence must never read as no difference.

Written up in full as case 1, which is now resolved — gerberdiff#17 was fixed in d71f6ce (2026-09-06), so an undefined aperture is an Error and exits 2 rather than dropping the flash in silence.

A3 itself stays open. That fix closed one instance; it did not give the differ a third state. has_changes is still boolean and layer status is still matched | added | removed, so a comparison that could not be made is still not expressible in the report. Fixing an instance is not the same as closing the adjudication, and conflating the two is how the record drifts.

A4 — orlab's version fallback was log-only resolved

reproduced 2026-09-05 · fixed in 6ebbaad, orlab#61 closed

Resolved: profile_exact is now a public attribute on both OpenRocketInstance and SimulationPool, so a caller can branch on whether it is running against an exact profile or a nearest-older fallback. Retained as case 3 in the record, where it is the first instance found in the drive layer rather than the verify layer.

The rule that produced them

Every adjudication here is a disagreement about how to say could not tell. That is not a coincidence — it is the one rule everything else follows from, showing up at the seams between tools that each solved it slightly differently.