Silence must never read as success
The rule everything else follows from — and the record of how often it is broken.
A check that could not run, could not reach its evidence, or could not decide must never be reportable as one that looked and found nothing.
This is the enabling condition, not a quality preference. A person catches a bad render by glancing at it; a script does not glance, and neither does an agent. Strip out the human who would have noticed, and the tool's own honesty is the only thing left holding the result up.
That is the whole reason this org can exist at all — so it is the first rule, and it binds drivers as hard as it binds checkers. A driver that swallows an engine error and returns a truncated artifact poisons the loop upstream of every check that would have caught it. The rule itself, and the discipline that follows from it, live in AGENTS.md §2. This page is the evidence.
How often it is violated is the point
As of 2026-09-06 this exact defect — no state for could not tell, defaulting to green, to a wrong verdict, or to a signal the caller cannot read — has been recorded nine times, across five unrelated domains, written at different times by the same author. Six are still open.
Every case here is tracked by an issue labelled
silence-defect.
That label is what links finding an instance to recording it, and it makes the
drift checkable rather than merely regrettable: if the label and this page
disagree, this page is wrong. It went stale twice before the label existed —
once by five cases, and once by two within a single day.
Nobody set out to build any of them that way. Assume you are doing it too.
| # | Where | Layer | Status |
|---|---|---|---|
| 1 | gerberdiff — dropped flash reads as no change | verify | resolved |
| 2 | netspec — no third verdict | verify | open |
| 3 | orlab — version fallback is log-only | drive | resolved |
| 4 | prusaslicer-first-layer-flow — OK when never installed | verify | open |
| 5 | orlab — unknown flight events dropped | drive | open |
| 6 | partspec — tool fault emitted as part fault | verify | open |
| 7 | partspec — undef dimension passes green | verify | open |
| 8 | partspec — measures a part whose data file is absent | verify | open |
| 9 | netspec — explicit engine request silently ignored | drive | resolved |
The record
1. A dropped Gerber flash reports as no change resolved
A flash selecting an undefined aperture is dropped with no diagnostic of
any severity. diff and geomdiff report
0 changes at exit 0 under --fail-on-diff,
and the JSON is byte-identical to diffing a board against a copy of itself.
Two things from that exercise generalise. The first reading was
wrong — the hypothesis under review was that warning-level
diagnostics leaked a silent pass; both engines in fact promote any
Error diagnostic to a raised GerberParseError and exit
2. The repro did not confirm the analysis; it replaced it. And
the tool already held the evidence: gerberdiff parse
prints nets: 2, apertures: 1 and a bounding box reaching the dropped
pad's centre rather than its edge. Nothing had to be measured that was
not already being computed — there was simply no state in which to say it. That
is what a missing third outcome costs: not a measurement, a verdict.
2. netspec reports "could not evaluate" as "the board is wrong" open
Verdict = Literal["pass", "fail"], green only when every rule is
green — so an unsupported rule collapses into fail,
saying the board is wrong when the truth is the tool could not check it. See
A2.
3. orlab's version fallback reached the caller only as a log line resolved
The first instance found in the drive layer rather than the
verify layer: the same failure reaches a caller through a binding exactly as it
does through a checker. get_profile() correctly returned
(profile, exact), but OpenRocketInstance.__init__
consumed exact, logged a warning and discarded it.
Two things the reproduction added to the suspicion. The single signal
was suppressible — setting the root logger to ERROR, an
ordinary application choice, removed the warning entirely, leaving no exception,
no return value and no attribute. And the obvious workaround was
wrong: comparing or_version to
profile.version_string looks equivalent, but
parse_version maps 24.12.RC.01 to (24, 12)
— an exact match whose strings differ — so the reconstruction reports a
fallback that is not one, on every RC and point-release build.
The fix exposes profile_exact on both
OpenRocketInstance and SimulationPool. Verified here by
mutation rather than by trusting the closed issue: discarding the value and
always reporting True fails 4 tests; implementing it as the naive
string comparison fails 2, including the test named for exactly that mutant.
Restored, 72 pass.
4. A G-code validator reports OK when the feature was never installed open
Outside this org, and the reason the rule is stated as a general one rather than
a house style: validate-gcode.py reports OK at exit
0 when the feature it verifies was never installed.
5. orlab drops unknown flight events, and the result looks complete open
Helper.get_events drops flight-event types orlab's enum does not
know and tells the caller only through a log line. The returned dict then looks
complete: a caller iterating it cannot tell "this simulation had no such event"
from "orlab did not recognise the event and silently dropped it", and the log line
is suppressed entirely by an application that raises its log level.
This is case 3's defect one layer up, and arguably worse — #61 degraded a whole run's fidelity in a way a user might notice, whereas this quietly removes rows from a result the caller is about to analyse. It matters most on exactly the jars #61 concerns: a newer OpenRocket on a nearest-older profile is where unknown event types are most likely to appear.
6. A partspec tool fault is emitted as a fault in the part open
When a measurement backend raises, the name lands in refused with a
reason — but the reason is prose, and it is the only signal
distinguishing a partspec bug from a defect in the part.
main(["measure", ...]) returns 0 on a monkeypatched
raise, so an agent reading the JSON must string-match English to tell "this part
has no centre of mass" from "this backend crashed".
The founding property one level in: a tool fault reported at exit 0,
in a block whose other entries mean "the part defeated this measurement", reads as
a statement about the part.
7. A dimension silently defaulted from undef passes green
open
An expression whose type error makes OpenSCAD substitute a default into a dimension. The engine prints one warning and exports a clean, watertight, single-solid mesh:
o = undef;
h = o + 1;
linear_extrude(h) square([40,30]);linear_extrude receives undef and falls back to a height
of 100. partspec measure reports
volume: 120000.0 mm3 — 40 × 30 × 100 — and check with
watertight() and solid_count(1) exits 0. The
part is 100 mm tall because a name was undefined, and partspec reports it green.
The scalar case is narrated by nothing at all on either engine — no
warning to guard on.
8. partspec measures and draws a part whose data file is absent open
check refuses a build whose depfile names a data file that is not on
disk. measure and render do not ask that question:
against a part importing a missing STL, check exits 4
while measure exits 0 with
volume: 7200.0 exact and render exits 0
with four PNGs. That volume is a bare 40 × 30 × 6 plate — the import contributes
nothing, which is the whole point. An agent measuring that number writes it into a
contract that then passes forever.
9. An explicit engine request is silently ignored resolved
NETSPEC_KICAD_CLI pointing at a path that does not exist is ignored
silently. Discovery falls through to the next candidate and the run succeeds
against a different engine than the one explicitly requested, with
nothing said on stdout, stderr or in the report, at exit 0.
The variable is presented as authoritative, not as a hint — it is what the tool's own error message tells you to set.
What the record is for
Three of the first four were found by running the code after a reading of it had concluded something different, twice concluding the wrong mechanism entirely. That is why AGENTS.md §7 requires a reproduction before a report, and why a fix is verified by breaking the thing it fixes and watching the check go red.
This list is what has been written up. It is a floor, not a census.