Silence must never read as success

1. A dropped Gerber flash reports as no change resolved

gerberdiff v0.29.1 · gerberdiff#17 · reproduced 2026-09-05 · fixed in d71f6ce, 2026-09-06

A flash selecting an undefined aperture is dropped with no diagnostic of any severity. diff and geomdiff report 0 changes at exit 0 under --fail-on-diff, and the JSON is byte-identical to diffing a board against a copy of itself.

Two things from that exercise generalise. The first reading was wrong — the hypothesis under review was that warning-level diagnostics leaked a silent pass; both engines in fact promote any Error diagnostic to a raised GerberParseError and exit 2. The repro did not confirm the analysis; it replaced it. And the tool already held the evidence: gerberdiff parse prints nets: 2, apertures: 1 and a bounding box reaching the dropped pad's centre rather than its edge. Nothing had to be measured that was not already being computed — there was simply no state in which to say it. That is what a missing third outcome costs: not a measurement, a verdict.

2. netspec reports "could not evaluate" as "the board is wrong" resolved

netspec · src/kicad_netspec/check.py:39 · netspec#19 · read from source 2026-09-05 · fixed in netspec 0.9.0, 2026-09-06 (D26)

Resolved: Verdict gained incomplete and error, and a skipped or unsupported rule now yields incomplete at exit 2 instead of fail at exit 1. Decided together with A1, since exit 2 had to stop meaning usage first. Note what the tests were already saying: test_a_part_absent_from_the_design_is_skipped_not_failed asserted verdict == "fail" — the name and the assertion contradicted each other, because fail was the only non-green value the type offered.

Verdict = Literal["pass", "fail"], green only when every rule is green — so an unsupported rule collapses into fail, saying the board is wrong when the truth is the tool could not check it. See A2.

3. orlab's version fallback reached the caller only as a log line resolved

orlab · orlab#61 · reproduced 2026-09-05 · fixed in 6ebbaad

The first instance found in the drive layer rather than the verify layer: the same failure reaches a caller through a binding exactly as it does through a checker. get_profile() correctly returned (profile, exact), but OpenRocketInstance.__init__ consumed exact, logged a warning and discarded it.

Two things the reproduction added to the suspicion. The single signal was suppressible — setting the root logger to ERROR, an ordinary application choice, removed the warning entirely, leaving no exception, no return value and no attribute. And the obvious workaround was wrong: comparing or_version to profile.version_string looks equivalent, but parse_version maps 24.12.RC.01 to (24, 12) — an exact match whose strings differ — so the reconstruction reports a fallback that is not one, on every RC and point-release build.

The fix exposes profile_exact on both OpenRocketInstance and SimulationPool. Verified here by mutation rather than by trusting the closed issue: discarding the value and always reporting True fails 4 tests; implementing it as the naive string comparison fails 2, including the test named for exactly that mutant. Restored, 72 pass.

4. A G-code validator reports OK when the feature was never installed resolved

prusaslicer-first-layer-flow · issue #1 · fixed in 30c2175, 2026-09-06

Resolved: the positional fallback is gone, and an absent marker gets its own exit code — 0 verified, 1 wrong, 2 the feature never ran. Worth recording what the fix nearly did: its first version left "the feature never ran" and "you gave me a bad path" both on 2, one code for two meanings, which is A1 in miniature. The regression test found it, on the run that was meant to confirm the fix.

Outside this org, and the reason the rule is stated as a general one rather than a house style: validate-gcode.py reports OK at exit 0 when the feature it verifies was never installed.

5. orlab drops unknown flight events, and the result looks complete resolved

orlab · src/orlab/core/helper.py:195-203 · orlab#63 · fixed in 96a0c92, 2026-09-06

Resolved: get_events returns an EventTimes mapping carrying unknown_event_types, so a caller can tell "no such event in this simulation" from "orlab could not interpret it". It subclasses dict, so existing callers are unaffected. The log line remains -- the point is that the log can be silenced and the field cannot.

Helper.get_events drops flight-event types orlab's enum does not know and tells the caller only through a log line. The returned dict then looks complete: a caller iterating it cannot tell "this simulation had no such event" from "orlab did not recognise the event and silently dropped it", and the log line is suppressed entirely by an application that raises its log level.

This is case 3's defect one layer up, and arguably worse — #61 degraded a whole run's fidelity in a way a user might notice, whereas this quietly removes rows from a result the caller is about to analyse. It matters most on exactly the jars #61 concerns: a newer OpenRocket on a nearest-older profile is where unknown event types are most likely to appear.

6. A partspec tool fault is emitted as a fault in the part resolved

partspec · partspec#371 · fixed in e5061f2, 2026-09-06

Resolved: a refused_by block maps each refused name to "part" or "tool", and a "tool" refusal now exits 4. Both halves were needed: a discriminator alone lets a consumer branch while the default answer still reads as success. The payload is still emitted in full, so one raising backend still costs one name and not the other thirteen.

When a measurement backend raises, the name lands in refused with a reason — but the reason is prose, and it is the only signal distinguishing a partspec bug from a defect in the part. main(["measure", ...]) returns 0 on a monkeypatched raise, so an agent reading the JSON must string-match English to tell "this part has no centre of mass" from "this backend crashed".

The founding property one level in: a tool fault reported at exit 0, in a block whose other entries mean "the part defeated this measurement", reads as a statement about the part.

7. A dimension silently defaulted from undef passes green accepted

partspec · both pinned engines (2021.01 and 2026.08.01) · partspec#308, remainder in #332 and #338

An expression whose type error makes OpenSCAD substitute a default into a dimension. The engine prints one warning and exports a clean, watertight, single-solid mesh:

o = undef;
h = o + 1;
linear_extrude(h) square([40,30]);

linear_extrude receives undef and falls back to a height of 100. partspec measure reports volume: 120000.0 mm3 — 40 × 30 × 100 — and check with watertight() and solid_count(1) exits 0. The part is 100 mm tall because a name was undefined, and partspec reports it green. The scalar case is narrated by nothing at all on either engine — no warning to guard on.

8. partspec measures and draws a part whose data file is absent resolved

partspec · both pinned engines · partspec#355 · fixed in 947d0b7, 2026-09-06

Resolved: all three verbs now refuse on one shared answer. Worth recording what the fix nearly got wrong: the issue's own "smallest fix" — refuse when the depfile lists anything missing — refuses a correct part. OpenSCAD evaluates a % subtree, so an absent file inside one reaches that field exactly as a real dependency does. Measured before writing anything: %import("gone.stl") and import("gone.stl") produce identical evidence and both measure 7200.0, and only the second is a fault. The narrowing moved into the shared answer, so the three verbs cannot diverge on it either.

check refuses a build whose depfile names a data file that is not on disk. measure and render do not ask that question: against a part importing a missing STL, check exits 4 while measure exits 0 with volume: 7200.0 exact and render exits 0 with four PNGs. That volume is a bare 40 × 30 × 6 plate — the import contributes nothing, which is the whole point. An agent measuring that number writes it into a contract that then passes forever.

9. An explicit engine request is silently ignored resolved

netspec 0.7.0 · netspec#17 · fixed in 1edc6c0, 2026-09-06 (D10.1)

NETSPEC_KICAD_CLI pointing at a path that does not exist is ignored silently. Discovery falls through to the next candidate and the run succeeds against a different engine than the one explicitly requested, with nothing said on stdout, stderr or in the report, at exit 0.

The variable is presented as authoritative, not as a hint — it is what the tool's own error message tells you to set.

10. An unmodelled trace reads as no change, and the two engines disagree resolved

gerberdiff v0.29.1 · gerberdiff#25 · reproduced 2026-09-06 · fixed in 0685cde, 2026-09-06 (A3)

The geometry engine does not model a stroke drawn with a macro or block aperture. That limitation was documented in docs/geometry-diff.md under "Known limitations" — the output simply never said so. A board with such a trace added reported 0 changes at exit 0 under --fail-on-diff, with the JSON byte-identical to comparing a board against a copy of itself.

What makes this one different from the rest: the raster engine saw the change. Same two directories, same question, opposite answers — diff reported 1/1 layers changed at exit 1 while geomdiff reported nothing at exit 0. A tool that contradicts itself is at least detectable; the danger was that only the quieter answer gated CI.

The pattern generalises past this one call site. The same bare return appeared five times in the emit path, one of them commented "matches renderer: silently skip over-deep nesting" — the silence was deliberate, consistent, and reviewed. Documenting a limitation is not the same as reporting it, and a known gap that the output cannot express is indistinguishable, at the point of use, from no gap at all.

Resolved: the engine records every operation it could not model, the comparison reports identical | different | indeterminate, and this case exits 2 naming the reason. Closes adjudication A3.

11. A PrusaSlicer that cannot start is reported as a failure of the code resolved

prusaslicer-py · prusaslicer_py/slicer.py, tests/conftest.py · prusaslicer-py#33 · reproduced 2026-09-06 · fixed in dfbc23e, 2026-09-10 (D14)

Resolved: discovery now answers in three states rather than two. PrusaSlicer.probe() runs --help and raises EngineUnusableError unless the engine started and answered, so absent, present and unusable and usable are separable, and conftest routes the middle one to the same skip-or-fail decision as the first — one function, because the half that gets dropped when the decision is duplicated is always the require half, which is the invisible one. The engine's own complaint is carried as a field rather than left in prose.

Engine discovery accepts a Flatpak when flatpak info exits 0. That establishes the app is installed, not that it can run. With an unwritable HOME the launcher fails before the engine starts:

$ flatpak info com.prusa3d.PrusaSlicer          # exit 0
$ flatpak run --command=prusa-slicer com.prusa3d.PrusaSlicer --help
error: mkdirat(.var): Permission denied         # exit 1

Discovery is unaffected, because it only ran info. PrusaSlicer() constructs and reports engine_kind="flatpak", so conftest.py — which skips engine tests by catching FileNotFoundError — does not skip. Same tree: 3 failed, 40 passed, 3 skipped against 43 passed, 3 skipped with a working HOME.

The machinery for this exists and does not fire. conftest's skip message reads "This is an environment fault, not a verdict on the code under test" — correct for an absent engine, absent for a present-but-unusable one. Under PRUSASLICER_PY_REQUIRE_ENGINE=1, whose whole job is asserting an engine was really exercised, the flag cannot tell "engine ran, code is wrong" from "engine never started".

Why this is here despite going red. It was first written up as not a case, on the reasoning that it fabricates no success — D5's artifact verification means no artifact, no green. That line is the wrong one, and case 2 is why: netspec reporting could not evaluate as the board is wrong also produced a red, and is case 2. Green-from-nothing is the sharpest form of this defect, not the definition of it. What the record tracks is the state a caller cannot distinguish, and the third outcome having no channel is the same defect whichever neighbouring answer it collapses into.

12. A build that greets you before it identifies itself reports the greeting as its version resolved

prusaslicer-py · prusaslicer_py/slicer.py:210 · prusaslicer-py#34 · reproduced 2026-09-06 · fixed in 00cd43c, 2026-09-10 (D13)

Resolved: the banner is matched rather than counted to — the first line matching ^PrusaSlicer-(\d\S*), on either stream — and a build that identifies itself nowhere raises VersionUnreadableError instead of returning something. The signature stays -> str: there is no line to return, so nothing is returned, and the third outcome is the exception rather than a widened return type. Note which narrowings were refused. Requiring based on Slic3r would answer could not tell for a version plainly present, and §2.1's third outcome is not licence to refuse more than the question requires.

PrusaSlicer has no --version flag — 2.9.6 answers Unknown option --version and exits 1 — so the version is read from the first line of --help. The native Windows console build prints a preamble first:

System OpenGL library successfully released
PrusaSlicer-2.9.6 based on Slic3r (with GUI support)

check_version() returns "System OpenGL library successfully released", at exit 0, as the version. Reproduced against a stub printing those two lines.

The Linux Flatpak has no preamble, which is why the code carries this comment: "the version is only ever printed as the first line of --help", followed by the Flatpak's banner as the example. A claim about every build, measured against one.

This one fabricates rather than misroutes, which puts it on the worse side of the severity axis from case 11. The signature is -> str, so there is no third outcome available: a version it could not read comes back looking exactly like one it could. Found by a sibling project's four-platform engine matrix — the defect needed a second platform to become visible, and neither project had one.

13. netspec throws away the cause it was handed and states a different one resolved

netspec · src/kicad_netspec/mcp.py:212 · netspec#35 · reproduced 2026-09-07 · fixed in e041686, 2026-09-10

Resolved: netspec-mcp now separates "the package is absent" from "the package is here and the symbol moved", using find_spec("mcp") as the observation, and prints upstream's own text in the second case. Both still exit 4, because both still mean netspec could not tell you anything about your board.

Under mcp 2.x, mcp/server/fastmcp.py is a stub that raises, and upstream is about as helpful as a library can be about it — naming the rename to MCPServer, the new import path, the migration guide, and the mcp<2 pin that keeps v1 code running. Reaching it took a fresh venv with the constraint overridden: netspec's own extra is mcp>=1.27,<2 and has been since the scaffold commit, the widening to <3 is an open pull request, and so no released netspec resolves 2.x. The typecheck that caught it ran on that branch, not on main.

$ python -c "import mcp.server.fastmcp"
ModuleNotFoundError: No module named 'mcp.server.fastmcp'. This is mcp 2.x, where
FastMCP was renamed to MCPServer (from mcp.server.mcpserver import MCPServer) and
other APIs changed; see the migration guide at ... or pin 'mcp<2' ...

The entry point caught ImportError, and ModuleNotFoundError is a subclass of it, so every word of that was discarded and replaced with netspec-mcp needs the mcp extra: pip install 'kicad-netspec[mcp]'. The extra was installed. The user is told to install what they already have.

Which is not confined to the unreleased pin, and that is why this is a case rather than a note on a bump. The handler was a bare except ImportError that read nothing off the exception, so it answered every import failure with the same sentence. An incomplete install — mcp 1.30.0, well inside the shipping <2 pin, with a transitive dependency of it missing — arrives at the same handler and gets the same wrong cause. 2.x is the sharpest illustration, not the reach.

This one misroutes rather than fabricates, which places it beside case 11 rather than case 12. The failure is real, exit 4 is the right channel for it under §2.2, and netspec genuinely could not run; only the cause was invented. The issue's own prose calls that reason fabricated — on this record's axis the word belongs to case 12, where a version that could not be read came back looking exactly like one that could.

It is worth saying where the line runs, because a neighbouring netspec issue sits on the other side of it and is deliberately not on this page. netspec#26 carries an environment fault in the exit code but not as a report field — an omission, and nothing it says is false. But omission is not the line, because case 3 is a pure failure to say something and is on this record. The line is whether the channel can be silenced, which is #26's own argument: case 3's only signal was a log line, and an application setting its root logger to ERROR removed it entirely, leaving nothing at all; #26's channel is an exit code, which no caller's configuration can turn off. #35 needs neither test — it held the right answer and stated a wrong one.

14. An engine that is installed reads as absent, and every engine test skips open

prusaslicer-py · prusaslicer_py/slicer.py:412, tests/conftest.py · prusaslicer-py#35 · read from source 2026-09-06, re-read on 00cd43c

_find_executable is shutil.which(cls._exec_name()) and nothing else, and _exec_name() answers prusa-slicer on every non-Windows platform. The Homebrew cask installs /Applications/PrusaSlicer.app, whose binary carries the application's name — PrusaSlicer — and puts nothing on PATH. Discovery reports the engine absent on a machine that has it.

The second-order effect is the worse half, and it is why this is fabricate rather than misroute. conftest decides whether to skip an engine test by constructing PrusaSlicer() and catching FileNotFoundError; on a cask install that raises, so every engine test skips on the one machine that could have run them. That is §2.4's a skipped test is not a passing test, and the same shape as partspec's 195 passed / 23 skipped with no OpenSCAD on any runner.

Whose green it is, precisely. Not prusaslicer-py's own CI: engine.yml's macOS job symlinks the bundle's binary onto /usr/local/bin/prusa-slicer and runs under PRUSASLICER_PY_REQUIRE_ENGINE, so a skip there is a hard failure. It is a user on a cask install who gets the green that verified nothing, and a workflow step is what stands between the two.

Not reproduced here, and recorded as case 2 is: there is no macOS host on this bench, so what is read from source is the discovery path and the skip cascade. What makes it more than a code reading is that the discovery miss was observed on a real macOS runner by a sibling project's engine matrix, immediately after Homebrew installed the engine. slicelab fixed its own discovery for it in 17f539f — per-platform candidates, because which name a bundle uses is a packaging decision — and reported the same shape across. This repository still has the single name.

The repository already knows the bundle layout — slicer.py handles .app when locating shapes resources, commenting that looking only beside the binary broke brew install --cask prusaslicer. The shape is understood for resources and not for discovery. Against case 11 it is the same module in the opposite direction: 11 is an engine found but unusable, this is an engine present but not found, and both are discovery answering with more confidence than it established.

15. A driver's own example reports its failure and exits 0 resolved

prusaslicer-py · examples/torus_example_attempt.py, examples/basic_slicing.py · prusaslicer-py#36 · reproduced 2026-09-11 · fixed in 389eed8, 2026-09-12

Resolved: both examples exit non-zero when they produce no G-code and when the shape they need is absent. All four exit sites are covered by tests that run the examples as real processes and assert the status — asserting it in-process is exactly what would not have caught this.

The example passed layer_height to additional_args, which builds --{key} verbatim, so the engine answered Unknown option --layer_height and produced nothing. The example caught the error, printed it, and ended:

$ python examples/torus_example_attempt.py
No G-code produced: PrusaSlicer exited 1 slicing .../torus.stl
Unknown option --layer_height
$ echo $?
0

The library was scrupulously correct throughout. slice_model raised SliceError carrying the engine's returncode, stdout and stderr, exactly as its D5 promises. Nothing was silenced by prusaslicer_py; the example dropped the answer. That is why this case is here rather than filed against the driver: an honest library does not save a caller who discards what it returns, and the file doing the discarding was the one whose job is to demonstrate the library working.

Both channels were not wrong — only the machine-readable one. The message is printed and accurate. A person running the example sees the failure; a smoke test reading $? sees success. This page's own opening is the argument for counting it: a person catches a bad render by glancing at it, and a script does not glance.

Two things from the fix generalise. The obvious repair was wrong: only layer_height becomes a real option by swapping the underscore — there is no --infill-density (it is --fill-density), no --print-speed (speeds are per-feature) and no --extruder-temperature (it is --temperature), so a mechanical _→- pass would have left the example broken while looking repaired. And two of the four demo values were the engine's own defaults, so checking that a value "came back" could not distinguish the option taking effect from it never being passed.

The examples are not run in CI, so the harm was to a person or an ad-hoc smoke test rather than a green pipeline. It became reachable only when an earlier fix removed an outer failure: until the driver stopped naming a Windows executable, the examples died earlier on Linux and never reached the engine. Removing an outer failure exposes an inner one, and a repository with no example smoke test does not find out.