A pattern, pulled out of four working tools
Four tools in this workshop have one habit in common, and it took a year of building them separately to notice. Each one would rather say nothing than say something plausible. That turns out to be the whole trick, and it is worth more than any of the four.
This is a pattern noticed, not a library published. The four tools are real, working and used daily. None of them is packaged, none is installable, and two of the four live in private repositories you cannot fetch. What is offered here is the idea and the working code of the half that is public.
A normal tool answers. You give it inputs, it gives you a number, and the number looks the same whether it was measured or invented. That is comfortable and it is how most estimates get into a plan.
These four do the opposite. Given a question they cannot honestly answer, they decline and say what is missing. Given an artifact that fails, they do not warn, they fail, and several of them rename the file so it cannot be used by accident.
The rule underneath it, learned the expensive way: a check that has only ever passed is not a check. Before any of these is trusted, it has to be shown failing something known to be bad. Several were written, passed everything, and turned out to be measuring nothing at all.
Every line below is real output from a real run, not an illustration.
Checks that a 3D model can actually be printed rather than merely being valid. It caught embossed digits on a measuring gauge that looked fine and were half sunk into the plate:
The fix was removing the digits, not raising the allowance.
A bending jig for bamboo rods, which measured itself and found the bend it imposes is smaller than the slop in its own hole. The rod could sit dead straight inside it:
Three separate faults, in a part that had passed every check it previously had, because those checks compared the part to its own description rather than to the rod.
Written for a second-hand hardware idea. Asked to value something it has no measured sales for, it declines and says what to go and measure:
It also refuses to treat an asking price as evidence, because an asking price is a wish. The idea it was built for was later abandoned on other grounds. The refusal outlived it.
Runs before anything reaches this site. It blocked a page whose fact-check had never run, and separately caught a page that named source files a reader had no way to obtain.
Five things, each of which was learned by getting it wrong first.
Fail, do not warn. A warning is read once and then filtered out. Failure stops the work. Where an artifact is produced, the failing one gets renamed so it cannot be picked up later by mistake.
Prove it can fire. Point it at something known to be broken and watch it fail before believing any pass. A gate written and never fired is decoration.
Measure the thing, not the recipe. Read the file that was produced, not the variables that produced it. A summary line recomputed from the same inputs will agree with itself forever while being wrong. Better still, measure by a different route than the one that made the number.
Refuse on missing data, not just on bad data. Most damage comes from confident numbers with nothing underneath. Silence is a usable answer; a plausible invention is not.
Watch the right thing. The subtlest failure here was a monitor that checked whether messages were being delivered, while the thing producing them had been dead for a week. It reported healthy every time. Delivery health was not content health.
A gate that is too strict lies in the other direction. One here was written to enforce a rule so rigidly that it reported a conflict between two rules that did not actually conflict. The tool was wrong and the rule was fine, and it took a while to believe that.
A gate can also fail to land. One was aimed at a part expected to be caught, the part passed, and the conclusion drawn was that the gate was broken. It was not: the test never reached the thing being guarded. Both readings of a non-firing gate have to be checked.
And the most common trap of all is a probe that reports success and never errors, so it looks like evidence. Three wrong conclusions in one day here came from measuring apparatus rather than from anything being measured.
Two of the four are in the public crackle repository: tools/qa_stl.py is the geometry gate, and validate.py holds a numbered set of rules that each fail loudly and were each shown firing before being trusted. Both are plain Python with no dependencies.
The other two, the publishing gate and the price tool, are in private repositories and are not fetchable. They are described above rather than offered, which is the honest version.
The pattern needs none of the code. It is one habit: make the tool decline.