The Error Without a Lesson
Expert professions hold a secret no one states: they do not progress by avoiding error, but because their errors benefit everyone. AI has just broken that mechanism.
Aviation did not become the safest activity in the world because planes stopped falling. It became so because every accident produced a report, every report an analysis, every analysis a rule, and every rule a generation of accidents avoided. Medicine has its morbidity and mortality conferences, where teams dissect their own failures before their peers. Law has the oldest of these machines: case law, which is nothing but the public catalogue of settled disagreements and corrected errors, offered to all who will plead afterwards. Everywhere, the same mechanism: the failure of one becomes the lesson of all.
This mechanism is so constitutive of the professions that it has become invisible. Yet it rests on precise, costly conditions, won at great expense: that the error be visible instead of buried, analyzable instead of shameful, its lesson published instead of kept private. The professions accepted a price few human activities consent to: exposing their failures so the collective could draw something from them. And it is precisely this circuit, this conversion of individual error into common good, that artificial intelligence is short-circuiting without anyone having decided it.
A profession does not progress by avoiding error. It progresses because its errors benefit everyone.
The machine that converts failure
Look at the machine closely, for its sophistication goes unnoticed. For an error to become a collective lesson, it must first be detected, which requires an outside gaze: the appellate judge, the conference peer, the investigator from the analysis bureau. It must then be attributable, not to punish, but to understand: an error whose genesis is unknown teaches nothing. It must finally be published in a reusable form: the annotated ruling, the accident report, the teaching case. At every step, an institution works, people are paid, time is devoted to what seems unproductive: examining what went wrong.
This unproductive work is in reality the most profitable investment the professions ever made. Every analyzed error spares its repetitions, and the spared repetitions number in the thousands. A landmark ruling avoids decades of ill-conceived litigation; an accident report changes the design of every following aircraft; a morbidity case changes a protocol worldwide. The economics are spectacular: the cost of one analysis against the cost of a thousand repetitions. It is this economy that justifies the price paid, including the most painful one, the public exposure of failure.
The most profitable work of the professions is the work that seems most unproductive: examining what went wrong.
The mute error
Now here is what happens with AI. An associate receives a generated analysis, spots an error in it, corrects it, and moves on. The gesture seems trivial, even virtuous: supervision worked. But follow that error's path. It was detected by a single person, corrected in silence, and its correction went nowhere. No report, no registry, no publication. Yet the error was rich in lessons: it said something about the system, its blind spots, the conditions under which it derails. That something is lost. And tomorrow, in a thousand other organizations, a thousand other associates will meet the same error and correct it in the same silence.
Comparison with human error illuminates the loss. When a professional errs, the error is idiosyncratic: it stems from their fatigue, their training, their particular bias. The collective lesson is useful but bounded. When a system used by thousands of organizations errs, the error is correlated: the same blind spot produces the same flaw everywhere, at the same moment, under the same conditions. It is exactly the type of error whose collective analysis would yield the most, since each lesson would apply to all users at once. And it is exactly the one no longer analyzed: dispersed into private corrections, invisible as a pattern, it exists for no one as a phenomenon.
A subtler loss follows. When the model's publisher fixes a defect, the fix arrives by update, without an autopsy. Behavior changes, no one knows why or what was at fault. It is the exact inverse of the professional culture of error: where aviation publishes what failed so everyone understands, the software fix erases what failed so no one worries. The error is repaired, the lesson is destroyed. The system improves, and the community that uses it learns nothing.
Each user silently corrects an error a thousand others will meet. The correction exists everywhere; the lesson exists nowhere.
Why no one sees it
This regression is invisible for a reason rooted in its very nature: quality rises while the loop breaks. Models err less than ever, deliverables are better, indicators reassure. But a profession's safety never rested on the rarity of errors: it rests on what is done with those that occur. A system that errs little but whose errors teach nothing is more dangerous, over time, than a system that errs more but whose every fault feeds the collective. The first accumulates a debt of ignorance that cannot be seen; the second accumulates a capital of lessons that cannot be seen either. The two invisibilities look alike; their trajectories diverge.
There is finally a second-order effect, the slowest and deepest. The professions did not only accumulate lessons: they trained readers of errors. The jurist who studies case law learns to reason on others' failures; the physician at the morbidity conference learns to dissect a decision; the investigator learns to trace a causal chain. This competence, reading error, is a craft in itself, and it is maintained by exercise. If errors cease to be exposed and analyzed, it is not only the stock of lessons that impoverishes: it is the very capacity to draw them that atrophies, for lack of material.
To measure what is at stake, imagine law without its machine. Suppose every court decision were rendered in private, notified only to the parties, never published nor commented; that appeals corrected the first judges' errors without anyone knowing their content. Law would still function, matter by matter. But it would cease to be a learning system: every court would repeat the others' errors, every lawyer would plead without precedent, and the profession's accumulated experience would shrink to what each person had personally lived. That is exactly the regime assisted work is entering: a regime of decisions rendered in private, corrected in private, and lost to all.
A system that errs little but teaches nothing is more dangerous than a system that errs and instructs.
Rebuilding the loop
The answer is obviously not to wish for more errors, nor to slow correction. It is to rebuild, around assisted work, the circuit the professions built around human work: detect, record, analyze, share. Concretely, this means the correction of an assisted output should never be a purely private gesture. What a reviewer spots should leave an exploitable trace: which error, in which context, on which type of task. These traces, aggregated, would give back to the collective what dispersion took from it: the view of patterns, the detection of recurring blind spots, the material for a case law of usage.
It is a design choice before being an organizational one, and here the way systems are built becomes decisive again. A tool that treats each session as an isolated event disperses lessons by architecture; a layer that writes each correction into an institutional memory capitalizes them by architecture. At MAX, this is one of the reasons memory is not a feature but a foundation: what the firm corrects must enrich what the firm knows, failing which every review is work lost to all who follow. The error loop will not be rebuilt by goodwill; it will be rebuilt where architecture makes it possible.
The professions took centuries to learn that the hidden error costs more than the exposed one. On that discovery they built their most precious institutions, the ones that turn failure into heritage. It would be paradoxical if, at the moment their instruments become the most powerful in their history, they let that heritage stop growing, not by decision, but by distraction: because the new errors, dispersed across millions of sessions, no longer belonged to anyone and no longer instructed anyone. A profession's progress was never the sum of its successes. It was always the sum of its errors understood.
A profession's progress was never the sum of its successes. It was always the sum of its errors understood.
