AI Vocabulary

Training corpus: how recently has the model learned?

A model was trained on texts gathered up to a given date, its cut-off. Beyond it, it knows nothing. This is not a lag an update would close: it is a property of how the model was built, and it bears directly on what it can say about the law in force.

What the date covers

Training a model means having it read a considerable quantity of text, gathered at a precise moment. That collection stops, training begins, and it runs for months. The model that emerges knows the world as it stood at the date of collection.

Between that date and the moment you interrogate it, a year commonly passes. A decision handed down 8 months ago, a statute in force this quarter, an administrative position clarified over the summer: the model is unaware of them, and nothing in its answer will say so.

The behaviour that misleads

A model asked about a development after its cut-off does not reply that it does not know. It answers from what it has, with the same assurance as on everything else.

On a legal question that produces an answer describing the earlier state of the law without flagging it. It is accurate as at its date and wrong today, and nothing in the reading distinguishes the two.

This is close to hallucination without being one. The model invents nothing: it faithfully restores a state of the world that is no longer the right one.

A frequent misconception

It is often believed that the model learns through use, and that information given today will correct its knowledge. It does not: what you give it serves to answer your question, then disappears.

Likewise, a new version of a model pushes the cut-off back without removing the problem: it moves it by a few months.

What fixes the problem

One mechanism only: supplying the source at the time of the question, either by attaching it yourself or by having the system fetch it from a corpus kept current.

That is precisely what a retrieval-before-answering arrangement does. The model stops answering from memory and answers from what has been put in front of it, which shifts the question from its own cut-off to the freshness of the corpus searched.

The right question to a provider therefore becomes: what corpus does the arrangement rely on, and how often is it updated? That bears on something verifiable, unlike a model's cut-off date.

What it does not solve

Supplying a recent source at the time of the question settles that case and no other. The model will know no more next time.

And the cut-off is not visible from outside. Providers rarely publish it prominently, and a model asked about its own cut-off gives an unreliable answer, that information not necessarily appearing in what it learned.

The cut-off is also not clean. Corpora contain documents of varying dates, and some fields are less well covered than others. A model may therefore know a development from January and be unaware of one from the previous November.

Why it matters to a lawyer

Because law is dated by nature, and your raw material changes several times a year. A model unaware of the last 6 months is unusable alone on any question of law in force.

The practical consequence is stark: on a question about the current state of the law, a model interrogated without a source does not merely risk error, it guarantees lag. What counts is what it does with a source you supply.