AI Vocabulary

Inference: what happens between your question and the answer?

Inference is the moment a model produces an answer from what it is given. It is the execution stage, to be distinguished from training, which happened once and for all before you ever used the system. Each inference starts from the same model and changes nothing in it.

Training and inference

Training is a long and costly process that produced the model. It is finished: the model you interrogate today is identical to yesterday's, and will remain so until the laboratory publishes a new version.

Inference is what happens each time you interrogate it. It takes seconds, it consumes computing resources, and it leaves no trace in the model.

The consequence everyone notices without naming it

The model does not learn from you. What you explained to it yesterday, it does not know today. What you corrected will not be corrected next time.

This is the source of a widespread frustration: the sense of training a colleague who starts afresh every morning. The impression is accurate, and it stems not from a design flaw but from the nature of inference itself.

When a system appears to remember, it is never the model remembering: it is a layer around it that retained something and hands it back at the next inference.

What the word covers on the provider's side

You will meet the term in two contexts that do not mean the same thing for you.

In a technical discussion, inference means the computation itself, and its cost: it is what is billed, it is what takes a few seconds, and it explains why a complex request is slower than a simple one.

In a contractual discussion, inference denotes the phase in which your data is transmitted to the engine — as opposed to training. An undertaking not to use data for training therefore says nothing about what happens during inference: retention, logging and operator access are separate questions to be asked separately.

A note on vocabulary, since the word appears in quotations: inference cost means the price of the computation at each request, as against training cost, which you never pay directly. It is therefore the only one of the two that appears on your invoice.

What it does not solve

That inference does not modify the model guarantees nothing about confidentiality. Your data may be retained, logged, read by an operator or used for later training, depending on what the contract provides — questions distinct from how the technology works.

And the model's fixed character does not make it stable over time. The laboratory may replace it with a new version without notice, and behaviour then changes overnight on tasks that were working.

Why it matters to a lawyer

Because it explains why context must be resupplied every time, and why that cost never amortises. Each inference starts from nothing plus what it is given, and nothing else.

It also lets you put the right question to a provider promising that its system learns from your usage: what exactly learns, since it is not the model? The answer describes the real architecture, or reveals that there is none.