AI Vocabulary

Orchestration: who decides what?

Orchestration is the layer that decides the order of operations: which model to call, with what information, in what sequence, and what to do with each result. It produces no text itself. It organises the work of those that do.

Where the term comes from

From distributed computing, where an orchestrator coordinates components that know nothing of one another. The musical image is apt: the instruments play, the conductor does not, and yet the outcome depends largely on them.

Applied to artificial intelligence, the orchestrator sits between your request and the model. It decides what the model sees, when it is called, and what becomes of its answer.

What it decides, concretely

On any moderately complex task, a dozen decisions are taken before the model produces anything. Should the request be split into stages? Which documents to attach, given that not everything can be attached? Should a source be retrieved before answering? Should the result be checked by a second pass?

None of those decisions belongs to the model. They are taken by the surrounding layer, and they determine a large share of the final quality.

One decision deserves isolating because it is invisible and decisive: what is left out. No system can pass an entire matter to the model; something must be selected, and that selection is made by the orchestration according to rules the user never sees.

Why it is the least visible subject

Because a user never sees orchestration. They see a request and a result; everything in between is hidden, and they naturally attribute the quality of the result to the model.

That is an attribution error comparable to judging an orchestra by the quality of its instruments. At equal models, two different orchestrations produce results with little in common.

This invisibility also explains why two organisations using the same model obtain very different results, and why comparing tools by the name of their engine tells you almost nothing.

What it does not solve

Good orchestration does not compensate for absent context. It can decide intelligently what to attach to a request; something must still exist to attach.

Nor does it remove the model's errors. It can make them more detectable, by providing for a verification pass or requiring each assertion to be tied to a source, but it does not prevent them.

A sign of maturity is worth watching for: does the orchestration know how to stop? An arrangement that always answers, whatever the quality of what it found, is orchestrating very little. One that finds it has nothing to answer with, and says so, has made a decision.

And orchestration adds complexity that has a cost: each additional stage lengthens response time and multiplies the points at which something can go wrong.

Why it matters to a lawyer

Because this is where the decisions that concern you are taken, and they are taken by the provider without your seeing them. Deciding that an answer will be produced without checking its sources is an orchestration decision, not a limitation of the model.

The useful question is therefore not which model is used, which changes every six months, but what the system does before and after calling it.