No. 104AI Systems

Context vs Behaviour

Diagnose an LLM failure first: missing knowledge needs better context; wrong behaviour needs better prompting or tuning.

Classify each failure before fixing it: missing facts need retrieval, misused facts need better prompting or tuning.

To an executive

When an AI feature gets things wrong, the fix depends on whether it lacks information or acts inconsistently. Diagnosing first avoids paying for the wrong remedy.

To an engineer

Build a baseline prompt and an evaluation set, then classify failures. Add retrieval when knowledge is missing, and fine-tune when format, tone or reasoning is inconsistent.

The idea

OpenAI's guide to optimising LLM accuracy offers a simple diagnostic. When a model gets things wrong, there are two different kinds of problem. Context problems mean the model does not have the knowledge it needs: missing, outdated or private information. Behaviour problems mean it has the knowledge but does not act consistently: wrong format, tone or reasoning steps.

The fixes differ. Retrieval-augmented generation (RAG) supplies missing context. Fine-tuning shapes behaviour. Prompt engineering is the starting point for both, and the approaches can be combined.

Why it works

It stops expensive guesses. Teams often fine-tune to add knowledge, or add retrieval to fix formatting, and get little for the effort.

It starts from evidence. The guide begins with a baseline prompt and an evaluation set, then looks at why each failure happened.

Where it applies

Improving an LLM feature that is not accurate enough.
Vendor proposals that offer fine-tuning or retrieval as the answer before the problem is understood.
Planning effort, since prompting is cheap, retrieval moderate and fine-tuning the most costly to maintain.

How to apply

Build a baseline. A clear prompt plus an evaluation set with known good answers.
Classify the failures. For each wrong answer, ask: did it lack facts, or did it mishandle facts it had?
Fix context with retrieval. Supply the right documents at the right time.
Fix behaviour with prompting first, then fine-tuning. Use a small, high-quality, representative dataset if you tune.
Re-measure after every change and combine methods only where each one helps.

Example scenario

A bank's assistant answered product questions wrongly about a third of the time, and a vendor proposed fine-tuning. The team classified a sample of failures first: most were questions about products launched after the model's training data, and the rest were answers in the wrong format.

They added retrieval from the product catalogue and a stricter output template. Accuracy rose sharply without any fine-tuning.

Questions to ask

Is the model wrong because it lacks information, or because it behaves inconsistently?
What does our baseline score on an evaluation set, and which failures dominate?
Have we exhausted prompting before reaching for retrieval or fine-tuning?

When not to use it

Failures that come from the task itself, such as ambiguous requirements or bad source data, which no model technique will fix.

How it misleads you

Real failures are often mixed. Forcing every failure into one bucket hides the cases that need both better context and better behaviour. Classify honestly, and accept that some answers need both fixes.