Begin with the decision the output supports
An answer can be fluent, relevant and still unsuitable for its purpose. A support draft may require the right policy reference and an appropriate escalation route. An information extraction task may depend on correctly preserving a document's conditions rather than merely finding a name. Evaluation criteria need to reflect those distinctions instead of treating every generated response as a general writing exercise.
Separate observable requirements from preferences. A required source citation can be checked differently from tone. An action performed without permission is different from an awkward sentence. If the customer must review every output, assess the review burden as well as the output itself. The strategic question is whether the whole workflow supports the promised benefit, not whether the model can sometimes produce an attractive answer.
Build comparisons around representative work
An evaluation set is a collection of inputs and expected judgements used to assess behaviour. Its usefulness depends on coverage, provenance and relevance. Routine requests matter, but so do incomplete records, conflicting documents, unusual formatting and requests beyond the intended scope. Keep examples used to tune the system separate from examples reserved to assess it, otherwise the comparison can reward familiarity rather than general performance.
Customer data needs appropriate permission and handling. Removing names does not necessarily remove identifying context, and access rights may differ between documents in the same collection. Artificially constructed inputs can help explore particular failures, but should not silently replace realistic workflow evidence. Record which conditions each example represents so that a broad-looking collection does not conceal repeated versions of the same easy task.
Match the review method to the claim
Deterministic checks work well for requirements such as valid output structure, permitted field values or the presence of a required reference. Human review is useful where correctness depends on context or professional judgement. Model-based grading can help organise larger comparisons, but the grader can make mistakes and favour particular styles. Agreement with a grader is not automatically agreement with the intended user.
Reviewers need written criteria and a way to record uncertainty. Where reasonable reviewers disagree, examine whether the task itself is ambiguous before treating disagreement as noise. Comparison with the existing process is also important: a replacement need not be perfect to be useful, but it must not introduce unacceptable failure modes. Public benchmarks can provide context without establishing suitability for a particular customer's documents, permissions or decisions.
Distinguish inconvenience from consequential failure
Not all errors deserve equal treatment. A formatting correction may be recoverable within the workflow; disclosing restricted information may not be. Systems that call tools or change external records need evaluation of permissions, action boundaries and recovery as well as language output. Prompt injection, where untrusted content attempts to redirect system behaviour, is particularly relevant when a product reads external documents or websites.
Abstention and escalation should be evaluated as product behaviours. Refusing an unsupported request can be the correct result, while refusing routine work can make a system unusable. Human oversight is not a complete answer unless the reviewer has enough information, time and authority to intervene. For specialised legal, health or financial uses, obtain appropriately qualified advice about the workflow and its consequences rather than treating general evaluation as professional assurance.
Turn results into an explicit release choice
A release comparison should make regressions visible alongside improvements. Changes to prompts, retrieval, models, tools and source documents can all alter behaviour. Keep enough configuration information to understand what was compared, and connect findings to the intended scope of release. A result for one document type does not establish performance across every account or language.
Offline evaluation and operational monitoring answer different questions. The first examines a controlled set of tasks; the second can reveal changing inputs, review patterns and failures in actual use. A consulting discussion should connect both to ownership: who can approve a release, restrict a feature or suspend a workflow when evidence is insufficient? Evaluation becomes strategically useful when it changes those decisions, rather than existing only as a reassuring slide.
A clear evaluation summary states the task, the comparison, the important failure types and the limits of the evidence. A single aggregate score rarely carries all of that information.