AI Engineering Principles & Practice · Post 002
Check the Work, Not Just the Words
When an AI system uses tools or changes data, its final response is only one part of the evidence. Verify the result—and inspect the activity trace when the actions matter.
The principle
An AI agent can say “Done” without completing the task. It can also complete an important step without mentioning it in the final response.
For systems that call tools, retrieve information, or change an external state, evaluate more than the wording of the final answer. Check whether the intended outcome actually occurred. When the process itself matters—for example, whether the agent checked a source, used the correct tool, or respected a restriction—inspect the recorded actions as well.
The record of an agent’s run may include its messages, tool calls, and intermediate results. The final state of the environment provides separate evidence of what changed. These views answer different questions: what the agent did, and whether the task was accomplished.
A practical example: updating a support ticket
Imagine an AI agent is asked to update a customer’s support ticket after a refund has been approved. Its final response says, “The ticket has been updated.”
That response alone does not establish that the task succeeded. Check evidence such as:
- Was the correct ticket opened?
- Did the system record the approved refund or update?
- Was the permitted tool used?
- Did the agent avoid making an unapproved change?
- Does the final response accurately describe the recorded result?
The checks should match the task. If the requirement is that a record changed, verify the record. If the requirement includes a required approval or safety step, inspect evidence that the step occurred.
Evaluate the outcome without overconstraining the path
Google Cloud described an evaluation where a grader marked three of eleven prompts as failures because the agent’s final responses did not mention required checks. Review of the tool-call logs showed that the agent had performed those checks. This illustrates why a final-message-only evaluation can miss relevant behavior.
At the same time, a test can be too rigid if it requires one exact sequence of tool calls. An agent may reach the correct, safe result through another valid route. A useful evaluation checks the required outcome and any actions that genuinely matter, while allowing reasonable alternatives.
Independent study exercise
Choose a task for an AI system that uses a tool or changes information. Write down:
- Expected outcome: What observable result would show that the task was completed?
- Evidence from the run: Which tool calls, inputs, or intermediate results matter?
- Valid alternatives: Could the agent succeed through more than one acceptable sequence of actions?
- Failure conditions: What would count as a failure even if the final response sounded confident?
- Repeatability: Which examples will you run again after changing the prompt, model, or tools?
Keep the test cases and expected outcomes together. Re-run them after changes, and review individual traces when a score does not explain what happened.
Research basis
- Google Cloud, “Four evaluation engineering lessons learned from building agent plugins” (October 16, 2026). The article describes checking execution traces when an evaluator’s score did not match the agent’s actions.
- Anthropic, “Demystifying evals for AI agents” (January 9, 2026). The guide distinguishes run transcripts from final environment outcomes and cautions against overly rigid graders that penalize valid approaches.
This post is an independent study reference based on publicly available engineering material. It is not an official guide or endorsement from the cited organizations.
AIYING — Accomplishing with AI
AI Engineering Principles & Practice · 002
