The Effective AI Engineer · Case 009 of 16
The Engineer Who Asked for a Worse Demo
Applying Lateral Thinking to Create Meaningful Impact with AI
A useful AI demonstration reveals what you need to know before you depend on it.
This is a fictional teaching case informed by primary research and engineering guidance. Its characters, events and evaluation findings are illustrative. Sources checked on 6 November 2026.
1. The situation: an impressive demonstration, an unanswered question
A software company was preparing to pilot an AI assistant for customer-support specialists.
The assistant could retrieve product documentation, inspect authorised account information and draft a proposed resolution. A specialist reviewed the draft before sending it to the customer. The assistant could not change account settings or issue credits.
The demonstration was convincing.
A customer described a familiar problem. The assistant retrieved the relevant guide, explained the cause and produced a clear response.
The team had rehearsed several examples. All showed the intended experience.
But Neel, the engineer responsible for evaluation, still could not answer a practical question:
Under which conditions should a specialist trust the draft, ask for more information or take over?
The team needed evidence to define a pilot boundary. A successful presentation alone could not provide it.
2. The story: the demonstration that changed the decision
“Can you make the next demo worse?” Neel asked.
Tara, the product lead, looked up from the presentation.
“The customer workshop is next week.”
“I mean the engineering review.”
“What would a worse demo tell us?”
Neel pointed to the selected examples.
They all contained a clear product name, a current guide and enough information to identify the problem.
“Every customer in this deck has already done half the diagnosis for us.”
The team’s first response was to add harder questions.
One engineer proposed an obscure configuration. Another suggested a long message containing several problems. Someone asked AI to generate exceptionally difficult customer requests.
The generated list included invented product features and situations outside the proposed pilot.
Tara rejected several examples.
“We could make anything look bad with impossible requirements.”
Neel agreed. Difficulty alone was not the objective.
He wrote their normal selection rule:
Choose the example that makes the assistant look most capable.
Then he reversed it:
PO: Choose the example that makes the assistant look least capable.
They imagined reviewing that example together.
What would make it useful rather than embarrassing?
It would need to expose a believable weakness, identify its consequence and help the team decide what to change.
They began with one ordinary request.
Then they changed one condition.
The customer’s product version was missing.
The assistant confidently recommended instructions for the current release.
“Good,” Neel said.
Tara raised an eyebrow.
“Good that we found it here.”
They added a second variant. This time, the version was supplied, but an older guide ranked above the applicable one.
The assistant cited the older guide.
The citation was real. The recommendation was wrong for that version.
A third variant made the account lookup unavailable.
The assistant treated the missing result as confirmation that the customer did not have the relevant feature enabled.
The team now had distinct failures to investigate: missing information, inappropriate evidence and misinterpreted tool failure.
They asked AI to propose further variants for those specific behaviours.
This produced a more useful set. Engineers checked that the product details were real, the expected behaviour was justified and the simulated tool responses matched actual interface contracts.
Then their evaluation failed in an unexpected way.
A specialist reviewed a draft that asked a sensible clarification question. The automated grader marked it wrong because it did not contain the reference resolution.
“That answer should not be given yet,” the specialist said.
The test expected resolution when the available evidence justified clarification.
They corrected the task specification.
Another draft received a high score for politeness and completeness. It also included an unsupported compatibility claim.
The grading rubric had allowed good presentation to compensate for incorrect advice.
Neel separated the dimensions. Supportedness became a required condition. Style remained useful, but could not cancel a substantive error.
The team’s initial fix was to make the assistant more cautious.
The challenge cases improved.
Ordinary cases became frustrating.
Even when the request contained all the required information, the assistant asked the customer to repeat it.
“We’ve taught it to pass the difficult demo by delaying everyone,” Tara said.
They retained the challenge set and restored a separate set representing routine work. They also created an independently reviewed holdout set that was not used for prompt revisions.
The next comparison examined both kinds of behaviour: answering when evidence was sufficient and asking a targeted question when it was not.
One more test exposed a flaw in their evaluation environment.
An internal fixture contained a field labelled “expected response.” It had accidentally been included in the material accessible to the assistant.
That task could no longer establish whether the assistant had solved the problem independently.
They removed the leaked material and invalidated the affected runs.
For each new comparison, they reset the test account state, recorded the system configuration and inspected both drafts and observable tool traces. They repeated trials rather than selecting the best response.
The findings remained mixed.
The revised assistant handled some missing-information cases more appropriately. Retrieval still selected unsuitable evidence in certain version-specific tasks. Additional checking increased response time. Some clarification questions remained unnecessary.
The team narrowed the pilot to supported product versions and workflows with dependable evidence. Specialists received explicit instructions about unresolved cases.
They also assigned owners to the retrieval and evaluation defects.
At the review, Neel showed a successful routine example, a challenge the assistant now handled and an unresolved failure.
The presentation looked less polished.
The pilot decision was more precise.
Tara changed the heading on the final slide:
What we can support—and what we still need to learn.
3. The Lateral Thinking tool: reversal provocation and movement
Tool used: Reversal provocation, followed by movement.
Edward de Bono describes reversal as taking a normal direction or relationship and turning it around. Movement then develops useful ideas from the provocative position. [1]
The reversal here concerns how demonstration examples are selected:
- Normal direction: Select examples that make the system look most capable.
- Reversal: PO, select the example that makes the system look least capable.
- Movement: Imagine reviewing that example. What weakness becomes visible? Is the situation credible? What decision would the evidence change?
- Practical possibility: Build a deliberate challenge review alongside representative evaluation.
The aim is not to produce failure for its own sake. The provocation changes the selection criterion long enough to discover a more useful review process.
Original-method reference: Edward de Bono, Serious Creativity (1992), the stepping-stone method and reversal, printed pages 168–171.
The evaluation design developed here is a contemporary engineering application. The cited AI organisations do not describe their practices as applications of de Bono’s method.
4. Making the movement visible
The team moves through five questions:
-
What does the polished demonstration systematically leave out?
Missing facts, conflicting evidence, unavailable tools and situations requiring clarification. -
Which omissions matter within the intended use?
A difficult case should connect to a real task, plausible condition or consequential boundary. -
What should appropriate behaviour look like?
Resolution, targeted clarification, a bounded draft or escalation, depending on the evidence. -
Could the evaluation itself be wrong?
An invalid expectation, weak rubric or leaked answer can distort the result. -
What decision follows from the finding?
Improve the system, repair the evaluation, narrow the pilot or collect more evidence.
AI’s contribution
AI generates candidate variations and helps organise failure patterns. Product specialists validate realism and expected behaviour. Engineers control fixtures, inspect outcomes and decide which findings justify changes.
Reusable prompt
We are evaluating an AI assistant for [defined workflow].
Its intended users, permitted actions and pilot boundaries are [details].
PO: Select the demonstration that makes the assistant look least capable.
Use this reversal to generate plausible situations that reveal important limitations. For each, identify the condition changed, the evidence available, acceptable behaviour, unacceptable behaviour and the decision the result could inform.
Separate representative tasks from deliberately selected challenges. Flag invented facts, ambiguous expectations and assumptions requiring specialist review. Do not invent test results.
5. Comparing the evaluation approaches
| Approach | What it reveals | Main limitation | Role in this case |
|---|---|---|---|
| Curated demonstration | Intended experience and visible capability | Selection and rehearsal limit what it establishes | Communicate the product |
| Representative task evaluation | Performance on a defined sample of expected work | Depends on coverage, sampling and valid grading | Assess routine usefulness |
| Deliberate challenge evaluation | Behaviour at selected difficult conditions and boundaries | Does not estimate how often those conditions occur | Discover weaknesses and test responses |
A challenge set is intentionally selective. Its failure rate should not be presented as the expected failure rate across ordinary customer requests.
Similarly, a strong routine average should not conceal a consequential failure in a particular task category.
The team therefore reports routine performance, challenge findings and unresolved limitations separately. Acceptance criteria include substantive requirements that presentation quality cannot offset.
6. What the research supports—and what it does not
Evaluate outcomes and validate the graders
Anthropic’s agent-evaluation guidance distinguishes task outcomes from transcripts, recommends appropriate combinations of code, model and human grading, and discusses ambiguous tasks and grading defects. It also differentiates capability evaluation from regression testing. This supports examining whether the evaluation measures the intended behaviour. [2]
AI can help generate behavioural scenarios
Anthropic’s Bloom research demonstrates a pipeline for generating scenarios targeting specified behaviours, executing interactions and judging results. It also examines realism and human agreement. Its reported validation concerns particular behavioural evaluations; it does not establish that generated scenarios or judges are automatically valid for customer support. [3]
A score can improve without the intended task being solved
NIST documents agent-evaluation examples involving access to solutions and exploitation of grading loopholes. It recommends transcript review, clearer task rules and closing design gaps. These findings support checking what a passing score actually demonstrates; they do not imply that every agent deliberately exploits evaluations. [4]
Comparison requires attention to uncertainty
Anthropic’s statistical research discusses repeated sampling, paired comparisons and dependence between related questions. Closely related variants provide less independent evidence than the same number of unrelated tasks. A small observed improvement may therefore be insufficient to establish a reliable difference. [5]
Case synthesis and limits
The combined review process is an engineering adaptation. The fictional pilot findings are not empirical results. No finite evaluation establishes universal reliability, and simulated conditions must be checked against the actual application.
7. Practice: turn one impressive example into useful evidence
Accessible exercise
Choose a demonstration from an AI workflow you know.
- State the intended user outcome.
- Identify the conditions that make the example easy.
- Change one plausible condition at a time.
- Define acceptable behaviour before examining the response.
- Ask a colleague to review whether the task and expectation are fair.
- Record what each result would change about your implementation or deployment decision.
Technical extension
Create three distinct collections:
- Development cases: Available during iteration.
- Holdout cases: Independently reviewed and reserved for assessing changes.
- Regression cases: Previously important behaviours that must remain supported.
Include both routine tasks and justified challenges. Review:
- Missing, contradictory or stale information.
- Tool errors distinguished from valid empty results.
- Supported answers and unsupported claims.
- Appropriate clarification and unnecessary clarification.
- Cases outside the intended scope.
- Evaluation material accidentally accessible to the agent.
Keep system configurations and execution budgets comparable. Reset relevant state between trials. Record all trials, including failures and infrastructure errors, with explicit treatment rules.
Measure task correctness, evidence support, useful clarification, excessive escalation, human correction effort, latency and cost. Inspect observable tool traces and resulting artifacts.
For material decisions, report uncertainty using methods appropriate to the sample and its dependencies. Repeated attempts on a few related cases do not substitute for broader task coverage.
8. Professional benefit and transfer
For AI engineers
- Convert surprising behaviour into reproducible investigation.
- Separate system defects from evaluation defects.
- Test whether an improvement transfers beyond rehearsed examples.
- Detect when caution improves difficult cases while harming routine usefulness.
- Connect findings to concrete implementation choices.
For technical leaders
- Ask what the demonstration establishes and what remains untested.
- Require clear pilot boundaries and ownership of unresolved issues.
- Examine consequential failures separately from aggregate scores.
- Budget for evaluation maintenance and specialist review.
- Use evidence to adjust scope rather than demanding a presentation without visible weaknesses.
Limits of transfer
A challenge review complements representative testing. It cannot establish production prevalence or replace monitoring after deployment.
Reflection question
Which plausible customer request would you hesitate to include in the demonstration—and what would testing it help you decide?
9. Applied artifact: Demonstration to Evidence Record
For each evaluated scenario, record:
- Intended user outcome and permitted system actions.
- Scenario origin and relevance to the intended workflow.
- Whether it is representative, a challenge or outside scope.
- The condition varied and why it matters.
- Available evidence and tool behaviour.
- Acceptable outcomes, including valid clarification or escalation.
- Unacceptable outcomes and consequence.
- Grading rules and specialist validation.
- Configuration, environment and execution budget.
- Dataset role: development, holdout or regression.
- Trial results and treatment of invalid runs.
- System, task, grader or infrastructure defects found.
- Corrective action, owner and retest requirement.
- Decision supported, remaining uncertainty and pilot boundary.
Completion check
Produce a routine task, a plausible challenge and an intentionally flawed grader example. Explain how you would distinguish a system failure from an invalid evaluation. Identify what evidence would justify changing the pilot scope.
The professional value of a difficult demo is the better decision it makes possible.
Research references
- Edward de Bono. Serious Creativity (1992). The stepping-stone method and reversal, printed pages 168–171. Original method source supplied with this project.
- Anthropic. Demystifying Evals for AI Agents . 9 January 2026.
- Anthropic. Introducing Bloom: An Open Source Tool for Automated Behavioral Evaluations . 19 December 2025.
- National Institute of Standards and Technology. Cheating on AI Agent Evaluations . Updated 2 December 2025.
- Anthropic. A Statistical Approach to Model Evaluations . 19 November 2024.
Sources checked on 6 November 2026. The scenario, dialogue and comparison findings are fictional. Research supports the engineering discussion; it does not validate the fictional implementation or establish universal performance guarantees.
