The Effective AI Engineer • Case 15 of 16
The Engineer Who Looked Beyond the Answer
Applying Lateral Thinking to Create Meaningful Impact with AI
Before improving the answer, examine what the person needs to accomplish.
An AI assistant can give a correct answer while leaving the user unable to complete the task. This case explores how an engineer uses Edward de Bono’s Concept Fan to develop alternatives beyond answer generation—and evaluate them through the work the user is trying to finish.
Research-grounded fiction: The people, organisation and events are illustrative. The thinking method comes from Edward de Bono. The workflow design and engineering exercises are contemporary adaptations. No measured improvement in completion, speed or cost is claimed.
1. The situation
A company was piloting an AI assistant to help employees complete internal requests.
Employees could ask questions such as:
- Which request form should I use?
- What supporting documents do I need?
- Where should I send the completed application?
The assistant searched approved guidance and generated answers with references.
Its responses were becoming clearer. Its evaluation scores were improving.
Yet employees still submitted incomplete requests, selected unsuitable forms and contacted the service desk for help.
The product team proposed another improvement to answer quality.
Leela, the AI engineer, asked a different question:
What happens after someone reads a correct answer?
Learning outcomes
By completing this case, you should be able to:
- Distinguish an AI feature’s output from the user’s intended outcome.
- Build a Concept Fan with directions, concepts and concrete ideas.
- Generate alternatives involving AI, conventional software, clearer information and human assistance.
- Compare designs through complete task performance.
- Document requirements, exceptions, ownership and remaining uncertainty.
2. The story
A correct answer left the task unfinished
During a user observation session, Leela watched an employee named Arun ask which form he needed for an equipment request.
The assistant supplied a clear explanation. It described several request categories and linked to the relevant guidance.
Arun read the answer twice. Then he opened another tab and searched for the form.
“Did the assistant answer your question?” Leela asked.
“Yes.”
“Can you complete the request?”
“I still don’t know which of these options applies to me.”
The answer explained the categories. Arun needed help identifying his situation and completing the correct process.
A better answer was only one possibility
Back with the team, Leela wrote their current objective:
Improve the assistant’s answer about request forms.
Then she wrote a broader objective:
Help an employee submit a complete request through the appropriate route.
The second objective changed the available design choices.
A colleague suggested a shorter answer. Another suggested a larger model.
Both might help. But both remained inside the same approach: generate explanatory text and ask the employee to translate it into action.
Leela introduced a Concept Fan.
“What does the answer help us achieve?” she asked.
“It helps the employee understand the process.”
“And what else could help someone complete the process?”
The team developed three broad directions:
- Help the employee understand the requirements.
- Help the employee identify the applicable route.
- Reduce the work required to prepare a complete submission.
Each direction produced different concepts and concrete ideas.
Understanding could involve explanations, examples or visible requirements.
Route selection could involve a small set of discriminating questions or an accessible comparison of options.
Preparation could involve checking required fields, showing missing attachments or preparing a reviewable draft.
The team now had alternatives that another prompt rewrite would not have produced.
The first alternative created another problem
Their first prototype replaced the long answer with a short sequence of questions.
It worked well for a familiar request.
Then an employee encountered an unusual case. None of the offered choices described the situation.
The prototype still required a selection.
“We have made the route easier to follow,” Leela said, “but we have also made it easier to follow the wrong route.”
They added an explicit “My situation is different” option.
Where the applicable route could not be established, the prototype preserved the employee’s information and offered a handover to the service desk.
The questions also needed maintenance. A clear interface could still encode an outdated rule.
The team assigned ownership for the routing logic and linked each decision to its approved requirement.
AI did not need to perform every step
The next prototype combined several approaches.
An employee could describe the situation in ordinary language. AI proposed a request category, with the relevant information visible for review.
Approved rules determined mandatory fields and required attachments. The interface showed what was missing.
AI could help draft a description, but the employee could edit or discard it.
Before submission, the employee reviewed the destination, contents and attachments.
Nothing was submitted merely because a draft had been generated.
For some routine cases, the team found that a direct form link with a short requirements checklist was sufficient.
Those cases did not need a conversation.
The comparison needed a different measure
The existing evaluation asked:
Is the assistant’s answer correct and helpful?
The new comparison also asked:
Can the employee complete the intended task correctly?
The team prepared representative tasks and compared:
- The current question-and-answer assistant.
- A clearer answer with a direct form link.
- A guided workflow with visible requirements.
- A combined approach offering explanation and guided preparation.
They observed route selection, completeness, corrections, completion time and requests for help. </
