AIYINGFrom AI Awareness to Accomplishment
Showing posts with label Concept Fan. Show all posts
Showing posts with label Concept Fan. Show all posts

Monday, November 9, 2026

Lateral AI #5: Expand the Map Before Choosing


Friday, November 6, 2026

The Effective AI Engineer #015: The Engineer Who Looked Beyond the Answer

The Effective AI Engineer • Case 15 of 16

The Engineer Who Looked Beyond the Answer

Applying Lateral Thinking to Create Meaningful Impact with AI

By Ravindra Dastikop



Before improving the answer, examine what the person needs to accomplish.

An AI assistant can give a correct answer while leaving the user unable to complete the task. This case explores how an engineer uses Edward de Bono’s Concept Fan to develop alternatives beyond answer generation—and evaluate them through the work the user is trying to finish.

Research-grounded fiction: The people, organisation and events are illustrative. The thinking method comes from Edward de Bono. The workflow design and engineering exercises are contemporary adaptations. No measured improvement in completion, speed or cost is claimed.

1. The situation

A company was piloting an AI assistant to help employees complete internal requests.

Employees could ask questions such as:

  • Which request form should I use?
  • What supporting documents do I need?
  • Where should I send the completed application?

The assistant searched approved guidance and generated answers with references.

Its responses were becoming clearer. Its evaluation scores were improving.

Yet employees still submitted incomplete requests, selected unsuitable forms and contacted the service desk for help.

The product team proposed another improvement to answer quality.

Leela, the AI engineer, asked a different question:

What happens after someone reads a correct answer?

Learning outcomes

By completing this case, you should be able to:

  • Distinguish an AI feature’s output from the user’s intended outcome.
  • Build a Concept Fan with directions, concepts and concrete ideas.
  • Generate alternatives involving AI, conventional software, clearer information and human assistance.
  • Compare designs through complete task performance.
  • Document requirements, exceptions, ownership and remaining uncertainty.

↑ Back to contents

2. The story

A correct answer left the task unfinished

During a user observation session, Leela watched an employee named Arun ask which form he needed for an equipment request.

The assistant supplied a clear explanation. It described several request categories and linked to the relevant guidance.

Arun read the answer twice. Then he opened another tab and searched for the form.

“Did the assistant answer your question?” Leela asked.

“Yes.”

“Can you complete the request?”

“I still don’t know which of these options applies to me.”

The answer explained the categories. Arun needed help identifying his situation and completing the correct process.

A better answer was only one possibility

Back with the team, Leela wrote their current objective:

Improve the assistant’s answer about request forms.

Then she wrote a broader objective:

Help an employee submit a complete request through the appropriate route.

The second objective changed the available design choices.

A colleague suggested a shorter answer. Another suggested a larger model.

Both might help. But both remained inside the same approach: generate explanatory text and ask the employee to translate it into action.

Leela introduced a Concept Fan.

“What does the answer help us achieve?” she asked.

“It helps the employee understand the process.”

“And what else could help someone complete the process?”

The team developed three broad directions:

  1. Help the employee understand the requirements.
  2. Help the employee identify the applicable route.
  3. Reduce the work required to prepare a complete submission.

Each direction produced different concepts and concrete ideas.

Understanding could involve explanations, examples or visible requirements.

Route selection could involve a small set of discriminating questions or an accessible comparison of options.

Preparation could involve checking required fields, showing missing attachments or preparing a reviewable draft.

The team now had alternatives that another prompt rewrite would not have produced.

The first alternative created another problem

Their first prototype replaced the long answer with a short sequence of questions.

It worked well for a familiar request.

Then an employee encountered an unusual case. None of the offered choices described the situation.

The prototype still required a selection.

“We have made the route easier to follow,” Leela said, “but we have also made it easier to follow the wrong route.”

They added an explicit “My situation is different” option.

Where the applicable route could not be established, the prototype preserved the employee’s information and offered a handover to the service desk.

The questions also needed maintenance. A clear interface could still encode an outdated rule.

The team assigned ownership for the routing logic and linked each decision to its approved requirement.

AI did not need to perform every step

The next prototype combined several approaches.

An employee could describe the situation in ordinary language. AI proposed a request category, with the relevant information visible for review.

Approved rules determined mandatory fields and required attachments. The interface showed what was missing.

AI could help draft a description, but the employee could edit or discard it.

Before submission, the employee reviewed the destination, contents and attachments.

Nothing was submitted merely because a draft had been generated.

For some routine cases, the team found that a direct form link with a short requirements checklist was sufficient.

Those cases did not need a conversation.

The comparison needed a different measure

The existing evaluation asked:

Is the assistant’s answer correct and helpful?

The new comparison also asked:

Can the employee complete the intended task correctly?

The team prepared representative tasks and compared:

  • The current question-and-answer assistant.
  • A clearer answer with a direct form link.
  • A guided workflow with visible requirements.
  • A combined approach offering explanation and guided preparation.

They observed route selection, completeness, corrections, completion time and requests for help. </

Thursday, November 5, 2026

The Effective AI Engineer #004: The Engineer Who Stopped Improving the Prompt

The Effective AI Engineer · Case 004 of 16

The Engineer Who Stopped Improving the Prompt

Applying Lateral Thinking to Create Meaningful Impact with AI

By Ravindra Dastikop



Research-grounded fiction. The characters, organization and trial observations are invented. Engineering sources checked on 5 November 2026.

1. The situation and impact sought

Arjun was improving a support assistant for a company that sold business software.

Customers asked whether a particular extension worked with their installation, which upgrade path was supported and where to find the relevant instructions.

The assistant searched approved documentation and drafted a response for a support specialist to review. It could not change a customer’s installation.

Its answers were readable. Some were also wrong.

The team’s immediate response was to improve the prompt. Arjun added instructions about accuracy, version numbers, evidence and uncertainty.

Yet the support specialists still had to reopen documents, identify missing details and correct recommendations.

Arjun had two weeks, access to approved support examples and help from a documentation owner. His objective was to reduce the total effort required to produce a correct, supported response.

The baseline was the existing process: a specialist searched the documentation and wrote the answer. Review time, corrections and unresolved questions mattered alongside response quality.

Back to contents ↑

2. The story

By Thursday, Arjun’s prompt had become a small manual.

It told the assistant to check the product version, consider exceptions, cite sources and avoid unsupported claims.

One addition improved the handling of older installations. Another made straightforward answers unnecessarily cautious.

A third produced longer explanations without making the underlying recommendation more dependable.

“Perhaps we need better examples,” someone suggested.

Arjun opened the prompt again.

Then Leela, the documentation owner, asked him to show her a failed case.

The customer’s question mentioned an extension name but omitted the installed software version.

The assistant had retrieved a relevant page. Unfortunately, that page described a different release.

Arjun pointed to his latest instruction.

“It already says to check compatibility.”

Leela looked at the record.

“Where would it get the missing version?”

They opened another failure.

This time the version was present. The retrieved excerpt contained the general compatibility statement but omitted the exception immediately below it.

A third answer used the right documentation but misunderstood a supported limitation.

Three similar-looking wrong answers had different causes.

Arjun closed the prompt editor.

On a whiteboard, he wrote:

Help the specialist resolve the customer’s compatibility question correctly.

Below it he wrote the idea that had occupied his week:

Improve the prompt.

Then he asked what that idea was supposed to accomplish.

“Help the model interpret the evidence correctly,” he said.

That was useful. It was also only one route.

What else could help the specialist reach a correct resolution?

Leela suggested making the necessary evidence easier to obtain.

A support specialist suggested identifying incomplete requests before drafting an answer.

Another colleague proposed letting the specialist send an approved compatibility statement directly when no explanation was needed.

The whiteboard began to spread.

Under make the evidence available, they listed preserving product and release information, retrieving complete compatibility sections and making exceptions visible.

Under resolve uncertainty earlier, they listed asking for the installed version, checking whether supplied details conflicted and distinguishing missing information from an unsupported configuration.

Under help people use the evidence, they listed clearer model instructions, worked examples and a review screen that placed the proposed answer beside its supporting passage.

Under avoid unnecessary generation, they listed direct links to approved statements and a simple compatibility lookup for cases covered by structured records.

For the first time that week, they were comparing different ways to achieve the outcome.

Arjun asked an AI assistant to expand the map.

It produced a long list. Several suggestions were merely new names for prompt rewriting.

He asked it to explain the mechanism behind each option and merge those that worked in substantially the same way.

The revised map was shorter and more useful.

One proposal remained questionable: fine-tune a model on historical support answers.

Leela pointed out that some of those answers reflected superseded releases.

They kept the idea as a possibility requiring additional evidence. They did not treat the training archive as an approved source.

The team selected three small experiments.

First, Arjun revised the prompt using examples of genuine interpretation errors.

Second, he changed the evidence preparation so that a compatibility statement retained its product, release and nearby exception.

Third, he added a clarification step when the installation version was missing.

They initially tested each change separately against the same development cases.

The prompt revision helped with some answers where the necessary evidence was already available.

It could not supply an absent installation version.

The evidence change exposed an exception that the original excerpts had omitted.

The clarification step prevented a premature recommendation, but also introduced another customer interaction.

That cost mattered.

On a complete request, an unnecessary clarification would make the process slower.

Arjun revised the step to ask only for information needed to distinguish between materially different answers.

Then a new failure appeared.

The response cited an approved document, but the cited passage did not actually support the recommendation.

The team separated two checks: whether the source was legitimate and whether it supported the specific claim.

Finally, they compared the most promising combination with the original assistant and the manual process on reserved cases.

The fictional observations supported a limited trial with specialists. They did not establish a general reduction in handling time.

At the review meeting, Arjun showed the whiteboard before the demonstration.

His manager glanced at the prompt editor.

“Did you finish improving it?”

“Yes,” Arjun said. “After we gave it a smaller job.”

Back to contents ↑

3. The thinking tool explained

The tool is the Concept Fan.

In Serious Creativity, Edward de Bono describes a way of generating alternatives by moving between an objective, broad directions, concepts and practical ideas.

An existing idea becomes a starting point. Ask what it achieves, move to a broader concept and seek other ways of achieving that concept. Continue broadening when necessary.

Then develop the alternatives into specific actions.

De Bono distinguishes this from merely dividing a subject into categories: the purpose is to generate ways of achieving an objective. See printed pages 129–133 in the supplied edition. [1]

For Arjun:

  • Objective: Help a specialist resolve a compatibility question correctly.
  • Direction: Make the necessary evidence available.
  • Concept: Preserve the conditions under which a statement applies.
  • Practical idea: Return the product, release and adjacent exception with the retrieved statement.

Another branch leads elsewhere:

  • Direction: Resolve uncertainty before recommending.
  • Concept: Obtain information that changes the answer.
  • Practical idea: Ask for the installed version when different versions have different compatibility outcomes.

Improving the prompt remains a valid branch. The tool reveals alternatives around it.

This case uses the Concept Fan directly. It does not require a PO provocation, and the technique should not be relabeled as PO.

Back to contents ↑

4. The thinking made visible, including AI’s contribution

Focus: Reduce the effort needed to produce a correct, supported resolution.

Restricting assumption: The next improvement must be another change to the prompt.

Upstream move: What does a better prompt achieve? It helps the model use available evidence appropriately.

Broader move: What else enables a correct resolution? Better evidence, sufficient customer information, clearer review and direct use of approved answers.

Downstream move: Turn each direction into implementable alternatives.

Direction Concept Practical alternative
Improve interpretation Demonstrate correct reasoning from supplied evidence Add a small set of reviewed examples
Improve evidence availability Preserve applicability and exceptions Supply complete, versioned compatibility passages
Resolve uncertainty Obtain answer-changing information Ask a targeted clarification question
Support verification Make claims easy to compare with evidence Show the supporting passage beside the draft
Avoid unnecessary generation Reuse approved information directly Offer a maintained compatibility lookup

AI helps expand the branches, identify duplicates and propose tests. People establish the objective, approve product facts and decide whether alternatives are meaningfully different.

Reusable prompt

Help me construct a Concept Fan.

Useful outcome: [outcome].
Current idea: [the solution we keep improving].
Verified constraints: [constraints].
Observed failures: [examples].

Explain what the current idea achieves. Move upstream to broader concepts and generate at least three distinct directions toward the outcome.

Under each direction, propose concepts and concrete actions. Include an option that requires no additional model call.

Explain how each action could produce the outcome. Merge cosmetic variations. Mark unverified assumptions and propose a small comparison.

Do not invent policies, results or claims that one architecture is universally superior.

A well-organized map can still contain weak ideas. Its branches require judgment and testing.

Back to contents ↑

5. The engineering decision

Arjun’s decision was to test a small combination of evidence preparation, targeted clarification and a focused prompt.

Alternative Potential benefit Cost or uncertainty Decision
Prompt changes alone Improve interpretation of available evidence Cannot establish facts absent from the inputs Retain as a controlled comparison
Better evidence preparation Make applicability and exceptions visible Requires document maintenance and evaluation Test on documented retrieval failures
Targeted clarification Resolve missing, consequential details Adds interaction and possible abandonment Use only when the missing detail changes the answer
Direct approved lookup Avoid unnecessary answer generation Limited coverage and maintenance burden Compare on structured compatibility questions
Fine-tuning Potentially improve recurring behavior Requires suitable data and separate validation Defer pending evidence of need

The experiments distinguish an evidence problem from an interpretation problem.

For diagnosis, the team can provide a manually verified passage and observe whether the answer improves. This is a controlled diagnostic condition, not a deployable retrieval system or an independent proof of the cause.

Changes are tested separately before combining them. Otherwise, the team cannot tell which change helped or introduced a regression.

The final comparison must include clarification, review and maintenance costs. A better draft alone does not establish a better support process.

Back to contents ↑

6. The evidence behind the case

The engineering foundations are established; the story’s particular design and results are fictional.

Anthropic’s context-engineering guidance treats instructions as one component of the information available to a model. It discusses examples, tools and other context alongside prompt wording. This supports examining the surrounding information rather than assuming every failure needs another instruction. [2]

Anthropic’s Contextual Retrieval research explains how isolated document chunks can lose information needed to identify their meaning and applicability. It also describes combining semantic and lexical retrieval. Its reported results concern its tested configurations; they do not predict this fictional support system’s performance. [3]

Its guidance on building effective agents recommends starting with simple approaches and adding complexity when justified. It describes retrieval, examples and predefined workflows among the available design choices. The article originated in 2024 and now flags changes in the tooling landscape; this case uses its architectural principles rather than a current implementation recipe. [4]

Anthropic’s evaluation guidance distinguishes tasks, repeated trials, graders and outcomes, and describes the complementary roles of automated and human assessment. These principles support comparing alternatives and checking the scoring process. [5]

None of these sources establishes that the authors use the Concept Fan, or that this technique improves AI engineering productivity. That connection is the series’ proposed application.

Back to contents ↑

7. Try the move

Choose a feature whose prompt you have repeatedly revised.

Write its useful outcome without mentioning the model, prompt or framework.

Select a few observed failures. For each, record:

  • What information was available.
  • What information was missing or misleading.
  • What the system did.
  • What a correct resolution required.

Construct a Concept Fan with several genuinely different directions. Develop at least one practical idea under each.

Accessible exercise: Use paper examples. Compare a revised instruction, a better evidence packet and a clarification question. Ask a colleague to explain what each changes.

Technical extension: Freeze a development set and reserve separate assessment cases. Compare the original system with individual changes before testing a combination.

Track supported correctness, appropriate clarification, unresolved cases, review effort, latency and cost. Record task categories and sample size; repeat trials where outputs vary.

Decide beforehand what would justify continuing, revising or stopping. Do not select the winner solely by fluent answers or one aggregate score.

Back to contents ↑

8. Take it into your work, including limits of transfer

For the engineer: Before the next prompt revision, identify which failure it is intended to address and whether the necessary evidence is available.

For the technical leader: Ask the team to show alternatives that change how the outcome is achieved, alongside the proposed prompt improvement.

The Concept Fan is useful when a team has become attached to one solution or keeps making local improvements without reconsidering the approach.

It can also become unnecessary overhead. If a specific instruction is demonstrably ambiguous and a small correction resolves it without regressions, make the correction.

Branches may overlap or combine. Their purpose is to expand the choice, not force every design into an exclusive category.

Alternative generation also needs a stopping point. Select a small number of informative comparisons, then let evidence guide the next decision.

Transfer question: What is your current solution one way of achieving—and what other ways have you not yet considered?

Back to contents ↑

9. Reusable artifact and sources

The Outcome to Options Sheet

Use these fields for your next design review:

  1. Useful outcome and intended user.
  2. Current process and baseline.
  3. Current solution being repeatedly improved.
  4. What that solution achieves.
  5. Broader directions toward the outcome.
  6. Concepts under each direction.
  7. Concrete actions under each concept.
  8. Verified facts and unresolved assumptions.
  9. Option requiring no additional model call.
  10. Smallest informative comparisons.
  11. Measures, costs and failure consequences.
  12. Decision, owner and review date.

References checked 5 November 2026

  1. Edward de Bono, Serious Creativity, 1992, printed pages 129–133, supplied project edition.
  2. Anthropic, Effective context engineering for AI agents, 29 September 2025.
  3. Anthropic, Introducing Contextual Retrieval, 19 September 2024.
  4. Anthropic, Building effective agents, originally published 19 December 2024.
  5. Anthropic, Demystifying evals for AI agents, 9 January 2026.

Publication note: Arjun, Leela, their colleagues and all trial observations are fictional. The scenario, prompt and reusable artifact are original contemporary applications. Sources support the underlying thinking method and engineering practices, not empirical results for this invented case.

Back to contents ↑

AIYING — Accomplishing with AI

Lateral Thinking · Engineering Judgment · Meaningful Impact

By Ravindra Dastikop · The Effective AI Engineer · Case 004 of 16

Accomplish with AI

Powered by Blogger.

Featured post

AI Engineering Hands-On Lab #001 — Build an AI Answer Reliability Test Bench Without Code

AIYING | Daily 24 — Post 009 AI Engineering Hands-On Lab #001 Build an AI Answer Reliability Test Bench — Without Writing Code Lea...

Search This Blog

List of blogs

Search This Blog

About

Translate

Instant Stories

Read in Your Language

Test

share this

ShareThis

Popular Posts