The Effective AI Engineer · Case 005 of 16
The Engineer Who Gave AI Less to Read
Applying Lateral Thinking to Create Meaningful Impact with AI
By Ravindra Dastikop
Research-grounded fiction. The characters, organization and experimental observations are invented. Engineering sources checked on 5 November 2026.
In this case
1. The situation and impact sought
Priya was building an internal assistant that helped engineers investigate software deployment failures.
For each incident, it received the incident conversation, deployment logs, service documentation and previous troubleshooting notes. It drafted an explanation and suggested checks for an engineer to review. It could not execute commands or change production systems.
Whenever the assistant missed something, the team added more material.
Soon, individual requests contained repeated logs, outdated suggestions and documentation for several service versions.
The material fitted inside the model’s supported context window. That did not establish that the assistant used it correctly.
Priya had two weeks to investigate the problem using approved historical incidents and a test environment.
Her objective was to help an engineer reach an evidence-supported next check with less investigation and correction effort. The existing assistant and the current manual process were the baselines.
A shorter input would be useful only if it preserved the evidence needed for that decision.
2. The story
The assistant’s latest explanation sounded convincing.
It attributed a deployment failure to a database connection problem and cited a timeout from the logs.
Dev, the reviewing engineer, shook his head.
“That timeout happened after the deployment had already failed.”
Priya inspected the input.
The timeout appeared repeatedly because several people had copied it into the incident conversation. The earlier configuration error appeared once.
She marked the sequence in her notebook.
At the next meeting, someone proposed adding the complete service handbook.
“Then it will have everything.”
Priya almost agreed.
Instead, she tried an uncomfortable thought: suppose the assistant could no longer read the incident archive.
What would remain?
A service name. A deployment identifier. A question. Perhaps a list of places where evidence could be found.
At first, the idea seemed to remove the assistant’s usefulness.
Then Priya imagined the next moment.
The assistant would have to ask for something specific.
Which deployment? Which service version? What was the first failing step? Where was the original error recorded?
The archive could remain available without appearing in every input.
She wrote three possibilities.
First, prepare a small evidence packet before asking for an explanation.
Second, let the assistant request additional records through a limited lookup process.
Third, separate the work: one step locates candidate evidence; another examines a selected set.
She asked an AI assistant to develop these directions and identify what each might lose.
It suggested compressing every incident into a short summary.
That sounded efficient.
Priya tried it on a development example.
The summary preserved the main error but omitted the fact that an earlier mitigation had already been attempted.
The next assistant recommended that same mitigation again.
Dev pointed to the original conversation.
“We made it shorter by removing the reason this suggestion was wrong.”
Priya stopped treating token count as the objective.
She replaced the free-form summary with a more explicit evidence packet:
- The question being investigated.
- Service and deployment identifiers.
- Relevant versions.
- An ordered sequence of selected events.
- Exact error excerpts.
- Actions already attempted and their observed results.
- Unresolved questions.
- References back to the original records.
She kept observations separate from participants’ hypotheses.
Then she tested the packet.
On one case, the assistant identified an earlier configuration failure without being distracted by repeated downstream timeouts.
On another, it missed a relevant event that the packet builder had excluded.
The first result was encouraging. The second exposed the new design’s weakness.
“We’ve moved part of the problem into selection,” Priya said.
She added a way to request surrounding records when the supplied evidence was insufficient. The test harness limited the number of retrieval rounds and recorded what was requested.
The assistant sometimes asked useful questions.
It also sometimes searched broadly without improving its explanation.
Priya added a stopping condition: when the available evidence did not support a next check within the defined limits, the assistant should identify the gap and hand the investigation back.
Then a colleague noticed another problem.
A document labeled “latest” described the current service, but the historical incident involved an older release.
Priya changed the evidence preparation to preserve applicability. Where historical documentation was unavailable, the packet stated that limitation.
For retrospective tests, she also excluded information added after the point at which the assistant was supposed to advise the engineer.
Knowing the eventual resolution would make the demonstration easier. It would make the comparison misleading.
The team compared the original broad input, a prepared packet and a packet with bounded follow-up retrieval.
They used the same model and task instructions where possible, and recorded selection effort, retrieved material, review time and unsupported claims.
The fictional results were mixed.
Focused packets helped on some cluttered incidents. Other incidents needed broader context. Follow-up retrieval recovered some missing evidence but added delay and more behavior to inspect.
Priya proposed a limited pilot for a defined incident category.
At the review, her manager asked how much less the assistant would read.
“That depends on the incident,” she said.
“What changed, then?”
“We stopped making it read everything before we knew what it needed.”
3. The thinking tool explained
The tool is escape provocation followed by deliberate movement.
In Serious Creativity, de Bono describes escape as identifying a normal feature taken for granted and temporarily removing it. The next step is movement toward a useful idea. [1]
For this case:
Taken-for-granted feature: The incident assistant receives the incident archive as part of its initial input.
Original contemporary provocation:
PO: The incident assistant cannot read the incident archive.
This is a thinking stimulus. It is not a deployment requirement.
Priya uses two movements:
- Moment to moment: Imagine the assistant facing a question without the archive. What must it request next?
- Extract a principle: Information can remain accessible without being loaded into every model call.
These movements generate distinct possibilities:
- Prepare a task-specific evidence packet.
- Retrieve additional evidence when a defined need arises.
- Separate evidence collection from interpretation.
The practical design restores selective access. A provocation does not have to survive literally in the resulting solution.
The PO is our original adaptation. Context selection and retrieval are established engineering practices; this case does not claim to invent them.
4. The thinking made visible, including AI’s contribution
Focus: Help an engineer identify an evidence-supported next check.
Restricting assumption: Giving the model more available material is the appropriate response to an omission.
Escape: Remove the incident archive from the initial input.
Movement: Explore what information the assistant would need to request, and how the system could provide it.
| Direction | Practical possibility | Main uncertainty |
|---|---|---|
| Prepare evidence in advance | Assemble a packet preserving sequence, applicability and source references | Will selection omit a decisive fact? |
| Obtain evidence progressively | Allow bounded requests for original records | Will requests recover useful information efficiently? |
| Separate responsibilities | Collect evidence before interpreting it | Will the handoff preserve qualifications and disagreements? |
AI helps generate packet designs, missing-evidence scenarios and retrieval questions.
Priya and domain reviewers establish which records support a claim, inspect omissions and decide whether a proposed check is justified.
Reusable prompt
Explore this escape provocation: PO, the incident assistant cannot read the incident archive.
Useful outcome: [outcome].
Available authorized sources: [sources].
Information available at the decision time: [details].
Constraints: [limits].
Use moment-to-moment movement and extract a useful principle.
Develop three different ways to supply sufficient evidence without loading the entire archive initially.
For each, identify what could be omitted, how missing evidence could be recovered and when the system should stop.
Preserve identifiers, event order, exceptions, attempted actions and source references where relevant.
Keep observations separate from hypotheses. Do not invent incident facts or claim that shorter context is always better.
5. The engineering decision
Priya selects a prepared evidence packet with a bounded route to additional records for a limited pilot.
| Alternative | Potential benefit | Cost or failure mode | Decision |
|---|---|---|---|
| Broad initial context | Retains material whose relevance is not yet known | Repetition, conflicting versions and difficult inspection | Keep as a baseline |
| Prepared evidence packet | Makes relevant evidence easier to inspect | Selection can omit necessary facts | Test on a defined incident category |
| Packet with follow-up retrieval | Can recover missing evidence | Added latency, retrieval failures and variable requests | Pilot within explicit limits |
| Compressed summary | Reduces repeated material | Can lose exact details, qualifications or failed attempts | Use only with evaluated preservation requirements |
The packet is a proposed design for this task, not a universal schema.
Its construction must be evaluated alongside answer generation. A model cannot reason from a decisive fact that the selection process has removed.
A useful diagnostic comparison supplies a manually verified evidence packet. If performance improves, selection deserves investigation. That does not independently establish a single cause or validate the automated packet builder.
Resource limits should support useful work. Arbitrarily cutting every incident to the same token count could remove essential evidence.
6. The evidence behind the case
Anthropic’s context-engineering guidance discusses selecting useful information, retrieving material when needed and managing accumulated context. It also warns that aggressive compaction can lose important details. This supports examining both what enters the input and what is removed. It does not establish that the shortest input performs best. [2]
Lost in the Middle found that the position of relevant information affected performance on the models and retrieval tasks studied. It provides a reason to test whether available evidence is used reliably. Published research on those models does not establish identical behavior for every contemporary model or incident workflow. [3]
Anthropic’s Contextual Retrieval research examines how isolated chunks can lose information needed to interpret them. Its approach preserves additional context during indexing. This reinforces a complementary lesson: selecting less material must not strip away applicability or meaning. Its reported results are specific to its experiments. [4]
The incident scenario, evidence packet and pilot decision are fictional adaptations. No source demonstrates that this invented system reduces investigation time or that the organizations use de Bono’s methods.
7. Try the move
Choose a document-heavy AI task.
Select several approved examples and identify what a competent reviewer needs to judge the answer.
For each example:
- State the question.
- Record the information available at the decision time.
- Identify necessary facts and qualifications.
- Preserve references to the originals.
- Mark disagreements and missing evidence.
Apply the escape: imagine the initial model call cannot receive the source collection. Develop three ways to obtain sufficient information.
Accessible exercise: Give a colleague a prepared packet. Ask them to identify a next action and record every additional item they need. Revise the packet around those observations.
Technical extension: Compare broad context, a selected packet and selected context with bounded follow-up retrieval.
Keep model and instructions comparable. Record preparation effort, retrieval calls, supported correctness, omissions, unresolved cases, review time, latency and cost. Include preparation or indexing costs when assessing overall value.
Use development and reserved assessment cases. Repeat variable trials.
Define unacceptable omissions and pilot limits before selecting a candidate. Evaluate useful outcomes rather than celebrating token reduction alone.
8. Take it into your work, including limits of transfer
For the engineer: Inspect one failure and ask whether the model lacked a fact, received conflicting evidence or failed to use information already present.
For the technical leader: Ask who owns evidence selection and how that process is evaluated.
This approach fits tasks with repetitive, mixed or selectively relevant source material.
It may fit less well when the task requires exhaustive review or relationships across a large body of evidence. A broader input can be the appropriate baseline or final choice.
Retrieval adds failure modes and maintenance. Summaries can remove important qualifications. A human-designed packet can encode the designer’s mistaken expectations.
Start with the least elaborate comparison that can inform the decision.
Transfer question: Which detail would make your shorter input dangerously incomplete—and how would the system recover it?
9. Reusable artifact and sources
The Context Selection Record
Document:
- User and useful outcome.
- Question being answered.
- Authorized sources.
- Applicable versions and decision time.
- Required evidence and qualifications.
- Observations, hypotheses and unresolved conflicts.
- Selection method.
- Exact excerpts and source references.
- Information deliberately omitted and why.
- Conditions for requesting more evidence.
- Retrieval limits and stopping conditions.
- Baseline and reserved assessment cases.
- Quality, effort, latency and cost measures.
- Unacceptable omissions.
- Pilot owner and review date.
References checked 5 November 2026
- Edward de Bono, Serious Creativity, 1992, “The Escape Method,” printed pages 165–167 in the supplied project edition.
- Anthropic, Effective context engineering for AI agents, 29 September 2025.
- Nelson F. Liu and colleagues, Lost in the Middle: How Language Models Use Long Contexts, 2023.
- Anthropic, Introducing Contextual Retrieval, 19 September 2024.
Publication note: Priya, Dev, their organization and all experimental observations are fictional. The PO, prompt and reusable artifact are original contemporary applications. Sources support the underlying methods and engineering considerations, not empirical results for this invented case.
