THE EFFECTIVE AI ENGINEER · CASE 007 OF 16
The Engineer Who Deleted Half the Tools
Applying Lateral Thinking to Create Meaningful Impact with AI
By Ravindra Dastikop
Engineering focus: Tool selection, interface contracts, error feedback and evaluation.
Case type: Research-grounded fiction. Characters, events and trial observations are invented.
Sources checked: 5 November 2026.
1. The situation and impact sought
Kabir was developing an assistant for a retail support team.
The assistant helped staff investigate return requests, check eligibility and prepare the next action. Creating a return required an authorised staff member’s approval.
Its tools were interfaces through which the model could request information or invoke application functions. The team had exposed twelve tools, largely reflecting existing backend endpoints.
Each addition had answered a reasonable need: find an order, inspect a shipment, retrieve policy exceptions or check a return’s status.
Yet the assistant sometimes selected overlapping tools, confused identifiers and repeated requests after incomplete responses.
The team’s usual repair was another instruction—or another tool.
Kabir had two weeks to test improvements in a staging environment using representative records and simulated failures.
The useful outcome: Complete the correct support task with the required approval, accurate records and manageable review effort.
A shorter tool list would count as progress only if it preserved useful capabilities.
2. The story
The assistant had twelve tools.
Kabir could explain why every one existed.
That was why removing any of them initially seemed unreasonable.
During one test, the assistant found the correct order but passed a shipment identifier into a field expecting an order identifier. The value was a valid string. It identified the wrong kind of object.
In another, a tool returned an empty list after a service timeout. The assistant interpreted this as evidence that no return existed.
“Let’s add a recovery tool,” suggested Sana, the support lead.
Kabir wrote down the suggestion. Then he looked at the tool catalogue.
They had built it one endpoint at a time. Nobody had reconsidered whether the catalogue should still follow that structure.
He asked a different question:
“Why must the assistant make every backend choice that our application already knows how to make?”
Sana pointed to a familiar sequence. Once staff had identified an order, they normally inspected its items, customer details and shipment information together.
The assistant performed those lookups separately.
That was one opportunity. But Kabir resisted turning it immediately into a single giant tool.
He explored three alternatives.
They could retain the existing tools and improve their names, arguments and responses.
They could offer a smaller set organised around support tasks.
Or they could keep a larger catalogue and expose relevant tools through discovery when needed.
An AI assistant helped compare the proposals. Its first recommendation combined investigation and return creation into one convenient operation.
Kabir rejected that boundary.
A request to investigate a return did not authorise creating one.
He tried consolidating only the related information retrieval. A read-only tool assembled order items and shipment details while preserving their identifiers and source status.
That removed several routine choices from the model.
The first version still failed.
When shipment information was unavailable, the response contained an order record and no shipment section. The assistant treated the missing section as “not shipped.”
Kabir changed the contract. Each component now reported whether it was available, absent or unavailable because of an error. Partial results were labelled.
Then the team tested a customer with several orders.
The convenient tool selected the most recent one.
The customer had meant an older purchase.
Kabir removed that assumption. Search could return candidates. The context tool required a resolved order identifier. When the user’s request remained ambiguous, the assistant had to clarify.
The revised interface contained six tools.
One searched for orders. One assembled order context. One checked eligibility against the applicable policy. One prepared a return draft without changing operational records. One created the approved return. One checked an existing request’s status.
The underlying services remained.
The change concerned the interface presented to the model.
A timeout during return creation produced another difficult test. The request might have succeeded even though the response had not arrived.
The assistant proposed trying again.
The backend needed a reliable way to recognise the same operation and check its status. A better tool name could not resolve uncertainty about whether a write had occurred.
Kabir added this case to the acceptance tests rather than hiding it inside a reassuring error message.
The team compared the original interface, a clarified twelve-tool version and the six-tool design. A separate discovery experiment examined whether a broader catalogue could remain usable without presenting everything initially.
The fictional findings were mixed. Consolidation helped some routine tasks, while unusual investigations still benefited from specialised access. Aggregate responses also cost more when they fetched information nobody needed.
Kabir proposed a limited pilot for the defined return workflow.
Sana read the change summary.
“You deleted half the tools.”
“From this interface,” Kabir replied. “We kept asking what the assistant needed to decide—and what the application could handle reliably.”
3. The thinking tool explained
The tool is Edward de Bono’s Creative Challenge.
Creative Challenge questions whether an established arrangement is the only useful way to proceed. It does not require proving that the existing arrangement is defective.
In Serious Creativity, de Bono distinguishes this from criticism and discusses challenging assumptions and arrangements that continue because nobody has reconsidered them. See the “Challenge” summary, printed page 314, in the supplied edition. [1]
Kabir challenges this assumption:
Each backend endpoint should become a separate tool available to the model.
The challenge opens several directions:
- Preserve the structure but clarify its contracts.
- Organise tools around meaningful tasks.
- Discover specialised tools when needed.
- Move predictable coordination into application code.
This is Creative Challenge, not PO. There is no deliberately provocative statement requiring movement from an apparently unreasonable position.
The modern tool-design application is this series’ adaptation.
4. The thinking made visible
| Thinking step | Application |
|---|---|
| Focus | Complete the correct return task with controlled changes and useful review. |
| Accepted arrangement | Expose backend operations individually. |
| Challenge | Must the model choose and coordinate every operation? |
| Generate alternatives | Clarify existing tools, consolidate selected tasks, or introduce discovery. |
| Preserve boundaries | Keep investigation, preparation and execution distinct where their authority differs. |
| Test | Compare actual outcomes, recovery and total effort. |
AI’s contribution
AI can identify overlapping descriptions, suggest alternative groupings and generate difficult test requests.
Its suggestions require review. A convenient combination may conceal an assumption, increase authority or make partial failures harder to interpret.
Reusable prompt
Examine this tool catalogue: [catalogue].
Our user task is [task]. Our constraints are [constraints].
Challenge the assumption that the current boundaries must remain unchanged. Generate three different designs: clarify the existing interface, consolidate selected operations, and discover specialised tools when needed.
For each, identify which decisions remain with the model, which move into code, what capabilities could be lost and how failures would be reported.
Preserve distinctions between reading, preparing and executing. Do not assume fewer tools are better. Propose a small comparison with observable success criteria.
5. The engineering decision
| Alternative | Potential benefit | Cost or difficulty | Critical uncertainty | Decision |
|---|---|---|---|---|
| Clarify twelve tools | Preserves granular capabilities. | Overlap and coordination remain. | Are unclear contracts the main problem? | Test. |
| Six task-oriented tools | Moves predictable coordination into code. | More backend orchestration and partial-result handling. | Does consolidation preserve necessary flexibility? | Limited trial. |
| Discover relevant tools | Retains a broader catalogue. | Discovery adds another selection step. | Can the assistant find the required capability? | Separate experiment. |
| One broad “handle return” tool | Simple entry point. | Conceals scope, assumptions and effects. | Can investigation occur without unintended execution? | Reject this proposed design. |
Define contracts beyond field types
A tool contract should explain:
- Its purpose and when to use it.
- Required identifiers and their meaning.
- Preconditions and permitted effects.
- The meaning of successful, empty and partial results.
- Recoverable failures and unresolved outcomes.
A structurally valid request can still contain the wrong identifier or an unjustified selection.
Preserve ambiguity
A tool should not silently select the newest order when several candidates could match. Either require a resolved identifier or return enough information for clarification.
Make errors useful
“No matching record,” “access denied” and “service unavailable” imply different next steps. Do not collapse them into an empty response.
Treat uncertain writes explicitly
If a response is lost after a mutation request, success may be unknown. The proposed design uses operation identifiers, status checks and backend deduplication where supported. These mechanisms require implementation and testing; a prompt cannot supply them.
The decision is a trial of the six-tool interface for a bounded workflow—not a rule that six tools, or fewer tools generally, produce better agents.
6. The evidence behind the case
Established guidance
Anthropic’s tool-engineering guidance recommends clear purposes, meaningful responses and evaluation with realistic tasks. It discusses selective consolidation rather than simply wrapping every existing endpoint. This supports examining interface boundaries; it does not establish the best catalogue for this fictional retailer. [2]
Anthropic’s advanced tool-use work describes discovering tools on demand. It provides an important alternative to removing capabilities from a large catalogue. Its reported outcomes are specific to its experiments and configurations. [3]
The cited Model Context Protocol specification defines tool descriptions, input schemas, optional output schemas and error signalling. These describe an interface, but schema compliance alone does not establish business correctness or authorisation. [4]
Anthropic’s agent-evaluation guidance distinguishes the interaction transcript from the resulting state and discusses repeated trials. That distinction matters when an assistant says a return was created: the test must inspect whether the correct record actually exists. [5]
Case adaptation
The retailer, twelve-to-six redesign, failures and pilot proposal are fictional teaching choices. The retry and approval arrangements are proposed engineering requirements for this scenario.
Uncertainty
Consolidation can increase response size, backend work and hidden coupling. Discovery can fail to surface an appropriate tool. Clear granular tools may outperform both.
The cited organisations are not being represented as users of de Bono’s methods.
7. Try the move
Accessible exercise
Choose a workflow with several functions or steps.
- List the decisions its user must make.
- Identify which decisions are meaningful and which are routine coordination.
- Challenge one existing boundary.
- Develop three alternatives.
- Walk through an ambiguous request and a partial failure.
- Check whether any convenience has introduced an unsupported assumption.
You can simulate the interfaces using written tool cards.
Technical extension
Compare the existing interface with clarified descriptions and a redesigned catalogue.
Keep the model, task set and permissions comparable where practical. Record differences that prevent direct comparison. Test discovery separately before combining improvements.
Include:
- Multiple matching records.
- Similar identifier types.
- Missing required information.
- Empty results and unavailable services.
- Partial responses.
- Expired approval.
- Timeouts after possible writes.
- Rare tasks needing specialised capabilities.
Check the final system state, not just the assistant’s wording.
Measure accepted task completion, incorrect selections, parameter errors, unintended changes, recovery, human corrections, latency and total cost.
Record tool-call counts as diagnostic information. Fewer calls are not automatically better.
Reserve validation tasks and repeat variable trials. Do not require one exact call sequence when several paths correctly satisfy the task.
Stop or revise if the redesign loses required capabilities, conceals uncertainty or makes consequential actions harder to control.
8. Take it into your work
For engineers
Design tools around the decisions an agent must make. Keep predictable coordination in code when that improves reliability, while retaining the flexibility the task needs.
Read tool traces and results. Failures may come from descriptions, semantics, backend behaviour or recovery—not simply model choice.
For technical leaders
Evaluate the interface and its operating responsibilities together. A shorter catalogue may require more orchestration, monitoring and maintenance.
Ask what became easier, what became hidden and who owns the resulting complexity.
Limits of transfer
Task-oriented consolidation can suit stable, repeated workflows. Exploratory work may benefit from granular capabilities or discovery.
A broad tool is especially difficult to assess when it mixes information gathering with consequential changes.
Transfer question
Which decisions in your tool interface genuinely require judgment—and which exist because your backend happened to be divided that way?
9. Your working artifact and references
Create a Tool Interface Decision Record
Record:
- User task and useful outcome.
- Existing tools and overlapping purposes.
- Assumption challenged.
- Three materially different interface designs.
- Decisions allocated to the model and to code.
- Identifier meanings and ambiguity handling.
- Preconditions, effects and approval requirements.
- Success, empty, partial and error responses.
- Retry and uncertain-outcome behaviour.
- Capabilities retained, deferred or removed.
- Baseline and reserved evaluation tasks.
- Outcome, effort, latency and cost measures.
- Failure criteria, owner and next decision.
References
Sources checked: 5 November 2026
- Edward de Bono, Serious Creativity (1992), “Challenge,” printed page 314. Supplied project edition.
- Anthropic, Writing effective tools for agents—with agents , 11 September 2025.
- Anthropic, Introducing advanced tool use , 24 November 2025.
- Model Context Protocol, Tools specification, revision 2025-11-25 .
- Anthropic, Demystifying evals for AI agents , 9 January 2026.
Editorial note: This is research-grounded fiction for professional learning. Sources support the technical discussion, not the fictional trial observations. The connection between Lateral Thinking and the engineering choices is this series’ educational adaptation. Platform capabilities and specifications can change.
No comments:
Post a Comment