THE EFFECTIVE AI ENGINEER
CASE #002 · CASE 2 OF 16
The Engineer Who Removed the Agent
Applying Lateral Thinking to Create Meaningful Impact with AI
By Ravindra Dastikop
Engineering focus: Architecture selection and the allocation of model judgment.
1. The situation and impact sought
Rohan’s team had built an agent that prepared onboarding checklists for customers of a software platform. It read requests, looked up account information and chose tools to assemble the checklist.
The prototype handled useful examples. Now the team needed to decide what deserved production investment.
Success meant an accurate checklist based on the customer’s plan, region and requested integrations, with unresolved information visible to a specialist. The system was not authorized to activate accounts or change subscriptions.
Rohan had a test environment, approved request examples and access-controlled account lookup services. He needed a baseline for correctness, specialist review time, response time and operating cost.
The question was whether model-directed planning was necessary for this particular task. Removing it was an option to investigate, not the project’s starting conclusion.
2. The story
On Wednesday, Rohan watched the agent complete a difficult example.
The request mentioned two products by their old names, referred to an earlier conversation and included a half-finished integration description. The agent found relevant records, asked for missing details and produced a useful draft.
His manager smiled. “That’s the flexibility we wanted.”
Rohan agreed.
At the next architecture review, he opened with that success. Then he asked the team to describe a normal request.
Maya, the onboarding specialist, drew five boxes: identify the account, confirm its plan, check regional availability, identify integrations and prepare the checklist.
“What determines the order?” Rohan asked.
“Dependencies,” she said. “You need the account before you can trust the plan lookup.”
“And when does the order need to change?”
They examined examples. Some needed clarification, but most still followed the same dependencies.
A colleague pointed at the prototype. “It already works. Why redesign it?”
“I want to know what else could work,” Rohan said.
He opened a fresh design note. In one column he listed the agent’s responsibilities. In another he wrote what each responsibility accomplished.
The agent chose when to look up the account. But the account was always needed.
It chose whether to check availability. But availability had to be checked for every relevant product.
It decided when the checklist was complete. But Maya already had explicit requirements for completeness.
There was still genuine uncertainty. Customers described integrations in inconsistent language. Sometimes the request referred to an undocumented arrangement. Neither a fixed checklist nor a confident model could settle that without more evidence.
Rohan asked an approved AI assistant to generate architectures that distributed those responsibilities differently. He included the successful difficult example and told it to preserve that capability in the comparison.
The first suggestions were three variations of the existing agent.
“Change who decides the next step,” he wrote. “Do not just change the model or framework.”
The next set contained a form with rules, a fixed workflow using a model for interpretation, and a workflow that escalated unusual requests to a bounded agent or specialist.
The model proposed routing by its own confidence. Maya objected.
“What does its confidence establish?”
They replaced that shortcut with observable conditions: missing fields, conflicting account records, unrecognized integration names and unsupported arrangements.
Rohan built the fixed workflow first.
The model interpreted the customer’s wording into a defined set of fields. Code checked the field types and allowed values. The backend verified that the operator could access the account. Required lookups followed a known dependency order. Rules built the checklist from verified records.
His first version looked reassuringly orderly.
Then Maya broke it.
A customer referred to a legacy integration with a nickname. The model mapped it to the wrong supported integration. The output passed the schema checks.
“Valid format,” she said, “wrong meaning.”
Rohan added a requirement to show the source phrase alongside the proposed mapping. Unrecognized or ambiguous descriptions went to review. They added aliases only after the relevant team confirmed them.
Another test exposed a changed regional-availability rule. The workflow had applied an outdated version. Rohan moved that rule behind a maintained service rather than embedding a copy in the prompt.
Neither defect had been solved merely by removing the agent.
For comparison, they ran the original agent, the fixed workflow and a form-and-rules baseline against reserved requests. Each run began with clean state and the same permitted records. They tested routine cases and exceptions separately.
The fixed workflow’s clearer execution path made the routine cases easier for the team to inspect. The form handled clean inputs without model calls. The original agent remained useful on some requests requiring exploratory investigation.
The fictional results supported a limited release of the workflow for the defined routine category, with specialists handling exceptions. An exception agent remained a candidate for a separate test; they did not deploy it simply because the architecture diagram had space for one.
At the final review, the manager asked, “So you took the intelligence out?”
Rohan pointed to the interpretation step, the verified records and Maya’s review queue.
“We changed where the decisions happen.”
The agent had been an achievement. Treating it as the only possible architecture would have limited what they could discover next.
3. The thinking tool explained
The tool is Creative Challenge.
In Serious Creativity, de Bono distinguishes it from criticism. A critical review asks whether something is inadequate. Creative Challenge explores whether the current approach is the only possible way, even when it works well. See the Challenge chapter, printed pages 104–105, and the summary on page 314.
Apply it by:
- Selecting an accepted design choice.
- Explaining why it exists and what function it serves.
- Treating its necessity as open to exploration.
- Generating alternatives that preserve the purpose.
- Comparing them with the existing approach afterward.
The five steps above are a practical adaptation, not a verbatim procedure from the book.
Rohan challenges the necessity of model-directed control. He does not need to declare the existing agent defective before considering other designs.
Removing a component is one possible result. Keeping it, narrowing it or discovering a more valuable use for it are equally legitimate outcomes.
No PO statement is used in this case. Creative Challenge is a distinct method, and the story should not turn it into a slogan that all agents are unnecessary.
4. The thinking made visible—including AI’s contribution
Focus: Prepare a correct, reviewable onboarding checklist.
Accepted choice: Let the model select the next step and tool.
Assumption examined: Variable customer language requires variable control over the entire process.
Challenge: Which decisions actually require model judgment, and which follow known requirements?
The alternatives change the location of decision-making:
- Customers supply structured inputs; rules handle the rest.
- A model interprets language; code controls the sequence.
- Code handles routine work; a bounded agent investigates selected exceptions.
- A model directs the broader process within enforced limits.
AI expands architecture options and helps describe their failure modes. Rohan supplies constraints and rejects superficial variants. Maya checks whether suggested designs preserve actual domain requirements.
Reusable prompt:
Apply Creative Challenge to this architecture.
Current approach: [design].
Useful outcome: [result].
Why the approach was chosen: [reasons].
Known task dependencies: [facts].
Uncertainties: [observations].
Constraints and authority: [limits].
Do not assume the current design is faulty.
Separate language interpretation, next-step selection, authorization and execution.
Generate four materially different allocations of these responsibilities. Include a non-agent option and a case where retaining the agent is justified.
For each, explain what flexibility is lost, what remains uncertain and the smallest comparison that could inform a decision.
Do not use model confidence alone as proof of correctness or invent performance results.
The prompt is an exploration aid. It cannot establish which architecture is best without implementation and evidence.
5. The engineering decision
| Architecture | Suitable conditions | Main burden | Decision in this story |
|---|---|---|---|
| Form and rules | Inputs and choices can be specified | Input friction and rule maintenance | Keep as baseline |
| Model interpretation with fixed workflow | Language varies but dependencies are known | Semantic errors and exception handling | Limited routine release |
| Workflow with exception agent | Some exceptions need bounded investigation | Routing, coordination and added evaluation | Test separately |
| Model-directed agent | Useful next steps depend on discoveries | Variable trajectories and execution controls | Retain as comparator and candidate for exploratory work |
“Single model call” does not mean deterministic behavior. A workflow with model interpretation still needs evaluation. A schema checks shape and permitted values; it does not prove that an integration was understood correctly.
Authorization, service validation, timeouts, retries and duplicate-action protection belong in execution code where applicable. Choosing fewer model-directed decisions does not remove ordinary distributed-system failures.
The selected scope deserves investment only if acceptable outcomes and review effort compare favourably with the simpler baseline. Maintenance and exception-handling costs count alongside inference.
6. The evidence behind the case
Anthropic distinguishes workflows with predefined code paths from agents that direct their own processes. It recommends beginning with a simple adequate solution and considering the cost and latency of additional autonomy [1].
OpenAI’s FDE account describes replacing custom implementations with shared Codex-based capabilities as the underlying product improved [2]. This supports reassessing architecture over time, not a general prescription to remove agents.
Measuring Agents in Production reports 20 case studies and a survey of 86 practitioners across 26 domains. Simple, controllable approaches were common in that sample [3]. The study does not prove that fixed workflows outperform agents on every task.
Anthropic’s evaluation guidance explains why actual outcomes, repeated trials and controlled environments matter for agent evaluation [4].
Our onboarding workflow and its fictional observations are original case adaptations. No cited source validates this particular design or claims that its authors used Creative Challenge.
7. Try the move
Select an AI workflow that already succeeds on some tasks.
Write down one design choice everyone treats as necessary. Explain its function before questioning it.
For a low-code exercise, list each decision and classify it as:
- A known requirement or dependency.
- Interpretation of uncertain information.
- A permission decision.
- Selection of a genuinely uncertain next step.
Ask AI for different allocations of those responsibilities. Review the classifications with a domain colleague.
For a technical experiment, implement one bounded alternative alongside the current approach and a simple baseline. Use the same information access and success criteria.
Reserve ordinary, ambiguous and unsupported examples. Test missing records, stale rules, access denial and interrupted services. Run repeated trials where model outputs vary. Keep test environments isolated and reset state.
Measure accepted checklist quality, unsupported mappings, unnecessary tool calls, review time, response time and cost. Report outcomes by category so strong routine performance cannot hide poor exception handling.
Before testing, define what would justify keeping the agent and what would justify narrowing its role. A learning experiment should permit either result.
8. Take it into your work—including limits of transfer
For the engineer: identify the smallest part of a task that actually requires model-directed exploration. Compare alternative placements of that judgment.
For the technical leader: reward justified architecture decisions, including retaining a complex approach when it earns its cost. A short diagram is not itself evidence of effectiveness.
Creative Challenge fits design reviews, successful prototypes and periodic architecture reassessment. It can be used before failure.
A fixed workflow may fit poorly when tools, evidence and useful next steps cannot reasonably be specified in advance. Overly rigid branching can create maintenance costs and brittle exception paths.
Prefer a bounded comparison when switching costs are large. During urgent incidents, restore service first and schedule broader exploration separately.
The story establishes no measured return on investment. Its intended lesson is a method for discovering and comparing alternatives.
Transfer question:
Which successful part of your architecture have you never explored alternatives to because it already works?
9. Reusable artifact and sources
Decision Allocation Record
- Useful outcome and acceptance criteria:
- Current architecture and reasons for it:
- Accepted design choice being challenged:
- Function that must be preserved:
- Known dependencies:
- Decisions requiring interpretation:
- Decisions requiring dynamic exploration:
- Authorization and execution boundaries:
- Three alternative allocations:
- Baseline and reserved test cases:
- Routine and exception results:
- Quality, review effort, latency and total cost:
- What would justify retaining the agent:
- Decision owner and reassessment date:
References
Checked 4 November 2026.
- Anthropic. Building effective agents. 19 December 2024, with updated-tooling notice.
- OpenAI. What FDE means at OpenAI. 22 October 2026.
- Pan et al. Measuring Agents in Production. ICML 2026.
- Anthropic. Demystifying evals for AI agents. 9 January 2026.
- Edward de Bono. Serious Creativity, 1992, Challenge chapter, printed pages 104–105; summary page 314, supplied project edition.
“It works” can be the starting point for exploring alternatives.
ReplyDeleteTry the Creative Challenge prompt in this case, then use the Decision Allocation Record to compare your current architecture with another approach.
Which decisions genuinely need model judgment? What evidence would justify keeping—or narrowing—the agent’s role?
Share one example from your work below.
— Ravindra Dastikop
AIYING — Accomplishing with AI