The Effective AI Engineer • Case 014 of 16
The Engineer Who Practised Switching It Off
Applying Lateral Thinking to Create Meaningful Impact with AI
By Ravindra Dastikop
An AI system’s readiness includes what happens when it stops helping.
Case under study: A support team discovers that its promised manual contingency cannot reliably handle the loss of AI assistance.
Thinking tool: Reversal—change the condition under which the team demonstrates readiness.
Engineering application: Test interruption, reduced service, workload limits and recovery.
Learning outcome: Replace an assumed contingency with an observable, bounded and rehearsed operating mode.
Professional benefit: Make service commitments that remain credible when AI is unavailable.
1. The situation
A service company was preparing to expand an AI assistant used by its support team.
The assistant read incoming requests, suggested a category, retrieved relevant guidance and drafted a response. Support specialists checked the draft before sending it. The existing ticketing system remained responsible for storing requests and recording completed work.
The launch review concentrated on answer quality, response time and operating cost. The team had also documented a contingency:
If AI becomes unavailable, support specialists will continue manually.
It sounded reasonable. The organisation had handled requests manually before introducing AI.
But the workflow had changed since then.
2. The story
A familiar contingency
Rohan, the engineer responsible for the integration, asked Maya, the support operations lead, a question during the launch rehearsal.
“Have we tested the manual route?”
“We used it for years,” Maya replied.
“Have we tested today’s manual route?”
The distinction mattered. The new interface opened requests inside an AI workspace. Category suggestions appeared automatically. Retrieved guidance arrived alongside the draft.
Several experienced specialists had moved to other responsibilities, and newer team members had learned the assisted process first. The old manual procedure described a workplace that no longer quite existed.
Reverse the launch condition
The team’s normal launch question was:
Can the service operate successfully with AI enabled?
Rohan wrote a provocation beneath it:
PO: The service must pass its launch test with AI switched off.
He wanted to reverse the condition under which the team judged readiness.
“If the assistant disappeared halfway through a shift,” he asked, “what would a specialist actually see?”
They arranged a controlled rehearsal in a test environment using representative, sanitised requests. The support team knew the scope, an observer could stop the exercise, and no real customer messages would be sent.
The manual route was behind a wait
Rohan disabled the model connection.
The first request opened. A loading indicator appeared.
It continued spinning.
The manual response button existed, but the interface did not reveal it until the AI operation finished or failed. The backend had a timeout; the person using the system had no useful explanation of what was happening.
“We have a manual route,” Maya said.
“Yes,” Rohan replied. “Behind a wait.”
Available information was inaccessible
The next specialist opened the manual guidance link. It led to a page requiring access that her role did not have.
The assistant normally retrieved the relevant material through a separate service identity.
The document existed. The person expected to use it during an interruption could not reach it.
A third request exposed another assumption. Without the suggested category, a newer specialist could not determine which team owned the case. The old routing guide used department names that had since changed.
These failures required a usable service when the model was absent.
Repair the route, then test its capacity
The team redesigned the interface so that a request remained accessible independently of AI drafting. The screen showed whether assistance was available, preserved the specialist’s work and offered an explicit manual route.
They repaired access to approved guidance and updated the routing procedure.
Then Maya raised a harder question.
“What happens when every request comes here manually?”
Their first rehearsal had involved only a small set of cases. A working manual interface did not establish that the team could absorb an entire assisted workload.
They ran a capacity exercise with a representative mix of requests, including difficult and incomplete ones. They measured handling time, work awaiting attention and the effect of interruptions on specialists.
The exercise did not support a promise of unchanged service.
Define an honest reduced service
The team defined a reduced service: urgent requests received priority, routine work could wait, and users would receive a clear acknowledgement with an honest indication of delay.
Queue limits and an overflow procedure prevented work from accumulating without an owner.
“Manual continuation is slower,” Maya said.
“Then our plan must say slower,” Rohan replied.
They also considered sending every failed request to a second model provider. They reviewed its separate data arrangements, behaviour, capacity and operating procedures.
They retained it as an option requiring its own evaluation. It did not become a dependable contingency merely because it could produce an answer in a demonstration.
Slow failure needed a different test
The next rehearsal tested a different failure: the assistant responded slowly rather than failing immediately.
Repeated attempts kept work waiting and increased traffic to the struggling dependency. The team introduced bounded attempts and a defined transition to reduced service.
They tested how requests already in progress were marked, handed over and reconciled.
Recovery could create duplicate work
Then they practised switching assistance back on.
An earlier request still had a delayed draft in flight. Meanwhile, a specialist had completed the manual response.
If the late result appeared as current work, someone could act on it again.
The team added checks against the ticket’s current state before presenting a delayed draft as actionable. Completed work remained completed. Unresolved work returned to the assisted route through a controlled process.
Test the dependency being bypassed
Their final rehearsal included loss of the AI workspace itself. This tested whether the manual route depended on the same failing component it was supposed to bypass.
Some activities could continue through the underlying ticketing system. Others required an explicit pause and escalation.
The team documented those limits.
At the revised launch review, Rohan demonstrated the assistant working.
Then Maya asked him to disable it.
This time, the audience could see what continued, what slowed down, what stopped and who took responsibility.
3. The Lateral Thinking tool: Reversal
Edward de Bono describes reversal as changing the normal direction or relationship of a situation. In Serious Creativity, he distinguishes reversal from simply doing without something, which he classifies as escape. [1]
“Imagine there is no AI” would be an escape. The reversal in this case concerns the launch criterion:
- Normal condition: demonstrate readiness through successful AI operation.
- Provocation: demonstrate readiness through operation with AI deliberately disabled.
The PO statement is a temporary thinking device. It does not establish a requirement that every AI product deliver its full function without a model.
Movement from the provocation produces a practical question:
Which commitments must the service still honour when assistance is unavailable?
That question leads to reduced service, accessible source material, clear ownership, capacity limits and tested recovery.
Method distinction: Failure injection, timeout design and operational rehearsals are established engineering practices. This case uses reversal to generate the design question. It does not present those practices as inventions of de Bono.
4. Making the thinking visible
-
Identify the accepted success condition.
The launch demonstration assumes AI is operating. -
Reverse that condition.
Ask the team to demonstrate readiness while assistance is disabled. -
Extract the useful principle.
Service commitments need explicit behaviour during loss of assistance. -
Follow a real work item.
Observe its arrival, visibility, ownership, processing and completion. -
Inspect dependencies.
Determine whether the alternative route shares the failed interface, identity service or data source. -
Test capacity.
Establish how much work the reduced service can actually handle. -
Design the return.
Reconcile pending, completed and uncertain work before normal operation resumes.
Reusable prompt
We are evaluating this AI-assisted workflow: [describe it].
Our normal readiness test is: [state the test].
Apply this reversal: “The workflow must demonstrate readiness with AI assistance deliberately disabled.”
Identify what users must still be able to do, what can be delayed and what must stop. Trace dependencies, access permissions, human capacity, pending work and recovery.
Propose several operating modes. State each mode’s limits and the evidence needed to test it.
Do not assume that manual work, cached answers or a second model will be available, accurate or sufficient. Distinguish a tested capability from an unverified contingency.
5. Comparing operating choices
| Choice | Potential benefit | What must be established |
|---|---|---|
| Wait and retry | Can recover from a brief interruption | Attempts and waiting are bounded; retries do not worsen overload. |
| Use another model | May retain some assisted capability | Behaviour, capacity, data handling and dependencies have been evaluated. |
| Continue manually | Preserves selected work through people | Access, skills, staffing, ownership and workload limits are adequate. |
| Offer reduced functionality | Keeps essential service available | Users understand the reduced scope and remaining results are suitable. |
| Queue work for later | Preserves requests during interruption | Storage, limits, priorities and resumption procedures are reliable. |
| Pause the affected operation | Avoids unsupported completion | Users receive a clear status and a defined next step. |
The appropriate choice depends on the task. An honest pause can be preferable to an apparently successful result that does not meet the service requirement.
A workflow may combine these choices—for example, continuing urgent work manually while queuing routine requests within defined limits. The combined behaviour also needs testing.
6. Research evidence and limits
Fallback can introduce its own failures
In Avoiding Fallback in Distributed Systems, AWS engineer Jacob Gabrielson explains why alternative execution paths can be difficult to exercise realistically and may worsen the conditions that caused the original failure. [2]
This supports scrutinising a contingency rather than assuming that an additional path improves reliability. It does not establish that every fallback should be removed.
Continuing work requires control of load
Google’s SRE chapter Addressing Cascading Failures explains how overload can spread between components. It discusses measures including graceful degradation and load shedding. [3]
The relevant lesson is that continuity needs capacity limits and controlled behaviour under pressure.
Application to this case
The manual route, prioritisation rules and recovery checks are proposed engineering adaptations for the fictional workflow. Neither source establishes that this particular design would work in another organisation.
The proposition to test
A rehearsed, bounded operating mode can make loss of AI assistance more manageable than an untested promise of continuity.
This case claims no measured improvement in availability, recovery time or customer satisfaction.
7. Practice and evaluation
Accessible exercise
Choose one task in which you regularly depend on AI.
- Identify what you must still accomplish if assistance disappears.
- Find the source information you would need.
- Check that you can access and interpret it.
- Complete a representative example without AI.
- Record additional time, uncertainty and work you cannot complete.
- Decide which commitments remain realistic.
Technical exercise
Run a controlled rehearsal of an AI-assisted workflow.
No comments:
Post a Comment