16 Cases. 16 Engineering Challenges. A New Way to Think About AI.
Don't just learn AI Engineering. Learn to think like an Effective AI Engineer.
AI Engineering is not simply about knowing models, prompts, agents, frameworks or programming tools.
It is also about asking the right questions, challenging assumptions, exploring alternatives and making decisions that create real value.
Introducing The Effective AI Engineer, a new case-based learning course from AIYING — Accomplishing with AI, hosted on Nisarga Innovations.
📚 Learn Through Cases, Not Just Concepts
The course brings together 16 practical, research-grounded cases, combining Edward de Bono's Lateral Thinking with contemporary AI Engineering practices.
Each case begins with a realistic engineering challenge and unfolds through a story that reveals a surprising way forward.
You will explore:
The engineering problem and its hidden assumptions.
An unexpected insight or alternative approach.
The thinking method behind the discovery.
Engineering choices, trade-offs and supporting evidence.
Practical exercises and reusable AI prompts.
🎯 Who Is This Course For?
🎓 Students and aspiring AI Engineers / AI Builders — Prepare for an AI career by developing problem-framing, design and decision-making skills alongside technical knowledge.
🌱 Freshers beginning their AI careers — Gain exposure to engineering situations, practical challenges and lessons that can help you approach your first projects with greater confidence.
💼 Experienced engineers, developers and architects — Revisit familiar practices, discover overlooked alternatives and develop fresh perspectives on AI system design and delivery.
💡 A Preview of the Cases
Case #001: The Engineer Who Changed the Question
Case #002: The Engineer Who Removed the Agent
Case #003: The Engineer Who Tested Before Building
Case #004: The Engineer Who Stopped Improving the Prompt
Case #007: The Engineer Who Deleted Half the Tools
These cases illustrate an important principle:
The most effective AI engineering decision is not always to build more. Sometimes it is to think differently.
🔜 Coming Soon!
Whether you are preparing to enter AI Engineering, taking your first steps, or bringing years of experience to the field, this course invites you to discover a different way of approaching AI engineering challenges.
A new developer has produced a working prototype with AI assistance.
The demonstration looks promising. Then a reviewer asks her to predict
what happens when one input changes.
The team now faces two connected questions:
Does the software behave correctly?
And can its owner understand, maintain and change it?
Version 1.0 · Evidence checked: 6 November 2026
Audience: freshers, students, freelancers, professionals,
workforce teams, faculty and technical leaders.
About this case
This is an original fictional teaching scenario
informed by published research and engineering accounts.
The organisation, people, dialogue, schedules, costs and ticket
records are invented for instruction. They are not reported events
at OpenAI, Anthropic or another real company.
Research findings are identified and referenced separately.
The accompanying Python exercise has been tested locally.
The proposed workplace review process has not been validated
as a learning intervention.
1. Learning objectives
After working through this case, you should be able to:
Distinguish a successful demonstration from evidence of correctness,
understanding and maintainable ownership.
Compare delivery options using business outcomes, review effort,
learning and reversibility.
Derive boundary and exception tests from an agreed business contract.
Explain a critical code path and apply that understanding to
a changed requirement.
Write a release recommendation with acceptance conditions,
named responsibilities and a fallback.
Preparation: 35–45 minutes. Discussion: 90 minutes. Optional lab: 60–120 minutes. Coding prerequisites: basic conditions, loops,
dictionaries and tests. A no-code pathway is provided.
At 9:10 on Monday, Asha Rao opened the review request she had created
late on Friday. She had joined FieldBridge Software six weeks earlier
after graduating. The small services company maintained an internal
dashboard for a client that repaired commercial equipment.
Its next opportunity was modest: help a coordinator review a daily
ticket export without manually copying each row into a priority sheet.
Meera, the delivery lead, wanted a demonstration on Tuesday afternoon.
The client had agreed to watch a prototype and consider a two-week
assisted pilot. A useful demonstration could create follow-on work.
A misleading one could make the client believe that exceptions
had been solved when the team had barely examined them.
Asha’s assignment was a CSV-in, report-out script. It would identify
ticket priorities under explicit rules and report rejected rows.
It would not send messages, update the ticket system, dispatch
technicians or make service commitments.
An approved AI assistant had helped Asha produce an early draft.
She asked it to implement the normal path, run two examples and
describe the result. On Friday, two open tickets received
sensible-looking priorities and a closed ticket disappeared
from the proposed queue.
She posted a screenshot. Meera replied that it looked promising.
The question that interrupted the demonstration
Dev, the senior developer reviewing the change, asked what would
happen to a normal-impact open ticket exactly sixty minutes old.
Asha said she thought it would become P2.
Dev asked her to trace the comparison before running the script.
She paused. Then he changed the capitalisation of the status field
and asked the same question.
Asha understood variables and loops. But she had accepted several
details because the assistant’s explanation sounded coherent.
She could describe the script’s intention more confidently than
its actual behaviour.
She was also unsure whether asking for time to investigate would
be interpreted as a failure to use AI effectively.
Dev did not propose banning the assistant. He wanted to know whether
the team had assigned ownership without providing a reliable way
to establish it. A screenshot was evidence about those exact inputs.
It did not show whether Asha could diagnose the next complaint.
What the coordinator needed
Leela, the client’s coordinator, received exports from several staff
members. Some cells contained leading spaces; others used upper-case
words. Her main concern was overlooked work, especially a ticket
blocking a customer’s operations.
She also wanted a count of records the script could not interpret.
Quietly discarding a row could make the queue look cleaner while
creating reconciliation work later.
The priority rules were simple: valid open blocked tickets were P1;
valid open normal-impact tickets aged sixty minutes or more were P2;
other valid open normal-impact tickets were P3.
Repeated identifiers raised another question. Should a later row
replace the first one? Leela chose a first-occurrence rule for the
exercise, with later records rejected for review. She understood
that this might preserve an earlier mistake and require a separate
correction process.
The team also agreed that blank age was unknown, not zero.
Invalid records had to remain visible in the rejection report.
Every input row needed to be accounted for.
Four legitimate concerns
Asha wanted to retain responsibility and develop competence.
Dev had only ninety review minutes before another deployment.
Meera needed to protect the schedule and client relationship.
Leela needed someone who could explain an unexpected result
and correct it.
No one had measured the complete assisted workflow. The fictional
manual estimate was twenty minutes per daily batch, but it had not
been systematically timed. Exporting data, reviewing rejections,
reconciling totals and approving actions would still take effort.
The alternatives
At 10:00, Meera asked the team to consider four options:
Keep the demonstration plan, generate more examples with AI
assistance and ask Dev to inspect the final file.
Ask Asha to rebuild the script without AI, accepting a narrower
demonstration or a delay.
Transfer delivery to Dev while Asha observes.
Keep Asha as the developer, freeze the scope and use the review
time for prediction, testing, explanation and a small unfamiliar
modification.
More examples could repeat the draft’s blind spots. A manual rebuild
could spend the window on syntax. A senior takeover could protect
tomorrow’s event while transferring ownership away from Asha.
The bounded review could expose important gaps, but ninety minutes
might not be enough.
Meera would authorise the demonstration. Dev would recommend whether
a pilot was technically supportable. Asha would own the prototype
and handover. Leela would control operational use.
By 10:15, Meera wanted a plan stating what would be shown on Tuesday
and what would have to wait.
No outcome is supplied at this stage. The title does not guarantee
that Asha passes the review.
What would you authorise: a synthetic demonstration, an assisted
pilot or neither?
Which delivery option would you choose, and why?
How would you allocate the ninety senior-review minutes?
What evidence would establish correctness and maintainable ownership?
What would stop the release, and what would the fallback be?
What additional evidence could change your recommendation?
Assignment: Write a 600–900 word decision memo,
supported by an option comparison and a one-page verification plan.
Cite exhibits, label assumptions and disclose AI assistance.
Coding is optional.
4. Exhibit 1: The business contract
These rules are synthetic teaching requirements, not industry-wide
service standards.
Input consists of CSV-style records with
id, status,
impact and age_minutes.
Trim the ID and preserve its case. Missing or empty IDs are rejected.
The first nonempty ID reserves that identifier, including when
its row is invalid or closed. Later occurrences of that trimmed,
case-sensitive ID are rejected.
Trim and lower-case status and impact. Allowed values are
open or closed, and
blocked or normal.
Age must contain only digits 0–9 after trimming.
Zero is valid. Blank, negative and fractional values are invalid.
Missing or non-string required fields are rejected.
Validate closed records too. Exclude only fully valid closed records.
For valid open records: blocked means P1; otherwise
age greater than or equal to 60 means P2;
otherwise P3.
Preserve accepted input order and do not mutate the input records.
Return accepted records, rejected records with reasons,
and an excluded count. Every input row belongs to exactly one category.
Error precedence:
missing ID → duplicate ID → invalid fields → invalid age →
valid closed exclusion → priority.
Rejection row numbers start at the first data row, excluding
the CSV header.
5. Exhibit 2: Synthetic tickets
Eight records for prediction and testing
Row
ID
Status
Impact
Age in minutes
1
T001
open
blocked
25
2
T002
open
normal
90
3
T003
closed
blocked
120
4
T004
" OPEN "
"normal "
60
5
T005
open
blocked
Empty cell
6
T006
open
normal
-5
7
T007
open
normal
20
8
T001
open
normal
30
Spaces in row 4 are intentional. Row 5 contains an empty cell,
not the words “empty cell”. Row 8 repeats the first identifier.
The original clean demonstration used only rows 1, 2 and 3.
Predict the full report before running the draft.
6. Exhibit 3: The draft code
This is a deliberately defective, author-created teaching draft
representing an artifact accepted too quickly.
It is not a transcript of an actual model response.
def triage(rows):
accepted = []
for row in rows:
if row.get("status") == "open":
age = int(row.get("age_minutes", "0") or "0")
priority = (
"P1" if row.get("impact") == "blocked"
else ("P2" if age > 60 else "P3")
)
accepted.append({
"id": row["id"],
"priority": priority
})
return {
"accepted": accepted,
"rejected": [],
"excluded": len(rows) - len(accepted)
}
Investigate before editing
For rows 4, 5, 6 and 8, distinguish the required behaviour
from the draft’s actual behaviour.
Test a normal-impact ticket at ages 59, 60 and 61.
Identify which rejected records could be hidden inside
the draft’s excluded count.
Explain why changing one comparison operator does not repair
the entire contract.
Propose a small counterexample that could disprove your explanation.
Learning and task completion are different outcomes
Shen and Tamkin studied 52 developers learning an unfamiliar Python
library. AI access was randomly assigned. The assisted group showed
lower immediate comprehension, without a statistically significant
average time advantage. The participants were not a sample of
fresh graduates. Findings about interaction patterns were exploratory
associations, not randomised comparisons of teaching methods.
[S1]
Assistance can also improve measured output
Cui and colleagues combined three workplace experiments involving
4,867 developers. They report more completed tasks with coding
assistance and larger gains among less experienced developers.
Those output measures do not themselves establish retained learning.
[S2]
The learning environment matters
Kazemitabaar and colleagues studied 69 learners aged 10–17 in a
structured introductory environment. AI-supported authoring improved,
while subsequent manual modification and assessments did not show
a general learning penalty. The population and learning conditions
differ materially from workplace onboarding.
[S3]
Engineering practice and measurement require context
OpenAI’s harness-engineering account describes investment in
environments and feedback around agents. It is a company-reported
engineering experience, not a controlled learning study or a release
policy for this fictional team.
[S4]
METR’s February 2026 methodological update explains how selection
and time-measurement issues weakened interpretation of its later
developer-productivity experiment.
[S5]
These sources support investigating several outcomes.
They do not establish a universal rule that AI always helps
or harms learning.
8. Practice guide and worked analysis
Pause here if using this as an assignment.
Complete your recommendation before opening the analysis.
Open the worked analysis
A conditional recommendation
Keep Asha responsible, freeze the scope and run a bounded review
of correctness and understanding. Authorise a synthetic,
read-only demonstration only if the checks are satisfactory.
Treat an assisted pilot as a separate decision.
The pilot needs an operational owner, actual workflow comparison,
agreed data handling and a correction process.
A successful demonstration does not establish those conditions.
The hidden assumptions
One assumption is that a completed artifact establishes its owner’s
competence. Another is that assessing competence requires a full
manual rebuild.
A more useful question is:
What evidence would let another person trust this developer
with the next change?
That moves the review toward prediction, diagnosis, modification
and communication. This is our teaching reframing, not a claim
that the proposed process is a proven instructional intervention.
Comparing the delivery options
Option
Benefit
Main limitation
Generate more and inspect
Preserves the current workflow.
Examples may repeat the original blind spots.
Rebuild without AI
Makes independent work visible.
May consume time on syntax rather than critical reasoning.
Senior takeover
Transfers responsibility to an experienced developer.
Uses scarce capacity and may remove the learning opportunity.
Bounded review
Examines correctness and understanding while retaining ownership.
Needs a clear contract, competent observation and stop rules.
A proposed ninety-minute plan
Minutes
Activity
Evidence produced
0–15
Agree the contract and predict outputs.
Written predictions and identified uncertainties.
15–35
Run the draft and investigate mismatches.
Input, expected result, observed result and cause.
35–60
Repair the draft and run contract checks.
Corrected behaviour and test results.
60–75
Close AI assistance and attempt a new requirement.
Independent prediction, change and explanation.
75–90
Review evidence and record the scope decision.
Acceptance conditions, owner, limits and fallback.
These timings are proposals, not measured optimal allocations.
If critical behaviour remains unresolved, narrow the demonstration
or use the manual fallback.
What Asha must understand
She need not memorise every library call. She should be able to
trace the boundary, distinguish invalid from excluded records,
explain duplicate handling, diagnose an unexpected result and
preserve important behaviour when a requirement changes.
When another decision is defensible
A senior takeover or delay may be appropriate when the deadline
cannot be met safely, no competent reviewer is available or the tool
would directly change live records. The recommendation depends on
the small, synthetic and reversible scope of this case.
Download the resource pack below.
The exercise uses local synthetic data and Python’s standard library.
It requires no paid AI account or external API.
The supplied reference was tested with Python 3.12.14.
Learner workflow
Predict the report for the eight records.
Run the starter and record mismatches.
Explain each cause before editing.
Repair the implementation while retaining its interface.
Run the checks and add a new contract-derived test.
Complete the independent transfer task before reading the reference.
From the extracted resource directory:
cd lab
python starter.py tickets.csv
python -m unittest -v test_triage
The starter is deliberately wrong, so initial failures are expected.
Where appropriate, substitute python3 or py
for python.
Reveal the expected full-batch report
Category
Expected result
Accepted
T001 P1; T002 P2; T004 P2; T007 P3, in that order.
The reference implementation passed fifteen named contract tests.
They cover boundaries, normalisation, malformed and missing values,
duplicate IDs, closed-row validation, empty input, accounting,
accepted order and input preservation.
The sample CSV and a threshold-change exercise were also checked.
Test logs and the reference implementation are in the instructor
resources.
These are local exercise results. They do not establish production
readiness, workplace savings or learning gains.
10. Independent transfer task
Change the normal-impact escalation threshold from
60 to 45 minutes.
Preserve P1 precedence and every validation rule.
Predict outcomes at ages 44, 45 and 46.
Predict the priority of a valid blocked ticket aged 10 minutes.
Identify existing test expectations that must change.
Identify behaviours that must remain unchanged.
Implement or specify the change without AI assistance.
Explain the results using the contract and observed behaviour.
Official documentation may be used for syntax if the assessment
permits it. The target is reasoning about the change, not memorising
an API.
Reveal the transfer-task check
Normal-impact ages 44, 45 and 46 become P3, P2 and P2.
A valid blocked ticket remains P1.
A previous expectation that a normal-impact 59-minute ticket is P3
must change under the new threshold.
Stop conditions
Stop the demonstration-ready claim if required priorities are wrong,
input accounting fails or the named owner cannot explain a critical
exception. Escalate business-rule ambiguity to the coordinator
and release ambiguity to the delivery lead.
11. Measuring business value and learning
Measure the work required to obtain a trustworthy result,
including export, execution, reconciliation, corrections and review.
Question
Useful evidence
Does the tool obey the contract?
Tests, representative records and reconciled totals.
Can its owner maintain it?
Prediction, diagnosis, novel change and explanation.
Does it help the coordinator?
Comparable end-to-end workflow time and error burden.
Is learning retained?
A parallel task after a delay, with assistance rules recorded.
What does support cost?
Reviewer time, rework, exceptions and escalations.
Net effort benefit per batch =
manual end-to-end minutes − assisted end-to-end minutes.
Include setup, training, review and support before claiming a pilot
benefit. The case’s twenty-minute manual estimate is unvalidated.
A delayed local assessment can guide mentoring.
It is not, by itself, a causal study of AI’s effect on learning.
Do not present improvement after repeated practice as proof
that the proposed review process caused it.
12. Learning and workplace pathways
Foundation and no-code
Trace four records through the rules on paper.
Explain a defect in ordinary language.
Submit a decision memo, corrected rule flow and reconciliation.
Practitioner
Repair the starter, add a test and submit a before-and-after report.
Include a handover note, actual test results and an independently
completed transfer task.
Advanced
Design a stronger correction policy for repeated IDs.
Compare first-occurrence rejection with latest-valid-record handling.
Specify provenance, audit events, access, recovery and safe updates
for a real integration.
Freelancers
Scope a bounded prototype review. Ask the client for approved rules
and synthetic examples. Deliver a contract, prototype, tests and
handover. Estimate implementation and review separately, and state
exclusions, acceptance conditions and future support ownership.
Workforce teams
Apply the same questions to an AI-assisted spreadsheet, report,
document workflow or automation: can the owner explain the important
decisions, detect exceptions and respond to a changed requirement?
Reusable tutor prompt
I will provide a business rule and my prediction.
Ask one question that tests my reasoning.
Do not give the answer before I attempt it.
Identify unsupported assumptions and suggest a small counterexample.
Reusable review prompt
Compare this draft with the agreed contract.
For each possible mismatch, propose an input that would expose it
and identify the rule it violates.
Separate definite defects from questions for the business owner.
Do not invent requirements or successful tests.
For assessment use, assign the narrative and exhibits first.
Withhold the worked guide and instructor resources until learners
submit their predictions and recommendations.
Collapsed sections on a public blog are not secure answer protection.
Opening question
Would you authorise tomorrow’s demonstration?
What exactly would you be authorising?
Record the answers before discussing code.
Separate a synthetic demonstration, an assisted pilot and
a live deployment.
Suggested ninety-minute class
Minutes
Activity
0–10
Opening vote and conditions.
10–25
Facts, assumptions and missing evidence.
25–40
Trace the draft against the contract.
40–60
Compare options and allocate review time.
60–75
Challenge recommendations and reveal evidence.
75–90
Transfer task and exit recommendation.
Discussion questions
What does the client need accomplished?
Which uncertainty is technical, organisational or educational?
When should a senior developer take over?
What evidence would justify a shorter review next time?
Could a senior developer pass the same demo while misunderstanding it?
What would change if the script updated live records?
Misconceptions to challenge
“She did not type it, so she cannot own it.”
Ask what maintenance responsibilities she can demonstrate.
“Tests passed, so she understands it.”
Ask for a fresh prediction and unfamiliar change.
“The research proves AI harms every fresher.”
Examine the population, task, outcome and limitations.
“The demonstration saved twenty minutes.”
Ask for a measured baseline and full workflow effort.
A defensible alternative recommendation can earn strong marks.
Assess the quality of evidence and reasoning rather than agreement
with the worked answer.
14. Assessment rubric
Criterion
Weight
Strong evidence
Problem and evidence
20%
Clear scope, exhibit use and qualified research claims.
Alternatives and judgment
20%
Credible options, trade-offs and conditions for changing the decision.
Workflow or technical design
20%
Feasible steps, correct contract and named responsibilities.
Verification and failure response
25%
Boundary and exception tests, transfer, stop conditions and fallback.
Professional communication
15%
Clear recommendation, assumptions, assistance disclosure and handover.
Score each criterion from 1 to 4:
1 = substantially incomplete;
2 = material gaps;
3 = sound and usable;
4 = well supported and explicit about limits.
Weighted score = sum of each weight × score ÷ 4.
Set pass requirements according to the teaching context.
An aggregate score does not override unresolved critical defects.
15. Research and verification record
The research examined learning, task output, engineering practice
and measurement limitations. Primary sources and counterevidence
were used to avoid treating one study as a universal workplace rule.
Evidence cutoff: 6 November 2026.
Research findings are separated from fictional operational details.
The reference passed fifteen named local contract tests.
The sample report and threshold-change exercise were checked.
The proposed mentoring process has not undergone a workplace trial.
No external expert review or independent statistical replication was obtained.
No production deployment or business saving is claimed.
The research dossier contains annotated sources, a claim register,
source cautions and maintenance triggers. Review the case when
central evidence is corrected, relevant follow-up research appears,
the runtime changes or learners identify a defect.
Cui, Z., Demirer, M., Jaffe, S., Musolff, L.,
Peng, S., and Salz, T.The Effects of Generative AI on High-Skilled Work:
Evidence from Three Field Experiments with Software Developers.
Microsoft Research publication record
.
DOI:
10.1287/mnsc.2025.00535
.
Used for the reported task-output finding, not a claim about learning.
Kazemitabaar, M., Chow, J., Ma, C. K. T.,
Ericson, B. J., Weintrop, D., and Grossman, T.Studying the effect of AI Code Generators on Supporting
Novice Learners in Introductory Programming.
CHI 2023.
Read the paper.
DOI: 10.1145/3544548.3580919.
Relevant sections: user study, results, discussion and limitations.
Lopopolo, R.Harness engineering: leveraging Codex in an agent-first world.
OpenAI, 11 February 2026.
Read the engineering account
.
First-party practice account; not a controlled learning study.
Becker, J., Rush, N., Cunningham, T.,
Rein, D., and Mahamud, K.We are Changing our Developer Productivity Experiment Design.
METR, 24 February 2026.
Read the methodological update
.
Used to qualify generalisations about developer productivity.
Sources accessed 6 November 2026.
Third-party publications retain their own rights.
These links open the author’s ChatGPT Library and may require
account access. For public distribution, upload the files to your
course platform or another public download location and replace
these links before publishing.
Reflection
What evidence would make you comfortable asking this person
to handle the next change?
Answer that question for your own AI-assisted work:
a program, spreadsheet, report, document workflow or automation.
Identify one prediction, one failure test and one change
you can explain independently.
.png)
