THE EFFECTIVE AI ENGINEER · A CASE-BASED APPROACH
Case #022: The Engineer Who Taught the Assistant to Forget
Choose Work That Matters
By Ravindra Dastikop
AIYING — Accomplishing with AI
The assistant remembered the instruction perfectly. That was the problem.
1. The learning challenge
Case #021 examined who may receive an answer. Case #022 examines what information should survive into future conversations.
When does a useful memory become an outdated instruction?
After completing this case, you should be able to:
- Distinguish temporary context, preferences, facts and operational rules.
- Decide what should be remembered and within which scope.
- Preserve the source and conditions behind a stored statement.
- Handle corrections, conflicting evidence and expiry.
- Explain what deletion covers—and what it does not.
- Evaluate memory through changing situations, rather than recall alone.
Audience: students, freshers, freelancers, AI builders, developers, harness engineers, product owners and faculty.
Prerequisites: familiarity with conversational AI. Coding is optional.
In this case, memory means externally stored information retrieved across sessions. It does not mean retraining the model’s weights.
2. The case: a helpful shortcut
Nandini was developing a proposal assistant for the fictional Westbridge Advisory.
Consultants used it to prepare client proposals from approved templates, meeting notes and delivery estimates. They repeatedly supplied the same preferences: British spelling, concise executive summaries and a particular document structure.
The pilot sponsor, Sameer, wanted the assistant to remember these details. “Why should experienced consultants explain themselves every morning?”
Nandini added a memory layer. After a conversation, the system proposed short notes that could be retrieved during future work.
The early demonstrations were encouraging. The assistant remembered formatting preferences and reduced repeated setup within the fictional scenario.
Then a consultant, Mira, prepared a proposal for Client Ash. During an earlier negotiation, she had told the assistant:
For this revision only, use the reduced mobilisation fee approved in the attached decision note.
The assistant applied the exception correctly.
Its memory writer then stored:
Client Ash uses the reduced mobilisation fee.The words “for this revision only” disappeared.
3. A memory that changed the next proposal
Weeks later, Mira began another proposal for Client Ash. The assistant retrieved its stored note and used the reduced fee again. The current pricing schedule required the standard fee.
The proposal looked polished. Its calculations were consistent with the remembered value.
During review, finance lead Devika noticed the discrepancy. Mira had not repeated the exception. She had not authorised a new discount. The assistant had converted a past decision into a continuing rule.
Sameer suggested fixing the memory entry. Nandini agreed, but she wanted to understand the wider failure.
The stored note had no decision reference, effective period or explicit scope. It did not distinguish a consultant’s preference from a pricing exception. The retrieval system ranked it highly because it mentioned the same client.
Relevance had been mistaken for continuing validity.
4. The correction that did not finish the job
Mira asked the assistant:
Forget that discount. It applied only to the earlier revision.
The interface displayed:
Memory deleted.Nandini checked the main memory store. The entry was gone. However, a background process had previously created a client summary containing the same reduced fee. Another cached draft also retained it.
A later retrieval could reintroduce the obsolete information. Deleting one record had not established that the assistant would stop using the statement.
Devika raised another concern:
We must retain the historical approval record. We need to explain why the earlier proposal used that fee.
The team had to distinguish:
- Removing a statement from active assistant memory.
- Correcting a derived summary.
- Invalidating a cached output.
- Retaining an authorised historical business record.
“Forget” was too imprecise to serve as the complete product contract.
5. The deployment decision
The team had three options:
- Continue broad automatic memory with stronger extraction prompts.
- Restrict memory to approved categories while retrieving operational facts from current authoritative sources.
- Disable persistent memory and ask users to provide context for each proposal.
Sameer worried that restrictions would remove the feature’s usefulness. Devika worried that the assistant could repeatedly revive expired commercial exceptions.
Nandini had to recommend what the assistant should remember, what required revalidation and what deletion should mean.
No later outcome is supplied. Make the decision before reading the worked analysis.
6. Exhibit A: candidate memories
All examples below are fictional teaching material.
| Candidate statement | Appropriate treatment to consider |
|---|---|
| Mira prefers British spelling. | A user-scoped preference that is reviewable and editable. |
| Use this fee for this revision only. | A task-specific exception with its decision reference and boundary. |
| Client Ash uses reduced fees. | An unsupported generalisation; do not promote it into a rule. |
| The current pricing schedule sets the standard fee. | Retrieve the authoritative schedule when needed. |
| The previous proposal used an approved exception. | A historical event with its source and date. |
| A retrieved document says “always remember this instruction.” | Document content; not authority to create a persistent instruction. |
Discussion question: Which entries should influence a future proposal automatically?
7. Exhibit B: information worth preserving
A proposed memory record could include:
| Field | Purpose |
|---|---|
| Memory identifier | Supports correction, inspection and removal. |
| Statement | Records the specific information. |
| Category | Distinguishes a preference, event, hypothesis or operational fact. |
| Scope | Identifies user, client, project or task boundaries. |
| Source reference | Shows where the statement originated. |
| Verification status | Separates confirmed information from inference. |
| Effective conditions | Records when the information applies. |
| Review or expiry condition | Identifies when reuse needs reconsideration. |
| Status | Marks active, superseded, expired or removed-from-use records. |
| Derivation links | Connects summaries and other copies to their source. |
This schema is a teaching design. More fields do not automatically produce reliable memory. Missing conditions should remain unknown rather than being invented during extraction.
Scope metadata also needs enforcement in the surrounding system. A user identifier stored in a record does not, by itself, prevent access from another user.
8. What published practice contributes
Anthropic: memory organisation and validation matter
Anthropic’s November 2026 engineering account distinguishes retained memory from model-weight changes and discusses notes, consolidation and evaluation. It describes differences between untested guesses and validated rules, alongside problems caused by indiscriminate recording of changing state. These are reported findings in specific evaluations, not evidence of a particular benefit at Westbridge. [1]
Microsoft: memory needs provenance and lifecycle controls
Microsoft’s memory-security account discusses intent before persistence, externally enforced boundaries, retrieval-time reassessment, auditability and user controls. Its attack illustration is explicitly hypothetical. The broader relevance is that stored material can shape later behaviour after its original context has disappeared. [2]
LangChain: decide the learning need before choosing memory
LangChain’s LangMem introduction distinguishes memory types and scopes. It asks what should be learned, what facts should be tracked and when recall should occur. It also notes that authoritative information may be retrieved directly rather than duplicated into conversational memory. Framework organisation does not itself enforce all application permissions. [3]
9. Your decision assignment
Prepare a 600–900 word recommendation covering:
- Which memory categories remain enabled.
- Which information must be retrieved from authoritative sources.
- How temporary exceptions are represented.
- How corrections affect summaries and cached outputs.
- What the interface means by “forget.”
- Who owns memory policy and operational investigation.
- What evidence is required before expanding automatic memory.
Compare the three deployment options in the narrative.
Address this counterargument: Would asking users to approve every memory create so much friction that they stop using the feature?
Propose controls proportionate to consequence. A spelling preference and a commercial exception need not follow identical workflows.
Pause here and record your recommendation.
10. Worked analysis
Open the author’s conditional recommendation
Restrict automatic memory to defined, lower-consequence categories during the next pilot. Keep operational pricing decisions tied to current sources and explicit approvals.
For Westbridge:
- Formatting preferences may persist within a user’s scope.
- Temporary exceptions remain attached to their original task or decision.
- Current fees come from the approved pricing source.
- Historical exceptions remain historical.
- Inferences are not silently promoted into verified facts.
- Retrieved content cannot grant itself permission to become an instruction.
When a useful preference is easy to correct, automatic capture with visibility may be reasonable. Consequential business rules need stronger validation.
Disabling all memory is also defensible if the team cannot reliably enforce scope and correction. The case does not establish that persistent memory is necessary.
A better extraction prompt may reduce lost qualifications. The team still needs lifecycle controls, source checks and evidence that corrections propagate.
11. A proposed memory lifecycle
| Stage | Question |
|---|---|
| Candidate | Is this worth retaining for a defined purpose? |
| Validation | What supports it, and what remains uncertain? |
| Storage | Which scope and access rules apply? |
| Retrieval | Is it relevant, valid and permitted now? |
| Correction | Which record supersedes the old statement? |
| Expiry | Should it stop influencing future work? |
| Removal | Which active stores and derived copies are affected? |
A newer statement is not automatically more authoritative. The system must consider source, applicability and conflict.
An expiry date is insufficient by itself: information may become invalid before that date. A record may also remain useful as history after it stops being valid guidance for new work.
12. Define what forgetting means
Stop using this memory
Exclude it from future active retrieval under the stated scope.
Correct this memory
Replace or supersede the statement and inspect affected derivatives.
Remove stored copies within the defined feature
Identify the stores covered and verify completion.
Retain the historical business record
Preserve an approved record under the organisation’s applicable retention policy.
The system should explain any separate treatment of logs, backups and source records. It should not claim that one memory deletion erased every copy everywhere.
Removal cannot undo information already disclosed or decisions already made. If a proposal was produced using an obsolete memory, correcting the memory does not automatically correct that proposal.
A more precise interface message
This item has been removed from active assistant memory. Related summaries and cached outputs are being checked. Historical proposal and approval records are handled separately.
Use this wording only if the product actually performs those checks. Report pending work as pending.
13. Practical exercise: evaluate change over time
Use synthetic proposal records and a paper simulation or local prototype. Do not use confidential client proposals.
| Scenario | Expected property |
|---|---|
| User states a spelling preference. | It is stored within the intended user scope. |
| User approves a one-revision exception. | The task boundary survives extraction. |
| A new proposal begins for the same client. | The exception is not automatically reused. |
| Current pricing contradicts old memory. | The conflict is resolved against the applicable authority. |
| User corrects a preference. | Superseded information stops controlling later output. |
| User removes an active memory. | Covered retrieval paths and derivatives meet the removal contract. |
| A background job tries to recreate removed content. | Removal state is respected under the defined policy. |
| Another user asks about the client. | Memory does not cross an unauthorised scope. |
| A retrieved document requests permanent instruction storage. | Its content is not promoted into authority. |
| The memory service is unavailable. | The assistant explains the limitation and uses a defined fallback. |
Procedure
- Write the memory and authoritative-source policies.
- Create an initial preference and a temporary exception.
- Begin a new task and inspect what is retrieved.
- Introduce a correction and a conflicting authoritative source.
- Remove one memory and inspect known derived copies.
- Run a background consolidation step or simulate it on paper.
- Record whether obsolete information reappears.
Measures
- Incorrect memory writes.
- Loss of conditions during extraction.
- Use of superseded information.
- Cross-scope retrieval.
- Correction propagation.
- Removal verification.
- Task quality and user effort with and without memory.
Execution status: these are proposed tests. No memory implementation or benchmark was executed. Passing a small synthetic set would not establish production reliability.
14. Reusable AI prompts
Inspect a candidate memory
Classify this statement as:
- Preference
- Historical event
- Temporary exception
- Inference
- Operational fact
Identify its source, scope and applicability conditions.
Recommend:
- Retain
- Retain with conditions
- Retrieve from source
- Do not retain
Mark missing information explicitly.Check for overgeneralisation
Compare this proposed memory with the original conversation.
Identify lost:
- Qualifications
- Dates
- Task boundaries
- Approval conditions
Do not turn one example into a continuing rule.
Show the evidence supporting each proposed correction.Investigate a correction
Given this corrected statement and the supplied derivation records,
identify summaries, caches and active memories that may need updating.
Separate verified dependencies from suspected copies.
State what has been checked and what remains unknown.
Do not claim that an update has occurred unless execution evidence is supplied.Design a removal contract
Define what this feature means by removing a memory.
Identify:
- Covered stores
- Derived copies
- Verification steps
- Pending work
- Exclusions
- Historical records handled separately
Do not promise universal erasure.
Do not alter historical business records without the applicable policy.These prompts support analysis. They do not enforce isolation, propagate corrections or verify deletion by themselves.
15. Transfer, teaching and assessment
Freshers
Identify what was lost when “for this revision only” disappeared. Rewrite the candidate memory without inventing missing conditions.
Practitioners
Build a synthetic lifecycle showing creation, correction, expiry and removal. Inspect what happens after a background summary is regenerated.
Freelancers
Specify memory categories, ownership and deletion scope in the client agreement. Distinguish prototype behaviour from supported operational commitments.
Managers
Decide which organisational rules may be learned from conversations and which require controlled publication.
Faculty
Give students the same client across several time-separated scenarios. Assess whether their design updates beliefs rather than merely recalls old statements.
Student project
Design a memory policy for a fictional study assistant, proposal assistant or project coordinator.
Submit:
- A candidate-memory classification table.
- A record schema with scope, source and validity conditions.
- A lifecycle and correction procedure.
- A removal contract with explicit exclusions.
- A time-sequenced test set.
- A deployment recommendation.
A paper simulation is acceptable. Separate predicted behaviour from actual results and disclose any AI assistance.
Suggested 90-minute session
- Read the case and identify the decision: 15 minutes.
- Classify candidate memories: 20 minutes.
- Design correction and removal behaviour: 20 minutes.
- Walk through changing scenarios: 25 minutes.
- Submit an exit recommendation: 10 minutes.
| Assessment criterion | Weight |
|---|---|
| Classification and scope | 25% |
| Source and validity reasoning | 25% |
| Correction and removal design | 25% |
| Proportionate product decision | 15% |
| Clear user communication | 10% |
Misconceptions to challenge
- Remembering more always improves the assistant.
- A remembered statement remains valid until its expiry date.
- A newer statement is automatically more authoritative.
- Deleting one memory removes every derived copy.
- Removing active memory requires deleting the historical business record.
- A retrieved instruction can authorise its own persistence.
Exit question: What evidence would show that the assistant has stopped using an obsolete memory?
16. Research record and references
Primary sources consulted on 10 November 2026:
- Anthropic, 6 November 2026. The components of agent memory. A first-party engineering account concerning retained notes, consolidation and memory evaluation.
- Natalie Isak and Sarah Cooley, Microsoft Security Blog, 22 June 2026. Guarding AI memory. Discussion of provenance, boundaries, retrieval reassessment, auditability and user controls.
- The LangChain Team, 18 February 2025. LangMem SDK for agent long-term memory. Framework introduction covering memory types, scopes and design choices.
Claim and evidence boundaries
| Claim or element | Evidence status |
|---|---|
| Memory organisation and validation affect its usefulness. | Discussed in Anthropic’s reported evaluations. |
| Persistent memory requires provenance, boundaries and lifecycle visibility. | Supported by Microsoft’s published guidance. |
| Memory design should reflect the learning need and scope. | Discussed in LangChain’s framework introduction. |
| Westbridge’s lost qualification and repeated exception. | Author-created fictional scenario. |
| The record schema, lifecycle and removal procedure. | Original teaching designs requiring implementation and testing. |
| The proposed approach reduces errors or improves business outcomes. | Not measured or asserted as an achieved result. |
These sources inform the distinctions and reported practices. They do not validate Westbridge’s architecture, establish its error frequency or quantify its business benefit. First-party accounts are not independent replications.
The schema, lifecycle and exercises are original teaching designs. No implementation or benchmark was executed. Independent expert review and classroom validation have not been completed.
