AIYINGFrom AI Awareness to Accomplishment
Showing posts with label Runtime Context. Show all posts
Showing posts with label Runtime Context. Show all posts

Tuesday, November 17, 2026

Lesson #004 — Training vs Inference: What Does an AI Model Learn?

AIYING · AI ENGINEERING: A PRACTITIONER’S APPROACH · LESSON #004

What Does a Model Learn—and What Happens During Inference?

By Ravindra Dastikop



Training changes learned parameters; inference uses them

In ordinary machine-learning training, an optimization process adjusts a model’s learned parameters using data and a learning objective. During ordinary inference, the trained model computes an output for an input without updating those learned parameters.

Parameters are internal numerical values, such as neural-network weights, that shape the model’s computations. A learning objective defines what the training process tries to improve. An objective can encourage useful behaviour without guaranteeing that every future output will be correct.

Google’s supervised-learning introduction illustrates this distinction using examples and predictions. Training methods vary; not all learning requires people to label every example.

For our document-help assistant, the practical implication is important: supplying a handbook passage in a request is not the same as retraining the model.

This lesson describes conventional fixed-parameter inference. Systems can separately implement online learning, saved memories, or later training pipelines; those mechanisms should be identified explicitly.

Worked example: the model and the handbook do different jobs

Our fictional training centre has a system brief, a baseline procedure, and candidate requirements. We have not yet chosen a production model or architecture.

Imagine testing an existing text-generation model with the approved source from earlier lessons:

Participants must bring a laptop. Loan devices are not provided.

The question is: “Can you lend me a computer for the workshop?”

The model’s prior training may help it interpret language and relate “computer” to “laptop.” The supplied passage provides the local policy. Our requirements determine what an acceptable response must preserve.

A candidate response is:

The supplied handbook says participants must bring a laptop and that loan devices are not provided.

This is an illustrative answer, not a measured model result.

Three mechanisms to distinguish
Mechanism What changes? Role in our case
Training or fine-tuning Learned parameters or trainable adapter parameters Can adapt model behaviour; not required merely to supply this passage.
Runtime context Information available in the current request Provides instructions, the question, and relevant policy text.
External storage Files, databases, or application memory Preserves approved document versions for later retrieval or loading.

A document in storage is not automatically available to the model. The application must retrieve, attach, or otherwise supply the information needed for the task.

A chat product may preserve history or saved memory and include some of it in later requests. That application behaviour does not by itself establish that the model’s weights changed.

How text generation fits

A token is a unit of text processing; it may be a word, part of a word, or punctuation. In a common decoder language model, generation proceeds by predicting a distribution over the next token using preceding context, then selecting a token and continuing. Hugging Face describes this as causal language modelling.

The output depends on learned parameters, the supplied context, and generation settings. Other AI tasks can produce a class label, numerical estimate, image, or other output rather than a sequence of text tokens.

Training can also lead models to memorize some data. Learned parameters are nevertheless not a version-controlled policy database, and plausible wording is not evidence of current policy.

Create Model–Context Map v0.1

  1. List the ingredients: model identifier, task instructions, user question, approved source version, and any external storage.
  2. Assign responsibility: mark which component supplies language capability, local facts, persistent records, and verification.
  3. Specify the update route: if the handbook changes, record who approves the revision and how the application supplies it. Do not silently overwrite the source used for earlier tests.
  4. Link checks: connect the map to R-01 correctness, R-02 source support, and R-03 missing-information handling from Lesson #003.
component_or_information:
location: model / runtime_context / external_storage
version_or_identifier:
how_it_reaches_the_model:
update_owner_and_procedure:
linked_requirement_ids:
evidence_or_test_status:

Optional observation: change context, not the model

In separate fresh conversations, use the same model and ask the computer-loan question with these three inputs:

  • A: The established no-loan passage.
  • B: A clearly labelled hypothetical alternative: “For this fictional exercise only, loan laptops are available by prior reservation.”
  • C: No handbook passage; instruct the assistant to say when the supplied information does not establish a policy.

In each request, instruct the model to answer only from the supplied source and identify supporting text. Record the actual outputs; compare them against the source and requirements.

B is a counterfactual test fixture, not a revision of our running case’s approved policy. Keep it separate. C tests handling of absent context; it does not erase the model’s prior knowledge.

If answers track A and B, you have observed context-sensitive behaviour—not proved permanent learning or perfect grounding. If they fail, preserve the failure for later evaluation lessons.

Verification: can another learner use your map to explain what needs updating when policy changes, where the source persists, and what the model actually receives?

Practitioner extension: inspect a real request at the application boundary, using sanitized data. Compare the source version you intended to supply with the content actually sent. Do not infer the request contents from the interface alone.

Case progress: System Brief → Baseline Record → Requirements Register → Model–Context Map. Preserve unresolved decisions and version identifiers.

Next lesson: Why Can an AI System Fail Even When Its Model Works?

Sources and scope: Google: Supervised Learning; Hugging Face: How Transformers solve tasks. Checked 17 November 2026. The running case, map, and context experiment are AIYING teaching adaptations. No model experiment was executed for this lesson.

AIYING · Ravindra Dastikop
AI Engineering: A Practitioner’s Approach · Lesson #004
See differently. Generate possibilities. Accomplish what matters.

Accomplish with AI

Powered by Blogger.

Featured post

How do I AI my first attempt? A Practical Guide to Getting Started

AIYING · HOW TO AI · EPISODE 001 How do I AI my first attempt? Turn an unfinished idea into one small version you can exa...

Search This Blog

List of blogs

Search This Blog

About

Translate

Instant Stories

Read in Your Language

Test

share this

ShareThis

Popular Posts