AI Engineering Principle #005: Give the Model the Right Context at the Right Time
AI Engineering Principles & Practice is an independent study series for students, freshers, and teams. Each post explores a practical principle for building, evaluating, or improving AI systems.
Good AI responses depend not only on the model, but also on the information made available to it for the task.
1. The principle
Give the model the information it needs for this task, in a clear and usable form.
That may include instructions, relevant conversation history, tool results, examples, or reference material. The goal is not to provide as much information as possible. The goal is to provide useful information that helps the model produce a grounded response.
2. Why context matters
A model can only use the information available in its current interaction. If an assistant must answer questions about a company handbook, for example, it needs access to the relevant handbook content. Without it, the model may give a generic answer or make an unsupported guess.
At the same time, adding large amounts of unrelated material can make the task harder to manage and consume the available context window. Context should be selected and organized around the user’s request.
Anthropic describes this work as context engineering: curating the information and tools provided to an AI system. OpenAI’s prompt-engineering guidance also discusses supplying relevant context, including through retrieval-augmented generation (RAG), when a model needs access to external or private knowledge.
3. A practical example: the handbook assistant
Imagine a learner asks: “How many days do I have to submit my travel expense claim?”
- Weak approach: Ask the model to answer from general knowledge. The answer may not match the organization’s policy.
- Better approach: Search the approved handbook, retrieve the section about expense claims, and provide that passage with the question.
- Stronger response: Answer using the retrieved policy, identify the relevant section, and say when the handbook does not contain enough information.
The model’s response is more useful because the context is relevant, authoritative, and connected to the task.
4. Try it yourself
Design a simple assistant that answers questions from a course handbook.
- Choose one realistic learner question.
- Find the exact handbook section that answers it.
- Write instructions telling the assistant to use the supplied section and acknowledge missing information.
- Test the assistant with a question answered by the section and one it does not answer.
- Record whether it used the source correctly, answered clearly, and avoided inventing details.
Reflection: What changed when you supplied the relevant passage? What information could you leave out without affecting the answer?
5. Key takeaway
Context is part of the system design. Select relevant information, present it clearly, and check whether the model’s answer is supported by that information. Good context can help an AI system respond more usefully; it does not remove the need to verify the result.
6. Sources for further study
AI Engineering Principles & Practice
An independent, research-grounded study reference for building, evaluating, and improving AI systems.
