AIYING · V1.0 · REGULAR VIEW
The AI Answer Looks Right. How Do You Know?
By Ravindra Dastikop
You ask AI to turn a textbook passage into five questions and an answer key.
Seconds later, everything is ready. The questions are clear. The answers sound confident. The formatting is neat.
You could use it immediately.
Then a question occurs to you:
Does the passage actually support every answer?
This post begins with that illustrative situation. It explores one practical idea: check an AI-generated answer by tracing it to the material it is supposed to represent.
A correct-looking answer can change the meaning
Consider this fictional source passage:
“The school garden receives rainwater collected from the roof. During dry weeks, students also water the plants using stored water.”
AI generates this question and answer:
Question: How is the school garden watered?
Answer: The school garden is watered entirely by rainwater collected from the roof.
The answer borrows familiar words from the passage. But it adds “entirely.” The passage does not establish that all the stored water comes from roof collection.
A more faithful answer would be:
“The garden receives rainwater collected from the roof. During dry weeks, students also use stored water.”
The difference is small in wording and significant in meaning.
Check the claim, then find its support
For a task that asks AI to answer according to a supplied text, examine each answer in three steps.
1. Identify what the answer claims.
Pay attention to words such as all, only, always, never and entirely. Check numbers, dates and statements about causes.
2. Locate the supporting passage.
Find the sentence or sentences that justify the claim. Similar vocabulary alone is insufficient; the meaning must match.
3. Decide how the answer relates to the source.
| Relationship | What it means | Your next step |
|---|---|---|
| Directly supported | The source explicitly supplies the information. | Check that the wording preserves its meaning. |
| Inferred | The answer draws a conclusion from the source. | Decide whether inference is allowed and explain the reasoning. |
| Not established | The source does not provide enough support. | Revise the answer or state that the information is missing. |
An answer may be true in the wider world and still be unsuitable for an exercise requiring answers strictly from the passage.
Ask AI to make checking easier
You can request an answer key that shows its basis:
REUSABLE PROMPT
Using only the passage below, create five questions and an answer key.
For each answer, include the exact supporting sentence or sentences from the passage. Label the answer as directly stated or inferred. If the passage does not establish an answer, say so.
Do not add outside facts. Keep the language suitable for [class level].
Passage: [paste the text].
This gives you a more inspectable draft. You still need to compare the quoted evidence with the original passage. AI can misquote a source or attach a sentence that does not fully support its answer.
Try it on one answer
Choose one short, nonconfidential passage and generate three questions.
Before using the answer key, select one answer and:
- Underline its factual claims.
- Find the supporting words in the passage.
- Remove or qualify anything the passage does not establish.
Your accomplishment is specific: one answer you can explain and defend using its source.
You can then repeat the check for the remaining answers.
Where does the difficulty appear for you?
Checking may become harder when an answer combines several sentences or requires an inference.
When you compare an AI-generated answer with its source, which is harder: finding the evidence or deciding whether the evidence is enough?
Share a short, nonconfidential example in a comment if you have one. That distinction could help shape our next practical exploration.
