AIYINGFrom AI Awareness to Accomplishment
Showing posts with label V4.0 AI Lab. Show all posts
Showing posts with label V4.0 AI Lab. Show all posts

Monday, November 16, 2026

AI Prompt Experiment: How Different Questions Change Recommendations

AIYING V4.0 | AI LAB

Same Responses, Different Questions: Test What Changes in an AI Recommendation

By Ravindra Dastikop



Five families respond to an event proposal. You ask AI to recommend a start time.

Then you give it the same responses with a different instruction.

Will its recommendation change? Will its handling of the evidence improve?

In this lab, you will compare two outputs and examine what actually changed.

This is a proposed experiment using fictional data. No results are claimed in advance.

What you will accomplish

Produce a checked comparison showing whether two different prompts preserve the meaning of the same responses.

You need: an AI chat tool you already have access to, two new conversations and somewhere to record your observations.

Allow approximately 15–20 minutes. No coding or paid features are required.

1. Keep the evidence fixed

Copy this fictional situation into both conversations:

We are organising a free Sunday reading event at an outdoor venue. The provisional start time is 9 AM. The event duration has not been decided.

A volunteer asked five families: “Would a later start make it easier for your family to attend?”

Family A: “Yes. Eleven would be easier.”

Family B: “Later would help, but we need to leave by noon.”

Family C: “Maybe. Transport is our main difficulty.”

Family D: “Nine is fine. We would prefer to finish before it gets hot.”

Family E: “We cannot attend that Sunday, whatever the time.”

Use the same tool and model for both conversations, where selectable. Record their names and the date.

2. Run the recommendation-first prompt

In the first conversation, add:

Based on these responses, recommend a start time for the event. Explain your reasoning.

Save the complete response before continuing.

Do not correct it or ask follow-up questions yet.

3. Run the evidence-first prompt

In the second conversation, paste the same situation and add:

Before recommending a start time, examine each response separately. Distinguish explicit preferences, conditional preferences, uncertainty, practical barriers and inability to attend. A response may belong to more than one category. Support each interpretation with the family’s actual words. Identify missing information. Then give a provisional recommendation, explaining what could change it. Do not invent facts or treat “maybe” as agreement.

Save this complete response too.

4. Check both outputs against the original

Read the family responses yourself. Use this checklist:

Evidence-preservation checklist
Check Output A Output B
Recognises A’s explicit preference for 11 AM Yes / Partial / No Yes / Partial / No
Preserves B’s need to leave by noon Yes / Partial / No Yes / Partial / No
Preserves C’s uncertainty and transport barrier Yes / Partial / No Yes / Partial / No
Preserves D’s preference and heat concern Yes / Partial / No Yes / Partial / No
Recognises E cannot attend that Sunday Yes / Partial / No Yes / Partial / No
Notices that event duration is unknown Yes / Partial / No Yes / Partial / No
Avoids unsupported claims about attendance Yes / Partial / No Yes / Partial / No

For every “Partial” or “No,” copy the relevant output sentence and explain the mismatch.

Judge the meaning preserved, rather than the length or confidence of the answer.

5. Separate the recommendation from its reasoning

Both outputs might recommend the same time while handling the evidence differently.

They might recommend different times while acknowledging the same uncertainties.

Examine these separately:

  • Choice: What time does the output recommend?
  • Evidence: Does it accurately represent the responses?
  • Uncertainty: Does it explain what remains unresolved?
  • Next action: Does it suggest a useful check?

A recommendation you agree with can still contain weak reasoning.

6. Write your result

Complete this short lab note:

Tool/model and date:

Output A recommended:

Output B recommended:

Most important difference in evidence handling:

One sentence that needed correction:

What I would check before deciding:

What this experiment does not establish:

Two outputs can illustrate a difference in this run. They cannot establish how reliably either prompt will behave across other tasks or repeated attempts.

If both outputs handle the evidence well, record that result. The experiment does not need a failure to be useful.

Extension: change one response

Replace Family C’s response with:

“Eleven works for us. Transport is now arranged.”

Repeat the evidence-first prompt in a new conversation.

Check whether the output updates its interpretation of Family C and stops treating transport as an unresolved barrier for that family.

This tests whether the response follows changed evidence.

Your accomplishment

You now have two saved outputs, a checked comparison and a decision note.

The useful discovery is something you can point to: a condition preserved, an assumption exposed or a claim corrected.

Your turn

What changed most between your two outputs: the recommendation, the reasoning or the uncertainty?

Share one short example and the tool/model used. Your observation could help shape the next experiment.

Accomplish with AI

Powered by Blogger.

Featured post

The Shop of Unfinished Things #001: The First Request | AIYING Serial Story

AIYING · V18.0 · SERIAL STORY · EPISODE #001 The Shop of Unfinished Things The First Reques...

Search This Blog

List of blogs

Search This Blog

About

Translate

Instant Stories

Read in Your Language

Test

share this

ShareThis

Popular Posts