3. AI for CS Students – Session 3 Companion Notes Where AI Fails: Limits, Risks, and the Discipline of Validation

AI for CS Students – Session 3 Companion Notes

Where AI Fails: Limits, Risks, and the Discipline of Validation

Aligned to VI Semester CS Curriculum (Customize college name before publishing)

---

1️⃣ Context: Why This Session Matters

AI tools are powerful.

They:

• Generate code

• Explain concepts

• Suggest optimizations

• Draft structured solutions

Because they sound confident, students often assume they are correct.

In engineering, confidence without correctness creates failure.

Understanding AI limitations is what separates:

• A tool user

• From a responsible engineer

This session focuses on technical limits — not hype.

---

2️⃣ Curriculum Mapping: Where Theory Protects You

AI failure becomes visible only when you apply subject knowledge.

Automata & Formal Languages

Helps test:

• Regular vs non-regular language classification

• Grammar validity

• State transition correctness

AI may generate plausible diagrams.
You verify formal correctness.

---

Complexity Analysis

Helps detect:

• Incorrect Big-O claims

• Misinterpreted recursion depth

• Hidden nested loops

AI often estimates complexity incorrectly.

You must derive it manually.

---

Information Security

Helps identify:

• Weak hashing

• Hardcoded secrets

• Insecure authentication flow

• API vulnerabilities

AI-generated code can be syntactically correct but insecure.

Security knowledge is your filter.

---

Data Mining / Blockchain

Helps analyze:

• Model bias

• Smart contract vulnerability

• Transaction logic flaws

• False positive rates

AI suggestions must be evaluated against domain constraints.

---

3️⃣ Where AI Helps

AI performs well at:

• Pattern recognition

• Draft generation

• Code scaffolding

• Summarization

• Syntax correction

It accelerates repetition.

It does not guarantee correctness.

---

4️⃣ Where AI Fails

AI struggles with:

• Formal proof

• Edge-case reasoning

• Worst-case complexity

• Security architecture

• Context-sensitive constraints

AI outputs are probabilistic — not logically guaranteed.

This distinction is critical.

---

5️⃣ Micro Case Study

A student asked AI:

“Classify this language and prove whether it is regular.”

AI responded:

• “This language is regular.”

• Provided a vague reasoning explanation.

• No formal pumping lemma proof.

The student accepted it.

In viva, examiner asked:

“Provide formal justification.”

Student could not derive proof.

AI gave plausible explanation.
But it did not provide rigorous proof.

Automata knowledge would have prevented this failure.

Lesson:

AI explanations must be validated formally.

---

6️⃣ Validation Discipline Framework

For any AI-generated output, apply this sequence:

Step 1 – Generate

Use AI to draft solution.

Step 2 – Manual Derivation

Re-derive complexity, grammar, or logic yourself.

Step 3 – Edge Case Testing

Test boundary conditions.

Step 4 – Security Review

Check for vulnerabilities.

Step 5 – Constraint Evaluation

Confirm assumptions hold under worst-case scenarios.

Without these steps, AI usage becomes risky.

---

7️⃣ Placement Articulation

Weak articulation:

“AI gave me the answer.”

Strong articulation:

“I used AI for initial reasoning but verified the solution using formal methods and complexity analysis.”

This signals:

• Ownership

• Technical depth

• Validation discipline

Interviewers value accountability.

---

8️⃣ AI Failure Validation Checklist

Before accepting AI output, ask:

□ Did I manually verify correctness?
□ Did I derive time and space complexity myself?
□ Did I check boundary conditions?
□ Did I review security implications?
□ Did I confirm no hidden assumptions exist?
□ Can I prove this in front of an examiner?

If any answer is “No,” do not rely on it.

---

9️⃣ Quick Revision Summary

• AI outputs are probabilistic, not guaranteed.

• Hallucination is common.

• Formal proof must be done manually.

• Security review is mandatory.

• Complexity claims require derivation.

• Accountability remains human.

---

🔟 2-Minute Interview Recap

• I use AI for acceleration, not final validation.

• I verify algorithm complexity manually.

• I test edge cases before acceptance.

• I review security implications carefully.

• I can defend every solution formally.

This framing reflects engineering maturity.

---

Final Reflection

The strongest engineers are not those who avoid AI.

They are those who:

• Use AI efficiently

• Detect its mistakes

• Apply theory rigorously

• Accept accountability

Understanding AI’s limits is your advantage.