DO this AI Thing Now | #015 | INNOVATION
FOR AI ENGINEERS & DEVELOPERS
Stop Broken JSON! Build Reliable AI Applications with Structured Outputs
Your LLM generates JSON. But can your application depend on its structure? Learn how to enforce a schema, validate responses, and handle failures.
1. The Real Engineering Problem
Large language models often produce useful information in unpredictable formats. A response may contain extra commentary, missing fields, inconsistent data types, or JSON that cannot be parsed.
These issues become important when LLM outputs feed APIs, databases, dashboards, automation workflows, or AI agents.
{
task: "Write report",
priority: high,
valid: true,
}
This is not valid JSON. Property names must be quoted, the value
high is not a valid unquoted JSON value, and the final
property has a trailing comma.
{
"task": "Write report",
"priority": 1,
"valid": true
}The second example is valid JSON and matches the field types in our example schema. However, formatting correctness does not prove that the generated information is factually accurate.
2. What Are Structured Outputs?
Structured Outputs allow developers to specify an expected response structure using a supported JSON Schema and request that a compatible model produce data conforming to that structure.
This is different from merely including the instruction "Please respond in JSON" in a prompt.
- Prompt-only JSON: Asks the model to follow a format.
- JSON mode: Helps ensure syntactically valid JSON, but does not necessarily enforce your complete schema.
- Structured Outputs: Uses supported schema constraints to enforce the expected response structure.
- Application validation: Independently checks the result and its business rules.
3. Practical Example: AI Task Extraction
Imagine an application that reads a short message and extracts a task, priority, and validity flag.
Input:
Desired output:
{
"task": "Prepare the quarterly report",
"priority": 1,
"valid": true
}
For this demonstration, priority 1 means high urgency,
priority 2 means normal urgency, and priority 3 means low urgency.
The valid field indicates whether the input contains
an actionable task. These are application-specific conventions.
4. Define the JSON Schema
Here is a schema defining the exact fields and data types.
{
"type": "object",
"properties": {
"task": {
"type": "string"
},
"priority": {
"type": "integer",
"enum": [1, 2, 3]
},
"valid": {
"type": "boolean"
}
},
"required": [
"task",
"priority",
"valid"
],
"additionalProperties": false
}This schema specifies three required fields, constrains priority to the values 1, 2, or 3, and disallows unexpected properties.
5. Hands-On Python Demonstration
The following example uses the OpenAI Python SDK and a compatible structured-output model. It also performs independent application-side validation.
Install dependencies:
pip install -U openai jsonschema
Configure your OPENAI_API_KEY as an environment variable.
Never place API keys directly inside published source code.
Python example:
import os
import json
from openai import OpenAI
from jsonschema import validate
client = OpenAI(
api_key=os.environ["OPENAI_API_KEY"]
)
schema = {
"type": "object",
"properties": {
"task": {"type": "string"},
"priority": {
"type": "integer",
"enum": [1, 2, 3]
},
"valid": {"type": "boolean"}
},
"required": [
"task", "priority", "valid"
],
"additionalProperties": False
}
response = client.responses.create(
model="gpt-4.1-mini",
instructions=(
"Extract the actionable task. "
"Use priority 1 for urgent, "
"2 for normal, 3 for low. "
"Set valid to true if an "
"actionable task exists. "
"Otherwise use an empty task "
"and set valid to false."
),
input=(
"Please prepare the quarterly "
"report. It is urgent."
),
text={
"format": {
"type": "json_schema",
"name": "task_extraction",
"strict": True,
"schema": schema
}
}
)
if response.status != "completed":
raise RuntimeError(
f"Response status: {response.status}"
)
if not response.output_text:
raise RuntimeError(
"No structured text returned"
)
data = json.loads(response.output_text)
validate(instance=data, schema=schema)
if data["valid"] and not data["task"].strip():
raise ValueError(
"Valid tasks must not be empty"
)
print(json.dumps(data, indent=2))This example uses the Responses API structured-output configuration. It assumes the request completes successfully and returns text rather than a refusal. Production systems must also handle API errors, refusals, incomplete responses, and retry policies. Confirm current model support and API syntax in the official documentation.
Illustrative result:
{
"task": "Prepare the quarterly report",
"priority": 1,
"valid": true
}The output above illustrates the expected result; it is not a claim that the example was executed during publication.
6. Test the Output
Reliable AI engineering requires more than one successful example. Test both the expected structure and your application rules.
from jsonschema import (
validate,
ValidationError
)
def is_schema_valid(value):
try:
validate(
instance=value,
schema=schema
)
return True
except ValidationError:
return False
assert is_schema_valid({
"task": "Write report",
"priority": 1,
"valid": True
})
assert not is_schema_valid({
"task": "Write report",
"priority": "high",
"valid": True
})
assert not is_schema_valid({
"task": "Write report",
"priority": 1
})
assert not is_schema_valid({
"task": "Write report",
"priority": 1,
"valid": True,
"extra": "unexpected"
})
print("Schema tests passed")These tests check a valid response, an incorrect type, a missing required field, and an unexpected extra property.
7. Where Structured Outputs Are Useful
Extract invoice fields, dates, categories, and entities into defined structures.
Produce structured plans, classifications, and intermediate results for workflows.
Generate task records, support classifications, and CRM-ready information.
Collect model assessments and test outcomes using consistent field definitions.
8. Engineering Limitations
Structured Outputs are useful, but they are not a complete reliability or security solution.
- Semantic errors: A schema-valid priority may still be wrong.
- Refusals: Safety-related refusals may not follow the requested schema.
- Incomplete generation: Token limits or interruptions may prevent a complete result.
- Schema support: Not every JSON Schema feature is supported by every model or API.
- Security: Structured outputs do not eliminate prompt injection or unsafe downstream actions.
- Business rules: Additional validation may be necessary for dates, identifiers, permissions, and domain-specific constraints.
- Cost and latency: Model selection and schema complexity may affect performance.
9. Practical Engineering Checklist
☐ Define the exact data contract.
☐ Choose a compatible model and API.
☐ Enable schema-enforced structured outputs.
☐ Validate the returned JSON independently.
☐ Check semantic and business-rule correctness.
☐ Handle refusals, incomplete outputs, and API errors.
☐ Test valid, invalid, and adversarial inputs.
☐ Monitor failures and maintain versioned schemas.
10. Further Learning
Official OpenAI Structured Outputs Guide:
https://platform.openai.com/docs/guides/structured-outputs
OpenAI API Reference:
Responses API Reference
JSON Schema Documentation:
https://json-schema.org/
11. Your AI Action Now
- Choose one LLM feature in an application or prototype.
- Define three output fields and their expected types.
- Enforce the structure with a compatible API.
- Test one valid and three invalid responses.
- Document what schema validation does not catch.
One schema. One test. One more reliable AI integration.
DO this AI Thing Now
Less AI talk. More AI action.
Explore practical AI actions, tools, and innovations for students, teachers, professionals, developers, engineers, researchers, and organizations.
AIYING — Accomplishing with AI
https://dastikop.blogspot.com
