DO this AI Thing Now | Less AI Talk. More AI Action.

DO this AI Thing Now #015 | Structured Outputs — Stop Broken JSON in AI Applications

DO this AI Thing Now | #015 | INNOVATION

FOR AI ENGINEERS & DEVELOPERS

Stop Broken JSON! Build Reliable AI Applications with Structured Outputs

Your LLM generates JSON. But can your application depend on its structure? Learn how to enforce a schema, validate responses, and handle failures.

Watch Episode #015 on YouTube

1. The Real Engineering Problem

Large language models often produce useful information in unpredictable formats. A response may contain extra commentary, missing fields, inconsistent data types, or JSON that cannot be parsed.

These issues become important when LLM outputs feed APIs, databases, dashboards, automation workflows, or AI agents.

Unreliable output:

{
  task: "Write report",
  priority: high,
  valid: true,
}

This is not valid JSON. Property names must be quoted, the value high is not a valid unquoted JSON value, and the final property has a trailing comma.

Valid structured result:

{
  "task": "Write report",
  "priority": 1,
  "valid": true
}

The second example is valid JSON and matches the field types in our example schema. However, formatting correctness does not prove that the generated information is factually accurate.

2. What Are Structured Outputs?

Structured Outputs allow developers to specify an expected response structure using a supported JSON Schema and request that a compatible model produce data conforming to that structure.

This is different from merely including the instruction "Please respond in JSON" in a prompt.

• Prompt-only JSON: Asks the model to follow a format.
• JSON mode: Helps ensure syntactically valid JSON, but does not necessarily enforce your complete schema.
• Structured Outputs: Uses supported schema constraints to enforce the expected response structure.
• Application validation: Independently checks the result and its business rules.

3. Practical Example: AI Task Extraction

Imagine an application that reads a short message and extracts a task, priority, and validity flag.

Input:

"Please prepare the quarterly report. It is urgent."

Desired output:

{
  "task": "Prepare the quarterly report",
  "priority": 1,
  "valid": true
}

For this demonstration, priority 1 means high urgency, priority 2 means normal urgency, and priority 3 means low urgency. The valid field indicates whether the input contains an actionable task. These are application-specific conventions.

4. Define the JSON Schema

Here is a schema defining the exact fields and data types.

{
  "type": "object",
  "properties": {
    "task": {
      "type": "string"
    },
    "priority": {
      "type": "integer",
      "enum": [1, 2, 3]
    },
    "valid": {
      "type": "boolean"
    }
  },
  "required": [
    "task",
    "priority",
    "valid"
  ],
  "additionalProperties": false
}

This schema specifies three required fields, constrains priority to the values 1, 2, or 3, and disallows unexpected properties.

5. Hands-On Python Demonstration

The following example uses the OpenAI Python SDK and a compatible structured-output model. It also performs independent application-side validation.

Install dependencies:

pip install -U openai jsonschema

Configure your OPENAI_API_KEY as an environment variable. Never place API keys directly inside published source code.

Python example:

import os
import json

from openai import OpenAI
from jsonschema import validate

client = OpenAI(
    api_key=os.environ["OPENAI_API_KEY"]
)

schema = {
    "type": "object",
    "properties": {
        "task": {"type": "string"},
        "priority": {
            "type": "integer",
            "enum": [1, 2, 3]
        },
        "valid": {"type": "boolean"}
    },
    "required": [
        "task", "priority", "valid"
    ],
    "additionalProperties": False
}

response = client.responses.create(
    model="gpt-4.1-mini",
    instructions=(
        "Extract the actionable task. "
        "Use priority 1 for urgent, "
        "2 for normal, 3 for low. "
        "Set valid to true if an "
        "actionable task exists. "
        "Otherwise use an empty task "
        "and set valid to false."
    ),
    input=(
        "Please prepare the quarterly "
        "report. It is urgent."
    ),
    text={
        "format": {
            "type": "json_schema",
            "name": "task_extraction",
            "strict": True,
            "schema": schema
        }
    }
)

if response.status != "completed":
    raise RuntimeError(
        f"Response status: {response.status}"
    )

if not response.output_text:
    raise RuntimeError(
        "No structured text returned"
    )

data = json.loads(response.output_text)

validate(instance=data, schema=schema)

if data["valid"] and not data["task"].strip():
    raise ValueError(
        "Valid tasks must not be empty"
    )

print(json.dumps(data, indent=2))

This example uses the Responses API structured-output configuration. It assumes the request completes successfully and returns text rather than a refusal. Production systems must also handle API errors, refusals, incomplete responses, and retry policies. Confirm current model support and API syntax in the official documentation.

Illustrative result:

{
  "task": "Prepare the quarterly report",
  "priority": 1,
  "valid": true
}

The output above illustrates the expected result; it is not a claim that the example was executed during publication.

6. Test the Output

Reliable AI engineering requires more than one successful example. Test both the expected structure and your application rules.

from jsonschema import (
    validate,
    ValidationError
)

def is_schema_valid(value):
    try:
        validate(
            instance=value,
            schema=schema
        )
        return True
    except ValidationError:
        return False

assert is_schema_valid({
    "task": "Write report",
    "priority": 1,
    "valid": True
})

assert not is_schema_valid({
    "task": "Write report",
    "priority": "high",
    "valid": True
})

assert not is_schema_valid({
    "task": "Write report",
    "priority": 1
})

assert not is_schema_valid({
    "task": "Write report",
    "priority": 1,
    "valid": True,
    "extra": "unexpected"
})

print("Schema tests passed")

These tests check a valid response, an incorrect type, a missing required field, and an unexpected extra property.

7. Where Structured Outputs Are Useful

Document Processing

Extract invoice fields, dates, categories, and entities into defined structures.

AI Agents

Produce structured plans, classifications, and intermediate results for workflows.

Business Applications

Generate task records, support classifications, and CRM-ready information.

Evaluation Systems

Collect model assessments and test outcomes using consistent field definitions.

8. Engineering Limitations

Structured Outputs are useful, but they are not a complete reliability or security solution.

• Semantic errors: A schema-valid priority may still be wrong.
• Refusals: Safety-related refusals may not follow the requested schema.
• Incomplete generation: Token limits or interruptions may prevent a complete result.
• Schema support: Not every JSON Schema feature is supported by every model or API.
• Security: Structured outputs do not eliminate prompt injection or unsafe downstream actions.
• Business rules: Additional validation may be necessary for dates, identifiers, permissions, and domain-specific constraints.
• Cost and latency: Model selection and schema complexity may affect performance.

9. Practical Engineering Checklist

☐ Define the exact data contract.

☐ Choose a compatible model and API.

☐ Enable schema-enforced structured outputs.

☐ Validate the returned JSON independently.

☐ Check semantic and business-rule correctness.

☐ Handle refusals, incomplete outputs, and API errors.

☐ Test valid, invalid, and adversarial inputs.

☐ Monitor failures and maintain versioned schemas.

10. Further Learning

Official OpenAI Structured Outputs Guide:
https://platform.openai.com/docs/guides/structured-outputs

OpenAI API Reference:
Responses API Reference

JSON Schema Documentation:
https://json-schema.org/

11. Your AI Action Now

10-Minute Challenge

1. Choose one LLM feature in an application or prototype.
2. Define three output fields and their expected types.
3. Enforce the structure with a compatible API.
4. Test one valid and three invalid responses.
5. Document what schema validation does not catch.

One schema. One test. One more reliable AI integration.

DO this AI Thing Now

Less AI talk. More AI action.

Explore practical AI actions, tools, and innovations for students, teachers, professionals, developers, engineers, researchers, and organizations.

Explore the YouTube Playlist

AIYING — Accomplishing with AI
https://dastikop.blogspot.com