AI ENGINEERING — ACTIVE CONCEPTS & PRACTICES | #010
Agent Orchestration: Designing Reliable Multi-Step AI Workflows
Chapter 2: Engineering AI Applications and Agents
AIYING — Accomplishing with AI
The Big Idea: Agent orchestration coordinates the sequence, dependencies, tools, decisions, state, and controls required to complete multi-step AI tasks.
An individual model response may be useful. An orchestrated workflow must deliver a verifiable outcome.
Engineering question: How do we make an AI workflow reliable when it involves multiple steps, external systems, failures, and human approvals?
Contents
- Why Agent Orchestration Matters
- UNDERSTAND — The Concept
- CONNECT — AI Engineering Relationships
- Agent Orchestration Architecture
- APPLY — Enterprise Procurement Workflow
- Worked Example: A Multi-Step Workflow
- Engineering Decisions and Trade-offs
- Active Industry Practices
- ACCOMPLISH — Practical Exercise
- Student and Practitioner Project
- Check Your Understanding
- Active Connections
- References and Further Reading
1. Why Agent Orchestration Matters
Consider an enterprise procurement manager asking:
This is not a single question. It is a sequence of dependent tasks.
The AI application may need to retrieve purchase-order information, inspect inventory, compare delivery alternatives, apply procurement policies, prepare a recommendation, and obtain approval before making changes.
Each step depends on information or decisions from previous steps.
What happens if the inventory service is unavailable? What if the purchase-order record is inconsistent? What if a human reviewer rejects the proposed action?
These are orchestration problems.
2. UNDERSTAND — The Concept
What is agent orchestration?
Agent orchestration is the coordination of AI models, tools, workflow steps, decisions, application state, and control mechanisms to accomplish a task.
The orchestrator may be implemented through conventional software logic, a workflow engine, an agent framework, or a combination of these.
Five essential responsibilities
| Responsibility | Engineering Question |
|---|---|
| Planning | Which steps are required? |
| Coordination | What should run next, and what depends on what? |
| State management | What has already happened? |
| Control | Which operations are permitted? |
| Recovery | What happens when a step fails? |
Workflow versus agent
A workflow generally follows a predefined control structure. An agent may dynamically select actions and tools based on intermediate observations.
In practice, production systems often combine both approaches.
For example, an agent may analyze a supplier delay while a deterministic workflow controls approvals, record updates, and completion status.
Common orchestration patterns
- Sequential: Execute dependent steps in order.
- Parallel: Run independent operations concurrently.
- Conditional routing: Choose a branch based on validated results.
- Iterative: Repeat a bounded process until a condition is satisfied.
- Supervisor-worker: Delegate specialized tasks to different components.
- Human-in-the-loop: Pause for an authorized decision.
- Event-driven: Resume when an external event occurs.
Not every problem needs multiple agents. A deterministic workflow with one model and a few tools may be simpler, cheaper, and easier to test.
3. CONNECT — AI Engineering Relationships
| Concept | Relationship to Orchestration |
|---|---|
| Context Engineering | Supplies relevant information to each step |
| Token Economics | Controls the cost of repeated model calls |
| Retrieval Engineering | Provides external evidence |
| Context Management | Preserves state and continuity |
| Structured Outputs | Defines data contracts between steps |
| Tool Calling | Invokes external capabilities |
| MCP | Provides standardized access to external capabilities |
| Evaluation | Tests both individual steps and overall outcomes |
| Observability | Traces execution, latency, cost, and failures |
| Governance | Establishes accountability and approval controls |
Active connection: Orchestration brings together the capabilities studied in earlier lessons into an operational AI application.
4. Agent Orchestration Architecture
↓
Workflow Controller / Orchestrator↓
Task Planning and State Initialization↓
Model and Tool Execution↓
Result Validation↓
Decision: Continue, Retry, Escalate, or Stop?↓
Human Approval When Required↓
Authorized Business Action↓
Outcome Verification and Audit RecordState transitions
A workflow should have explicit states, such as:
CREATED | v RUNNING | +--> WAITING_FOR_APPROVAL | | | v | RUNNING | +--> FAILED | +--> COMPLETED
Real implementations may need additional states such as cancelled, timed out, rejected, or requiring manual intervention.
Why state matters
State enables a workflow to resume after interruptions, avoid repeating completed actions, and distinguish a pending operation from a successful one.
State should be stored in a durable application system when the workflow must survive process restarts or extended human approval delays.
5. APPLY — Enterprise Procurement Workflow
Scenario: Recovering from a Supplier Delay
A manufacturing company uses an AI assistant to investigate delayed material deliveries.
The assistant has access to fictional procurement, inventory, and supplier systems.
Step 1 — Retrieve purchase order
The system obtains the current order status through an authorized lookup tool.
Step 2 — Check inventory
The application retrieves stock availability from alternative locations.
Step 3 — Evaluate alternatives
The AI analyzes available evidence and prepares possible recovery options.
Step 4 — Apply business constraints
Deterministic checks verify inventory eligibility, policy requirements, and permitted actions.
Step 5 — Request approval
If a transfer or supplier change requires approval, the workflow pauses for the designated decision-maker.
Step 6 — Execute authorized action
The application invokes the approved operation and verifies its result.
Step 7 — Complete and record
The workflow reports the outcome and records the relevant execution history.
Important: The assistant should never describe an operation as completed merely because the model proposed it.
6. Worked Example: A Multi-Step Workflow
The following is a fictional design illustration, not a live enterprise integration.
Sample workflow state
{
"workflow_id": "WF-1048",
"po_id": "PO-1048",
"status": "waiting_for_approval",
"order_status": "delayed",
"alternative_stock_available": true,
"recommended_action": "internal_transfer",
"approval_status": "pending",
"execution_status": "not_started"
}
Illustrative orchestration pseudocode
async def process_order(po_id, user):
order = await get_order(po_id, user)
validate_order(order)
if order["status"] != "delayed":
return {"status": "completed",
"action": "none"}
inventory = await check_inventory(order, user)
validate_inventory(inventory)
proposal = await recommend_action(
order, inventory
)
validate_proposal(proposal)
approval_id = await request_approval(
proposal, user
)
# Persist state and stop execution.
# Resume only after a verified approval event.
return {
"status": "waiting_for_approval",
"approval_id": approval_id,
"execution_status": "not_started"
}
This pseudocode illustrates a workflow boundary. A production implementation must authenticate the approver, persist state, resume through a controlled event, enforce authorization, and verify any eventual write operation.
What makes the workflow reliable?
- Each step has defined inputs and outputs.
- External results are validated.
- State is recorded before suspension.
- Approvals are explicit.
- Retries are bounded and controlled.
- Side-effecting actions use duplicate-prevention safeguards.
- Completion is based on verified execution results.
7. Engineering Decisions and Trade-offs
| Decision | Benefit | Trade-off |
|---|---|---|
| Fixed workflow | Predictability and testability | Less flexibility |
| Dynamic agent planning | Adapts to varied tasks | Harder to constrain and evaluate |
| Parallel execution | Reduced latency | Dependency and concurrency risks |
| Persistent state | Resumability | Storage and consistency complexity |
| Automatic retries | Failure recovery | Cost and duplicate-action risks |
| Human approval | Stronger oversight | Additional time |
| Multiple specialized agents | Division of responsibilities | Coordination overhead |
Common failure modes
- Infinite or excessive execution loops
- Incorrect tool selection
- Loss of workflow state
- Contradictory intermediate results
- Repeated actions after retries
- Unauthorized execution
- Unbounded token consumption
- Incorrect completion reporting
- Prompt injection through retrieved data or tool results
Recommended engineering controls
- Maximum step and execution budgets
- Timeouts and bounded retries
- Checkpointing and durable state
- Idempotency for operations with side effects
- Structured contracts between components
- Explicit approval boundaries
- Least-privilege tool permissions
- Execution tracing and audit records
- Outcome-based evaluations
8. Active Industry Practices
Established engineering approaches
- State-machine and graph-based workflows
- Task dependency management
- Tool execution with validation
- Human approval checkpoints
- Persistent workflow state
- Execution tracing and monitoring
Evolving AI engineering practices
- Model-directed workflow routing
- Supervisor and specialist-agent architectures
- Agent evaluation using execution traces
- Policy-aware tool orchestration
- Adaptive planning with bounded autonomy
- Durable execution for long-running agents
Relevant technologies
Developers may evaluate frameworks such as LangGraph, Microsoft Agent Framework, OpenAI Agents SDK, and conventional workflow engines.
These technologies have different architectures and operational characteristics. Framework selection should follow the requirements of the application rather than determine the architecture in advance.
Industry applications
Agent orchestration is relevant to customer support, software engineering, incident response, research assistants, enterprise operations, procurement, document processing, and multi-step business automation.
Practice status: Established workflow-engineering principles combined with evolving agentic AI architectures.
Reviewed: November 2026.
9. ACCOMPLISH — Practical Exercise
Exercise: Design a Procurement Recovery Workflow
Use this fictional scenario:
Purchase order: PO-2056
Status: Delayed
Required quantity: 500 units
Alternative location stock: 350 units
Proposed action: Partial internal transfer
Approval: Required
Your tasks
- Define the workflow objective.
- List the required tools and data sources.
- Identify sequential and parallel steps.
- Define workflow states and transitions.
- Specify validation checkpoints.
- Identify actions requiring approval.
- Design failure and retry handling.
- Define completion and escalation conditions.
- Explain how the remaining 150-unit shortage will be handled.
Reusable AI-assisted prompt
Act as an enterprise AI workflow architect. Design an agent orchestration workflow for a delayed purchase order. Scenario: - Order: PO-2056 - Required quantity: 500 units - Alternative stock: 350 units - Supplier delivery: delayed - Internal transfer: approval required Provide: 1. Workflow objective. 2. Required tools and data. 3. Step-by-step execution plan. 4. State transitions. 5. Validation rules. 6. Approval checkpoints. 7. Failure recovery strategy. 8. Cost and execution limits. 9. Observability requirements. 10. Completion and escalation criteria. Distinguish AI-generated recommendations from verified system actions. Do not claim any real action was executed.
Accomplishment: Produce an orchestration design that can be reviewed for reliability, authorization, recovery, and measurable completion.
10. Student and Practitioner Project
Mini Project: Build a Stateful Procurement Agent
Objective: Prototype a multi-step AI workflow using simulated enterprise systems.
Project requirements
- Create fictional purchase-order and inventory datasets.
- Define lookup and simulated action tools.
- Implement a workflow controller.
- Persist workflow state between steps.
- Use structured inputs and outputs.
- Introduce an approval checkpoint.
- Simulate a tool timeout and a service failure.
- Test workflow resumption after interruption.
- Prevent duplicate side effects.
- Record execution traces and evaluate outcomes.
Suggested evaluation metrics
| Metric | What It Measures |
|---|---|
| Task completion rate | Successful completion of authorized workflows |
| Step accuracy | Correct execution of required stages |
| Recovery success | Resumption after failures |
| State consistency | Correctness of persisted workflow information |
| Approval compliance | Whether required approvals are enforced |
| Duplicate action rate | Repeated side effects after retries |
| Execution cost | Model and tool resource consumption |
| End-to-end latency | Total workflow processing time |
Project deliverable: A workflow diagram, working prototype or design, sample execution traces, evaluation results, and an engineering recommendation.
11. Check Your Understanding
- What is agent orchestration?
- How is orchestration different from tool calling?
- What is the difference between a fixed workflow and a dynamic agent?
- Why is persistent state important?
- When should workflow steps run in parallel?
- How should approval checkpoints be implemented?
- Why are retries dangerous for write operations?
- How would you measure end-to-end workflow reliability?
Reflection: If every individual AI step succeeds but the overall workflow performs an unauthorized action, has the system succeeded? What does this reveal about orchestration quality?
12. Active Connections
- #001 — Context Engineering
- #002 — Token Economics
- #003 — Embeddings in Practice
- #004 — Retrieval Engineering
- #005 — Chunking and Reranking
- #006 — Context Management
- #007 — Structured Outputs
- #008 — Tool Calling
- #009 — Model Context Protocol (MCP)
Coming next — #011: Agent Evaluation: Measuring Whether AI Workflows Actually Succeed.
13. References and Further Reading
- LangGraph — Overview
- OpenAI Agents SDK — Documentation
- Microsoft Agent Framework — Documentation
- Temporal — Durable Execution Documentation
- Model Context Protocol — Official Documentation
Framework capabilities and implementation guidance evolve. Verify the current documentation when building production systems.
Final Takeaway
AI Engineering — Active Concepts & Practices
Active Concept #010 — Agent Orchestration
AIYING — Accomplishing with AI
Understand → Connect → Apply → Accomplish
