Orchestrator Agent is a multi-tenant, policy-aware planning & execution engine.

It owns:

- Turning intents into executable task graphs

- Routing tasks to agents/tools/LLMs

- Enforcing auth, policy, telemetry, resiliency

- Containerized microservice on AKS

- Stateless HTTP/gRPC front-end behind ILB + API Mgmt

- Background workers on AKS for long tasks

- State externalized in Cosmos DB/SQL + Redis

- Event-driven via Service Bus

Ingress & Normalization: UI/API sends request with user claims, intent, goals, context.

Intent Classification & Routing: Router model selects domain & capabilities.

Plan Generation: Planner LLM creates JSON task graph, validated by schema & policy.

Policy & Guardrail Check: Policy Engine vets plan for compliance.

Execution: Tasks emitted to Service Bus; workers execute & stream results.

Aggregation & Response: Outputs synthesized by GPT-4o; final response streamed back.

- Agents are containerized services on AKS

- Standard POST /execute contract with run, step, tenant IDs

- Agents do domain-specific work (Databricks jobs, MMM, RAG, external calls)

- Publish structured outputs (KPIs, tables, insights)

- Orchestrator is domain-agnostic, focusing on DAGs, policies, SLAs

- GPT-4o for planning & decomposition with JSON output

- Lightweight open-weight model for fallback routing

- Optional self-critique with second GPT-4o pass

- Strict separation: LLM proposes plans; deterministic code executes

"LLM proposes route plans; deterministic services validate & execute."

- Multi-tenant isolation via per-tenant configs & Entra claims

- Observability with OpenTelemetry, Azure Monitor, Logs & Metrics

- Auditability: user identity, full plan, tool calls, LLM prompts

- Resiliency: timeouts, retries, circuit breakers, degraded mode fallback

"The Orchestrator Agent is a DAG-based, policy-aware, LLM-assisted workflow engine on AKS, using GPT-4o and open-weight models for planning and synthesis, delegating side-effects to governed agents and tools for enterprise-grade flexibility."