Agent Orchestrator
Architecture Design Doc

1System overview

AO runs multiple coding agents (Claude Code, Codex, Aider, OpenCode, Cursor, Grok, Kimi) simultaneously. One call — ao spawn <project> <issue> — creates an isolated workspace, launches an agent inside a managed runtime, and wires up the feedback loops so that PR reviews and CI failures route back to the right agent automatically.

The defining design choice is that every external dependency is a swappable plugin. Where the agent runs, which AI tool it is, how code is isolated, where issues live, how PRs and CI are read, how you get notified, and how a human attaches to a terminal — all eight are pluggable interfaces resolved at startup. The orchestration engine in packages/core knows nothing concrete about tmux, GitHub, or Claude; it only knows the interfaces.

flowchart TB
  subgraph CORE["packages/core — Orchestration Engine"]
    direction TB
    SM["Session Manager
CRUD · spawn · restore · reconcile"]
    LM["Lifecycle Manager
30s poll · state machine · reactions"]
    REG["Plugin Registry
discover · resolve · instantiate"]
    SM --- LM
    REG --- SM
    REG --- LM
  end

  RT["RUNTIME
tmux · process"]:::runtime
  AG["AGENT
claude-code · codex · aider
opencode · cursor · grok · kimi"]:::agent
  WS["WORKSPACE
worktree · clone"]:::workspace
  TR["TRACKER
github · linear · gitlab"]:::tracker
  SCM["SCM
github · gitlab"]:::scm
  NO["NOTIFIER
desktop · slack · discord
webhook · composio · …"]:::notifier
  TE["TERMINAL
iterm2 · web"]:::terminal

  RT  --- REG
  AG  --- REG
  WS  --- REG
  TR  --- REG
  SCM --- REG
  NO  --- LM
  TE  --- SM

  CFG["agent-orchestrator.yaml"]:::cfg --> REG
  LM --> WEB["Next.js Dashboard
mux WebSocket → React + xterm.js"]:::web

  classDef runtime fill:#2a1c14,stroke:#f2845c,color:#ffd9c7;
  classDef agent fill:#16223a,stroke:#6ea8ff,color:#cfe2ff;
  classDef workspace fill:#11281f,stroke:#4fd1a1,color:#c4f0df;
  classDef tracker fill:#2a2410,stroke:#f5c451,color:#f6e6b0;
  classDef scm fill:#1f1830,stroke:#9d7bff,color:#ddd0ff;
  classDef notifier fill:#2a1320,stroke:#f06595,color:#ffc9de;
  classDef terminal fill:#102528,stroke:#38c7d4,color:#bdf0f5;
  classDef cfg fill:#1b2330,stroke:#6b7689,color:#e6ebf2;
  classDef web fill:#151b27,stroke:#6ea8ff,color:#e6ebf2;
    

flowchart LR
  A["agent-orchestrator.yaml"] --> B["Config Loader
Zod validate"]
  B --> C["Plugin Registry"]
  C --> D["Session Manager"]
  D --> E["Lifecycle Manager
poll loop"]
  E --> F["Events"]
  F --> G["Notifiers"]
  D -. flat files .-> H[("~/.agent-orchestrator")]
  E --> I["Next.js API routes"]
  I --> J["mux WebSocket"]
  J --> K["React Dashboard
+ xterm.js"]
  classDef d fill:#151b27,stroke:#2f3a4f,color:#e6ebf2;
  class A,B,C,D,E,F,G,H,I,J,K d;
    

2Monorepo & build order

AO is a pnpm workspace (pnpm 9.15.4, package.json) with roughly 30 packages. Cross-package dependencies use the workspace:* protocol so the whole graph builds from source.

# root build — pnpm -r build, dependency-ordered
packages/
  core/            # engine: types, config, registry, session + lifecycle managers
  cli/             # the `ao` command — depends on every plugin
  web/             # Next.js 15 dashboard (App Router, React 19, xterm.js)
  ao/              # global CLI wrapper (thin shim around cli)
  notifier-macos/  # native macOS notifier helper
  integration-tests/
  plugins/         # 25 plugin packages, one per implementation
    agent-claude-code  agent-codex  agent-aider  agent-opencode
    agent-cursor  agent-grok  agent-kimicode
    runtime-tmux  runtime-process
    workspace-worktree  workspace-clone
    tracker-github  tracker-linear  tracker-gitlab
    scm-github  scm-gitlab
    notifier-desktop  notifier-slack  notifier-discord  notifier-webhook
    notifier-composio  notifier-dashboard  notifier-openclaw
    terminal-iterm2  terminal-web

flowchart TD
  CORE["@aoagents/ao-core
types · config · registry
session + lifecycle managers"]:::core
  PLUGINS["25 plugin packages
@aoagents/ao-plugin-*"]:::plug
  CLI["@aoagents/ao-cli
the `ao` command"]:::leaf
  WEB["@aoagents/ao-web
Next.js dashboard"]:::leaf
  AO["ao
global shim"]:::leaf

  CORE --> PLUGINS
  CORE --> CLI
  CORE --> WEB
  PLUGINS --> CLI
  CLI --> AO

  classDef core fill:#16223a,stroke:#6ea8ff,color:#cfe2ff;
  classDef plug fill:#11281f,stroke:#4fd1a1,color:#c4f0df;
  classDef leaf fill:#151b27,stroke:#2f3a4f,color:#e6ebf2;
    

Build order is therefore core → plugins → cli / web (cli and web build in parallel once their deps are ready). Core is the only barrel-exported package (packages/core/src/index.ts) and forms the stable public API every plugin imports as @aoagents/ao-core.

3The 8 plugin slots

All slot interfaces live in packages/core/src/types.ts. Each is async-first (every I/O method returns a Promise), and plugins never call each other directly — they communicate only through the Session object and the lifecycle manager.

The registry & the PluginModule contract

The plugin registry (packages/core/src/plugin-registry.ts) discovers, resolves, and instantiates plugins. Built-ins are a hard-coded table keyed by slot:name (plugin-registry.ts:39); external plugins come from the config's plugins[] via npm package or local path (plugin-registry.ts:533). Every import is normalized to a PluginModule shape (plugin-registry.ts:303), accepting either a direct module or a default export.

Every plugin default-exports a PluginModule<T> (types.ts:1747): a manifest, a create() factory, and an optional detect() for binary availability. Config is validated once in create() and captured in a closure — never re-validated per call.

import type { PluginModule, Runtime } from "@aoagents/ao-core";

export const manifest = {
  name: "tmux",            // must match the {name} package suffix
  slot: "runtime" as const, // `as const` preserves the literal type
  description: "tmux session runtime",
  version: "0.1.0",
};

export function create(config?: Record<string, unknown>): Runtime {
  // validate config here, capture in closure
  return { /* …Runtime methods… */ };
}

export function detect(): boolean { /* is the binary installed? */ }

export default { manifest, create, detect } satisfies PluginModule<Runtime>;

The Session object (types.ts:280) is the shared currency that flows through every slot. Key fields: id, projectId, status (legacy), lifecycle (canonical truth, types.ts:297), activity, branch, pr, workspacePath, runtimeHandle, and a free-form metadata map.

4Session lifecycle

A session's truth is stored as a canonical lifecycle with separate state and reason fields (plus parallel pr and runtime sub-states). The dashboard's familiar status labels are derived from this canonical truth, never stored as the source.

flowchart LR
  spawning --> working --> pr_open
  pr_open --> ci_failed
  pr_open --> review_pending
  ci_failed --> changes_requested
  review_pending --> approved
  changes_requested --> approved
  approved --> mergeable --> merged --> cleanup --> done
  classDef s fill:#16223a,stroke:#6ea8ff,color:#cfe2ff;
  classDef bad fill:#2a1320,stroke:#f06595,color:#ffc9de;
  classDef good fill:#11281f,stroke:#4fd1a1,color:#c4f0df;
  class spawning,working,pr_open,review_pending s;
  class ci_failed,changes_requested bad;
  class approved,mergeable,merged,cleanup,done good;
    

stateDiagram-v2
  [*] --> not_started: spawn_requested
  not_started --> working: agent_acknowledged
  working --> idle: task quiet
  idle --> working: new activity
  working --> needs_input: awaiting_user_input
  idle --> needs_input: awaiting_user_input
  needs_input --> working: input received
  working --> detecting: runtime probe ambiguous
  idle --> detecting: runtime_lost
  detecting --> working: runtime alive
  detecting --> stuck: probe_failure
  detecting --> terminated: runtime_lost / agent_process_exited
  working --> stuck: error_in_process
  stuck --> working: recovered
  idle --> done: research_complete
  working --> done: research_complete
  idle --> terminated: pr_merged / auto_cleanup
  working --> terminated: manually_killed
  stuck --> terminated: probe_failure / error_in_process
  done --> [*]
  terminated --> [*]
    

5Session manager

The session manager (packages/core/src/session-manager.ts, factory createSessionManager at :377) owns session CRUD and all disk persistence. It is the only component that mutates session metadata files.

Stale-runtime reconciliation

During list() enrichment, if a session's tmux/process runtime is found dead, the manager persists the fact to disk and parks the session in detecting with reason runtime_lost (session-manager.ts:2303) — crucially before emitting any event, so a mid-event crash can't lose the update. It does not decide the session is terminated; the lifecycle manager's probe pipeline makes that call on the next poll. This is the concrete enforcement of invariant #1735.

6Storage model — flat files, no database

~/.agent-orchestrator/
├─ config.yaml                              # global config — all registered projects
├─ running.json                             # current `ao start` PID, port, projects
├─ last-stop.json                           # sessions killed by stop/Ctrl+C (for restore)
├─ portfolio/                               # preferences.json, registered.json
└─ projects/{projectId}/
   ├─ orchestrator.json                     # getOrchestratorPath()  paths.ts:139
   ├─ sessions/{sessionId}.json          # getSessionPath()       paths.ts:144
   ├─ worktrees/{sessionId}/             # isolated git workspace paths.ts:124
   ├─ code-reviews/  feedback-reports/

7End-to-end data flow: ao start → dashboard

ao start (packages/cli/src/commands/start.ts) spawns the dashboard as a managed daemon child (start.ts:939), which in turn launches the Next.js server plus a dedicated direct-terminal WebSocket server (packages/web/server/start-all.ts:108), listening on DIRECT_TERMINAL_PORT (default 14801, direct-terminal-ws.ts:103).

The browser opens one multiplexed WebSocket (packages/web/server/mux-websocket.ts) carrying three logical channels:

• sessions — SessionBroadcaster polls /api/sessions/patches every 3 s (mux-websocket.ts:138) and broadcasts lightweight SessionPatch[].
• terminal — TerminalManager attaches a node-pty to tmux attach-session, streaming live output with a 50 KB ring buffer for late subscribers (:466).
• notifications — NotificationBroadcaster tails the JSONL store every 1 s (:319).

sequenceDiagram
  autonumber
  actor U as User
  participant CLI as ao start (CLI)
  participant DASH as Dashboard daemon
  participant NEXT as Next.js :3000
  participant WS as mux WS :14801
  participant API as /api/sessions/patches
  participant SM as Session Manager
  participant LM as Lifecycle Manager
  participant BR as Browser (React + xterm)

  U->>CLI: ao start
  CLI->>DASH: spawnManagedDaemonChild("dashboard")
  DASH->>NEXT: start Next.js (PORT)
  DASH->>WS: start direct-terminal WS (14801)
  CLI->>LM: start poll loop (30s)
  loop every 30s
    LM->>SM: list() + enrich PRs
    SM-->>LM: sessions (canonical lifecycle)
    LM->>LM: determineStatus → reactions
  end
  U->>BR: open http://localhost:3000
  BR->>NEXT: SSR initial page
  BR->>WS: connect mux, subscribe(sessions,notifications)
  loop every 3s
    WS->>API: GET /api/sessions/patches
    API->>SM: listCached(project)
    SM-->>API: SessionPatch[]
    API-->>WS: patches
    WS-->>BR: {ch:sessions, snapshot}
  end
  BR->>WS: {ch:terminal, type:open, tmuxName}
  WS->>WS: node-pty ⇄ tmux attach-session
  WS-->>BR: {ch:terminal, type:data} (stream)
  BR->>BR: xterm renders + kanban updates
    

8Prompt assembly (3 layers)

buildPrompt() (packages/core/src/prompt-builder.ts:191) composes a worker agent's system prompt from three layers, returning { systemPrompt, taskPrompt? }:

flowchart TB
  L1["Layer 1 — Base prompt
BASE_AGENT_PROMPT (repo) or
BASE_AGENT_PROMPT_NO_REPO · prompt-builder.ts:21"]:::l
  L2["Layer 2 — Config context
project · repo · branch · issue
buildConfigLayer() :111"]:::l
  L3["Layer 3 — User rules
agentRules + agentRulesFile
readUserRules() :162"]:::l
  L1 --> OUT["systemPrompt"]:::o
  L2 --> OUT
  L3 --> OUT
  classDef l fill:#16223a,stroke:#6ea8ff,color:#cfe2ff;
  classDef o fill:#11281f,stroke:#4fd1a1,color:#c4f0df;
    

1. Base prompt — system instructions (session lifecycle, ao report commands, git/PR workflow). Two variants: full, or _NO_REPO when the project has no repo configured (prompt-builder.ts:60).
2. Config context — project name/repo/branch/tracker and the task (issue id + pre-fetched issue details), built by buildConfigLayer() (:111). An optional orchestrator back-channel block injects an ao send {sessionId} "…" command (:205).
3. User rules — inline agentRules and/or an external agentRulesFile resolved relative to the project (:162).

The orchestrator session gets a separate prompt rendered from a markdown template with conditional section blocks (generateOrchestratorPrompt(), packages/core/src/orchestrator-prompt.ts:189) — this is what ao start injects via --append-system-prompt.

9Cross-platform abstractions

AO ships first-class on macOS, Linux, and Windows. All OS branching is centralized in packages/core/src/platform.ts (full reference in docs/CROSS_PLATFORM.md).

On Windows the default runtime is process (ConPTY, no tmux); killProcessTree always uses taskkill /F (WM_CLOSE fails for headless processes), while Unix kills the negative-PID process group first. getShell() prefers pwsh → powershell.exe → cmd.exe and honors the AO_SHELL escape hatch.

10Code map — where to look