diff --git a/README.md b/README.md index 27398e1a8..2d9f4dd2a 100644 --- a/README.md +++ b/README.md @@ -1,221 +1,223 @@ -# ReverbCode +
-The orchestration layer for parallel AI coding agents. ReverbCode is a -Go-backed daemon that supervises many coding-agent sessions at once, each in -its own `git worktree`, and routes the feedback they need (CI failures, review -comments, merge conflicts) back to the right agent automatically. It ships with -an `ao` CLI and an Electron supervisor that both drive the same daemon over -loopback. +

+ Agent Orchestrator +

-The Go module and packages remain `agent-orchestrator`; "ReverbCode" is the -public name. +

Agent Orchestrator

-See [`docs/architecture.md`](docs/architecture.md) for the backend mental model -and [`AGENTS.md`](AGENTS.md) for the contributor / worker contract. For current -progress (what's shipped vs. in flight) see [`docs/STATUS.md`](docs/STATUS.md). +

The orchestration layer for parallel AI coding agents

-## What it does +

+ Stars + Contributors + Twitter + Discord + License: Apache-2.0 +

-- **Agent-agnostic.** A 23-adapter platform under - `backend/internal/adapters/agent/` (`claude-code`, `codex`, `cursor`, - `opencode`, `aider`, `amp`, `goose`, `copilot`, `grok`, `qwen`, `kimi`, - `crush`, `cline`, `droid`, `devin`, `auggie`, `continue`, `kiro`, `kilocode`, - and more), registered through a shared registry with common - activity-dispatch / hook utilities. Worker and orchestrator defaults are set - per project. -- **Isolated workspaces.** Worker and orchestrator sessions spawn into their own - `git worktree` (`backend/internal/adapters/workspace/gitworktree/`), launched - inside a `zellij` runtime adapter (`backend/internal/adapters/runtime/`) so - every session has its own attachable terminal. -- **Live PR observation.** The provider-neutral SCM observer - (`backend/internal/observe/scm/`) polls each session's PR with ETag guards and - semantic diffing, tracking CI/check runs and review threads, and feeds those - facts into the lifecycle manager, which sends the owning agent nudges for CI - failures, review feedback, and merge conflicts. GitHub is the implemented - provider today. -- **Durable facts, derived status.** The SQLite store - (`backend/internal/storage/sqlite/`) persists a small set of session facts - plus PR/check/comment rows; display status is computed at read time, never - stored. DB triggers append every user-visible change to `change_log`, and a - CDC poller/broadcaster (`backend/internal/cdc/`) feeds in-process subscribers - and an SSE replay endpoint. -- **Loopback-only daemon.** The HTTP daemon (`backend/internal/httpd`) controls - projects, sessions, orchestrators, and hook callbacks over `127.0.0.1` with no - auth, CORS, or TLS by design. -- **Lifecycle manager + reaper** (`backend/internal/lifecycle/`, - `backend/internal/observe/reaper/`) reduce runtime/activity/PR observations - into the durable session state and reclaim dead sessions. +

An Agentic IDE that supervises parallel AI coding agents in isolated workspaces, with complete control and automatic feedback loops from CI failures, review comments, and merge conflicts.

-## How it works +

+ Agent Orchestrator Dashboard +

-1. Register a local git repo as a project (`ao project add`). -2. Spawn a worker session (`ao spawn`), or an orchestrator that fans work out - across sessions. Each session gets its own `git worktree` and a `zellij` - pane. -3. The agent develops, tests, and opens a PR from inside its worktree. -4. The SCM observer watches that PR and routes feedback back to the agent: a CI - failure, a requested change, or a merge conflict becomes a nudge to the agent - that owns the PR. -5. You inspect, attach a terminal, and merge from the CLI or the Electron app; - human attention is needed only where the loop can't resolve on its own. +### See AO in Action (before the re-write) -## Extensibility + + + + + + + + + +
+First

+Visit +		 +Second

+Visit +
+Third

+Visit +		 +Fourth

+Visit +
-The backend is organized around inbound/outbound port contracts -(`backend/internal/ports/`) with swappable adapters under -`backend/internal/adapters/`: +[Features](#features) • [Quick Start](#quick-start) • [Architecture](#architecture) • [Documentation](#documentation) • [Contributing](#contributing) -| Port | Implemented adapters | -| --------- | --------------------------------------------- | -| Agent | 23 harnesses (see above) | -| Runtime | `zellij` | -| Workspace | `git worktree` | -| SCM | GitHub | -| Tracker | GitHub (adapter present; no runtime loop yet) | -| Reviewer | `claude-code` | -| Notifier | port defined; no shipped adapter yet | +
-See [`docs/STATUS.md`](docs/STATUS.md) for which lanes are live at runtime. +--- -## Quick start +## Features -Requirements: Go 1.25+, [`zellij`](https://zellij.dev/) on `PATH` for the -runtime adapter, and `gh` (or `GITHUB_TOKEN`) if you want the SCM observer to -authenticate against GitHub. The SQLite driver is the pure-Go -`modernc.org/sqlite` — no system SQLite library is required. +| Feature | Description | +| :--- | :--- | +| **Agent-Agnostic Platform** | 23+ agent adapters including [Claude Code](https://code.claude.com/docs/en/overview), [OpenAI Codex](https://openai.com/), [Cursor](https://cursor.com/), [OpenCode](https://opencode.ai/), [Aider](https://aider.chat/), [Amp](https://ampcode.com/manual), [Goose](https://goose-docs.ai/), [GitHub Copilot](https://github.com/features/copilot), [Grok](https://x.ai/grok), [Qwen Code](https://github.com/QwenLM/qwen-code), [Kimi Code](https://www.kimi.com/code), [Cline](https://cline.bot/), [Continue](https://www.continue.dev/), [Kiro](https://kiro.dev/), and more | +| **Isolated Workspaces** | Each session spawns into its own git worktree with dedicated runtime | +| **Platform-Native Runtimes** | tmux on Darwin/Linux, conpty on Windows for optimal performance | +| **Live PR Observation** | Provider-neutral SCM observer with automatic feedback routing | +| **Automatic Feedback Routing** | CI failures, review comments, and merge conflicts routed to the owning agent | +| **Durable Facts Storage** | SQLite persists immutable facts with display status derived at read time | +| **CDC Broadcasting** | DB triggers append changes to change_log, broadcasted via SSE | +| **Desktop Experience** | Native Electron app with React UI and live terminal streaming | +| **Loopback-Only Daemon** | HTTP control over 127.0.0.1 with no auth, CORS, or TLS by design | -```bash -cd backend -go build -o /tmp/ao ./cmd/ao +### Supported Agents -# Start the daemon and wait for /readyz. -/tmp/ao start +Works with 23+ CLI-based coding agents including Claude Code, OpenAI Codex, Cursor, OpenCode, Aider, Amp, Goose, GitHub Copilot, Grok, Qwen Code, Kimi Code, Crush, Cline, Droid, Devin, Auggie, Continue, Kiro, and Kilo Code. -# Register a local git repo as a project. The id defaults to the lowercased -# base of --path; pass --id explicitly when the directory name doesn't match. -/tmp/ao project add --path /path/to/your/repo --id your-repo --name your-repo \ - --worker-agent codex --orchestrator-agent codex +**If it runs in a terminal, it runs on Agent Orchestrator.** -# Spawn a worker session running the project's worker agent. -/tmp/ao spawn --project your-repo --prompt "Refactor the auth module" +--- -# Inspect what's running. -/tmp/ao status -/tmp/ao session ls -``` +## Quick Start -### Electron app (dev) +### Prerequisites -The desktop supervisor lives under `frontend/` and is started separately: +| Requirement | Minimum | Recommended | +|-------------|---------|-------------| +| Go | 1.25+ | Latest | +| Node.js | 20+ | Latest LTS | +| Git | Any | Latest | +| pnpm | Any | Latest | -```bash -cd frontend -npm install -npm run dev # electron-forge start -``` +**Optional:** +- `tmux` (Darwin/Linux) - For Unix runtime +- `gh` (GitHub CLI) - For authenticated GitHub API calls -Heads-up: `npm run dev` does **not** start the daemon for you. Start it first -(`ao start`, see above) — the renderer attaches to the running daemon over -loopback (`127.0.0.1:3001` by default, the `AO_PORT` from the table below). -Without a daemon the app opens but shows its daemon-not-ready state. +### Installation -For renderer-only UI work without the Electron shell, use -`npm run dev:web` (Vite in a regular browser). +Download the latest release for your platform: -## CLI surface +| Platform | Download | +|----------|----------| +| **Windows** | [Setup.exe](https://github.com/AgentWrapper/agent-orchestrator/releases/latest) | +| **macOS** | [Agent Orchestrator.dmg](https://github.com/AgentWrapper/agent-orchestrator/releases/latest) | +| **Linux** | [Agent Orchestrator.AppImage](https://github.com/AgentWrapper/agent-orchestrator/releases/latest) | -The CLI is intentionally thin: every product command resolves to a daemon HTTP -route. Run `ao --help` for the authoritative flag shape; the table -below groups what's on `main` today. +**Direct Download:** [Latest Release](https://github.com/AgentWrapper/agent-orchestrator/releases/latest) -| Lane | Command | Purpose | -| ------------ | ------------------------------------ | ---------------------------------------------------------------------------------- | -| Daemon | `ao start` | Start the daemon in the background and wait for `/readyz`. | -| Daemon | `ao stop` | Graceful shutdown via loopback `POST /shutdown`. | -| Daemon | `ao status` | Report PID/port/health/readiness from `running.json`. | -| Daemon | `ao daemon` | Hidden internal entrypoint used by `ao start`. | -| Project | `ao project add` | Register a local git repo as a project. | -| Project | `ao project ls` | List registered projects. | -| Project | `ao project get ` | Fetch one project. | -| Project | `ao project set-config ` | Update per-project config. | -| Project | `ao project rm ` | Remove a project. | -| Session | `ao spawn` | Spawn a worker session in a registered project. | -| Session | `ao session ls` | List sessions (filter by project, include terminated). | -| Session | `ao session get ` | Fetch one session. | -| Session | `ao session kill ` | Terminate a session. | -| Session | `ao session rename ` | Rename a session. | -| Session | `ao session restore ` | Relaunch a terminated session. | -| Session | `ao session cleanup` | Reclaim eligible workspaces for terminated sessions. | -| Session | `ao session claim-pr ` | Attach an existing PR to a session. | -| Orchestrator | `ao orchestrator ls` | List orchestrator sessions. | -| Messaging | `ao send` | Send a message to a running agent session. | -| Preview | `ao preview [url]` | Open a URL (or the workspace `index.html`) in the session's desktop browser panel. | -| Utility | `ao doctor` | Local health checks (config, data dir, DB, `git`, `zellij`). | -| Utility | `ao completion ` | Generate bash/zsh/fish/powershell completions. | -| Utility | `ao version` | Print build metadata. | -| Internal | `ao hooks ` | Hidden adapter hook callback. | +--- -See [`docs/cli/`](docs/cli/) for the daemon-control intent and command shape. +## Telemetry -## Configuration +Agent Orchestrator collects minimal telemetry for reliability and product understanding. Data is stored locally by default; remote transmission is opt-in via environment variables. [Read the full telemetry policy](docs/telemetry.md). -All configuration is env-driven; the daemon takes no config file. The bind -host is hard-coded to `127.0.0.1` — the daemon has no auth, CORS, or TLS, and -exposing it beyond loopback would be a security regression. - -| Var | Default | Purpose | -| --------------------- | ------------------------------------------------- | --------------------------------------------------------------------------- | -| `AO_PORT` | `3001` | Bind port; daemon fails fast if taken. | -| `AO_REQUEST_TIMEOUT` | `60s` | Per-request timeout (Go duration). | -| `AO_SHUTDOWN_TIMEOUT` | `10s` | Graceful-shutdown hard cap. | -| `AO_RUN_FILE` | `/agent-orchestrator/running.json` | PID + port handshake path. | -| `AO_DATA_DIR` | `/agent-orchestrator/data` | SQLite DB, WAL files, managed state. | -| `AO_AGENT` | `claude-code` | Compatibility agent adapter id validated at daemon startup. | -| `AO_SESSION_ID` | _(unset)_ | Set inside spawned sessions; read by `ao send` and `ao hooks`. | -| `GITHUB_TOKEN` | _(unset)_ | Used by the GitHub SCM and tracker adapters. Falls back to `gh auth token`. | - -Health check: - -```bash -curl localhost:3001/healthz -curl localhost:3001/readyz -``` +--- ## Architecture -The daemon is a long-running supervisor. Adapters observe external facts (PR -state, agent activity, runtime liveness); the lifecycle manager reduces those -into a small set of durable session facts (`activity_state`, `is_terminated`, -PR rows). Display status is _derived_ from those facts at read time — it is -never stored. SQLite triggers append every user-visible change to `change_log`, -and the CDC poller broadcasts those events to in-process subscribers and an -SSE stream. +Agent Orchestrator is a long-running Go daemon built around **inbound/outbound port contracts** with swappable adapters. -Full mental model and load-bearing rules: [`docs/architecture.md`](docs/architecture.md). -Package-by-package ownership: [`docs/backend-code-structure.md`](docs/backend-code-structure.md). +**Core mental model:** OBSERVE external facts → UPDATE durable facts → DERIVE display status / ACT + +**Key components:** +- **Frontend** - Electron + React UI with TanStack Router/Query and shadcn/ui +- **Backend Daemon** - Go-based HTTP server with controllers, services, and adapters +- **Runtime** - Platform-specific: `tmux` on Darwin/Linux, `conpty` on Windows +- **Storage** - SQLite with change-data-capture (CDC) for real-time updates +- **Adapters** - 23+ agent adapters, git worktree workspace, GitHub SCM integration + +For detailed architecture diagrams, data flows, and load-bearing rules, see [architecture.md](docs/architecture.md). + +--- + +## Documentation + +| Document | Description | +|----------|-------------| +| [Architecture](docs/architecture.md) | System architecture, data flows, and load-bearing rules | +| [Backend Code Structure](docs/backend-code-structure.md) | Package-by-package ownership and dependency rules | +| [AGENTS.md](AGENTS.md) | Contributor and worker-agent contract | +| [Agent Adapter Contract](docs/agent/README.md) | Agent adapter interface and hook behavior | + +--- ## Testing -The local gate is the backend Go build and race-enabled test suite: - ```bash -cd backend && go build ./... && go test -race ./... +# Backend tests +cd backend +go test -race ./... + +# Frontend tests +cd frontend +pnpm test + +# Full CI validation locally +npx @redwoodjs/agent-ci run --all ``` -GitHub Actions is the authoritative pre-merge gate; mirror its commands here -when in doubt. See [`AGENTS.md`](AGENTS.md) for the regen workflow when -touching the daemon API surface (`npm run sqlc`, `npm run api`). +--- -## Status and roadmap +## Configuration -Progress tracking lives in [`docs/STATUS.md`](docs/STATUS.md): what is shipped -on `main` today, what is still in flight, and the linked -[`rewrite`](https://github.com/aoagents/agent-orchestrator/milestone/1) -milestone on GitHub. +All configuration is environment-driven. The daemon takes no config file. + +| Variable | Default | Purpose | +|----------|---------|---------| +| `AO_PORT` | `3001` | HTTP bind port | +| `AO_REQUEST_TIMEOUT` | `60s` | Per-request timeout | +| `AO_SHUTDOWN_TIMEOUT` | `10s` | Graceful shutdown cap | +| `AO_RUN_FILE` | `~/.ao/running.json` | PID/port handshake | +| `AO_DATA_DIR` | `~/.ao/data` | SQLite data directory | +| `AO_AGENT` | `claude-code` | Compatibility agent adapter | +| `GITHUB_TOKEN` | - | GitHub auth token | + +### Health Checks + +```bash +curl localhost:3001/healthz # Liveness probe +curl localhost:3001/readyz # Readiness probe +``` + +--- ## Contributing -Repo layout and the worker contract live in [`AGENTS.md`](AGENTS.md). Keep -changes surgical, follow the package boundaries documented in -[`docs/backend-code-structure.md`](docs/backend-code-structure.md), and prefer -adding daemon HTTP routes over leaking storage / runtime into the CLI. +We love contributions! Join our community on Discord to get started. + +### Join us on Discord + +[![Discord](https://img.shields.io/badge/Discord-join%20the%20community-5865F2?style=for-the-badge&logo=discord&logoColor=white&logoSize=auto)](https://discord.com/invite/UZv7JjxbwG) + +**Daily contributor sync:** Every day at **10:00 PM IST** + +Get your issues verified by core contributors, ask questions, share progress, and learn from the community. New contributors are always welcome! + +**Why join Discord?** + +- Get your issues and PRs verified by core contributors before investing time +- Learn from experienced contributors in daily sync calls +- Share your progress and get feedback +- Get help troubleshooting in real-time +- Stay updated on the latest developments and roadmap + +### Quick Start + +1. **Join the Discord** - Connect with the community and get guidance +2. **Read the contributor contract** - See [AGENTS.md](AGENTS.md) for repo layout, daemon/API boundaries, and coding conventions +3. **Pick a focused problem** - Browse [open issues](https://github.com/AgentWrapper/agent-orchestrator/issues) and choose one small enough for a focused PR +4. **Open a clear PR** - Keep changes narrow, explain user-visible impact, link issues, include tests +5. **Iterate with contributors** - Use review feedback to tighten the PR until verified + +--- + +## License + +Apache License 2.0 - see [LICENSE](LICENSE) for details. + +--- + +
+ +**[Star us on GitHub](https://github.com/AgentWrapper/agent-orchestrator)** • **[Report Issues](https://github.com/AgentWrapper/agent-orchestrator/issues)** • **[Discussions](https://github.com/AgentWrapper/agent-orchestrator/discussions)** + +Made with love by the Agent Orchestrator community + +
diff --git a/ao-dashboard-preview.png b/ao-dashboard-preview.png new file mode 100644 index 000000000..27832975e Binary files /dev/null and b/ao-dashboard-preview.png differ diff --git a/ao-logo.svg b/ao-logo.svg new file mode 100644 index 000000000..cfe33f5df --- /dev/null +++ b/ao-logo.svg @@ -0,0 +1,25 @@ + + + + + + + + + + + + + + + + + + + + + + + + + \ No newline at end of file diff --git a/docs/agent/README.md b/docs/agent/README.md index 89385c054..6ddc3424c 100644 --- a/docs/agent/README.md +++ b/docs/agent/README.md @@ -1,124 +1,945 @@ -# Agent Adapter PRD +# Agent Adapter Contract -## Goal +This document defines the contract between Agent Orchestrator and CLI-based coding agent adapters. Every agent must implement the contract defined in `backend/internal/ports/agent.go` to be usable by AO. -Agent adapters let AO run and observe different CLI coding agents without hardcoding agent-specific behavior into the spawn engine. Every CLI coding agent must implement the contract in `backend/internal/ports/agent.go`. +## Table of Contents -The important current slice is hook-derived session info. AO should know a running worker's native agent session id, title, and summary from agent hooks installed in the per-session worktree, not from scanning agent transcript/cache files. +- [Overview](#overview) +- [Agent Contract](#agent-contract) +- [Hook System](#hook-system) +- [Session Metadata](#session-metadata) +- [Agent Lifecycle](#agent-lifecycle) +- [Implementation Guide](#implementation-guide) +- [Supported Agents](#supported-agents) +- [Testing](#testing) -## Current Decisions +--- -- AO only needs to derive session info for AO-managed sessions. -- Hook installation happens at worktree/session creation time. -- `SessionInfo` reads normalized metadata persisted in AO's session store. -- `SessionInfo` must not infer display info by reading agent transcript/cache files. -- `SummaryIsFallback` is removed from `ports.SessionInfo`. -- `TranscriptPath` is removed from `ports.SessionInfo`. -- `Title` and `Summary` are both first-class fields. -- `Title` is derived from the user prompt hook. -- `Summary` is derived from the stop/final assistant hook. -- Agent adapter `Metadata` should stay nil/empty unless an adapter has a real extra field that does not belong in the normalized contract. +## Overview + +Agent adapters let AO run and observe different CLI coding agents without hardcoding agent-specific behavior into the spawn engine. The adapter contract provides: + +- **Agent discovery** — Find and validate the agent binary +- **Launch configuration** — Build the native agent command +- **Session restoration** — Resume existing sessions +- **Hook integration** — Install and manage agent hooks +- **Metadata extraction** — Surface session title, summary, and native session ID + +```mermaid +graph TB + AO[Agent Orchestrator] -->|uses| Adapter[Agent Adapter] + Adapter -->|implements| Contract[ports.Agent Contract] + + Contract --> Methods[Required Methods] + Methods --> GetConfig[GetConfigSpec] + Methods --> GetLaunch[GetLaunchCommand] + Methods --> GetStrategy[GetPromptDeliveryStrategy] + Methods --> GetHooks[GetAgentHooks] + Methods --> GetRestore[GetRestoreCommand] + Methods --> SessionInfo[SessionInfo] + Methods --> Uninstall[UninstallHooks] + + Adapter --> Agent[Native CLI Agent] + Adapter --> Hooks[Agent Hook Config] + +``` + +--- ## Agent Contract -The shared contract lives in `backend/internal/ports/agent.go`. +### Port Interface -Required adapter behavior: +The `ports.Agent` interface (defined in `backend/internal/ports/agent.go`) specifies the required adapter methods: -- `GetConfigSpec` describes user-facing agent config. -- `GetLaunchCommand` builds the native agent command. -- `GetPromptDeliveryStrategy` says whether the prompt is passed in argv or sent after launch. -- `GetAgentHooks` installs or merges AO hooks into the agent's workspace-local hook config. -- `GetRestoreCommand` builds a native resume command when restore is supported. -- `SessionInfo` returns normalized metadata: - - `AgentSessionID` - - `Title` - - `Summary` - - optional adapter-specific `Metadata` +```go +// Agent is the interface all coding agent adapters must implement. +type Agent interface { + // GetConfigSpec describes user-facing agent configuration. + GetConfigSpec() ConfigSpec -Implementation layout: + // GetLaunchCommand builds the native agent command for spawning a new session. + GetLaunchCommand(ctx context.Context, cfg LaunchConfig) ([]string, error) -- Agent-specific hook installation should live beside the agent adapter in `backend/internal/adapters/agent//hooks.go`; the hook commands are defined in code, not embedded template files. -- Launch, restore, and session-info behavior can stay in the main agent implementation unless the file grows enough to justify another split. -- Every file an adapter writes into the session worktree must be covered by a sibling self-ignoring `.gitignore` written via `hookutil.EnsureWorkspaceGitignore`. Hook files are untracked, and `git worktree remove` (never run with `--force`) refuses on any untracked file — an uncovered hook file makes every session workspace permanently undeletable. The registry conformance test (`registry.TestGetAgentHooksFootprintIsGitignored`) enforces this for all adapters. + // GetPromptDeliveryStrategy reports how the prompt is delivered. + GetPromptDeliveryStrategy() PromptDeliveryStrategy -## Metadata Keys + // GetAgentHooks installs or merges AO hooks into the agent's workspace-local config. + GetAgentHooks(ctx context.Context, cfg WorkspaceHookConfig) error -Hook callbacks persist these normalized keys in the session metadata JSON blob: + // GetRestoreCommand builds a command to resume an existing session. + GetRestoreCommand(ctx context.Context, cfg RestoreConfig) ([]string, bool, error) -- `agentSessionId`: native agent session id. -- `title`: display title, derived from the first user prompt hook for the session. -- `summary`: display summary, derived from the final assistant message exposed to the stop hook. + // SessionInfo returns normalized session metadata from the session record. + SessionInfo(ctx context.Context, session SessionRef) (SessionInfo, bool, error) -The original spawn prompt may remain in metadata as `prompt` for launch/debug fallback, but `title` is the preferred display title once hook metadata lands. - -## Hook Methodology - -Agent adapters install hooks into the worktree-local config owned by the native agent. - -Codex is the exception: Codex (0.136+) only loads project-local `.codex/` hook config from trusted directories, and for linked git worktrees it sources hook declarations from the matching folder in the root checkout — never from AO's per-session worktree. The Codex adapter therefore passes its hooks on the launch command as `-c 'hooks.=[...]'` session-flag config (plus `--dangerously-bypass-hook-trust`, since session-flag hooks have no persisted trust hash), and marks the worktree as a trusted project for the invocation with `-c 'projects={""={trust_level="trusted"}}'` so spawns into never-before-trusted repos don't hang on Codex's interactive directory-trust prompt. Its `GetAgentHooks` writes nothing; it only strips entries older AO versions left in the worktree-local `.codex/hooks.json`. - -Hook callbacks run through hidden AO CLI commands: - -```text -ao hooks + // UninstallHooks removes AO hooks from the workspace. + UninstallHooks(ctx context.Context, workspacePath string) error +} ``` -The callback: +### Method Details -1. Reads the native hook JSON payload from stdin. -2. Reads the AO session id from `AO_SESSION_ID` (exits 0 immediately for non-AO sessions). -3. Derives a normalized activity state from the agent + event (`activitydispatch.Derive`); events with no activity meaning report nothing. -4. POSTs the state to the daemon at `POST /api/v1/sessions/{id}/activity`; the daemon owns the store and fans out `session.updated` via CDC. -5. Always exits 0 — a failed delivery must never break the user's agent. Failures are appended to `hooks.log` under `AO_DATA_DIR` and surfaced by the `hooks-log` check in `ao doctor`. +#### GetConfigSpec() -The daemon also records the FIRST callback per spawn/restore (`first_signal_at`); a live session that has never signaled past a grace period derives the `no_signal` display status instead of a confident `idle`, so a broken hook pipeline is visible on the dashboard. The downgrade only applies to harnesses with a registered activity deriver (`activitydispatch.SupportsHarness`, injected into the session service at daemon wiring) — for a hook-less adapter, permanent silence is normal and stays `idle`. Known limitation: neither Codex nor Claude Code derives an activity state from `SessionStart`, so a restored session the user never prompts has nothing to signal and shows `no_signal` once the grace passes; a receipt-only session-start signal would close that gap. +Returns a specification of user-facing configuration options: -Persisting hook-derived metadata (`agentSessionId`, `title`, `summary`) into the session row is **not implemented yet** — until it is, adapters whose restore needs the native session id (e.g. `codex resume`) fall back to a fresh launch. +```go +type ConfigSpec struct { + // Permissions is the permission mode the agent supports. + Permissions PermissionMode -The spawn engine inserts the AO session row before launching the durability provider so early startup hooks can update an existing row. If launch fails after insertion, spawn deletes the row during rollback. + // Model is the optional model identifier (e.g., "claude-3-5-sonnet-20241022"). + Model string -The hook commands are a bare `ao hooks ...` on purpose: worktree-committed hook files stay machine-portable, and adapters recognize their own entries by command prefix for install/dedup/uninstall. To make the bare `ao` resolve to the daemon that installed the hooks (not a foreign or legacy `ao` earlier on the user's PATH), the session manager pins each spawned session's `PATH` with the daemon executable's directory first. When the pin cannot be applied (executable unresolvable or not named `ao`), the daemon logs a warning at spawn. Hook delivery failures are best-effort appended to `hooks.log` under `AO_DATA_DIR` (agents swallow hook stderr), and `ao doctor` warns when the `ao` on PATH is not the running binary. + // SupportsSystemPrompt indicates the agent accepts a system prompt. + SupportsSystemPrompt bool -## Restore Boundary - -Session display info and native restore are separate concerns. - -Some agents may still need transcript-derived or deterministic native ids for `GetRestoreCommand` until restore is redesigned for that agent. Do not remove restore support just because `SessionInfo` stops reading transcripts. - -For `SessionInfo`, transcript/cache files are not an acceptable source of title or summary. - -## UI And Events - -The workspace adapter prefers: - -- `metadata.title` as session title. -- `metadata.summary` as session description. -- `metadata.prompt` only as fallback. - -Hook metadata changes publish `session.updated`. The frontend listens to `session.created`, `session.terminated`, and `session.updated` and invalidates the workspace query. - -## Acceptance Criteria - -Agent adapter behavior: - -- Agent hook installation preserves user hooks and deduplicates AO hooks. -- Hook callbacks persist native session id, title, and summary. -- `SessionInfo` returns normalized fields from persisted metadata. -- `SessionInfo` does not read transcripts or caches for title/summary. -- Adapter-specific metadata stays nil/empty unless a concrete feature requires it. - -Engine and UI: - -- Spawn installs hooks before launching the native agent. -- The session row exists before launch so hooks can merge metadata. -- Launch failure after row insertion deletes the row. -- Metadata updates publish `session.updated`. -- The dashboard refreshes title/summary without a manual reload. - -Verification: - -```sh -(cd backend && go test ./...) -(cd frontend && npm run typecheck) + // SupportsRestore indicates the agent supports session restoration. + SupportsRestore bool +} ``` + +#### GetLaunchCommand() + +Builds the native agent launch command: + +```mermaid +flowchart LR + Input[LaunchConfig] --> Validate[Validate Config] + Validate --> Resolve[Resolve Binary] + Resolve --> Build[Build Command] + Build --> Apply[Apply Permissions] + Apply --> Model[Add Model Flag] + Model --> System[Add System Prompt] + System --> Prompt[Add Prompt] + Prompt --> Output[Launch Command] + +``` + +**Input (`LaunchConfig`):** +- `SessionID` — AO session identifier +- `ProjectID` — AO project identifier +- `Prompt` — User prompt to send +- `Config` — Agent-specific configuration +- `Permissions` — Permission mode override +- `WorkspacePath` — Path to session worktree + +**Output:** Executable command line as argv array + +#### GetPromptDeliveryStrategy() + +Reports how the prompt is delivered to the agent: + +```go +type PromptDeliveryStrategy string + +const ( + // StrategyArgv means the prompt is passed as a command-line argument. + StrategyArgv PromptDeliveryStrategy = "argv" + + // StrategyStdin means the prompt is written to stdin after launch. + StrategyStdin PromptDeliveryStrategy = "stdin" +) +``` + +#### GetAgentHooks() + +Installs AO hooks into the agent's workspace-local configuration: + +```mermaid +flowchart TD + Input[WorkspaceHookConfig] --> Read[Read Existing Config] + Read --> Merge[Merge AO Hooks] + Merge --> Dedup[Deduplicate Entries] + Dedupe --> Preserve[Preserve User Hooks] + Preserve --> Write[Write Updated Config] + Write --> Gitignore[Update .gitignore] + +``` + +**Requirements:** +- Preserve all user hooks +- Deduplicate AO hook entries +- Make AO hooks machine-portable (use `ao hooks ...` command) +- Ensure all written files are covered by `.gitignore` + +#### GetRestoreCommand() + +Builds a command to resume an existing session: + +**Input (`RestoreConfig`):** +- `Session` — Complete session record with metadata +- `Permissions` — Permission mode to apply +- `SystemPrompt` — System prompt to re-apply + +**Output:** +- `cmd` — Restore command argv +- `ok` — True if restore is supported +- `err` — Error if restore fails + +#### SessionInfo() + +Returns normalized session metadata from the stored session record: + +```go +type SessionInfo struct { + // AgentSessionID is the native agent's session identifier. + AgentSessionID string + + // Title is the display title derived from the first user prompt. + Title string + + // Summary is the display summary derived from the final assistant message. + Summary string + + // Metadata contains adapter-specific extra fields. + Metadata map[string]any +} +``` + +**Important:** `SessionInfo` must read from the `session.Metadata` map (populated by hooks), not from scanning agent transcript/cache files. + +#### UninstallHooks() + +Removes AO hooks from the workspace while preserving user hooks. + +--- + +## Hook System + +### Hook Overview + +AO uses agent hooks to receive activity signals and session metadata from running agents: + +```mermaid +sequenceDiagram + participant AO as AO Daemon + participant Adapter as Agent Adapter + participant Agent as Native Agent + participant Hook as Hook Command + + Note over AO: Spawning session + AO->>Adapter: GetAgentHooks(config) + Adapter->>Hook: Write hook config + Hook-->>Adapter: Config written + + AO->>Agent: Launch agent + Agent->>Agent: Running in worktree + + Note over Agent: Agent triggers event + Agent->>Hook: Execute hook callback + Hook->>Hook: Read stdin payload + Hook->>AO: POST /api/v1/sessions/{id}/activity + AO->>AO: Update activity_state + AO->>Hook: 200 OK + Hook-->>Agent: Exit 0 + + Agent->>Agent: Continue work +``` + +### Hook Events + +AO supports hooks for these agent events: + +| Event | Purpose | Activity State | +|-------|---------|----------------| +| `SessionStart` | Session initialized | - | +| `UserPromptSubmit` | User submitted prompt | `active` | +| `AssistantMessage` | Assistant response | - | +| `ToolUse` | Agent used a tool | `active` | +| `Stop` | Session stopped | `idle` or `exited` | + +### Hook Contract + +All hook callbacks follow this contract: + +```bash +# Hook command format +ao hooks + +# Input (stdin) +# JSON payload from agent (agent-specific format) + +# Output (stdout) +# None (discarded) + +# Exit code +# 0 = success (always, even on delivery failure) +``` + +### Hook Behavior + +```mermaid +flowchart TD + Trigger[Agent Event Triggered] --> Execute[Execute Hook Command] + Execute --> ReadEnv[Read AO_SESSION_ID
from environment] + ReadEnv --> CheckSession{Is AO Session?} + CheckSession -->|No| Exit[Exit 0 - ignore] + CheckSession -->|Yes| ReadStdin[Read JSON Payload
from stdin] + ReadStdin --> Derive[Derive Activity State
from event] + Derive --> Post[POST to Daemon
/api/v1/sessions/{id}/activity] + Post --> Success{Success?} + Success -->|Yes| Update[Daemon updates
activity_state] + Success -->|No| Log[Log to hooks.log
under AO_DATA_DIR] + Update --> Exit + Log --> Exit + +``` + +**Critical rules:** +1. **Always exit 0** — Hook failure must never break the user's agent +2. **Log failures** — Append to `hooks.log` under `AO_DATA_DIR` +3. **Best-effort delivery** — The daemon may be temporarily unavailable +4. **Idempotent** — Duplicate hook deliveries are safe + +### Hook Installation Patterns + +Different agents use different hook mechanisms: + +#### Claude Code / Compatible Agents + +```json +// .claude/hooks.json +{ + "SessionStart": ["ao hooks claude-code SessionStart"], + "UserPromptSubmit": ["ao hooks claude-code UserPromptSubmit"], + "Stop": ["ao hooks claude-code Stop"] +} +``` + +#### Factory Droid + +```json +// .factory/hooks.json +{ + "hooks": [ + { + "event": "agent:beforeThinking", + "command": "ao hooks droid beforeThinking" + }, + { + "event": "agent:afterThinking", + "command": "ao hooks droid afterThinking" + } + ] +} +``` + +#### Codex (Session Flags) + +```bash +# Codex passes hooks as session flags +codex -c 'hooks.SessionStart=["ao hooks codex SessionStart"]' \ + -c 'hooks.Stop=["ao hooks codex Stop"]' \ + --dangerously-bypass-hook-trust +``` + +--- + +## Session Metadata + +### Metadata Keys + +Hook callbacks persist normalized keys in the session metadata JSON blob: + +```go +const ( + // MetadataKeyAgentSessionID is the native agent's session identifier. + MetadataKeyAgentSessionID = "agentSessionId" + + // MetadataKeyTitle is the display title from the first user prompt. + MetadataKeyTitle = "title" + + // MetadataKeySummary is the display summary from the final assistant message. + MetadataKeySummary = "summary" +) +``` + +### Metadata Flow + +```mermaid +sequenceDiagram + participant Hook as Hook Callback + participant Daemon as AO Daemon + participant Store as SQLite Store + participant UI as Dashboard + + Note over Hook: User submits first prompt + Hook->>Daemon: POST /activity
{state: "active", title: "..."} + Daemon->>Store: Update session metadata
metadata.title = "..." + Store->>Store: CDC: session.updated + Store->>UI: SSE: session.updated + UI->>UI: Update session title + + Note over Hook: Agent finishes work + Hook->>Daemon: POST /activity
{state: "idle", summary: "..."} + Daemon->>Store: Update session metadata
metadata.summary = "..." + Store->>Store: CDC: session.updated + Store->>UI: SSE: session.updated + UI->>UI: Update session summary +``` + +### Metadata Persistence + +```mermaid +flowchart LR + Hook[Hook Callback] --> POST[POST Activity] + POST --> LCM[Lifecycle Manager] + LCM --> Update[Update Session Record] + Update --> Metadata[Set Metadata Fields] + Metadata --> CDC[CDC Broadcast] + CDC --> UI[Dashboard Updates] + +``` + +--- + +## Agent Lifecycle + +### Spawn Flow + +```mermaid +sequenceDiagram + participant User as User + participant UI as Dashboard + participant API as HTTP API + participant Mgr as Session Manager + participant Adapter as Agent Adapter + participant Runtime as Runtime + participant Agent as Agent Process + + User->>UI: Click "Spawn Session" + UI->>API: POST /sessions + API->>Mgr: Spawn(config) + + Note over Mgr: 1. Create session row + Mgr->>Mgr: Insert session with empty metadata + + Note over Mgr: 2. Prepare workspace + Mgr->>Adapter: GetAgentHooks(config) + Adapter->>Adapter: Write hook config + Adapter-->>Mgr: Hooks installed + + Note over Mgr: 3. Build launch command + Mgr->>Adapter: GetLaunchCommand(config) + Adapter-->>Mgr: Launch command argv + + Note over Mgr: 4. Execute in runtime + Mgr->>Runtime: Execute(agent command) + Runtime->>Agent: Spawn process + + Note over Agent: Agent starts + Agent->>Agent: Execute SessionStart hook + Agent->>API: POST /activity (agentSessionId) + API->>API: Update metadata + + Agent-->>Runtime: Running + Runtime-->>Mgr: Session active + Mgr-->>API: Session response + API-->>UI: Session created +``` + +### Restore Flow + +```mermaid +sequenceDiagram + participant User as User + participant UI as Dashboard + participant API as HTTP API + participant Mgr as Session Manager + participant Adapter as Agent Adapter + participant Runtime as Runtime + participant Agent as Agent Process + + User->>UI: Click "Restore Session" + UI->>API: POST /sessions/{id}/restore + API->>Mgr: Restore(session) + + Note over Mgr: Check adapter supports restore + Mgr->>Adapter: GetRestoreCommand(config) + + alt Restore supported + Adapter-->>Mgr: Restore command + Mgr->>Runtime: Execute(restore command) + Runtime->>Agent: Spawn process + Agent-->>Mgr: Running + Mgr-->>API: Session restored + else Restore not supported + Adapter-->>Mgr: (cmd, false, nil) + Mgr->>Mgr: Fresh spawn instead + Mgr-->>API: Session created (new) + end + + API-->>UI: Session response +``` + +### Activity Flow + +```mermaid +stateDiagram-v2 + [*] --> Spawning: Spawn() + Spawning --> Idle: Agent starts + Idle --> Active: User prompt + Active --> Active: Agent working + Active --> Waiting: Needs input + Waiting --> Active: User responds + Active --> Idle: Agent completes + Idle --> Exited: Agent exits + Active --> Exited: Agent crashes + Exited --> [*] + + note right of Active + Hook: UserPromptSubmit + Hook: ToolUse + activity_state = "active" + end note + + note right of Waiting + Hook: WaitingForInput + activity_state = "waiting_input" + end note + + note right of Idle + Hook: Stop + activity_state = "idle" + end note + + note right of Exited + Hook: Stop + activity_state = "exited" + is_terminated = true + end note +``` + +--- + +## Implementation Guide + +### Adapter Template + +```go +package myagent + +import ( + "context" + "fmt" +) + +// Plugin is the MyAgent adapter. +type Plugin struct { + // Adapter state (e.g., resolved binary path) +} + +// New returns a ready-to-register MyAgent adapter. +func New() *Plugin { + return &Plugin{} +} + +// GetConfigSpec describes the agent configuration. +func (p *Plugin) GetConfigSpec() ports.ConfigSpec { + return ports.ConfigSpec{ + Permissions: ports.PermissionReadWrite, // or ReadOnly, WriteOnly + SupportsSystemPrompt: true, + SupportsRestore: true, + } +} + +// GetLaunchCommand builds the native agent command. +func (p *Plugin) GetLaunchCommand(ctx context.Context, cfg ports.LaunchConfig) ([]string, error) { + // 1. Validate config + if err := cfg.Config.Validate(); err != nil { + return nil, fmt.Errorf("myagent: %w", err) + } + + // 2. Resolve binary + binary, err := p.resolveBinary(ctx) + if err != nil { + return nil, err + } + + // 3. Build command + cmd := []string{binary} + + // Add flags (model, permissions, etc.) + if cfg.Config.Model != "" { + cmd = append(cmd, "--model", cfg.Config.Model) + } + + // Add system prompt if supported + if cfg.SystemPrompt != "" { + cmd = append(cmd, "--system-prompt", cfg.SystemPrompt) + } + + // Add prompt + if cfg.Prompt != "" { + cmd = append(cmd, "--prompt", cfg.Prompt) + } + + return cmd, nil +} + +// GetPromptDeliveryStrategy reports how the prompt is delivered. +func (p *Plugin) GetPromptDeliveryStrategy() ports.PromptDeliveryStrategy { + return ports.StrategyArgv // or StrategyStdin +} + +// GetAgentHooks installs AO hooks. +func (p *Plugin) GetAgentHooks(ctx context.Context, cfg ports.WorkspaceHookConfig) error { + // 1. Read existing config + config, err := p.readConfig(cfg.WorkspacePath) + if err != nil { + return err + } + + // 2. Add AO hooks (preserving user hooks) + config = p.addHooks(config, cfg.SessionID) + + // 3. Write updated config + if err := p.writeConfig(cfg.WorkspacePath, config); err != nil { + return err + } + + // 4. Ensure .gitignore coverage + if err := p.ensureGitignore(cfg.WorkspacePath); err != nil { + return err + } + + return nil +} + +// GetRestoreCommand builds a restore command. +func (p *Plugin) GetRestoreCommand(ctx context.Context, cfg ports.RestoreConfig) ([]string, bool, error) { + // 1. Extract native session ID from metadata + sessionID := cfg.Session.Metadata[ports.MetadataKeyAgentSessionID] + if sessionID == "" { + return nil, false, nil // Restore not supported + } + + // 2. Build restore command + binary, err := p.resolveBinary(ctx) + if err != nil { + return nil, false, err + } + + cmd := []string{ + binary, + "--resume", sessionID, + // Re-apply permissions and system prompt + } + + return cmd, true, nil +} + +// SessionInfo returns normalized metadata. +func (p *Plugin) SessionInfo(ctx context.Context, session ports.SessionRef) (ports.SessionInfo, bool, error) { + info := ports.SessionInfo{ + AgentSessionID: session.Metadata[ports.MetadataKeyAgentSessionID], + Title: session.Metadata[ports.MetadataKeyTitle], + Summary: session.Metadata[ports.MetadataKeySummary], + } + + // Return hasData=true if any field is populated + hasData := info.AgentSessionID != "" || info.Title != "" || info.Summary != "" + + return info, hasData, nil +} + +// UninstallHooks removes AO hooks. +func (p *Plugin) UninstallHooks(ctx context.Context, workspacePath string) error { + // Remove AO hook entries while preserving user hooks + config, err := p.readConfig(workspacePath) + if err != nil { + return err + } + + config = p.removeHooks(config) + + return p.writeConfig(workspacePath, config) +} +``` + +### Hook Implementation + +```go +// Add hooks to the agent's config +func (p *Plugin) addHooks(config map[string]any, sessionID string) map[string]any { + // Define AO hook commands + aoHooks := map[string]string{ + "SessionStart": "ao hooks myagent SessionStart", + "UserPromptSubmit": "ao hooks myagent UserPromptSubmit", + "Stop": "ao hooks myagent Stop", + } + + // Merge with existing config, preserving user hooks + for event, command := range aoHooks { + if config[event] == nil { + config[event] = []string{command} + } else { + // Deduplicate: check if AO hook already exists + hooks := config[event].([]string) + found := false + for _, h := range hooks { + if strings.HasPrefix(h, "ao hooks myagent") { + found = true + break + } + } + if !found { + config[event] = append(hooks, command) + } + } + } + + return config +} +``` + +### Gitignore Enforcement + +```go +import "github.com/aoagents/agent-orchestrator/backend/internal/adapters/agent/hookutil" + +func (p *Plugin) ensureGitignore(workspacePath string) error { + // Every file the adapter writes must be gitignored + filesToIgnore := []string{ + ".myagent/config.json", + ".myagent/hooks.json", + } + + return hookutil.EnsureWorkspaceGitignore(workspacePath, ".myagent/") +} +``` + +--- + +## Supported Agents + +### Current Adapters + +AO currently supports 23+ agent adapters: + +```mermaid +mindmap + root((Agent Adapters)) + Anthropic + Claude Code + OpenAI + Codex + Cursor + Cursor + OpenCode + OpenCode + Aider + Aider + Amp + Amp Code + Goose + Goose + GitHub + Copilot + xAI + Grok + Alibaba + Qwen Code + Moonshot + Kimi Code + Reverb + Crush + Cline + Cline + Factory + Droid + Devin + Augment + Auggie + Continue + Continue + Kiro + Kiro + Kilo + Kilo Code + Agy + Agy + Roo + Roo Code + Windsurf + Windsurf +``` + +### Adapter Locations + +``` +backend/internal/adapters/agent/ +├── agy/ +├── aider/ +├── amp/ +├── augie/ +├── claudecode/ +├── clone/ +├── codex/ +├── continue/ +├── cursor/ +├── devin/ +├── droid/ +├── grok/ +├── goose/ +├── kilo/ +├── kimi/ +├── kiro/ +├── opencode/ +├── qwen/ +├── roo/ +├── crush/ +├── windsurf/ +└── ... +``` + +### Adapter Capabilities + +| Agent | Permissions | System Prompt | Restore | Hooks | +|-------|-------------|---------------|---------|-------| +| Claude Code | ✓ | ✓ | ✓ | ✓ | +| Codex | ✓ | ✓ | ✓ | ✓ (session flags) | +| Cursor | ✓ | ✓ | ✗ | ✗ | +| OpenCode | ✓ | ✓ | ✗ | ✗ | +| Aider | ✓ | ✓ | ✓ | ✓ | +| Amp | ✓ | ✓ | ✓ | ✓ | +| Grok | ✓ | ✓ | ✓ | ✓ (Claude compat) | +| ... | ... | ... | ... | ... | + +--- + +## Testing + +### Unit Tests + +```go +func TestGetLaunchCommand(t *testing.T) { + p := New() + cfg := ports.LaunchConfig{ + SessionID: "test-session", + Prompt: "Write a function", + Config: domain.AgentConfig{ + Model: "gpt-4", + }, + } + + cmd, err := p.GetLaunchCommand(context.Background(), cfg) + assert.NoError(t, err) + assert.Contains(t, cmd, "myagent") + assert.Contains(t, cmd, "--model", "gpt-4") + assert.Contains(t, cmd, "Write a function") +} +``` + +### Integration Tests + +```go +func TestAgentE2E(t *testing.T) { + // Skip if agent not installed + if !agentInstalled(t) { + t.Skip("myagent not installed") + } + + // Create test project + ctx := context.Background() + project := createTestProject(t) + + // Spawn session + session := spawnSession(t, ctx, project.ID, "myagent", "Hello") + + // Wait for activity + waitForActivity(t, ctx, session.ID, "active") + + // Verify metadata + info := getSessionInfo(t, ctx, session.ID) + assert.NotEmpty(t, info.AgentSessionID) + assert.NotEmpty(t, info.Title) + + // Kill session + killSession(t, ctx, session.ID) +} +``` + +### Conformance Tests + +```go +// TestGetAgentHooksFootprintIsGitignored verifies all adapter-written +// files are covered by .gitignore. +func TestGetAgentHooksFootprintIsGitignored(t *testing.T) { + registry := agentRegistry.New() + for _, adapter := range registry.List() { + t.Run(adapter, func(t *testing.T) { + tmpDir := t.TempDir() + cfg := ports.WorkspaceHookConfig{ + WorkspacePath: tmpDir, + SessionID: "test-session", + } + + agent := registry.Get(adapter) + err := agent.GetAgentHooks(context.Background(), cfg) + assert.NoError(t, err) + + // Verify all written files are gitignored + verifyGitignore(t, tmpDir) + }) + } +} +``` + +### Hook Tests + +```go +func TestHookDelivery(t *testing.T) { + // Mock daemon endpoint + server := mockActivityServer(t) + defer server.Close() + + // Simulate hook callback + payload := map[string]any{ + "state": "active", + "title": "Test session", + } + + cmd := exec.Command("ao", "hooks", "myagent", "UserPromptSubmit", "test-session") + cmd.Env = append(cmd.Env, + fmt.Sprintf("AO_SESSION_ID=%s", "test-session"), + fmt.Sprintf("AO_DAEMON_URL=%s", server.URL), + ) + + stdin, _ := cmd.StdinPipe() + json.NewEncoder(stdin).Encode(payload) + stdin.Close() + + err := cmd.Run() + assert.NoError(t, err) + + // Verify daemon received the activity + assert.ActivityReceived(t, server, "active") +} +``` + +--- + +## Summary + +**Key points:** + +1. **Port contract** — All agents implement `ports.Agent` +2. **Hooks are critical** — Activity signals and metadata flow through hooks +3. **Always gitignore** — Every adapter-written file must be covered +4. **Metadata from hooks** — `SessionInfo` reads from metadata, not files +5. **Preserve user hooks** — Never delete user configuration +6. **Exit 0 always** — Hook failure must never break the agent + +**Implementation checklist:** + +- [ ] Implement `ports.Agent` interface +- [ ] Write hook installation/removal +- [ ] Ensure all files are gitignored +- [ ] Implement `SessionInfo` from metadata +- [ ] Add unit tests +- [ ] Add integration tests +- [ ] Add conformance tests +- [ ] Register in daemon wiring diff --git a/docs/architecture.md b/docs/architecture.md index 8e5b6ffcc..4d6ea9901 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -1,103 +1,856 @@ -# Agent Orchestrator backend architecture +# Agent Orchestrator Architecture -The backend is a long-running Go daemon that supervises coding-agent sessions. -The current model is intentionally small: session rows persist only durable facts, -and display status is derived at read time. +Agent Orchestrator is a long-running Go daemon that supervises multiple parallel AI coding agent sessions. Each session runs in an isolated git worktree with its own runtime, while the daemon coordinates lifecycle, observes external state, and routes feedback. -## Mental model +## Table of Contents + +- [Mental Model](#mental-model) +- [System Overview](#system-overview) +- [Core Architectural Principles](#core-architectural-principles) +- [Component Architecture](#component-architecture) +- [Data Flows](#data-flows) +- [Persistence and CDC](#persistence-and-cdc) +- [Status Derivation](#status-derivation) +- [Lifecycle Management](#lifecycle-management) +- [Observation Loops](#observation-loops) +- [HTTP Layer](#http-layer) +- [Terminal Multiplexing](#terminal-multiplexing) + +--- + +## Mental Model + +The fundamental architecture follows a simple three-stage pipeline: + +```mermaid +flowchart LR + A[OBSERVE
External Facts] --> B[UPDATE
Durable Facts] + B --> C[DERIVE
Display Status / ACT] -``` -OBSERVE external facts → UPDATE durable facts → DERIVE display status / ACT ``` -The durable session facts are: +**Key insight:** Display status is never stored. It is computed at read time from durable facts. -- `activity_state` — what the agent last reported or what the runtime observer - can safely conclude (`active`, `idle`, `waiting_input`, `exited`). -- `is_terminated` — whether the session should be treated as over. -- PR facts in the `pr`, `pr_checks`, and `pr_comment` tables. +### Durable Session Facts -The UI status is not stored. `service.Session` computes it from the session -record plus PR facts while assembling controller-facing read models. +The only persistent session state is: -## Package layout +- `activity_state` — What the agent last reported (`active`, `idle`, `waiting_input`, `exited`) +- `is_terminated` — Whether the session should be treated as over +- PR facts — `pr`, `pr_checks`, `pr_comment` tables + +### What is NOT Durable + +Display status like `working`, `needs_input`, `ci_failed`, `mergeable` are **computed at read time** by the service layer from the durable facts above. + +--- + +## System Overview + +```mermaid +graph TB + subgraph Frontend + FE[Electron + React UI] + CLI[ao CLI] + end + + subgraph HTTP["HTTP Daemon (127.0.0.1)"] + Controllers[REST Controllers] + SSE[SSE Events] + Terminal[Terminal WebSocket] + end + + subgraph Core["Core Services"] + SessionSvc[Session Service] + ProjectSvc[Project Service] + PRSvc[PR Service] + ReviewSvc[Review Service] + SessionMgr[Session Manager] + LCM[Lifecycle Manager] + end + + subgraph Observe["Observation Layer"] + SCMObserver[SCM Observer] + Reaper[Runtime Reaper] + end + + subgraph Storage["Persistence Layer"] + SQLite[(SQLite DB)] + CDC[CDC Poller] + Broadcaster[Event Broadcaster] + end + + subgraph Adapters["Adapters"] + AgentAdapter[Agent Adapters] + RuntimeAdapter[Runtime tmux/conpty] + WorkspaceAdapter[Workspace git worktree] + SCMAdapter[SCM GitHub] + end + + FE -->|REST/SSE| Controllers + CLI -->|REST| Controllers + Controllers --> SessionSvc + Controllers --> ProjectSvc + Controllers --> PRSvc + + SessionSvc --> SessionMgr + SessionMgr --> LCM + SessionMgr --> AgentAdapter + SessionMgr --> RuntimeAdapter + SessionMgr --> WorkspaceAdapter + + LCM --> SQLite + LCM --> AgentAdapter + + SCMObserver --> SCMAdapter + SCMObserver --> SQLite + SCMObserver --> LCM + + Reaper --> RuntimeAdapter + Reaper --> SQLite + Reaper --> LCM + + CDC -->|poll| SQLite + CDC --> Broadcaster + Broadcaster --> SSE + + Terminal --> RuntimeAdapter ``` -backend/internal/domain shared vocabulary and API status value types -backend/internal/ports inbound/outbound interfaces -backend/internal/service/{project,session,pr,review} - controller-facing services and read-model assembly -backend/internal/session_manager internal session command manager -backend/internal/lifecycle runtime/activity/spawn/termination session fact reducer -backend/internal/observe/scm SCM (GitHub) observer loop feeding PR facts -backend/internal/observe/reaper runtime liveness observation loop -backend/internal/storage SQLite persistence and DB-triggered CDC -backend/internal/cdc change-log poller and broadcaster -backend/internal/httpd daemon HTTP surface (REST + SSE + terminal mux) -backend/internal/terminal WebSocket terminal multiplexer -backend/internal/adapters agent/tmux+conpty runtime/git-worktree/GitHub SCM + tracker adapters -backend/internal/daemon production wiring and shutdown -backend/internal/config daemon env/default config + +--- + +## Core Architectural Principles + +### 1. Port-Based Design + +Core code never depends on concrete implementations. All external systems are accessed through port interfaces defined in `backend/internal/ports/`: + +```mermaid +graph LR + Core[Core Services] -->|consumes| Ports[Port Interfaces] + Adapters[Adapters] -->|implement| Ports + External[External Systems] -->|wrapped by| Adapters + ``` -## Status derivation +### 2. Durable Facts, Derived Status -`service.Session` selects the display PR from all PR snapshots for a session, then -applies this rough precedence: +Storage layer persists minimal facts. Service layer computes display status on-demand: -1. `is_terminated` → `terminated`, except merged PRs display `merged`. -2. `activity_state=waiting_input` → `needs_input`. -3. Open PR facts drive PR pipeline statuses: `ci_failed`, `draft`, - `changes_requested`, `mergeable`, `approved`, `review_pending`, `pr_open`. -4. `activity_state=active` → `working`. -5. A signal-capable harness that has never sent a hook callback past the - ~90s spawn grace → `no_signal` (a broken hook pipeline is visible rather - than reported as a confident `idle`). Hook-less harnesses stay `idle`. -6. Everything else → `idle`. +```mermaid +flowchart LR + SQLite[(SQLite)] -->|raw facts| Service[Session Service] + Service -->|compute| Status[Display Status] + Service -->|enrich| UI[Dashboard/UI] -## Lifecycle manager + SQLite -->|activity_state| Service + SQLite -->|is_terminated| Service + SQLite -->|PR facts| Service + SQLite -->|runtime_handle| Service -`lifecycle.Manager` is the write path for session lifecycle facts and lifecycle-owned agent nudges: +``` -- runtime observations can mark a session terminated only when runtime and - process are both clearly dead and recent activity does not contradict that; - failed/unknown probes do not persist a special state. -- activity signals update `activity_state`; `exited` also marks the session - terminated. -- PR observations do not write PR rows here, but after the PR service persists - them lifecycle sends actionable agent nudges for CI failures, review feedback, - and merge conflicts. +### 3. Observer Pattern -## PR manager +Observation is separated from action: -`pr.Manager` records SCM observations into the PR/check/comment tables, then -forwards the observation to lifecycle for agent nudges. A merged PR marks the -owning session terminated through the lifecycle manager; other PR facts are -consumed at read time for display status. +- **Observe layer** — SCM Observer, Runtime Reaper poll external state +- **Lifecycle layer** — Reduces observations into durable facts +- **Service layer** — Computes display status from facts -## Session manager +### 4. Change Data Capture -`session_manager.Manager` performs internal session mutations: +All durable changes flow through a CDC pipeline: -- `Spawn` creates a row, creates workspace/runtime resources, and reports the - handles to the lifecycle manager. -- `Kill` marks the row terminated, then tears down runtime/workspace resources. -- `Restore` relaunches a terminated session and clears `is_terminated` via the - spawn-completed path. +```mermaid +flowchart LR + DB[(SQLite)] -->|triggers| ChangeLog[change_log table] + ChangeLog -->|tail| Poller[CDC Poller] + Poller -->|Event| Broadcaster[Event Broadcaster] + Broadcaster -->|fan-out| Subscribers[Subscribers] + Subscribers -->|SSE| Clients[Dashboard Clients] -`service.Session` is the controller-facing boundary. It delegates commands to -`session_manager.Manager` and attaches derived display status on read paths. +``` + +--- + +## Component Architecture + +### Package Layout + +``` +backend/internal/ +├── domain/ # Shared vocabulary and durable fact records +├── ports/ # Inbound/outbound interfaces +├── service/ # Controller-facing services +│ ├── project/ # Project CRUD +│ ├── session/ # Session read-model assembly +│ ├── pr/ # PR observation service +│ └── review/ # Code review service +├── session_manager/ # Internal session command engine +├── lifecycle/ # Durable session fact reducer +├── observe/ # Observation loops +│ ├── scm/ # SCM (GitHub) observer +│ └── reaper/ # Runtime liveness observer +├── storage/ # SQLite persistence +│ └── sqlite/ # DB, migrations, queries, stores +├── cdc/ # Change-log poller and broadcaster +├── httpd/ # HTTP API, controllers, terminal mux +├── terminal/ # Terminal session protocol +├── adapters/ # Concrete adapter implementations +│ ├── agent/ # 23+ agent harnesses +│ ├── runtime/ # tmux/conpty runtimes +│ ├── workspace/ # git worktree +│ ├── scm/ # GitHub +│ └── tracker/ # GitHub tracker +├── daemon/ # Production wiring +└── config/ # Environment-based configuration +``` + +### Core Data Flow + +```mermaid +sequenceDiagram + participant UI as Dashboard + participant HTTP as HTTP Controller + participant Svc as Session Service + participant Mgr as Session Manager + participant LCM as Lifecycle Manager + participant Agent as Agent Adapter + participant Runtime as Runtime Adapter + participant WS as Workspace Adapter + participant DB as SQLite + participant CDC as CDC Broadcaster + + UI->>HTTP: POST /sessions + HTTP->>Svc: Spawn(config) + Svc->>Mgr: Spawn(config) + + Note over Mgr: 1. Create session row + Mgr->>DB: Insert session + DB->>CDC: trigger change_log + CDC->>UI: SSE session.created + + Note over Mgr: 2. Create workspace + Mgr->>WS: Create(project, branch) + WS->>WS: git worktree add + + Note over Mgr: 3. Launch runtime + Mgr->>Runtime: Create(session) + Runtime->>Runtime: Start tmux/conpty + + Note over Mgr: 4. Start agent + Mgr->>Agent: GetLaunchCommand() + Agent-->>Mgr: launch command + Mgr->>Runtime: Execute(agent command) + + Note over Mgr: 5. Mark spawned + Mgr->>LCM: MarkSpawned(handle) + LCM->>DB: Update activity_state + DB->>CDC: trigger change_log + CDC->>UI: SSE session.updated + + Mgr-->>Svc: Session(created) + Svc-->>HTTP: Session response + HTTP-->>UI: 201 Created +``` + +--- + +## Data Flows + +### Session Spawn Flow + +```mermaid +flowchart TD + Start([User spawns session]) --> Validate[Validate project config] + Validate --> CreateRow[Create session row in SQLite] + CreateRow --> CreateWS[Create git worktree] + CreateWS --> CreateRT[Launch runtime tmux/conpty] + CreateRT --> GetCmd[Get agent launch command] + GetCmd --> ExecAgent[Execute agent in runtime] + ExecAgent --> MarkSpawned[MarkSpawned in LCM] + MarkSpawned --> Trigger1[CDC: session.created] + Trigger1 --> Trigger2[CDC: session.updated] + Trigger2 --> Done([Session running]) + +``` + +### Observation Flow + +```mermaid +flowchart TD + subgraph SCM["SCM Observer Loop"] + Poll1[Poll PRs every 30s] + Poll1 --> Fetch[Fetch from GitHub API] + Fetch --> Diff[Semantic diff vs local] + Diff --> Changed{Changed?} + Changed -->|Yes| WritePR[Write PR/check/comment] + Changed -->|No| Wait1[Wait for tick] + WritePR --> NotifyLCM[Notify Lifecycle Manager] + NotifyLCM --> Trigger1[CDC event] + Trigger1 --> Wait1 + Wait1 --> Poll1 + end + + subgraph Reaper["Runtime Reaper Loop"] + Poll2[Poll every 5s] + Poll2 --> Probe[Probe each runtime] + Probe --> Report[Report fact to LCM] + Report --> Trigger2[CDC event] + Trigger2 --> Wait2[Wait for tick] + Wait2 --> Poll2 + end + + LCM[Lifecycle Manager] -->|consumes| NotifyLCM + LCM -->|consumes| Report + +``` + +### Feedback Routing Flow + +```mermaid +sequenceDiagram + participant SCM as SCM Observer + participant LCM as Lifecycle Manager + participant Agent as Agent Adapter + participant Runtime as Runtime Adapter + + SCM->>SCM: Observe PR comment + SCM->>LCM: ApplySCMObservation() + LCM->>LCM: Detect actionable feedback + LCM->>Agent: SendNudge(feedback) + + SCM->>SCM: Observe CI failure + SCM->>LCM: ApplySCMObservation() + LCM->>LCM: Detect actionable feedback + LCM->>Agent: SendNudge(CI failure) + + SCM->>SCM: Observe merge conflict + SCM->>LCM: ApplySCMObservation() + LCM->>LCM: Detect actionable feedback + LCM->>Agent: SendNudge(merge conflict) + + Note over Agent,Runtime: Agent receives nudges via
runtime messages or hooks +``` + +--- ## Persistence and CDC -SQLite is the durable store. User-visible table changes are captured by database -triggers into `change_log`; the Go store does not manually emit CDC events. A -poller tails `change_log` and publishes live events to in-process subscribers. +### SQLite Schema -## Load-bearing rules +```mermaid +erDiagram + projects ||--o{ sessions : owns + sessions ||--o{ pull_requests : owns + pull_requests ||--o{ pr_checks : has + pull_requests ||--o{ pr_review_threads : has + pull_requests ||--o{ pr_comments : has + sessions ||--o{ notifications : has + change_log }|--|| projects : tracks + change_log }|--|| sessions : tracks + change_log }|--|| pull_requests : tracks -- Do not store display status. -- Keep session status facts small: `activity_state`, `is_terminated`, and PR - facts are the durable inputs. -- Do not treat failed probes as death. -- Do not force-delete registered dirty worktrees. + projects { + string id PK + string name + string repo + jsonb config + } + + sessions { + string id PK + string project_id FK + string harness + string activity_state + boolean is_terminated + jsonb metadata + } + + pull_requests { + string id PK + string session_id FK + integer number + string state + string title + boolean draft + boolean mergeable + } + + pr_checks { + string id PK + string pr_id FK + string name + string status + string conclusion + } + + change_log { + bigint seq PK + string table_name + string row_id + string operation + jsonb old_data + jsonb new_data + } +``` + +### CDC Pipeline + +```mermaid +flowchart LR + DB[(SQLite)] -->|INSERT/UPDATE/DELETE| Trigger[DB Trigger] + Trigger -->|append| ChangeLog[change_log] + ChangeLog -->|poll| Poller[CDC Poller] + Poller -->|decode| Decoder[Event Decoder] + Decoder -->|Event| Broadcaster[Broadcaster] + Broadcaster -->|callback| Sub1[Terminal Fanout] + Broadcaster -->|callback| Sub2[SSE Writer] + Broadcaster -->|callback| Sub3[Cache Invalidation] + + Poller -->|watermark| Watermark[seq tracking] + Watermark -->|resume position| Poller + +``` + +--- + +## Status Derivation + +### Display Status Precedence + +The `service.Session` computes display status from durable facts using this precedence (highest to lowest): + +```mermaid +flowchart TD + CheckTerm{is_terminated?} + CheckTerm -->|Yes| PRMerged{PR merged?} + CheckTerm -->|No| CheckWait{activity_state
== waiting_input?} + + PRMerged -->|Yes| Merged[merged] + PRMerged -->|No| Terminated[terminated] + + CheckWait -->|Yes| NeedsInput[needs_input] + CheckWait -->|No| CheckPR{Has PR facts?} + + CheckPR -->|Yes| PRPipeline[PR Pipeline Check] + CheckPR -->|No| CheckActive{activity_state
== active?} + + PRPipeline --> PRState{PR State} + PRState -->|ci failed| CIFailed[ci_failed] + PRState -->|draft| Draft[draft] + PRState -->|changes requested| Changes[changes_requested] + PRState -->|not mergeable| Conflict[merge_conflict] + PRState -->|mergeable| Mergeable[mergeable] + PRState -->|approved| Approved[approved] + PRState -->|review pending| ReviewPending[review_pending] + PRState -->|open| PROpen[pr_open] + + CheckActive -->|Yes| Working[working] + CheckActive -->|No| CheckSignal{Signal capable
&& no signal?} + + CheckSignal -->|Yes| NoSignal[no_signal] + CheckSignal -->|No| Idle[idle] + +``` + +### PR Pipeline States + +```mermaid +flowchart LR + PR[Open PR] --> CI{CI Status} + CI -->|failing| CIFailed[ci_failed] + CI -->|pending| CIPending[ci_pending] + CI -->|passing| Review{Reviews} + + Review -->|changes requested| Changes[changes_requested] + Review -->|approved| Mergeable{Mergeable?} + + Mergeable -->|conflict| Conflict[merge_conflict] + Mergeable -->|yes| Merged[Mergeable] + + PR -.->|draft| Draft[Draft State] + +``` + +--- + +## Lifecycle Management + +### Lifecycle Manager Responsibilities + +The `lifecycle.Manager` is the **canonical write path** for all session lifecycle facts: + +```mermaid +flowchart TD + subgraph Inputs["Observation Inputs"] + RuntimeObs[Runtime Observations] + ActivitySignals[Agent Activity Signals] + SCMObs[SCM Observations] + end + + subgraph LCM["Lifecycle Manager"] + Reducer[Fact Reducer] + StateMachine[Activity State Machine] + Termination[Termination Logic] + Nudge[Agent Nudge Engine] + end + + subgraph Outputs["Durable Facts"] + ActivityState[activity_state] + IsTerminated[is_terminated] + PRFacts[PR Facts Table] + end + + RuntimeObs --> Reducer + ActivitySignals --> Reducer + SCMObs --> Reducer + + Reducer --> StateMachine + StateMachine --> Termination + Termination --> ActivityState + Termination --> IsTerminated + + SCMObs --> Nudge + Nudge -->|route| Agent[Agent Adapter] + +``` + +### Session State Machine + +```mermaid +stateDiagram-v2 + [*] --> Spawning: Spawn() + Spawning --> Active: MarkSpawned + Active --> Idle: activity_state = idle + Active --> Working: activity_state = active + Active --> Waiting: activity_state = waiting_input + Active --> Exited: activity_state = exited + Working --> Active: work completes + Waiting --> Active: user responds + Idle --> Active: agent starts work + Exited --> Terminated: process exit + Active --> Terminated: Kill() + Waiting --> Terminated: Kill() + Idle --> Terminated: Kill() + Terminated --> [*] + + note right of Active + Agent is working + Runtime alive + end note + + note right of Waiting + Agent needs input + Waiting for user + end note + + note right of Terminated + Session over + Runtime cleaned up + end note +``` + +### Termination Guardrails + +The lifecycle manager only terminates when **all** conditions are met: + +```mermaid +flowchart TD + Check{Can terminate?} + Check -->|No| Keep[Keep running] + + Check -->|Yes| AllDead{Runtime AND
process dead?} + AllDead -->|No| Keep + AllDead -->|Yes| NoRecent{No recent
activity?} + NoRecent -->|No| Keep + NoRecent -->|Yes| NoPR{No merged PR
ownership?} + NoPR -->|No| Keep + NoPR -->|Yes| Terminate[Mark terminated] + + Terminate --> Cleanup[Trigger cleanup] + Cleanup --> CDC[CDC event] + CDC --> UI[Dashboard update] + +``` + +**Key principle:** Failed probes are NOT proof of death. A session is only terminated when the runtime and process are **both** clearly dead and recent activity doesn't contradict that. + +--- + +## Observation Loops + +### SCM Observer + +```mermaid +flowchart TD + Start([Observer Start]) --> Immediate[Immediate Poll] + Immediate --> Loop{Tick every 30s} + + Loop --> ListRepos[List active repos] + ListRepos --> CheckCreds{Credentials
available?} + CheckCreds -->|No| Disabled[Disabled mode] + CheckCreds -->|Yes| Fetch[Fetch PRs via ETags] + + Fetch --> ListPRs[List open PRs] + ListPRs --> Discover[Discover new PRs] + Discover --> FetchDetailed[Fetch detailed PR data] + FetchDetailed --> FetchChecks[Fetch CI checks] + FetchChecks --> FetchReviews[Fetch review threads] + + FetchReviews --> Write[Write to SQLite] + Write --> Notify[Notify Lifecycle] + Notify --> Trigger[CDC event] + + Disabled --> Loop + Trigger --> Loop + +``` + +### Runtime Reaper + +```mermaid +flowchart TD + Start([Reaper Start]) --> Loop{Tick every 5s} + + Loop --> List[List non-terminated
sessions] + List --> ForEach[For each session] + + ForEach --> GetHandle{Has runtime
handle?} + GetHandle -->|No| Skip[Skip session] + GetHandle -->|Yes| Probe[Probe runtime] + + Probe --> Result{Probe result} + Result -->|Error| ReportFailed[Report ProbeFailed] + Result -->|Alive| ReportAlive[Report ProbeAlive] + Result -->|Dead| ReportDead[Report ProbeDead] + + ReportFailed --> Apply[ApplyRuntimeObservation] + ReportAlive --> Apply + ReportDead --> Apply + + Apply --> LCM[Lifecycle Manager] + LCM --> Update[Update facts] + Update --> CDC[CDC event] + + Skip --> NextSession{More sessions?} + CDC --> NextSession + NextSession -->|Yes| ForEach + NextSession -->|No| Loop + +``` + +### Observation Integration + +```mermaid +flowchart LR + subgraph External["External State"] + GitHub[GitHub API] + Runtimes[tmux/conpty] + end + + subgraph Observers["Observation Layer"] + SCM[SCM Observer] + Reaper[Runtime Reaper] + end + + subgraph Core["Core Processing"] + LCM[Lifecycle Manager] + PRMgr[PR Manager] + end + + subgraph Storage["Persistence"] + SQLite[(SQLite)] + end + + GitHub --> SCM + Runtimes --> Reaper + + SCM --> PRMgr + PRMgr --> SQLite + PRMgr --> LCM + + Reaper --> LCM + LCM --> SQLite + +``` + +--- + +## HTTP Layer + +### API Structure + +```mermaid +flowchart TD + subgraph HTTPD["HTTP Daemon"] + Router[Router + Middleware] + + Router --> API[REST API] + Router --> Events[SSE Events] + Router --> Terminal[Terminal WebSocket] + end + + subgraph Controllers["Controllers"] + Sessions[Sessions Controller] + Projects[Projects Controller] + PRs[PRs Controller] + Reviews[Reviews Controller] + end + + subgraph Services["Services"] + SessionSvc[Session Service] + ProjectSvc[Project Service] + PRSvc[PR Service] + ReviewSvc[Review Service] + end + + API --> Sessions + API --> Projects + API --> PRs + API --> Reviews + + Sessions --> SessionSvc + Projects --> ProjectSvc + PRs --> PRSvc + Reviews --> ReviewSvc + + Events -->|subscribe| CDC[CDC Broadcaster] + Terminal --> TerminalMux[Terminal Manager] + +``` + +### Request Flow + +```mermaid +sequenceDiagram + participant Client + participant Router + participant Controller + participant Service + participant Manager + participant Store + participant DB + + Client->>Router: POST /api/v1/sessions + Router->>Router: Middleware (auth, logging) + Router->>Controller: handler(w, r) + Controller->>Controller: decode JSON + Controller->>Service: Spawn(config) + Service->>Manager: Spawn(config) + Manager->>Store: Create session + Store->>DB: INSERT INTO sessions + DB->>Store: session record + Store->>Manager: session record + Manager->>Manager: Create workspace + Manager->>Manager: Launch runtime + Manager->>Service: Session response + Service->>Controller: enriched session + Controller->>Controller: encode JSON + Controller->>Client: 201 Created + Session +``` + +--- + +## Terminal Multiplexing + +### Terminal Architecture + +```mermaid +flowchart TD + subgraph Frontend + Browser[Browser Terminal] + end + + subgraph HTTPD + WS[WebSocket Handler] + end + + subgraph Terminal + Mux[Terminal Mux] + Sessions[Session States] + end + + subgraph Runtime + TMux[tmux Runtime] + ConPTY[conpty Runtime] + end + + Browser -->|WebSocket| WS + WS -->|attach| Mux + Mux --> Sessions + Sessions -->|create| TMux + Sessions -->|create| ConPTY + + TMux -->|PTY attach| Mux + ConPTY -->|loopback dial| Mux + + Mux -->|frame| WS + WS -->|binary| Browser + +``` + +### Attach Flow + +```mermaid +sequenceDiagram + participant Client as Browser + participant WS as WebSocket Handler + participant Mux as Terminal Mux + participant Runtime as tmux/conpty + + Client->>WS: WebSocket upgrade + WS->>Mux: Attach(session, rows, cols) + Mux->>Runtime: Attach(handle, rows, cols) + + Runtime->>Runtime: Create PTY + Runtime->>Runtime: Spawn tmux attach + + loop Data Loop + Runtime->>Mux: PTY output + Mux->>WS: Binary frame + WS->>Client: WebSocket message + + Client->>WS: User input + WS->>Mux: Input frame + Mux->>Runtime: Write to PTY + end + + Client->>WS: Close + WS->>Mux: Detach + Mux->>Runtime: Close PTY +``` + +--- + +## Load-Bearing Rules + +These rules are **load-bearing** — changing them breaks fundamental architectural assumptions: + +1. **Never store display status** — Status is derived from durable facts at read time +2. **Never treat failed probes as death** — A failed probe is a fact, not a termination signal +3. **Never force-delete dirty worktrees** — User data safety over cleanup convenience +4. **All app state under ~/.ao** — No OS-default app-data locations +5. **Daemon binds to 127.0.0.1 only** — No network exposure, ever +6. **CLI is thin** — All logic lives in the daemon, CLI is just an HTTP client +7. **CDC is source-truth for events** — DB triggers write to change_log, poller fans out +8. **Adapters are leaves** — Adapters never import core packages, only ports and domain +9. **Hooks are gitignored** — Every file an adapter writes must be in .gitignore +10. **Migrations never change** — Add new migrations, never modify existing ones + +--- + +## Summary + +Agent Orchestrator's architecture is designed around: + +- **Separation of concerns** — Observation, persistence, and display are distinct layers +- **Port-based design** — Core code depends on interfaces, not implementations +- **Durable minimalism** — Store only facts, compute everything else +- **Event-driven updates** — CDC broadcasts changes to all subscribers +- **Isolation** — Each session in its own worktree with its own runtime +- **Safety** — Conservative termination, path validation, gitignored hooks + +This architecture enables parallel AI agents to work safely while maintaining complete visibility and control. diff --git a/docs/backend-code-structure.md b/docs/backend-code-structure.md index 2fc1a7db8..1910cb2b7 100644 --- a/docs/backend-code-structure.md +++ b/docs/backend-code-structure.md @@ -1,392 +1,942 @@ # Backend Code Structure -This document describes package ownership for the Go backend. It is about where -code belongs. See [architecture.md](architecture.md) for lifecycle behavior, -status derivation, persistence, CDC, and invariants. +This document describes package ownership for the Go backend. It defines where code belongs, how packages interact, and the architectural boundaries that keep the system maintainable. -## Goal +## Table of Contents -The backend is a local daemon that supervises coding-agent sessions. The code -needs clear homes for product workflows, protocol surfaces, persistence, and -replaceable external systems without turning any single package into a catch-all. +- [Overview](#overview) +- [Architecture Layers](#architecture-layers) +- [Package-by-Package Ownership](#package-by-package-ownership) +- [Interface Placement Rules](#interface-placement-rules) +- [Import Graph](#import-graph) +- [Adding New Code](#adding-new-code) +- [Examples](#examples) -The current structure is a layered hybrid: +--- -- `domain` holds shared product vocabulary and durable fact records. -- `service/*` owns controller-facing product use cases and read models. -- `session_manager` owns internal session mutations and resource orchestration. -- `lifecycle` owns the durable session fact reducer. -- `ports` defines narrow capability interfaces consumed by core code. -- `adapters/*` implements those capabilities with real external systems. -- `storage/sqlite` and `cdc` own persistence and change delivery. -- `httpd` and `cli` own protocol concerns. -- `daemon` wires the production graph together. +## Overview -## Package Roles +The backend is a **layered hybrid** architecture with clear separation between core business logic and external concerns: + +```mermaid +graph TB + subgraph CLI["CLI Layer"] + CLI[internal/cli] + end + + subgraph HTTP["HTTP Layer"] + HTTPD[internal/httpd] + end + + subgraph Services["Service Layer"] + Project[internal/service/project] + Session[internal/service/session] + PR[internal/service/pr] + Review[internal/service/review] + end + + subgraph Core["Core Layer"] + SessionMgr[internal/session_manager] + Lifecycle[internal/lifecycle] + Observe[internal/observe/*] + end + + subgraph Data["Data Layer"] + Domain[internal/domain] + Ports[internal/ports] + Storage[internal/storage/sqlite] + CDC[internal/cdc] + end + + subgraph Infra["Infrastructure Layer"] + Terminal[internal/terminal] + Adapters[internal/adapters/*] + Daemon[internal/daemon] + Config[internal/config] + end + + CLI -->|calls| HTTPD + HTTPD -->|calls| Services + Services -->|calls| Core + Services -->|uses| Data + Core -->|uses| Data + Core -->|uses| Infra + HTTPD -->|uses| Data + +``` + +### Key Architectural Principles + +1. **Domain stays pure** — No infrastructure dependencies +2. **Ports define contracts** — Interfaces consumed by core, implemented by adapters +3. **Services orchestrate** — Controller-facing use cases over core and data +4. **Adapters are leaves** — Implement ports, don't import core +5. **CLI/HTTP stay thin** — Just protocol handling, all logic in daemon + +--- + +## Architecture Layers + +### Layer Interactions + +```mermaid +flowchart LR + Protocol[Protocol Layer
CLI + HTTP] -->|uses| Services[Service Layer
Use Cases] + Services -->|commands| Core[Core Layer
Session Manager + Lifecycle] + Services -->|queries| Data[Data Layer
Domain + Ports + Storage] + Core -->|reads/writes| Data + Core -->|invokes| Infra[Infrastructure
Adapters + Terminal] + Infra -->|implements| Ports + +``` + +### Dependency Rules + +```mermaid +graph TD + Direction[Dependency Direction] + Direction --> Down["Top-down only"] + Direction --> No["No upward dependencies"] + + CLI[CLI] -->|OK| HTTP[HTTP] + HTTP -->|OK| Services[Services] + Services -->|OK| Core[Core] + Services -->|OK| Storage[Storage] + Core -->|OK| Adapters[Adapters] + + Bad1[Adapters] -.->|FORBIDDEN| Services + Bad2[Storage] -.->|FORBIDDEN| HTTP + Bad3[Core] -.->|FORBIDDEN| CLI + +``` + +--- + +## Package-by-Package Ownership ### `internal/domain` -`domain` is AO's shared product language. Keep it stable and free of -infrastructure imports. +**Purpose:** Shared product vocabulary and durable fact records. The single source of truth for domain concepts. -Belongs here: +```mermaid +graph TD + Domain[internal/domain] --> Contains[Contains] + Contains --> IDs[Shared IDs
ProjectID, SessionID, IssueID] + Contains --> Enums[Status Enums
SessionStatus, ActivityState] + Contains --> Records[Durable Records
SessionRecord, PRRecord] + Contains --> Vocab[Product Vocabulary
PR, Project, Review concepts] -- shared IDs such as `ProjectID`, `SessionID`, and `IssueID`; -- shared enums and status vocabulary; -- durable fact records that multiple packages must agree on; -- PR, tracker, project, and session vocabulary that is not transport-specific. + DoesNot[Does NOT Contain] --> HTTP[HTTP DTOs] + DoesNot --> CLI[CLI Output] + DoesNot --> Generated[sqlc Generated Rows] + DoesNot --> External[External Payloads
GitHub, Claude, etc.] -Does not belong here: - -- HTTP request/response DTOs; -- CLI output shapes; -- OpenAPI wrapper/envelope types; -- sqlc generated rows; -- GitHub, tmux, Claude, Codex, or OpenCode payloads; -- one-resource controller helper types. - -Rule of thumb: if AO would still use the concept after replacing HTTP, the CLI, -SQLite, GitHub, the tmux/conpty runtime, and every agent adapter, and more than one package needs -the exact vocabulary, it may belong in `domain`. - -### `internal/service/*` - -`service` packages are the controller-facing application boundary. - -Current examples: - -```txt -internal/service/project -internal/service/session -internal/service/pr -internal/service/review ``` -Belongs here: +**Belongs here:** +- Shared IDs: `ProjectID`, `SessionID`, `IssueID` +- Enums and status vocabulary +- Durable fact records used across packages +- PR, tracker, project, session vocabulary -- resource use cases called by HTTP controllers and CLI-backed API flows; -- resource read models and command/result types; -- display-model assembly, such as session status derived from session and PR - facts; -- resource-specific validation and user-facing errors; -- small store interfaces consumed by the service. +**Does NOT belong here:** +- HTTP request/response DTOs +- CLI output shapes +- OpenAPI wrapper types +- sqlc generated rows +- External system payloads (GitHub, tmux, agent-specific) -Does not belong here: +**Rule of thumb:** If AO would still use the concept after replacing HTTP, CLI, SQLite, GitHub, tmux, and every agent adapter, it belongs in domain. -- low-level runtime/workspace/agent process control; -- raw sqlc generated rows as public service results; -- HTTP routing, path parsing, status-code decisions, or OpenAPI generation; -- concrete external adapter details. - -For example, project API concepts live in `internal/service/project`, not in -`domain` and not in a top-level `internal/project` package. - -### `internal/session_manager` - -`session_manager` owns internal session commands: spawn, restore, kill, cleanup, -and send-related orchestration over runtime, workspace, agent, storage, -messenger, and lifecycle dependencies. - -Belongs here: - -- multi-step session mutations; -- rollback/cleanup sequencing when spawn partially succeeds; -- resource teardown safety; -- internal errors such as not found, terminated, or not restorable. - -Does not belong here: - -- HTTP request decoding; -- CLI formatting; -- controller-facing list/get read-model assembly; -- terminal WebSocket framing. - -The split is intentional: `service/session` is the product/API boundary; -`session_manager` is the internal command engine. - -### `internal/lifecycle` - -`lifecycle` is the canonical write path for durable session lifecycle facts. It -reduces runtime observations, activity signals, spawn completion, termination, -and PR observations into small persisted facts. - -Belongs here: - -- updates to lifecycle-owned session facts; -- guardrails around runtime/activity observations; -- lifecycle-triggered agent nudges for actionable PR facts. - -Does not belong here: - -- display status persistence; -- HTTP/CLI DTOs; -- direct adapter implementation details; -- PR row persistence. - -The UI status is derived at read time by service code. Do not store display -status in lifecycle or SQLite. +--- ### `internal/ports` -`ports` contains narrow capability interfaces and shared adapter-facing structs. -It connects core code to replaceable systems. +**Purpose:** Narrow capability interfaces that connect core code to replaceable external systems. -Current capability examples: +```mermaid +graph LR + Core[Core Code] -->|consumes| Ports[Ports Interfaces] + Adapters[Adapters] -->|implement| Ports -- `Runtime` -- `Workspace` -- `Agent` -- `AgentResolver` -- `AgentMessenger` -- `PRWriter` + Ports --> Examples[Examples] + Examples --> Runtime[Runtime
Create, Destroy, IsAlive, Attach] + Examples --> Workspace[Workspace
Create, Destroy, ValidatePath] + Examples --> Agent[Agent
GetLaunchCommand, GetAgentHooks] + Examples --> SCM[SCM
ListPRs, FetchPR, FetchChecks] + Examples --> PR[PR
WriteSCMObservation] -Belongs here: - -- interfaces consumed by core packages and implemented by adapters; -- capability structs such as `RuntimeConfig`, `WorkspaceConfig`, and - `SpawnConfig`; -- vocabulary needed at the boundary between core orchestration and adapters. - -Does not belong here: - -- resource read models like project/session API responses; -- HTTP request/response DTOs; -- sqlc rows; -- concrete adapter options; -- one-off interfaces that only a single package needs internally. - -Keep `ports` capability-oriented. It should not become the dumping ground for -every manager, DTO, and resource contract. - -### `internal/adapters/*` - -Adapters are concrete implementations of external systems. - -Current examples: - -```txt -internal/adapters/agent/claudecode -internal/adapters/agent/codex -internal/adapters/agent/opencode -internal/adapters/runtime/tmux -internal/adapters/runtime/conpty -internal/adapters/runtime/runtimeselect -internal/adapters/runtime/ptyexec -internal/adapters/workspace/gitworktree -internal/adapters/scm/github -internal/adapters/tracker/github ``` -Adapters should be leaves in the import graph. They translate external behavior -into AO ports and domain concepts; they should not own product workflows. +**Belongs here:** +- Interfaces consumed by core packages, implemented by adapters +- Capability structs: `RuntimeConfig`, `WorkspaceConfig`, `SpawnConfig` +- Vocabulary at the boundary between core and adapters -Good: +**Does NOT belong here:** +- Resource read models (belongs in `service/*`) +- HTTP request/response DTOs (belongs in `httpd`) +- sqlc rows (belongs in `storage/sqlite`) +- One-off internal interfaces + +**Key Port Interfaces:** + +| Port | Purpose | Implementations | +|------|---------|-----------------| +| `Runtime` | Process isolation | `tmux`, `conpty` | +| `Workspace` | Git worktree management | `gitworktree` | +| `Agent` | Agent launching | 23+ agent adapters | +| `SCM` | PR/CI observation | `github` | +| `Tracker` | Issue tracking | `github` (adapter only) | +| `AgentMessenger` | Agent communication | Agent hooks | +| `PRWriter` | PR persistence | `pr.Manager` | + +--- + +### `internal/service/*` + +**Purpose:** Controller-facing application boundary. Owns product use cases and read-model assembly. + +```mermaid +graph TD + subgraph Services + Project[project] + Session[session] + PR[pr] + Review[review] + end + + subgraph Responsibilities + UseCases[Use Cases] + ReadModels[Read Models] + Validation[Validation] + Errors[User-Facing Errors] + end + + Project --> Responsibilities + Session --> Responsibilities + PR --> Responsibilities + Review --> Responsibilities + + subgraph NotHere + LowLevel[Low-level runtime control] + RawSQL[Raw sqlc rows] + HTTP[HTTP routing] + end -```txt -session_manager -> ports.Runtime -adapters/runtime/tmux -> ports + domain -adapters/workspace/gitworktree -> ports + domain -daemon -> adapters + services + storage ``` -Avoid: +**Current service packages:** + +```mermaid +graph LR + Controllers[HTTP Controllers] -->|call| Services + + Services --> Project[internal/service/project
Project CRUD] + Services --> Session[internal/service/session
Session read-model assembly] + Services --> PR[internal/service/pr
PR observation/actions] + Services --> Review[internal/service/review
Code review] + + Services -->|delegate to| SessionMgr[session_manager] + Services -->|query| Store[storage stores] -```txt -domain -> adapters -service/session -> adapters/runtime/tmux -httpd/controllers -> storage/sqlite/store -adapters/* -> httpd ``` +**Belongs here:** +- Resource use cases called by HTTP controllers and CLI +- Resource read models and command/result types +- Display-model assembly (e.g., session status derivation) +- Resource-specific validation and user-facing errors +- Small store interfaces consumed by the service + +**Does NOT belong here:** +- Low-level runtime/workspace/agent process control +- Raw sqlc generated rows as public results +- HTTP routing, path parsing, status-code decisions +- Concrete external adapter details + +**Example:** Project concepts live in `internal/service/project`, not in `domain` and not in `internal/project`. + +--- + +### `internal/session_manager` + +**Purpose:** Internal session command engine. Owns multi-step session mutations and resource orchestration. + +```mermaid +graph TD + Service[service/session] -->|commands| Mgr[session_manager] + Mgr -->|orchestrates| Resources[Resources] + + Resources --> Workspace[Workspace Adapter] + Resources --> Runtime[Runtime Adapter] + Resources --> Agent[Agent Adapter] + Resources --> Storage[Storage Store] + Resources --> Lifecycle[Lifecycle Manager] + Resources --> Messenger[Agent Messenger] + + Mgr -->|owns| Operations[Operations] + Operations --> Spawn[Spawn
Create all resources] + Operations --> Kill[Kill
Teardown all resources] + Operations --> Restore[Restore
Relaunch terminated session] + Operations --> Send[Send
Message to agent] + +``` + +**Belongs here:** +- Multi-step session mutations with rollback +- Resource sequencing (workspace → runtime → agent) +- Resource teardown safety and cleanup +- Internal errors: not found, terminated, not restorable + +**Does NOT belong here:** +- HTTP request decoding +- CLI formatting +- Controller-facing list/get read-model assembly +- Terminal WebSocket framing + +**Intentional split:** `service/session` is the product/API boundary; `session_manager` is the internal command engine. + +--- + +### `internal/lifecycle` + +**Purpose:** Canonical write path for durable session lifecycle facts. Reduces observations into minimal persisted state. + +```mermaid +graph LR + subgraph Inputs + RuntimeObs[Runtime Observations
from Reaper] + ActivitySignals[Activity Signals
from Agent Hooks] + SCMObs[SCM Observations
from SCM Observer] + end + + subgraph Lifecycle + LCM[Lifecycle Manager] + Reducer[Fact Reducer] + StateMachine[State Machine] + Nudge[Agent Nudge Engine] + end + + subgraph Outputs + ActivityState[activity_state] + IsTerminated[is_terminated] + PRFacts[PR Facts] + Nudges[Agent Nudges] + end + + RuntimeObs --> LCM + ActivitySignals --> LCM + SCMObs --> LCM + + LCM --> Reducer + Reducer --> StateMachine + StateMachine --> ActivityState + StateMachine --> IsTerminated + + SCMObs --> Nudge + Nudge --> Nudges + +``` + +**Belongs here:** +- Updates to lifecycle-owned session facts +- Guardrails around runtime/activity observations +- Lifecycle-triggered agent nudges for actionable PR facts + +**Does NOT belong here:** +- Display status persistence (use service layer instead) +- HTTP/CLI DTOs +- Direct adapter implementation details +- PR row persistence (use `pr.Manager`) + +**Key invariant:** The UI status is derived at read time by service code. Do not store display status in lifecycle or SQLite. + +--- + +### `internal/observe/*` + +**Purpose:** Observation loops that poll external state and report facts to lifecycle. + +```mermaid +graph TD + subgraph Observe + SCM[observe/scm
SCM Observer] + Reaper[observe/reaper
Runtime Reaper] + end + + subgraph External + GitHub[GitHub API] + Runtimes[tmux/conpty] + end + + subgraph Internal + LCM[Lifecycle Manager] + Store[SQLite Store] + end + + SCM -->|polls| GitHub + SCM -->|writes| Store + SCM -->|notifies| LCM + + Reaper -->|probes| Runtimes + Reaper -->|reports to| LCM + +``` + +**Current observation packages:** +- `internal/observe/scm` — SCM (GitHub) observer loop +- `internal/observe/reaper` — Runtime liveness observation loop + +**Belongs here:** +- Polling loops and observation logic +- External state transformation into domain facts +- Observation error handling and retry logic + +**Does NOT belong here:** +- Product workflow decisions (belongs in service layer) +- Direct storage writes (use lifecycle instead) + +--- + ### `internal/storage/sqlite` -`storage/sqlite` owns SQLite setup, migrations, sqlc generated code, and store -implementations. +**Purpose:** SQLite setup, migrations, queries, and store implementations. -Belongs here: +```mermaid +graph TD + Storage[storage/sqlite] --> Components[Components] -- connection setup and PRAGMAs; -- goose migrations; -- sqlc queries and generated code; -- table-specific store methods; -- transactions and CDC-triggered persistence behavior. + Components --> Setup[Connection Setup
PRAGMAs] + Components --> Migrations[Goose Migrations] + Components --> SQLC[sqlc Queries + Generated Code] + Components --> Stores[Table-specific Stores] + Components --> Trans[Transactions + CDC] -Does not belong here: + Components -->|NOT| HTTP[HTTP Response Types] + Components -->|NOT| CLI[CLI Formatting] + Components -->|NOT| Product[Product Display Status Rules] -- HTTP response types; -- CLI output formatting; -- product display status rules; -- external adapter logic. +``` -Generated sqlc types should stay behind store methods. Services and lifecycle -code should work with domain records or service read models, not generated rows. +**Belongs here:** +- Connection setup and PRAGMAs +- Goose migrations +- sqlc queries and generated code +- Table-specific store methods +- Transactions and CDC-triggered persistence behavior + +**Does NOT belong here:** +- HTTP response types +- CLI output formatting +- Product display status rules +- External adapter logic + +**Rule:** Generated sqlc types should stay behind store methods. Services should work with domain records or service read models, not generated rows. + +--- ### `internal/cdc` -`cdc` owns `change_log` polling and event broadcasting. SQLite triggers append -durable events to `change_log`; the poller tails that table and fans events out -to subscribers. +**Purpose:** Change-log polling and event broadcasting. -Belongs here: +```mermaid +graph LR + SQLite[(SQLite)] -->|triggers| ChangeLog[change_log] + ChangeLog -->|tail| Poller[CDC Poller] + Poller -->|Event| Broadcaster[Broadcaster] + Broadcaster -->|fan-out| Subs[Subscribers] -- event type definitions for the CDC stream; -- poller and broadcaster logic; -- subscriber fan-out behavior. + Subs --> SSE[SSE Writer] + Subs --> Term[Terminal Fanout] + Subs --> Cache[Cache Invalidation] -Does not belong here: +``` -- terminal byte streams; -- product workflow decisions; -- database schema ownership. +**Belongs here:** +- Event type definitions for the CDC stream +- Poller and broadcaster logic +- Subscriber fan-out behavior + +**Does NOT belong here:** +- Terminal byte streams (belongs in `internal/terminal`) +- Product workflow decisions (belongs in service layer) +- Database schema ownership (belongs in `storage/sqlite`) + +--- ### `internal/terminal` -`terminal` owns the terminal session protocol and PTY attach management used by -the HTTP mux. The runtime is selected by `runtimeselect`: tmux on Darwin/Linux, -conpty on Windows. Every client that opens a pane gets its own attach Stream. On -unix, tmux spawns `tmux attach` on a local PTY via ptyexec; on Windows, conpty -dials the session's loopback pty-host directly. Either way the runtime owns -screen state and scrollback and replays its init handshake plus a full repaint -per attach, so there is no shared per-pane buffer. +**Purpose:** Terminal session protocol and PTY attach management used by the HTTP terminal mux. -Belongs here: +```mermaid +graph TD + HTTP[httpd] -->|WebSocket| Terminal[terminal] + Terminal -->|creates| Attach[Attach Streams] + Attach -->|wraps| PTY[PTY Sessions] -- per-client attachment lifecycle (liveness gating, re-attach backoff); -- input/output framing independent of HTTP; -- PTY-backed attach handling and terminal protocol tests. + PTY --> Unix[Unix: tmux attach
via ptyexec] + PTY --> Windows[Windows: conpty
loopback dial] -`httpd` adapts WebSocket connections to terminal interfaces; `terminal` should -not import `httpd`. + Terminal -->|manages| State[Session States] + State --> Liveness[Liveness gating] + State --> Backoff[Re-attach backoff] + +``` + +**Belongs here:** +- Per-client attachment lifecycle +- Input/output framing independent of HTTP +- PTY-backed attach handling and terminal protocol tests + +**Does NOT belong here:** +- HTTP-specific concerns (belongs in `httpd`) +- HTTP routing or WebSocket upgrade logic + +**Note:** `httpd` adapts WebSocket connections to terminal interfaces. `terminal` should not import `httpd`. + +--- ### `internal/httpd` -`httpd` is the HTTP protocol adapter. +**Purpose:** HTTP protocol adapter. Handles routing, middleware, and request/response encoding. -Belongs here: +```mermaid +graph TD + HTTPD[httpd] --> Components[Components] -- routing and middleware; -- HTTP request decoding and response encoding; -- path/query parameter handling; -- status-code mapping; -- API error envelopes; -- OpenAPI generation and serving; -- WebSocket upgrade handling for terminal mux. + Components --> Routing[Routing + Middleware] + Components --> Decode[Request Decoding] + Components --> Encode[Response Encoding] + Components --> Errors[API Error Envelopes] + Components --> OpenAPI[OpenAPI Generation] + Components --> WS[WebSocket for Terminal] -Controllers call service managers and translate service results/errors into HTTP -responses. Controllers should not reach directly into concrete adapters or the -SQLite store. + Components -->|calls| Services[service/*] -HTTP-only request/response wrappers belong in `httpd` or -`httpd/controllers`. Application read models shared by controller and CLI flows -belong in the owning `service/*` package. + Services -->|NOT| Adapters[Direct Adapter Access] + Services -->|NOT| Storage[Direct SQLite Access] + +``` + +**Belongs here:** +- Routing and middleware +- HTTP request decoding and response encoding +- Path/query parameter handling +- Status-code mapping +- API error envelopes +- OpenAPI generation and serving +- WebSocket upgrade handling for terminal mux + +**Does NOT belong here:** +- Direct adapter or SQLite store access +- Application read models shared with CLI (belongs in `service/*`) + +**Rule:** Controllers call service managers and translate service results/errors into HTTP responses. + +--- ### `internal/cli` -`cli` owns the user-facing `ao` command. It should stay thin: +**Purpose:** User-facing `ao` command. Thin client over the daemon HTTP API. -- discover the local daemon; -- call the daemon's loopback HTTP API; -- format command output; -- start/stop/status/doctor process control. +```mermaid +graph LR + CLI[cli] --> Operations[Operations] -The CLI should not duplicate daemon business logic. If a command needs product -behavior, put the behavior in the daemon service/API path and have the CLI call -that path. + Operations --> Discover[Discover Daemon] + Operations --> Call[Call HTTP API] + Operations --> Format[Format Output] + Operations --> Control[Process Control
start/stop/status/doctor] + + Operations -->|NOT| Direct[Direct Storage/DB Access] + Operations -->|NOT| Runtime[Direct Runtime Control] + Operations -->|NOT| Adapters[Direct Adapter Calls] + +``` + +**Belongs here:** +- Daemon discovery +- HTTP API calls +- Command output formatting +- Process control: start/stop/status/doctor + +**Does NOT belong here:** +- Duplicate daemon business logic (put in daemon service/API) +- Direct storage, runtime, or adapter access + +**Rule:** If a command needs product behavior, put it in the daemon and have the CLI call that API path. + +--- + +### `internal/adapters/*` + +**Purpose:** Concrete implementations of ports interfaces. Wraps external systems. + +```mermaid +graph TD + Ports[Ports Interfaces] -->|implemented by| Adapters[Adapters] + + Adapters --> Agent[agent/*
23+ harnesses] + Adapters --> Runtime[runtime/*
tmux, conpty] + Adapters --> Workspace[workspace/*
gitworktree] + Adapters --> SCM[scm/*
github] + Adapters --> Tracker[tracker/*
github] + + Agent --> Codex[codex] + Agent --> Claude[claude-code] + Agent --> Cursor[cursor] + Agent --> Aider[aider] + Agent -->|... 20+| More[more agents] + +``` + +**Adapter principles:** +- Adapters are leaves in the import graph +- Adapters translate external behavior into AO ports/domain concepts +- Adapters should not own product workflows +- All adapter-written files must be gitignored + +**Good dependencies:** +``` +session_manager → ports.Runtime +adapters/runtime/tmux → ports + domain +adapters/workspace/gitworktree → ports + domain +daemon → adapters + services + storage +``` + +**Avoid:** +``` +domain → adapters +service/session → adapters/runtime/tmux +httpd/controllers → storage/sqlite/store +adapters/* → httpd +``` + +--- ### `internal/daemon` -`daemon` is the production composition root. It wires config, logging, SQLite, -CDC, lifecycle, reaper, runtime, terminal manager, services, HTTP, and shutdown. +**Purpose:** Production composition root. Wires all dependencies together. -Belongs here: +```mermaid +graph TD + Daemon[daemon] --> Responsibilities[Responsibilities] -- production dependency construction; -- adapter registration; -- startup/shutdown sequencing; -- cross-component wiring. + Responsibilities --> Wire[Dependency Construction] + Responsibilities --> Register[Adapter Registration] + Responsibilities --> Startup[Startup Sequencing] + Responsibilities --> Shutdown[Shutdown Sequencing] + Responsibilities --> Cross[Cross-component Wiring] -Does not belong here: + Responsibilities -->|NOT| Business[Business Logic
(put in service/lifecycle)] + Responsibilities -->|NOT| Adapter[Adapter Implementation
(put in adapters/*)] -- business logic that should be testable in service, lifecycle, or manager - packages; -- adapter implementation details. - -## Interface Placement - -Prefer interfaces near their consumers, except for shared capabilities. - -- If only one package consumes an abstraction, define the smallest interface in - that package. -- If multiple core packages consume a replaceable capability, define it in - `ports`. -- If HTTP controllers need a resource service, use the owning `service/*` - manager interface. -- Return concrete types from constructors unless callers genuinely need an - interface. - -## Current Tree - -The current main-line shape is: - -```txt -backend/ - cmd/ao/ # CLI entrypoint - main.go # daemon entrypoint compatibility - sqlc.yaml - - internal/domain/ # shared product vocabulary and durable facts - internal/ports/ # capability interfaces - internal/service/ - project/ # project API/use-case boundary - session/ # session API/use-case boundary - pr/ # PR observation/action service - review/ # code-review API/use-case boundary - internal/session_manager/ # internal session command engine - internal/lifecycle/ # durable lifecycle fact reducer - internal/observe/scm/ # SCM (GitHub) observer loop - internal/observe/reaper/ # runtime liveness observation loop - internal/storage/sqlite/ # DB, migrations, queries, generated sqlc, stores - internal/cdc/ # change_log poller and broadcaster - internal/terminal/ # terminal session protocol and PTY handling - internal/httpd/ # HTTP API, controllers, OpenAPI, terminal mux - internal/cli/ # user-facing ao command - internal/daemon/ # production wiring and shutdown - internal/config/ # daemon env/default config - internal/adapters/ # concrete agent/runtime/workspace/SCM/tracker adapters ``` +**Belongs here:** +- Production dependency construction +- Adapter registration +- Startup/shutdown sequencing +- Cross-component wiring + +**Does NOT belong here:** +- Business logic (belongs in service, lifecycle, or manager packages) +- Adapter implementation details (belongs in adapters/*) + +--- + +### `internal/config` + +**Purpose:** Environment-based daemon configuration. + +```mermaid +graph LR + Config[config] --> Sources[Sources] + + Sources --> Env[Environment Variables] + Sources --> Defaults[Built-in Defaults] + Sources --> Validate[Validation] + + Config -->|provides| Settings[Settings] + + Settings --> Port[AO_PORT] + Settings --> Timeout[AO_REQUEST_TIMEOUT] + Settings --> DataDir[AO_DATA_DIR] + Settings --> RunFile[AO_RUN_FILE] + Settings --> Agent[AO_AGENT] + +``` + +**Key environment variables:** +- `AO_PORT` — HTTP bind port (default: 3001) +- `AO_REQUEST_TIMEOUT` — Per-request timeout (default: 60s) +- `AO_SHUTDOWN_TIMEOUT` — Graceful shutdown cap (default: 10s) +- `AO_RUN_FILE` — PID/port handshake (default: ~/.ao/running.json) +- `AO_DATA_DIR` — SQLite data directory (default: ~/.ao/data) +- `AO_AGENT` — Compatibility agent adapter (default: claude-code) +- `GITHUB_TOKEN` — GitHub authentication + +--- + +## Interface Placement Rules + +```mermaid +graph TD + Question{Where to define
interface?} + + Question --> Single{Only one package
consumes it?} + Single -->|Yes| InPackage[Define in consuming
package] + Single -->|No| Multiple{Multiple core packages
need it?} + + Multiple -->|Yes| Ports[Define in ports] + Multiple -->|No| HTTP{HTTP controllers
need it?} + + HTTP -->|Yes| Service[Define in service/*] + HTTP -->|No| Concrete[Return concrete type
from constructor] + +``` + +**Rules:** + +1. **Single consumer** → Define in the consuming package (smallest interface) +2. **Multiple core consumers** → Define in `ports` (shared capability) +3. **HTTP controllers need resource** → Use `service/*` manager interface +4. **Return from constructor** → Return concrete type unless genuinely needed + +**Examples:** + +```go +// Good: Interface near single consumer +type sessionGetter interface { + GetSession(ctx context.Context, id SessionID) (SessionRecord, bool, error) +} + +// Good: Shared capability in ports +type Runtime interface { + Create(ctx context.Context, cfg RuntimeConfig) (RuntimeHandle, error) + Destroy(ctx context.Context, handle RuntimeHandle) error + IsAlive(ctx context.Context, handle RuntimeHandle) (bool, error) +} + +// Good: Service interface for controllers +type Manager interface { + List(ctx context.Context) ([]Project, error) + Add(ctx context.Context, cfg Config) (Project, error) + Remove(ctx context.Context, id string) error +} +``` + +--- + +## Import Graph + +```mermaid +graph TD + CLI[cli] --> HTTPD[httpd] + HTTPD --> Services[service/*] + HTTPD --> Terminal[terminal] + + Services --> SessionMgr[session_manager] + Services --> Lifecycle[lifecycle] + Services --> Storage[storage/sqlite] + Services --> Domain[domain] + Services --> Ports[ports] + + SessionMgr --> Ports + SessionMgr --> Adapters[adapters/*] + SessionMgr --> Lifecycle + + Lifecycle --> Ports + Lifecycle --> Storage + Lifecycle --> Domain + + Observe[observe/*] --> Ports + Observe --> Storage + Observe --> Lifecycle + + Storage --> Domain + Storage --> Ports + + Adapters --> Ports + Adapters --> Domain + + CDC[cdc] --> Storage + Terminal --> Ports + + Daemon[daemon] --> All[All packages] + +``` + +**Key patterns:** +- All arrows point downward (no cycles) +- Adapters and domain are leaves +- CLI and HTTPD don't touch storage directly +- Everything depends on ports and domain + +--- + ## Adding New Code -Use these defaults: +### New HTTP Route -- New HTTP route: add controller/API code in `httpd`, call a `service/*` - package, and update OpenAPI generation/spec tests. -- New product resource: put shared IDs/vocabulary in `domain`, use cases and - read models in `service/`, storage in `storage/sqlite`, and external - system seams in `ports`. -- New adapter: implement a `ports` interface under `adapters//` - and wire it in `daemon`. -- New persisted fact: add a migration, sqlc query, store method, domain record or - event vocabulary, and CDC behavior when the UI/API must observe it. -- New CLI command: keep command parsing/formatting in `cli`; call the daemon API - rather than reimplementing daemon behavior. +```mermaid +flowchart LR + AddRoute[Add HTTP Route] --> Route[Register in httpd] + Route --> Call[Call service/*] + Call --> Update[Update OpenAPI] + Update --> Test[Add tests] -## Project Routes Example +``` -Project-owned concepts live in `internal/service/project`: +**Steps:** +1. Add controller in `httpd/controllers/` +2. Call a `service/*` package +3. Update OpenAPI generation +4. Add spec tests -- project read models; -- project add/remove command types; -- project validation and user-facing errors; -- the `Manager` contract consumed by HTTP controllers. +### New Product Resource -`internal/httpd/controllers` remains responsible for: +```mermaid +flowchart TD + NewResource[New Resource] --> Domain[Add IDs/vocab to domain] + Domain --> Service[Create service/resource] + Service --> Storage[Add storage queries] + Storage --> Ports[Add ports if needed] + Ports --> Adapter[Implement adapter if needed] -- route registration; -- JSON decoding/encoding; -- HTTP status codes and error envelopes; -- mapping service errors to responses. +``` -When a type is ambiguous, ask whether it is a product use-case/read model or an -HTTP wire wrapper. Product use-case/read models belong in `service/project`; -HTTP wire wrappers belong in `httpd`. +**Steps:** +1. Add shared IDs/vocabulary to `domain` +2. Create use cases in `service/` +3. Add storage in `storage/sqlite` +4. Add ports if external system needed +5. Implement adapter in `adapters//` +6. Wire in `daemon` + +### New Adapter + +```mermaid +flowchart LR + NewAdapter[New Adapter] --> Port[Implement port interface] + Port --> Hooks[Implement hooks if agent] + Hooks --> Gitignore[Add .gitignore entries] + Gitignore --> Wire[Wire in daemon] + Wire --> Test[Add conformance tests] + +``` + +**Steps:** +1. Implement a `ports` interface under `adapters//` +2. For agents: implement hooks with gitignored files +3. Wire in `daemon` +4. Add conformance tests + +--- + +## Examples + +### Example: Adding a Session Command + +```go +// In internal/service/session/service.go +func (s *Service) MyNewCommand(ctx context.Context, id SessionID) (Result, error) { + // 1. Validate input + // 2. Call session_manager + // 3. Enrich result + // 4. Return read model +} + +// In internal/httpd/controllers/sessions.go +func (c *SessionsController) myNewCommand(w http.ResponseWriter, r *http.Request) { + // 1. Decode request + // 2. Call service + // 3. Encode response +} +``` + +### Example: Adding a Port Interface + +```go +// In internal/ports/myfeature.go +package ports + +type MyFeature interface { + DoSomething(ctx context.Context, cfg Config) (Result, error) +} + +// In internal/adapters/myfeature/impl.go +package impl + +import "github.com/aoagents/agent-orchestrator/backend/internal/ports" + +type Impl struct { ... } + +func (i *Impl) DoSomething(ctx context.Context, cfg ports.Config) (ports.Result, error) { + // Implementation +} +``` + +### Example: Service Layer Pattern + +```go +// In internal/service/myresource/service.go +package service + +// Service is the controller-facing boundary +type Service struct { + manager *manager.Manager // Internal command engine + store Store // Storage interface +} + +// New constructs the service +func New(mgr *manager.Manager, store Store) *Service { + return &Service{manager: mgr, store: store} +} + +// List returns enriched read models +func (s *Service) List(ctx context.Context) ([]MyResource, error) { + records, err := s.store.List(ctx) + if err != nil { + return nil, err + } + return s.enrich(records), nil +} + +// Create performs a use case +func (s *Service) Create(ctx context.Context, cfg Config) (MyResource, error) { + // Business logic + result, err := s.manager.Create(ctx, cfg) + if err != nil { + return MyResource{}, err + } + return s.enrichOne(result), nil +} +``` + +--- + +## Summary + +**Key takeaways:** + +1. **Domain** stays pure — shared vocabulary only +2. **Ports** define contracts — interfaces for external systems +3. **Services** orchestrate — controller-facing use cases +4. **Adapters** are leaves — implement ports, no core imports +5. **CLI/HTTP** stay thin — protocol handling only +6. **Daemon** wires it all — composition root + +**Always ask:** +- Does this belong in domain (shared concept)? +- Does this belong in ports (shared capability)? +- Does this belong in service (use case)? +- Does this belong in adapters (external system)? + +**Never:** +- Put HTTP types in domain +- Put display status in storage +- Put business logic in CLI +- Import core from adapters +- Import adapters from services diff --git a/docs/telemetry.md b/docs/telemetry.md new file mode 100644 index 000000000..37a961fad --- /dev/null +++ b/docs/telemetry.md @@ -0,0 +1,90 @@ +# Telemetry + +Agent Orchestrator includes telemetry for understanding product usage, reliability, and failure modes. Telemetry is implemented as **best-effort structured events** and is controlled by environment variables. + +## What We Collect + +Telemetry events are structured records that capture: + +- **Event Name** — The type of event (e.g., session lifecycle events, daemon operations, errors) +- **Source** — The component that emitted the event +- **Timestamp** — When the event occurred +- **Level** — Severity level (Debug, Info, Warn, Error) +- **Context** — Project ID, Session ID, and Request ID when applicable +- **Payload** — Event-specific metadata + +**We do not collect:** + +- Code from your repositories +- File contents or workspace data +- Authentication credentials or API keys +- Personal information beyond what is necessary for operational analytics + +## Storage and Transmission + +### Local Storage (Default) + +By default, all telemetry events are stored locally in a SQLite database at: + +``` +~/.ao/data/telemetry.db +``` + +No data leaves your machine unless you explicitly configure remote telemetry. + +### Remote Telemetry (Opt-In) + +You may optionally configure remote telemetry via PostHog by setting the `POSTHOG_API_KEY` environment variable. When configured: + +- Events are transmitted to PostHog for aggregate analytics +- Transmission is best-effort — failures do not affect daemon operation +- Events are batched to minimize network overhead + +## Configuration + +Telemetry behavior is controlled by these environment variables: + +| Variable | Default | Purpose | +| -------------------- | ------------------------- | ------------------------------------------------------ | +| `AO_TELEMETRY_LEVEL` | `info` | Minimum event level to emit (debug, info, warn, error) | +| `POSTHOG_API_KEY` | unset | PostHog API key for remote telemetry | +| `POSTHOG_HOST` | `https://app.posthog.com` | PostHog host endpoint | +| `AO_DATA_DIR` | `~/.ao/data` | Directory for local telemetry database | + +## Disabling Telemetry + +To completely disable telemetry: + +```bash +export AO_TELEMETRY_LEVEL=none +``` + +This prevents both local storage and any remote transmission of telemetry events. + +## Event Examples + +Typical telemetry events include: + +- Session spawned, terminated, or restored +- Daemon started or stopped +- Agent harness lifecycle events +- HTTP request errors +- Runtime failures +- SCM observation errors + +These events help us understand: + +- How agents are being used +- Where failures occur +- How to improve reliability +- Which features need attention + +## Privacy Commitment + +- Local telemetry is stored on your machine only +- Remote telemetry is opt-in via explicit environment variable configuration +- We do not collect code, file contents, or credentials +- Events are designed for aggregate product analytics, not individual surveillance +- PostHog configuration respects your privacy settings and data retention policies + +For questions or concerns about telemetry, please open an issue on GitHub or join our Discord community. diff --git a/screenshots/first.png b/screenshots/first.png new file mode 100644 index 000000000..c6e157506 Binary files /dev/null and b/screenshots/first.png differ diff --git a/screenshots/image.png b/screenshots/image.png new file mode 100644 index 000000000..6bc3f99c9 Binary files /dev/null and b/screenshots/image.png differ diff --git a/screenshots/second.png b/screenshots/second.png new file mode 100644 index 000000000..7eaaf9eb0 Binary files /dev/null and b/screenshots/second.png differ diff --git a/screenshots/third.png b/screenshots/third.png new file mode 100644 index 000000000..847585cfb Binary files /dev/null and b/screenshots/third.png differ