docs: document the LCM + Session Manager lane (architecture + status)
Add docs/ for newcomers: an index, an architecture deep-dive (the OBSERVE→DECIDE→ACT loop, the canonical state model, the package layout, every component, and the load-bearing invariants), and a status/roadmap (what's done PR-by-PR, what's left, the integration to-dos + carried-forward items, the open cross-lane contract questions, and where to plug in). Link them from the README. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
harshitsinghbhandari
parent
6cf7c0400d
commit
1258a3ef14
|
|
@ -2,3 +2,6 @@
|
|||
|
||||
Rewrite of the agent-orchestrator: a long-running Go backend daemon (`backend/`)
|
||||
paired with an Electron + TypeScript frontend (`frontend/`).
|
||||
|
||||
See [`docs/`](docs/README.md) for architecture and status — start with the
|
||||
Lifecycle Manager + Session Manager lane in [`docs/architecture.md`](docs/architecture.md).
|
||||
|
|
|
|||
|
|
@ -0,0 +1,34 @@
|
|||
# agent-orchestrator (rewrite) — docs
|
||||
|
||||
The agent-orchestrator is being rebuilt as a long-running **Go backend daemon**
|
||||
(`backend/`) plus an **Electron + TypeScript frontend** (`frontend/`). The
|
||||
backend supervises a fleet of coding-agent sessions and keeps one true status
|
||||
per session.
|
||||
|
||||
This folder documents the **Lifecycle Manager (LCM) + Session Manager (SM)
|
||||
lane** — the deterministic core of the backend that is now implemented (behind
|
||||
fakes) on the `feat/lcm-sm-contracts` integration branch.
|
||||
|
||||
## Start here
|
||||
|
||||
| Doc | What it covers |
|
||||
|-----|----------------|
|
||||
| [architecture.md](architecture.md) | How the lane works: the OBSERVE→DECIDE→ACT loop, the canonical state model, the package layout, every component, and the load-bearing invariants. Read this first. |
|
||||
| [status.md](status.md) | What's done (PR by PR), what's left, the integration to-dos, the open cross-lane contract questions, and how to build/test. |
|
||||
|
||||
## The one-paragraph mental model
|
||||
|
||||
The backend is a **stateless supervisor over external ground truth**: git/GitHub
|
||||
own PR/CI/review truth, the agent's own files own its activity, and the backend
|
||||
owns no agent state. Its whole job is, per session: **OBSERVE** raw facts →
|
||||
**DECIDE** one canonical status via pure, deterministic functions → **ACT**
|
||||
(persist + fire reactions). The LCM is that reducer; the SM is the
|
||||
explicit-mutation plumbing (spawn/kill/restore/cleanup) that feeds it.
|
||||
|
||||
## Where this lane fits
|
||||
|
||||
Other lanes (built by other people, in parallel) provide the real adapters this
|
||||
lane depends on through narrow interfaces: the **persistence layer + CDC**, the
|
||||
**SCM poller**, the **runtime/agent/workspace plugins**, the **backend API +
|
||||
OpenAPI**, and the **frontend store**. See [status.md](status.md#integration)
|
||||
for the hand-off points.
|
||||
|
|
@ -0,0 +1,187 @@
|
|||
# LCM + Session Manager — architecture
|
||||
|
||||
This is the deterministic core of the backend daemon. It supervises agent
|
||||
sessions and keeps exactly one true status per session.
|
||||
|
||||
## 1. Mental model: OBSERVE → DECIDE → ACT
|
||||
|
||||
The backend owns no agent state. git/GitHub own PR/CI/review truth; the agent's
|
||||
own files own its activity. The job, per session, is one loop:
|
||||
|
||||
```
|
||||
OBSERVE → DECIDE → ACT
|
||||
(impure, external) (pure, total) (impure)
|
||||
raw facts one canonical status persist + react
|
||||
```
|
||||
|
||||
In the rewrite the **OBSERVE** step lives *outside* the LCM (separate owners),
|
||||
and the LCM is a **synchronous reducer** invoked with facts:
|
||||
|
||||
```
|
||||
SCM poller ─ ApplySCMObservation ──┐
|
||||
reaper ─ ApplyRuntimeObservation┤
|
||||
activity hooks ─ ApplyActivitySignal ───┼─▶ LCM: load canonical
|
||||
Session Mgr ─ OnSpawnCompleted ──────┘ → pure DECIDE
|
||||
─ OnKillRequested → diff → persist (merge-patch)
|
||||
reaper tick ─ TickEscalations → if transition: react (ACT)
|
||||
```
|
||||
|
||||
The LCM **never polls**. The reaper (a timer, owned elsewhere) drives liveness
|
||||
sampling and duration-based escalation by calling in.
|
||||
|
||||
## 2. Canonical state model — the crown jewel
|
||||
|
||||
The **only** thing persisted per session is `CanonicalSessionLifecycle`
|
||||
(`backend/internal/domain/lifecycle.go`). The single-word display status is
|
||||
**derived on read and never stored** — this is the most important invariant; it
|
||||
prevents canonical truth and display from drifting.
|
||||
|
||||
```
|
||||
CanonicalSessionLifecycle
|
||||
Version schema version of the record shape
|
||||
Revision monotonic write counter (optimistic-concurrency token)
|
||||
Session (state, reason) working/idle/needs_input/stuck/detecting/done/terminated
|
||||
PR (state, reason) none/open/merged/closed
|
||||
Runtime (state, reason) unknown/alive/exited/missing/probe_failed
|
||||
Activity last-known agent activity (+ timestamp, source) ← decider input
|
||||
Detecting anti-flap quarantine memory (nil unless quarantined) ← decider input
|
||||
```
|
||||
|
||||
`DeriveLegacyStatus` (`domain/status.go`) is the **sole producer** of the
|
||||
display `SessionStatus`. Precedence: terminal/hard session states map directly
|
||||
(they outrank PR facts) → a merged PR wins → an open PR maps by reason → else the
|
||||
soft session state. So an idle worker with a CI-failing open PR displays
|
||||
`ci_failed`, but a `needs_input` session shows `needs_input` regardless of the PR.
|
||||
|
||||
`Session` (`domain/session.go`) is the read-model: a `SessionRecord`
|
||||
(persistence shape, identity + lifecycle + metadata) plus the derived `Status`.
|
||||
The **Session Manager is the single producer of `Status`** — it attaches it on
|
||||
read; the store and API never recompute or persist it.
|
||||
|
||||
## 3. Package layout (`backend/internal/`)
|
||||
|
||||
```
|
||||
domain/ the vocabulary (imports only the std lib → no cycles)
|
||||
lifecycle.go CanonicalSessionLifecycle + all sub-states/enums
|
||||
status.go SessionStatus + DeriveLegacyStatus (sole display producer)
|
||||
session.go SessionRecord (persisted) + Session (read-model) + id types
|
||||
decide/ the PURE core — total, deterministic, zero I/O
|
||||
types.go LifecycleDecision + Probe/OpenPR/Detecting inputs + tuning consts
|
||||
decide.go the deciders + the anti-flap quarantine + HashEvidence
|
||||
ports/ the boundaries (interfaces + DTOs)
|
||||
inbound.go LifecycleManager, SessionManager (we implement)
|
||||
outbound.go LifecycleStore, Notifier, AgentMessenger, Runtime/Agent/Workspace
|
||||
facts.go SCMFacts, RuntimeFacts, ActivitySignal, SpawnOutcome, KillReason
|
||||
lifecycle/ the LCM implementation (DECIDE + ACT)
|
||||
manager.go the Apply* pipeline, per-session lock, patch diffing
|
||||
decide_bridge.go fact→decide-input translation + the composition rules
|
||||
reactions.go the reaction table + escalation engine + TickEscalations
|
||||
session/ the SM implementation (explicit mutations)
|
||||
manager.go Spawn/Kill/Restore/Cleanup/List/Get/Send + rollback
|
||||
```
|
||||
|
||||
`domain` + `ports` are the committed, stabilized **integration boundary**.
|
||||
Everything else implements behind it.
|
||||
|
||||
## 4. The pure DECIDE core (`domain/decide`)
|
||||
|
||||
Total, deterministic, side-effect-free functions — the highest-value test
|
||||
surface (table-tested to 100%). Key ones:
|
||||
|
||||
- `ResolveProbeDecision` — runtime/process liveness. An explicit kill
|
||||
short-circuits to terminal; a **failed probe is never read as death** (routes
|
||||
to `detecting`), as does any probe disagreement; only runtime-dead +
|
||||
process-dead + no-recent-activity reaches `killed`.
|
||||
- `ResolveOpenPRDecision` — the PR ladder: `ci_failing` → `changes_requested` →
|
||||
`mergeable` → `approved` → `review_pending` → idle-beyond → else `pr_open`.
|
||||
- `ResolveTerminalPRStateDecision` — merged → `merged` (park idle awaiting a
|
||||
human decision); closed → `idle`.
|
||||
- `CreateDetectingDecision` — the **anti-flap quarantine**. Counts attempts and
|
||||
hashes the *timestamp-stripped* evidence; escalates to `stuck` only after 3
|
||||
consecutive unchanged-evidence ticks **or** 5 minutes since first entering
|
||||
detecting (`StartedAt` is preserved across the whole episode). Changing
|
||||
evidence resets the counter.
|
||||
|
||||
## 5. The LCM (`lifecycle`)
|
||||
|
||||
Implements `ports.LifecycleManager`. Every `Apply*`/`On*` entrypoint runs the
|
||||
same pipeline (`manager.go`):
|
||||
|
||||
```
|
||||
withLock(session): ← per-session serialization
|
||||
load canonical → decideFn (build sparse patch) → if changed: persist → load after
|
||||
return transition (before, after)
|
||||
```
|
||||
then, **after the lock releases**, `react()` fires the mapped reaction.
|
||||
|
||||
- **Per-session serialization** — `keyedMutex` hands out one lock per session id
|
||||
(parallel across sessions, serial within one). Entries are reference-counted
|
||||
and evicted when the last holder releases, so the map stays bounded.
|
||||
- **Composition rules** (`decide_bridge.go`) — two observers must not fight over
|
||||
the session axis. Liveness (runtime probes) owns the runtime + death/detecting
|
||||
axis; activity owns working/idle/waiting. `isLivenessOwned` decides when a
|
||||
healthy probe may *recover* a state (e.g. `detecting → working`) vs. when it
|
||||
must not clobber an activity-owned `needs_input`/`blocked`. A high-confidence
|
||||
activity signal may resolve a `detecting` session; an open PR writes only the
|
||||
PR axis and lets `DeriveLegacyStatus` surface it.
|
||||
- **Detecting-memory lifecycle** — a decision with `Detecting == nil` clears the
|
||||
persisted quarantine memory (`LifecyclePatch.ClearDetecting`) so a stale prior
|
||||
can't leak into a later episode.
|
||||
- **ACT — reactions + escalation** (`reactions.go`) — on a genuine status
|
||||
transition, `react()` maps it to a reaction (`send-to-agent` / `notify`;
|
||||
`auto-merge` exists but is off by default) and dispatches it. A
|
||||
per-`(session,reaction)` escalation tracker counts attempts; it escalates
|
||||
(notifies a human and silences further auto-dispatch) when a numeric cap or a
|
||||
duration is exceeded. The `ci-failed` budget is persistent across CI
|
||||
oscillation within an open PR and re-arms on genuine recovery. `TickEscalations`
|
||||
(called by the reaper) fires the duration-based escalations the synchronous
|
||||
LCM can't wake itself for; it notifies outside the lock.
|
||||
|
||||
## 6. The Session Manager (`session`)
|
||||
|
||||
Implements `ports.SessionManager` — the explicit-mutation plumbing. It never
|
||||
derives/observes lifecycle state; it routes outcomes to the LCM.
|
||||
|
||||
- **Spawn** — `Workspace.Create` → build prompt → `Runtime.Create` (env
|
||||
`AO_SESSION_ID`/`AO_PROJECT_ID`/`AO_ISSUE_ID`) → **seed** the initial record
|
||||
(`not_started`/`spawn_requested`) via the store → `LCM.OnSpawnCompleted`.
|
||||
Eager rollback unwinds prior steps on failure; an `OnSpawnCompleted` failure
|
||||
routes the seeded orphan to terminal-errored (the store has no delete; a later
|
||||
`Cleanup` reclaims it).
|
||||
- **Kill** — `LCM.OnKillRequested` → `Runtime.Destroy` → `Workspace.Destroy`,
|
||||
honoring the **worktree-remove safety**: after `git worktree prune`, a still-
|
||||
registered path is never `rm -rf`'d (it may hold the agent's uncommitted work)
|
||||
— the refusal is surfaced, not forced.
|
||||
- **Restore** — reopen via `PatchLifecycle` (not re-seed): session →
|
||||
`not_started`, PR → `cleared_on_restore`; relaunch with the agent's resume
|
||||
command; runtime is rolled back on a post-create failure.
|
||||
- **List/Get** — read records and attach the derived `Status`. **Send** — via
|
||||
`AgentMessenger`. **Cleanup** — tear down terminal/stale sessions, skipping
|
||||
paths with uncommitted work.
|
||||
|
||||
## 7. Load-bearing invariants
|
||||
|
||||
1. **Persist canonical; derive display.** Never store the display status.
|
||||
2. **One authority for death.** Only the DECIDE pipeline (via `detecting`) writes
|
||||
inferred terminal states; the SM's explicit-kill path goes through
|
||||
`OnKillRequested`. Everything else that notices a dead runtime persists
|
||||
`detecting`, never `terminated`.
|
||||
3. **Failed probe ≠ dead.** Timed-out/errored probes route to `detecting`.
|
||||
4. **Evidence-hash debounce** prevents flapping signals from terminating live
|
||||
work; the 5-minute cap is a whole-episode wall-clock safety net.
|
||||
5. **PR facts dominate** the soft session states once a PR exists.
|
||||
6. **Merge-patch persistence** — writes touch only changed keys; the store is the
|
||||
single disk writer (atomic write + lock + CDC).
|
||||
7. **Sticky activity states** (`waiting_input`/`blocked`) do not decay by clock.
|
||||
8. **Worktree-remove safety** on teardown.
|
||||
|
||||
## 8. Concurrency & testing
|
||||
|
||||
- Within a session, the per-session lock serializes the load→decide→persist
|
||||
read-modify-write. `react()` runs *outside* the lock (so a busy-waiting
|
||||
send-to-agent never holds the session mutex) — see `status.md` for the
|
||||
integration-time follow-up this implies.
|
||||
- Tests use **in-memory fakes** for every outbound port, so the LCM and SM are
|
||||
fully testable with no real adapters. The SM tests drive the **real**
|
||||
`lifecycle.Manager` for spawn/kill round-trips, so the SM↔LCM contract is
|
||||
genuinely exercised. The `decide` package is table-tested in isolation.
|
||||
|
|
@ -0,0 +1,98 @@
|
|||
# LCM + Session Manager — status & roadmap
|
||||
|
||||
Where the lane stands, what's left, and where to plug in.
|
||||
|
||||
## Branch model
|
||||
|
||||
`feat/lcm-sm-contracts` is the **lane integration branch**: each sub-PR below
|
||||
branched off it and merged **into** it. The whole lane lands on `main` as one
|
||||
unit once it's ready. Sub-PRs were reviewed against the integration branch;
|
||||
the eventual lane→main merge is a single cumulative review.
|
||||
|
||||
## Done — implementation complete (behind fakes)
|
||||
|
||||
| Area | What landed | PR |
|
||||
|------|-------------|----|
|
||||
| Skeleton | `backend/` (Go) + `frontend/` (Electron/TS) | #1 (on `main`) |
|
||||
| Contracts + CI | `domain/` + `ports/`; Go + gitleaks workflows | #2 |
|
||||
| Pure DECIDE core | the deciders + anti-flap quarantine + exhaustive truth-table tests | #4 |
|
||||
| LCM — pipeline | `Apply*` pipeline, per-session serialization, store integration, composition rules, detecting-memory lifecycle | #5 |
|
||||
| LCM — reactions | reaction table + escalation engine + real `TickEscalations` | #6 |
|
||||
| Session Manager | spawn / kill / restore / cleanup / list, eager rollback, worktree-remove safety | #7 |
|
||||
|
||||
`gofmt` / `go build` / `go vet` / `go test -race` all green across `domain`,
|
||||
`domain/decide`, `lifecycle`, and `session`. The `decide` core is at 100%
|
||||
statement coverage; the impl packages cover the load-bearing logic including the
|
||||
error/rollback paths.
|
||||
|
||||
### Build & test
|
||||
|
||||
```
|
||||
cd backend
|
||||
gofmt -l . # must print nothing
|
||||
go build ./...
|
||||
go vet ./...
|
||||
go test -race ./...
|
||||
go test -cover ./...
|
||||
```
|
||||
|
||||
## Not done — the integration phase
|
||||
|
||||
Everything above runs against **in-memory fakes**. Making it a live system means
|
||||
swapping fakes for real adapters (built by other lanes) behind the existing
|
||||
ports, and resolving the carried-forward items below.
|
||||
|
||||
### Carried-forward items (must be addressed as real adapters land)
|
||||
|
||||
- **`react()` out-of-lock dispatch.** Reactions fire after the per-session lock
|
||||
releases (deliberate, so a busy-waiting send-to-agent doesn't hold the mutex).
|
||||
Under a live daemon with concurrent observers this can dispatch on a stale
|
||||
snapshot / out of order. Give `react()` a per-session ordering (a small react
|
||||
queue) or re-check the triggering state before dispatching. Documented in
|
||||
`lifecycle/reactions.go`.
|
||||
- **`ExpectedRevision` optimistic-concurrency is unused.** The in-process
|
||||
per-session mutex covers a single daemon. Multi-writer or CDC-driven setups
|
||||
must use the `LifecyclePatch.ExpectedRevision` CAS the contract already exposes.
|
||||
- **Store `Seed` + `Get` need a real implementation.** The Session Manager added
|
||||
two record-with-identity methods to `LifecycleStore`; the real persistence
|
||||
layer must implement them (create-with-identity that rejects an existing id;
|
||||
full-record read by id). Documented in `ports/outbound.go`.
|
||||
|
||||
### Real adapters needed (other lanes)
|
||||
|
||||
| Port | Real adapter | Owning lane |
|
||||
|------|--------------|-------------|
|
||||
| `LifecycleStore` | persistence layer (flat-file/KV + atomic write + lock + CDC) | persistence |
|
||||
| `SCMFacts` producer | SCM poller (batch PR/CI/review enrichment) | SCM |
|
||||
| `Runtime` / `Agent` / `Workspace` | tmux runtime, claude-code/codex agent, git-worktree workspace | coding-agents |
|
||||
| `Notifier` | desktop/Slack notifier | notifications |
|
||||
| `AgentMessenger` | tmux inject with busy-detect + delivery verify | coding-agents |
|
||||
| `SessionManager` consumer | backend API (routes/controllers) + OpenAPI | API |
|
||||
|
||||
### Open cross-lane contract questions
|
||||
|
||||
- **SCM facts** — does `SCMFacts` match what the poller can cheaply produce
|
||||
(batch enrichment, CI log tail as a pointer)?
|
||||
- **Persistence** — is `LifecycleStore` + `LifecyclePatch` the right boundary?
|
||||
Per-session lock vs. the `ExpectedRevision` CAS?
|
||||
- **API** — is the `SessionManager` interface + the `Session` read-model
|
||||
OpenAPI-friendly?
|
||||
|
||||
### Land the lane → `main`
|
||||
|
||||
A final cumulative review of `feat/lcm-sm-contracts` vs. `main`, then merge the
|
||||
complete lane in one unit.
|
||||
|
||||
## Where to plug in (for someone picking this up)
|
||||
|
||||
- **Implementing a real adapter?** Write it to satisfy the matching interface in
|
||||
`ports/`, then construct the `lifecycle.Manager` / `session.Manager` with it in
|
||||
place of the fake. Nothing in `domain`/`lifecycle`/`session` should need to
|
||||
change.
|
||||
- **Changing decision behavior?** It lives in `domain/decide` (pure) — add a
|
||||
truth-table case first; nothing there does I/O.
|
||||
- **Adding a reaction?** Extend the table in `lifecycle/reactions.go` and map the
|
||||
triggering status in `reactionEventFor`.
|
||||
- **Don't** persist the display status, conclude death outside the probe
|
||||
pipeline, or `rm -rf` a still-registered worktree — see the invariants in
|
||||
[architecture.md](architecture.md#7-load-bearing-invariants).
|
||||
Loading…
Reference in New Issue