* rebase: forward branch onto main + resolve activity-events kind union conflict * feat(core): wire scm/runtime/agent plugin-call failure events Adds activity-event evidence for previously-silent failure paths in lifecycle-manager.ts so the RCA agent can answer 'why did X happen?': - scm.batch_enrich_failed (line 617 catch) - scm.detect_pr_succeeded (line 658 success path) - scm.detect_pr_failed (line 664 catch) - scm.review_fetch_failed (line 1517 catch) - scm.poll_pr_failed (line 1132 catch) - runtime.probe_failed (line 938 catch) - agent.process_probe_failed (lines 1054 + 1139 catches, with where field) - agent.activity_probe_failed (line 1062 outer catch) Plus 6 new tests covering the call shapes. Invariants preserved (per CLAUDE.md): - B1 state-mutate-before-emit: each emit follows existing observer call - B2 never throws: recordActivityEvent best-effort by design - B3 re-entrancy guard unchanged - B4 Promise.allSettled semantics unchanged * feat(core): wire reaction lifecycle activity events Adds AE evidence around reaction triggers, escalations, and failures so RCA can answer 'did AO try to auto-fix this? did it succeed?': - reaction.action_succeeded (combined for send-to-agent / notify / auto-merge, with data.action variant) — fires after each successful reaction action - reaction.send_to_agent_failed — fires in the previously-silent catch when sessionManager.send throws inside a send-to-agent reaction - reaction.escalated — fires alongside the existing notifyHuman escalation with data.escalationCause = 'max_retries' | 'max_duration' Plus 3 new tests covering the call shapes. Invariants preserved: emits land after the existing notifyHuman/return paths so state mutation order is unchanged. * feat(core): wire auto-cleanup, poll-cycle, detecting escalation events Adds AE evidence around session destruction, poll loop failures, and the detecting→stuck transition so RCA can answer 'when did my session get cleaned up?', 'did the polling loop crash?', and 'why did AO mark this session stuck?': - session.auto_cleanup_deferred — agent busy, cleanup deferred - session.auto_cleanup_completed — kill succeeded, runtime + worktree gone - session.auto_cleanup_failed (level=error) — kill threw, session stays merged - lifecycle.poll_failed (level=error) — pollAll outer catch fired - detecting.escalated — first cycle that promotes detecting→stuck, with cause = max_attempts | max_duration. Guarded by detectingEscalatedAt metadata so it fires once per escalation, not on every poll while stuck. Plus 5 new tests covering the call shapes and the idempotency guard. Invariants preserved: - Auto-cleanup events fire AFTER existing observer.recordOperation (B1) - detecting.escalated emits ONCE per escalation (invariant B9 in .context/lifecycle-manager-instrumentation.md) - poll_failed emits inside the existing pollAll catch — flow unchanged * feat(core): wire report_watcher.triggered activity event Adds AE evidence when the report watcher fires (no_acknowledge / stale_report / agent_needs_input). RCA: 'AO thinks my agent is stuck — why?' - report_watcher.triggered (level=warn) — emitted alongside the existing observer.recordOperation, only when a trigger is non-null (per invariant in .context/lifecycle-manager-instrumentation.md §B9) Plus 1 test exercising the no_acknowledge trigger path. * fix(core): one-shot guard on report_watcher.triggered AE emit Live-observed regression: report_watcher.triggered fired 116 times in production over a few hours because the emit was unguarded and re-fired every 30s poll while a trigger stayed active. Symptom was massive event flood for stuck/no-acknowledge/stale conditions. Fix: gate the emit on the existing isNewTrigger variable (same one-shot guard pattern used for detecting.escalated). The observer.recordOperation above remains unguarded by design (it's a metric/heartbeat); the AE trail is for actionable evidence only. Adds a regression test that drives the same trigger across two polls and asserts the AE event fires only on the first. * fix(core): address Greptile feedback on PR #1620 Two findings from Greptile (issue same as Codex P2 #1): 1. scm.batch_enrich_failed omitted projectId/sessionId — when the lifecycle worker is project-scoped (deps.projectId set), this event is effectively project-scoped too. Without projectId, queries like `ao events list --project todo-app --type scm.batch_enrich_failed` return zero results, defeating the purpose of the instrumentation. Fix: pass scopedProjectId when set. Unscoped (multi-project) supervisors still leave projectId null because the batch crosses project boundaries. 2. Misleading field name pendingSinceMs in session.auto_cleanup_deferred data — the local variable of the same name is a Unix epoch timestamp, but the data field stored `Date.now() - pendingSinceMs` (an elapsed duration). RCA agents would mis-interpret it as a timestamp and compute a 1970-era "pending since" date. Renamed to pendingElapsedMs. * fix(core): address Codex review on PR #1620 - lifecycle.poll_failed: keep summary generic, route raw error text through `data.errorMessage` only. sanitizeSummary just truncates; sanitizeData redacts credential URLs. Since FTS5 indexes summary, interpolating subprocess error output (which can include https://x-oauth-basic:TOKEN@github.com/... from git/gh) made credentials persistently searchable. - reaction.escalated: expand escalationCause to "max_retries" | "max_attempts" | "max_duration" and mirror the trigger checks. Numeric escalateAfter is an attempt-count gate, not a duration; previously got misattributed to "max_duration" whenever retries was unset (built-in defaults use {escalateAfter: 2}). Adds two regression tests as guards for both behaviors. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(core): replace import() type annotation with import type to satisfy lint CI's @typescript-eslint/consistent-type-imports rule rejects inline `typeof import("../activity-events.js")` inside the vi.mock factory. Hoist it to a top-level `import type * as ActivityEventsModule` so the type lives in a proper import declaration; vi.mock factory resolution is unaffected (type-only imports emit no runtime code). Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(core): keep report_watcher.triggered summary generic to plug FTS leak auditResult.message for the agent_needs_input trigger embeds the free-form report.note supplied via `ao report --note "..."`. Since sanitizeSummary only truncates and FTS5 indexes the summary column, a note containing a credential URL would be persistently searchable from the events DB. Same class of bug as the prior poll_failed fix. Summary becomes generic ("<trigger> triggered"); the full message continues to flow through `data.message` where sanitizeData redacts credential URLs. Adds a regression test that seeds a needs_input report with a credential-bearing note and asserts the summary stays clean. Reported by @ashish921998 in PR #1620. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(core): redact token-shaped secrets in activity-event data (P1) Both `summary` and `data` columns are FTS5-indexed (events-db.ts:58-59). Prior fixes moved raw error/report text from `summary` to `data.message` / `data.errorMessage`, on the assumption that sanitizeData() would scrub it. That assumption was incomplete: sanitizeData only redacted credential URLs and entire values under sensitive *key* names. Token-shaped substrings (`Bearer …`, `ghp_…`, `sk-…`, JWTs, `AKIA…`, ALL_CAPS_TOKEN=value) under non-sensitive keys like `message`/`errorMessage` were stored as-is and made searchable via FTS. Adds a TOKEN_PATTERNS array applied to every string value during sanitization, plus a 500-char per-string cap (matching sanitizeSummary's existing precedent — limits blast radius if a new token format slips past the patterns). Patterns cover: Bearer headers, GitHub PATs (classic + fine-grained), OpenAI/Anthropic sk- keys, Slack xox- tokens, AWS access key IDs, JWTs, and ENV-style assignments scoped to ALL_CAPS keys ending in TOKEN/PASSWORD/SECRET/etc. Tests: - 10 new sanitizeString unit tests (one per token shape + prose-preservation regression guard + 500-char cap + nested array/object recursion) - 1 new FTS5 integration test that drives recordActivityEvent → real SQLite → both direct row read and FTS MATCH must return zero token leakage Test fixtures use string concatenation across the prefix boundary so literal token shapes don't appear in source (gitleaks pre-commit guard). Reported by @ashish921998 in PR #1620. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(core): bound credential-URL regex to prevent ReDoS (CodeQL alert) CodeQL flagged CREDENTIAL_URL_RE as polynomial: input shaped like `http://http://http://...` with no terminating `@` caused O(n²) backtracking because the unbounded `[^@\s]+` greedily spanned multiple `http://` prefixes before failing at end-of-string and walking back. Two-part fix: 1. Exclude `/` from the userinfo character class — this is also semantically correct since RFC 3986 userinfo cannot contain unencoded `/`. 2. Add a hard length cap (200 chars) on the userinfo segment as a belt-and- braces guard against future pathological inputs. The fix is observable: 14KB pathological input completes in single-digit ms post-fix vs multiple seconds pre-fix. Adds a regression test that runs the pathological input through the full sanitize pipeline and asserts <100ms completion. Reported by GitHub Advanced Security on PR #1620. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com> * fix(core): replace CREDENTIAL_URL_RE regex with linear scan The bounded {1,200} quantifier in CREDENTIAL_URL_RE let credential URLs with >200-char userinfo pass through unredacted. Since data is FTS5-indexed, those credentials became searchable (P1 from PR #1620 review). Replace the regex with a simple linear scan (redactCredentialUrls) that: - Has no length limit — scans until @, space, or / - Is O(n) with no regex backtracking (fixes CodeQL polynomial-regex alert) - Matches http:// and https:// case-insensitively (preserves old /gi behavior) Adds regression tests for: - >200-char userinfo bypass - URLs without userinfo (no false positives) - Multiple credential URLs in one string - Pathological ReDoS-shaped input still completes in <100ms --------- Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com> Co-authored-by: AO Bot <ao-bot@composio.dev> |
||
|---|---|---|
| .. | ||
| __tests__ | ||
| src | ||
| CHANGELOG.md | ||
| README.md | ||
| package.json | ||
| rollup.config.ts | ||
| tsconfig.build.json | ||
| tsconfig.json | ||
| vitest.config.ts | ||
README.md
@aoagents/ao-core
Core services, types, and configuration for the Agent Orchestrator system.
What's Here
src/types.ts— All TypeScript interfaces (Runtime, Agent, Workspace, Tracker, SCM, Notifier, Terminal, Session, events)src/services/— Core services (SessionManager, LifecycleManager, PluginRegistry)src/config.ts— Configuration loading + Zod schemassrc/utils/— Shared utilities (shell escaping, metadata parsing, etc.)
Key Files
src/types.ts — The Source of Truth
Every interface the system uses is defined here. If you're working on any part of the orchestrator, start by reading this file.
Main interfaces:
Runtime— where sessions execute (tmux, docker, k8s)Agent— AI coding tool adapter (claude-code, codex, aider)Workspace— code isolation (worktree, clone)Tracker— issue tracking (GitHub Issues, Linear)SCM— PR/CI/reviews (GitHub, GitLab)Notifier— push notifications (desktop, Slack, webhook)Terminal— human interaction UI (iTerm2, web)Session— running agent instance (state, metadata, handles)OrchestratorEvent— events emitted by lifecycle managerPluginModule— what every plugin exports
src/services/session-manager.ts — Session CRUD
Handles session lifecycle:
spawn(config)— create new session (workspace + runtime + agent)list(projectId?)— list all sessionsget(sessionId)— get session detailskill(sessionId)— terminate sessioncleanup(projectId?)— kill completed/merged sessionssend(sessionId, message)— send message to agent
Data flow in spawn():
- Load project config
- Validate issue exists via
Tracker.getIssue()(if issueId provided, fails-fast if not found) - Reserve session ID
- Determine branch name
- Create workspace via
Workspace.create() - Generate prompt via
Tracker.generatePrompt() - Build layered worker prompt via
buildPrompt()intosystemPrompt+taskPrompt - Persist
systemPromptFilefor the session and, for OpenCode workers, writeOPENCODE_CONFIG - Build launch command via
Agent.getLaunchCommand() - Create runtime session via
Runtime.create() - Run
Agent.postLaunchSetup()(optional) - Write metadata file
- Return Session object
Note: If issue validation fails (not found, auth error), spawn fails before creating any resources (no workspace, no runtime, no session ID). This prevents spawning sessions with broken issue references.
Worker sessions keep persistent instructions in the prompt file. OpenCode workers consume that file through OPENCODE_CONFIG, while OpenCode orchestrators continue to project their system prompt into workspace AGENTS.md.
src/services/lifecycle-manager.ts — State Machine + Reactions
Polls sessions, detects state changes, triggers reactions:
State machine:
spawning → working → pr_open → ci_failed/review_pending/approved → mergeable → merged
Reactions:
ci-failed→ send fix prompt to agentchanges-requested→ send review comments to agentapproved-and-green→ notify human (or auto-merge)agent-stuck→ notify human
Polling loop:
- For each session: check agent activity state (
Agent.getActivityState()) - If PR exists: check CI status (
SCM.getCISummary()), review state (SCM.getReviewDecision()) - Update session status based on state
- Trigger reactions if state changed
- Emit events
src/services/plugin-registry.ts — Plugin Discovery + Loading
Loads plugins and provides access to them:
register(plugin, config?)— register a plugin instanceget<T>(slot, name)— get plugin by slot + namelist(slot)— list all plugins for a slotloadBuiltins(config?)— load built-in plugins (runtime-tmux, agent-claude-code, etc.)loadFromConfig(config)— load built-ins today; external plugin descriptors are the marketplace extension point
Built-in plugins (loaded by default):
- runtime-tmux, runtime-process
- agent-claude-code, agent-codex, agent-aider, agent-cursor, agent-kimicode, agent-opencode
- workspace-worktree, workspace-clone
- tracker-github, tracker-linear, tracker-gitlab
- scm-github, scm-gitlab
- notifier-desktop, notifier-discord, notifier-slack, notifier-composio, notifier-openclaw, notifier-webhook
- terminal-iterm2, terminal-web
src/config.ts — Configuration Loading
Loads and validates agent-orchestrator.yaml:
Main config sections:
- Runtime data paths are auto-derived from the config location under
~/.agent-orchestrator/{hash}-{projectId}/ port— web dashboard port (default 3000, set different values for multiple projects)terminalPort— terminal WebSocket port (auto-detected if not set)directTerminalPort— direct terminal WebSocket port (auto-detected if not set)defaults— default plugins (runtime, agent, workspace, notifiers)plugins— installer-managed external plugin descriptors (registry, npm, or local)projects— per-project config (repo, path, branch, symlinks, reactions, agentRules)notifiers— notification channel config (Slack webhooks, etc.)notificationRouting— which notifiers get which priority eventsreactions— auto-response config (ci-failed, changes-requested, approved-and-green, etc.)
Zod schemas validate all config at load time.
Common Tasks
Adding a Field to Session
- Edit
src/types.ts→Sessioninterface - Edit
src/services/session-manager.ts→ initialize field inspawn() - Rebuild:
pnpm --filter @aoagents/ao-core build
Adding an Event Type
- Edit
src/types.ts→EventTypeunion - Emit the event:
eventEmitter.emit()in relevant service - Add reaction handler (optional):
src/services/lifecycle-manager.ts
Adding a Reaction
- Edit
src/services/lifecycle-manager.ts→ add handler function - Wire it up in the polling loop
- Add config schema in
src/config.tsif new reaction type
Feedback Tools (v1)
@aoagents/ao-core exports two structured feedback tool contracts:
bug_reportimprovement_suggestion
Both share the same required input fields:
titlebodyevidence(array of strings)sessionsourceconfidence(0..1)
Example:
import { FEEDBACK_TOOL_NAMES, FeedbackReportStore, getFeedbackReportsDir } from "@aoagents/ao-core";
const reportsDir = getFeedbackReportsDir(configPath, projectPath);
const store = new FeedbackReportStore(reportsDir);
const saved = store.persist(FEEDBACK_TOOL_NAMES.BUG_REPORT, {
title: "SSO login loop",
body: "Google SSO redirects back to /login repeatedly.",
evidence: ["trace_id=abc123", "screenshot: login-loop.png"],
session: "ao-22",
source: "agent",
confidence: 0.84,
});
Storage format:
- Reports are persisted under
~/.agent-orchestrator/{hash}-{projectId}/feedback-reports - Each report is a typed key=value file (
report_<timestamp>_<id>.kv) for easy inspection - A deterministic dedupe key (
sha256, 16 hex chars) is generated from normalized tool+content
Migration notes:
- No migration needed for existing AO installs
- The
feedback-reportsdirectory is created lazily on first persisted report
Testing
# Run all core tests
pnpm --filter @aoagents/ao-core test
# Run in watch mode
pnpm --filter @aoagents/ao-core test -- --watch
# Run specific test
pnpm --filter @aoagents/ao-core test -- session-manager.test.ts
Tests are in src/__tests__/:
session-manager.test.ts— session CRUD, spawn, cleanuplifecycle-manager.test.ts— state machine, reactionsplugin-registry.test.ts— plugin loading, resolutiontmux.test.ts— tmux utility functions (not a plugin test)prompt-builder.test.ts— prompt generation utilities
Building
# Build core
pnpm --filter @aoagents/ao-core build
# Typecheck
pnpm --filter @aoagents/ao-core typecheck
This package is a dependency of all other packages. Build it first if working on the codebase.
Architecture Notes
Why flat metadata files?
- Debuggability:
cat ~/.agent-orchestrator/<hash>-my-app/sessions/app-3shows full state - No database dependency (survives crashes, easy to inspect)
- Backwards-compatible with bash script orchestrator
Why polling instead of webhooks?
- Simpler (no webhook setup, no ngrok for local dev)
- Works offline (CI/review state is fetched, not pushed)
- Survives orchestrator restarts (no missed events)
Why plugin slots?
- Swappability: use tmux locally, docker in CI, k8s in prod
- Testability: mock plugins for tests
- Extensibility: users can add custom plugins (e.g., company-specific notifier)