diff --git a/.changeset/activity-events-webhooks-mux.md b/.changeset/activity-events-webhooks-mux.md new file mode 100644 index 000000000..6f4dcc73f --- /dev/null +++ b/.changeset/activity-events-webhooks-mux.md @@ -0,0 +1,18 @@ +--- +"@aoagents/ao-core": minor +"@aoagents/ao-web": minor +--- + +Wire activity events into webhook ingress and the mux WebSocket terminal server (sub-issue of #1511, follows #1620). + +- `api.webhook_unverified` (warn) — signature verification failed; data includes `slug`, `remoteAddr`, `candidateCount` (never the failed signature) +- `api.webhook_rejected` (warn) — payload exceeded `maxBodyBytes`; data includes counts and `maxBodyBytes` (never the body) +- `api.webhook_received` (info|warn) — accepted webhook; data includes `projectIds`, `matchedSessions`, `parseErrorCount`, `lifecycleErrorCount` (never the body) +- `api.webhook_failed` (error) — outer pipeline crash with `errorMessage` +- `ui.terminal_connected` / `ui.terminal_disconnected` — one event per mux WS connection lifecycle +- `ui.terminal_heartbeat_lost` (warn) — fires once on 3 missed pongs (was console-only) +- `ui.terminal_pty_lost` (warn) — fires when PTY exits with subscribers attached (distinguishes "PTY died" from "user closed browser") +- `ui.terminal_protocol_error` (warn) — invalid mux client message +- `ui.session_broadcast_failed` (warn) — emitted on the healthy→failing transition only (re-arms after a successful poll), so a long outage produces one event, not 20/min + +`api.webhook_unverified` is the security-audit event; treat 401s on webhooks as a signal worth retaining for the full 7-day window. diff --git a/.changeset/issue-1660-recovery-metadata-events.md b/.changeset/issue-1660-recovery-metadata-events.md new file mode 100644 index 000000000..4ad89f8b0 --- /dev/null +++ b/.changeset/issue-1660-recovery-metadata-events.md @@ -0,0 +1,6 @@ +--- +"@aoagents/ao-cli": patch +"@aoagents/ao-core": minor +--- + +Wire activity events for the recovery subsystem, metadata-corruption detection, and agent-report apply path. New event kinds: `recovery.session_failed`, `recovery.action_failed`, `metadata.corrupt_detected`, `api.agent_report.session_not_found`, `api.agent_report.transition_rejected`. Adds `"recovery"` to the `ActivityEventSource` union. Lets RCA reconstruct `ao recover` invocations, find every silent metadata overwrite, and audit rejected agent transitions. Adds `ao events list --source` and `--kind` so these forensic event queries are available from the CLI. diff --git a/.changeset/launch-orchestrator-clean.md b/.changeset/launch-orchestrator-clean.md new file mode 100644 index 000000000..a13c2f708 --- /dev/null +++ b/.changeset/launch-orchestrator-clean.md @@ -0,0 +1,8 @@ +--- +"@aoagents/ao-core": minor +"@aoagents/ao-web": minor +--- + +feat: "Launch Orchestrator (clean context)" action on the orchestrator session page + +Adds a `Relaunch (clean)` action on the orchestrator session page that replaces the project's canonical orchestrator with a fresh one — killing the existing orchestrator, deleting its metadata, and spawning a new session with no carryover state. Backed by a new `SessionManager.relaunchOrchestrator(config)` method that ignores `orchestratorSessionStrategy`. Removes the now-redundant Orchestrator Selector page (`/orchestrators?project=X`) — there is only ever one orchestrator per project, so a selector page is no longer meaningful. Closes #1900 and #1080. diff --git a/.changeset/quiet-sqlite-rebuild.md b/.changeset/quiet-sqlite-rebuild.md new file mode 100644 index 000000000..79dbfd120 --- /dev/null +++ b/.changeset/quiet-sqlite-rebuild.md @@ -0,0 +1,7 @@ +--- +"@aoagents/ao": patch +"@aoagents/ao-core": patch +"@aoagents/ao-cli": patch +--- + +Rebuild missing better-sqlite3 native bindings during ao postinstall and replace noisy activity-events native-binding failures with a one-line diagnostic. diff --git a/CLAUDE.md b/CLAUDE.md index 42c583e49..731ee041f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -111,7 +111,7 @@ spawning -> working -> pr_open -> ci_failed / review_pending +-> mergeable -> merged -> cleanup -> done ``` -**Stale runtime reconciliation:** `sm.list()` detects dead runtimes (tmux/process gone) during enrichment and persists `runtime_lost` reason to disk. This maps to legacy status `killed`. Without this, sessions with dead runtimes would show stale "active" status indefinitely. +**Stale runtime reconciliation:** `sm.list()` detects dead runtimes (tmux/process gone) during enrichment and persists `detecting` state with `runtime_lost` reason to disk. The lifecycle manager's `resolveProbeDecision` pipeline is the single authority on terminal decisions — `sm.list()` never writes `terminated` directly (#1735). ### Data Flow @@ -224,7 +224,7 @@ Strong success criteria let you loop independently. Weak criteria ("make it work - Kanban board filters client-side via `projectSessions` memo ### Key invariants -- `sm.list()` persists `runtime_lost` lifecycle to disk when enrichment detects dead runtimes — this is the only place stale runtime state gets reconciled +- `sm.list()` persists `detecting` state (not `terminated`) to disk when enrichment detects dead runtimes — terminal decisions are made only by the lifecycle manager's probe pipeline (#1735) - `deriveLegacyStatus()` maps canonical lifecycle to legacy status — new terminal reasons must be added here - Tab completions merge local config + global config to show all projects @@ -450,8 +450,8 @@ import { validateUrl, // Webhook URL validation readLastJsonlEntry, // Efficient JSONL log tail (native agent JSONL) readLastActivityEntry, // Read last AO activity JSONL entry - checkActivityLogState, // Extract waiting_input/blocked from AO JSONL (with staleness cap) - getActivityFallbackState, // Last-resort fallback: entry state + age-based decay + checkActivityLogState, // Extract sticky waiting_input/blocked from AO JSONL + getActivityFallbackState, // Last-resort fallback: actionable states + liveness age decay recordTerminalActivity, // Shared recordActivity impl (classify + dedup + append) classifyTerminalActivity, // Classify terminal output via detectActivity appendActivityEntry, // Low-level JSONL append @@ -460,7 +460,7 @@ import { normalizeAgentPermissionMode, // Normalize permission mode strings DEFAULT_READY_THRESHOLD_MS, // 5 min — ready→idle threshold DEFAULT_ACTIVE_WINDOW_MS, // 30s — active→ready window - ACTIVITY_INPUT_STALENESS_MS, // 5 min — waiting_input/blocked expiry + ACTIVITY_INPUT_STALENESS_MS, // Deprecated compatibility export; actionable states no longer expire by wallclock PREFERRED_GH_PATH, // /usr/local/bin/gh CI_STATUS, ACTIVITY_STATE, SESSION_STATUS, // Constants type Session, type ProjectConfig, type RuntimeHandle, diff --git a/ao-pr1483 b/ao-pr1483 deleted file mode 160000 index 2abb1c1e3..000000000 --- a/ao-pr1483 +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 2abb1c1e3dd804da34baf8262fa44c6766421032 diff --git a/hermes-agent b/hermes-agent deleted file mode 160000 index 27eeea055..000000000 --- a/hermes-agent +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 27eeea0555a0a15b19cea615f46d1f8b88b9dbc1 diff --git a/libkrunfw b/libkrunfw deleted file mode 160000 index 351d354b4..000000000 --- a/libkrunfw +++ /dev/null @@ -1 +0,0 @@ -Subproject commit 351d354b4b3b3e45f38e29897af8acec9966fd41 diff --git a/packages/ao/bin/postinstall.js b/packages/ao/bin/postinstall.js index feb8f21e2..58caae5fc 100644 --- a/packages/ao/bin/postinstall.js +++ b/packages/ao/bin/postinstall.js @@ -11,19 +11,28 @@ * If not (common with nvm/fnm/volta), rebuilds from source via npx node-gyp. * See: https://github.com/ComposioHQ/agent-orchestrator/issues/987 * - * 3. Clears stale Next.js runtime cache (.next/cache) from @composio/ao-web + * 3. Verifies better-sqlite3 has a native binding for this Node ABI. + * Node majors can ship new NODE_MODULE_VERSION values before better-sqlite3 + * publishes matching prebuilds; global installs must rebuild from source. + * See: https://github.com/ComposioHQ/agent-orchestrator/issues/1822 + * + * 4. Clears stale Next.js runtime cache (.next/cache) from @composio/ao-web * after a version upgrade, so `ao start` serves fresh dashboard assets. * Writes a version stamp (.next/AO_VERSION) to skip cleanup on subsequent runs. */ import { chmodSync, existsSync, readFileSync, rmSync, writeFileSync } from "node:fs"; -import { execSync } from "node:child_process"; +import { execFileSync, execSync } from "node:child_process"; import { resolve, dirname } from "node:path"; import { fileURLToPath } from "node:url"; const __dirname = dirname(fileURLToPath(import.meta.url)); -function findPackageUp(startDir, ...segments) { +function isWindows() { + return process.platform === "win32"; +} + +export function findPackageUp(startDir, ...segments) { let dir = resolve(startDir); while (true) { const candidate = resolve(dir, "node_modules", ...segments); @@ -35,12 +44,12 @@ function findPackageUp(startDir, ...segments) { return null; } -function resolveNodeModulesPackage(fromDir, ...segments) { +export function resolveNodeModulesPackage(fromDir, ...segments) { const packageDir = resolve(fromDir, "node_modules", ...segments); return existsSync(resolve(packageDir, "package.json")) ? packageDir : null; } -function findWebDir() { +export function findWebDir() { const directWebDir = findPackageUp(__dirname, "@aoagents", "ao-web"); if (directWebDir) return directWebDir; @@ -50,8 +59,112 @@ function findWebDir() { return resolveNodeModulesPackage(cliDir, "@aoagents", "ao-web"); } -// --- 1 & 2. Fix node-pty spawn-helper permissions and verify ABI (non-Windows only) --- -if (process.platform !== "win32") { +export function findBetterSqlite3Dir() { + const directBetterSqlite3Dir = findPackageUp(__dirname, "better-sqlite3"); + if (directBetterSqlite3Dir) return directBetterSqlite3Dir; + + const cliDir = findPackageUp(__dirname, "@aoagents", "ao-cli"); + if (!cliDir) return null; + + const coreDir = resolveNodeModulesPackage(cliDir, "@aoagents", "ao-core"); + if (!coreDir) return null; + + return ( + resolveNodeModulesPackage(coreDir, "better-sqlite3") ?? findPackageUp(coreDir, "better-sqlite3") + ); +} + +export function betterSqlite3BindingCandidates( + packageDir, + { + platform = process.platform, + arch = process.arch, + modules = process.versions.modules, + nodeVersion = process.versions.node, + } = {}, +) { + return [ + resolve(packageDir, "build", "better_sqlite3.node"), + resolve(packageDir, "build", "Debug", "better_sqlite3.node"), + resolve(packageDir, "build", "Release", "better_sqlite3.node"), + resolve(packageDir, "out", "Debug", "better_sqlite3.node"), + resolve(packageDir, "Debug", "better_sqlite3.node"), + resolve(packageDir, "out", "Release", "better_sqlite3.node"), + resolve(packageDir, "Release", "better_sqlite3.node"), + resolve(packageDir, "build", "default", "better_sqlite3.node"), + resolve(packageDir, "compiled", nodeVersion, platform, arch, "better_sqlite3.node"), + resolve(packageDir, "addon-build", "release", "install-root", "better_sqlite3.node"), + resolve(packageDir, "addon-build", "debug", "install-root", "better_sqlite3.node"), + resolve(packageDir, "addon-build", "default", "install-root", "better_sqlite3.node"), + resolve( + packageDir, + "lib", + "binding", + `node-v${modules}-${platform}-${arch}`, + "better_sqlite3.node", + ), + ]; +} + +export function hasBetterSqlite3Binding(packageDir, options = {}) { + const fileExists = options.existsSync ?? existsSync; + return betterSqlite3BindingCandidates(packageDir, options).some((candidate) => + fileExists(candidate), + ); +} + +export function betterSqlite3RebuildCommand(packageDir, env = process.env) { + const packageManager = + `${env.npm_config_user_agent ?? ""} ${env.npm_execpath ?? ""}`.toLowerCase(); + if (packageManager.includes("npm") && !packageManager.includes("pnpm")) { + return { command: "npm", args: ["rebuild"], display: `cd ${packageDir} && npm rebuild` }; + } + return { + command: "pnpm", + args: ["--dir", packageDir, "rebuild"], + display: `pnpm --dir ${packageDir} rebuild`, + }; +} + +function checkBetterSqlite3Binding() { + const betterSqlite3Dir = findBetterSqlite3Dir(); + if (!betterSqlite3Dir) { + console.warn( + "⚠️ better-sqlite3 package not found; skipping activity-events native binding check", + ); + return; + } + + const abi = process.versions.modules; + if (hasBetterSqlite3Binding(betterSqlite3Dir)) { + console.log( + `✓ better-sqlite3 native binding present for Node ${process.version} (ABI v${abi})`, + ); + return; + } + + const { command, args, display } = betterSqlite3RebuildCommand(betterSqlite3Dir); + try { + execFileSync(command, args, { + cwd: betterSqlite3Dir, + stdio: "ignore", + timeout: 120000, + shell: isWindows(), + windowsHide: true, + }); + console.log( + `✓ better-sqlite3 native binding rebuilt for Node ${process.version} (ABI v${abi})`, + ); + } catch { + console.warn( + `⚠️ better-sqlite3 rebuild failed for Node ${process.version} (ABI v${abi}) — activity events may be unavailable. Manual fix: ${display}`, + ); + } +} + +function fixNodePty() { + if (isWindows()) return; + const nodePtyDir = findPackageUp(__dirname, "node-pty"); if (nodePtyDir) { const spawnHelper = resolve( @@ -84,7 +197,11 @@ if (process.platform !== "win32") { }, ); } catch { - console.log("⚠️ node-pty prebuilt binary incompatible with Node.js " + process.version + ", rebuilding..."); + console.log( + "⚠️ node-pty prebuilt binary incompatible with Node.js " + + process.version + + ", rebuilding...", + ); try { execSync("npx --yes node-gyp rebuild", { cwd: nodePtyDir, @@ -100,27 +217,43 @@ if (process.platform !== "win32") { } } -// --- 3. Clear stale Next.js runtime cache after version upgrade --- -try { - const webDir = findWebDir(); - if (webDir) { - const pkgPath = resolve(webDir, "package.json"); - if (existsSync(pkgPath)) { - const pkg = JSON.parse(readFileSync(pkgPath, "utf8")); - const version = pkg.version; - const cacheDir = resolve(webDir, ".next", "cache"); - const stampPath = resolve(webDir, ".next", "AO_VERSION"); +function clearDashboardCache() { + try { + const webDir = findWebDir(); + if (webDir) { + const pkgPath = resolve(webDir, "package.json"); + if (existsSync(pkgPath)) { + const pkg = JSON.parse(readFileSync(pkgPath, "utf8")); + const version = pkg.version; + const cacheDir = resolve(webDir, ".next", "cache"); + const stampPath = resolve(webDir, ".next", "AO_VERSION"); - if (existsSync(cacheDir)) { - rmSync(cacheDir, { recursive: true, force: true }); - console.log("✓ Cleared stale .next/cache"); - } - if (existsSync(resolve(webDir, ".next"))) { - writeFileSync(stampPath, version, "utf8"); - console.log(`✓ Dashboard version stamp set to ${version}`); + if (existsSync(cacheDir)) { + rmSync(cacheDir, { recursive: true, force: true }); + console.log("✓ Cleared stale .next/cache"); + } + if (existsSync(resolve(webDir, ".next"))) { + writeFileSync(stampPath, version, "utf8"); + console.log(`✓ Dashboard version stamp set to ${version}`); + } } } + } catch (err) { + console.warn(`⚠️ Could not clear dashboard cache (non-critical): ${err.message}`); } -} catch (err) { - console.warn(`⚠️ Could not clear dashboard cache (non-critical): ${err.message}`); +} + +export function runPostinstall() { + // --- 1 & 2. Fix node-pty spawn-helper permissions and verify ABI (non-Windows only) --- + fixNodePty(); + + // --- 3. Ensure better-sqlite3 has a native binding for this Node ABI --- + checkBetterSqlite3Binding(); + + // --- 4. Clear stale Next.js runtime cache after version upgrade --- + clearDashboardCache(); +} + +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + runPostinstall(); } diff --git a/packages/cli/__tests__/commands/events.test.ts b/packages/cli/__tests__/commands/events.test.ts new file mode 100644 index 000000000..e96633234 --- /dev/null +++ b/packages/cli/__tests__/commands/events.test.ts @@ -0,0 +1,87 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { Command } from "commander"; + +const { mockQueryActivityEvents, mockSearchActivityEvents, mockGetActivityEventStats } = vi.hoisted( + () => ({ + mockQueryActivityEvents: vi.fn(), + mockSearchActivityEvents: vi.fn(), + mockGetActivityEventStats: vi.fn(), + }), +); + +vi.mock("@aoagents/ao-core", () => ({ + queryActivityEvents: (...args: unknown[]) => mockQueryActivityEvents(...args), + searchActivityEvents: (...args: unknown[]) => mockSearchActivityEvents(...args), + getActivityEventStats: (...args: unknown[]) => mockGetActivityEventStats(...args), + droppedEventCount: () => 0, + isActivityEventsFtsEnabled: () => true, +})); + +import { registerEvents } from "../../src/commands/events.js"; + +describe("events command", () => { + let program: Command; + let consoleLogSpy: ReturnType; + + beforeEach(() => { + program = new Command(); + program.exitOverride(); + registerEvents(program); + + consoleLogSpy = vi.spyOn(console, "log").mockImplementation(() => {}); + mockQueryActivityEvents.mockReset(); + mockSearchActivityEvents.mockReset(); + mockGetActivityEventStats.mockReset(); + mockQueryActivityEvents.mockReturnValue([]); + }); + + afterEach(() => { + consoleLogSpy.mockRestore(); + }); + + it("filters list output by source and --kind alias", async () => { + await program.parseAsync([ + "node", + "test", + "events", + "list", + "--source", + "recovery", + "--kind", + "metadata.corrupt_detected", + "--limit", + "1", + "--json", + ]); + + expect(mockQueryActivityEvents).toHaveBeenCalledWith( + expect.objectContaining({ + source: "recovery", + kind: "metadata.corrupt_detected", + limit: 1, + }), + ); + expect(consoleLogSpy).toHaveBeenCalledWith(expect.stringContaining('"source": "recovery"')); + expect(consoleLogSpy).toHaveBeenCalledWith( + expect.stringContaining('"kind": "metadata.corrupt_detected"'), + ); + }); + + it("keeps --type as the existing event-kind filter", async () => { + await program.parseAsync([ + "node", + "test", + "events", + "list", + "--type", + "recovery.session_failed", + "--json", + ]); + + expect(mockQueryActivityEvents).toHaveBeenCalledWith( + expect.objectContaining({ + kind: "recovery.session_failed", + }), + ); + }); +}); diff --git a/packages/cli/__tests__/scripts/postinstall.test.ts b/packages/cli/__tests__/scripts/postinstall.test.ts new file mode 100644 index 000000000..9d277e854 --- /dev/null +++ b/packages/cli/__tests__/scripts/postinstall.test.ts @@ -0,0 +1,64 @@ +import { describe, expect, it } from "vitest"; + +import { + betterSqlite3BindingCandidates, + betterSqlite3RebuildCommand, + hasBetterSqlite3Binding, +} from "../../../ao/bin/postinstall.js"; + +describe("ao postinstall better-sqlite3 native binding detection", () => { + const env = { + platform: "darwin", + arch: "arm64", + modules: "141", + nodeVersion: "25.9.0", + }; + + it("checks the current Node ABI binding path", () => { + const packageDir = "virtual-better-sqlite3"; + const candidates = betterSqlite3BindingCandidates(packageDir, env); + + expect(candidates.some((candidate) => candidate.includes("node-v141-darwin-arm64"))).toBe(true); + }); + + it("reports the binding present when a mocked candidate exists", () => { + const packageDir = "virtual-better-sqlite3"; + const candidates = betterSqlite3BindingCandidates(packageDir, env); + const currentAbiBinding = candidates.find((candidate) => + candidate.includes("node-v141-darwin-arm64"), + ); + if (!currentAbiBinding) { + throw new Error("expected current ABI binding candidate"); + } + + const existingFiles = new Set([currentAbiBinding]); + + expect( + hasBetterSqlite3Binding(packageDir, { + ...env, + existsSync: (candidate: string) => existingFiles.has(candidate), + }), + ).toBe(true); + }); + + it("reports the binding missing when no mocked candidate exists", () => { + expect( + hasBetterSqlite3Binding("virtual-better-sqlite3", { + ...env, + existsSync: () => false, + }), + ).toBe(false); + }); + + it("uses pnpm to rebuild inside the resolved better-sqlite3 package", () => { + expect( + betterSqlite3RebuildCommand("virtual-better-sqlite3", { + npm_config_user_agent: "pnpm/9.15.4 npm/? node/v25.9.0 darwin arm64", + }), + ).toEqual({ + command: "pnpm", + args: ["--dir", "virtual-better-sqlite3", "rebuild"], + display: "pnpm --dir virtual-better-sqlite3 rebuild", + }); + }); +}); diff --git a/packages/cli/src/commands/events.ts b/packages/cli/src/commands/events.ts index 87430d2a3..bbb72a7bd 100644 --- a/packages/cli/src/commands/events.ts +++ b/packages/cli/src/commands/events.ts @@ -9,6 +9,7 @@ import { type ActivityEvent, type ActivityEventLevel, type ActivityEventKind, + type ActivityEventSource, } from "@aoagents/ao-core"; interface JsonEnvelope { @@ -80,7 +81,12 @@ export function registerEvents(program: Command): void { .description("List recent activity events") .option("-p, --project ", "Filter by project ID") .option("-s, --session ", "Filter by session ID") - .option("-t, --type ", "Filter by event kind (e.g. session.spawned, lifecycle.transition)") + .option( + "-t, --type ", + "Filter by event kind (e.g. session.spawned, lifecycle.transition)", + ) + .option("--kind ", "Alias for --type") + .option("--source ", "Filter by event source (e.g. lifecycle, recovery, api)") .option("--log-level ", "Filter by log level (debug, info, warn, error)") .option("--since ", "Show events from last N minutes/hours/days (e.g. 30m, 2h, 1d)") .option("-n, --limit ", "Max results", "50") @@ -91,15 +97,21 @@ export function registerEvents(program: Command): void { if (sinceRaw) { since = parseSinceDuration(sinceRaw); if (!since) { - console.error(chalk.yellow(`Warning: unrecognised --since format "${sinceRaw}" (use e.g. 30m, 2h, 1d). No time filter applied.`)); + console.error( + chalk.yellow( + `Warning: unrecognised --since format "${sinceRaw}" (use e.g. 30m, 2h, 1d). No time filter applied.`, + ), + ); } } const limit = parseInt(opts["limit"] ?? "50", 10); + const kind = opts["type"] ?? opts["kind"]; const results = queryActivityEvents({ projectId: opts["project"], sessionId: opts["session"], - kind: opts["type"] as ActivityEventKind, + kind: kind as ActivityEventKind, + source: opts["source"] as ActivityEventSource, level: opts["logLevel"] as ActivityEventLevel, since, limit, @@ -111,7 +123,8 @@ export function registerEvents(program: Command): void { query: { projectId: opts["project"] ?? null, sessionId: opts["session"] ?? null, - kind: opts["type"] ?? null, + kind: kind ?? null, + source: opts["source"] ?? null, level: opts["logLevel"] ?? null, since: sinceRaw ?? null, limit, diff --git a/packages/core/package.json b/packages/core/package.json index 0b83ed14b..3ac2696d8 100644 --- a/packages/core/package.json +++ b/packages/core/package.json @@ -91,7 +91,7 @@ "zod": "^3.24.0" }, "optionalDependencies": { - "better-sqlite3": "^11.0.0" + "better-sqlite3": "^12.10.0" }, "devDependencies": { "@rollup/plugin-typescript": "^12.3.0", @@ -99,8 +99,8 @@ "@types/node": "^25.2.3", "@vitest/coverage-v8": "^4.0.18", "rollup": "^4.60.1", - "tsx": "^4.21.0", "tslib": "^2.8.1", + "tsx": "^4.21.0", "typescript": "^5.7.0", "vitest": "^4.0.18" }, diff --git a/packages/core/src/__tests__/activity-log.test.ts b/packages/core/src/__tests__/activity-log.test.ts index 858cfea74..6d8288b8d 100644 --- a/packages/core/src/__tests__/activity-log.test.ts +++ b/packages/core/src/__tests__/activity-log.test.ts @@ -9,9 +9,26 @@ import { appendActivityEntry, recordTerminalActivity, getActivityLogPath, - ACTIVITY_INPUT_STALENESS_MS, + getActivityFallbackState, } from "../activity-log.js"; -import type { ActivityState } from "../types.js"; +import type { ActivityDetection, ActivityLogEntry, ActivityState } from "../types.js"; + +const minutesAgo = (minutes: number): string => new Date(Date.now() - minutes * 60_000).toISOString(); + +const toActivityResult = ( + entry: ActivityLogEntry, +): { entry: ActivityLogEntry; modifiedAt: Date } => ({ + entry, + modifiedAt: new Date(entry.ts), +}); + +const detectWithProcessCheck = ( + isProcessRunning: boolean, + activityResult: { entry: ActivityLogEntry; modifiedAt: Date } | null, +): ActivityDetection | null => { + if (!isProcessRunning) return { state: "exited", timestamp: new Date() }; + return checkActivityLogState(activityResult) ?? getActivityFallbackState(activityResult, 30_000, 5 * 60_000); +}; describe("classifyTerminalActivity", () => { it("returns active state with no trigger", () => { @@ -56,13 +73,20 @@ describe("checkActivityLogState", () => { expect(result?.state).toBe("blocked"); }); - it("returns null for stale waiting_input entry", () => { - const staleTs = new Date(Date.now() - ACTIVITY_INPUT_STALENESS_MS - 1000).toISOString(); + it("returns waiting_input even when older than the former wallclock cap", () => { const result = checkActivityLogState({ - entry: { ts: staleTs, state: "waiting_input", source: "terminal" }, + entry: { ts: minutesAgo(10), state: "waiting_input", source: "terminal" }, modifiedAt: new Date(), }); - expect(result).toBeNull(); + expect(result?.state).toBe("waiting_input"); + }); + + it("returns blocked even when older than the former wallclock cap", () => { + const result = checkActivityLogState({ + entry: { ts: minutesAgo(6), state: "blocked", source: "terminal" }, + modifiedAt: new Date(), + }); + expect(result?.state).toBe("blocked"); }); it("returns null for non-critical states", () => { @@ -82,6 +106,77 @@ describe("checkActivityLogState", () => { }); }); +describe("getActivityFallbackState", () => { + it("returns waiting_input for a 10-minute-old entry instead of decaying to idle", () => { + const result = getActivityFallbackState( + toActivityResult({ ts: minutesAgo(10), state: "waiting_input", source: "terminal" }), + 30_000, + 5 * 60_000, + ); + + expect(result?.state).toBe("waiting_input"); + }); + + it("returns blocked for a 6-minute-old entry instead of decaying to idle", () => { + const result = getActivityFallbackState( + toActivityResult({ ts: minutesAgo(6), state: "blocked", source: "terminal" }), + 30_000, + 5 * 60_000, + ); + + expect(result?.state).toBe("blocked"); + }); + + it("returns blocked for a 1-minute-old entry with unchanged behavior", () => { + const result = getActivityFallbackState( + toActivityResult({ ts: minutesAgo(1), state: "blocked", source: "terminal" }), + 30_000, + 5 * 60_000, + ); + + expect(result?.state).toBe("blocked"); + }); + + it("lets a newer active entry override an older waiting_input entry", async () => { + const tmpDir = await mkdtemp(join(tmpdir(), "ao-test-")); + try { + await mkdir(join(tmpDir, ".ao"), { recursive: true }); + const waitingEntry: ActivityLogEntry = { + ts: minutesAgo(6), + state: "waiting_input", + source: "terminal", + }; + const activeEntry: ActivityLogEntry = { + ts: new Date(Date.now() - 1000).toISOString(), + state: "active", + source: "terminal", + }; + await writeFile( + getActivityLogPath(tmpDir), + `${JSON.stringify(waitingEntry)}\n${JSON.stringify(activeEntry)}\n`, + "utf-8", + ); + + const activityResult = await readLastActivityEntry(tmpDir); + const result = getActivityFallbackState(activityResult, 30_000, 5 * 60_000); + + expect(activityResult?.entry.state).toBe("active"); + expect(result?.state).toBe("active"); + } finally { + await rm(tmpDir, { recursive: true, force: true }); + } + }); + + it("returns exited when the process check fails before a stale waiting_input can fall through", () => { + const result = detectWithProcessCheck( + false, + toActivityResult({ ts: minutesAgo(6), state: "waiting_input", source: "terminal" }), + ); + + expect(result?.state).toBe("exited"); + }); +}); + describe("readLastActivityEntry", () => { let tmpDir: string; @@ -144,6 +239,25 @@ describe("readLastActivityEntry", () => { const result = await readLastActivityEntry(tmpDir); expect(result).toBeNull(); }); + + it("falls back to the previous complete line when a read races a truncated tail", async () => { + await mkdir(join(tmpDir, ".ao"), { recursive: true }); + const completeEntry: ActivityLogEntry = { + ts: minutesAgo(10), + state: "waiting_input", + source: "terminal", + trigger: "approve?", + }; + await writeFile( + getActivityLogPath(tmpDir), + `${JSON.stringify(completeEntry)}\n{"ts":"${new Date().toISOString()}","state":`, + "utf-8", + ); + + const result = await readLastActivityEntry(tmpDir); + + expect(result?.entry).toEqual(completeEntry); + }); }); describe("recordTerminalActivity", () => { diff --git a/packages/core/src/__tests__/agent-report.test.ts b/packages/core/src/__tests__/agent-report.test.ts index 7e3fddce1..f49483b9d 100644 --- a/packages/core/src/__tests__/agent-report.test.ts +++ b/packages/core/src/__tests__/agent-report.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect, beforeEach, afterEach } from "vitest"; +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; import { mkdirSync, readFileSync, rmSync } from "node:fs"; import { join } from "node:path"; import { tmpdir } from "node:os"; @@ -18,17 +18,25 @@ import { } from "../agent-report.js"; import { writeMetadata, writeCanonicalLifecycle, readMetadataRaw } from "../metadata.js"; import { createInitialCanonicalLifecycle } from "../lifecycle-state.js"; +import { recordActivityEvent } from "../activity-events.js"; import type { CanonicalSessionLifecycle } from "../types.js"; +vi.mock("../activity-events.js", () => ({ + recordActivityEvent: vi.fn(), +})); + let dataDir: string; +let tempRoot: string; beforeEach(() => { - dataDir = join(tmpdir(), `ao-test-agent-report-${randomUUID()}`); + tempRoot = join(tmpdir(), `ao-test-agent-report-${randomUUID()}`); + dataDir = join(tempRoot, "projects", "project-alpha", "sessions"); mkdirSync(dataDir, { recursive: true }); + vi.mocked(recordActivityEvent).mockClear(); }); afterEach(() => { - rmSync(dataDir, { recursive: true, force: true }); + rmSync(tempRoot, { recursive: true, force: true }); }); function seedWorkerSession( @@ -433,6 +441,13 @@ describe("applyAgentReport", () => { sessionState: "terminated", }, }); + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + projectId: "project-alpha", + sessionId, + kind: "api.agent_report.transition_rejected", + }), + ); }); it("throws when the session does not exist", () => { @@ -442,6 +457,13 @@ describe("applyAgentReport", () => { now: new Date(), }), ).toThrow(/not found/); + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + projectId: "project-alpha", + sessionId: "missing-session", + kind: "api.agent_report.session_not_found", + }), + ); }); // 260 atomic-write cycles are slow on Windows (rename + AV scan); bump the diff --git a/packages/core/src/__tests__/events-db.test.ts b/packages/core/src/__tests__/events-db.test.ts new file mode 100644 index 000000000..f1f21237a --- /dev/null +++ b/packages/core/src/__tests__/events-db.test.ts @@ -0,0 +1,72 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +import { + __resetActivityEventsDbWarningForTests, + emitActivityEventsDbUnavailableWarning, + formatActivityEventsDbUnavailableWarning, +} from "../events-db.js"; + +describe("activity-events DB unavailable warning", () => { + const originalArgv = process.argv; + const originalDebug = process.env["AO_DEBUG"]; + + beforeEach(() => { + __resetActivityEventsDbWarningForTests(); + vi.spyOn(console, "warn").mockImplementation(() => {}); + delete process.env["AO_DEBUG"]; + process.argv = ["node", "ao"]; + }); + + afterEach(() => { + process.argv = originalArgv; + if (originalDebug === undefined) { + delete process.env["AO_DEBUG"]; + } else { + process.env["AO_DEBUG"] = originalDebug; + } + vi.restoreAllMocks(); + }); + + it("formats missing native binding errors without the bindings search path", () => { + const message = formatActivityEventsDbUnavailableWarning( + new Error( + "Could not locate the bindings file. Tried:\n → /tmp/build/Release/better_sqlite3.node", + ), + ); + + expect(message).toBe( + `[ao] activity-events disabled: better-sqlite3 not compiled for Node ${process.version} (ABI v${process.versions.modules}). Run \`pnpm rebuild better-sqlite3\` or use a supported Node version.`, + ); + expect(message).not.toContain("Tried:"); + expect(message).not.toContain("/tmp/build/Release"); + }); + + it("prints the runtime warning once per process", () => { + process.env["AO_DEBUG"] = "1"; + const err = new Error( + "Could not locate the bindings file. Tried:\n → /tmp/better_sqlite3.node", + ); + + emitActivityEventsDbUnavailableWarning(err); + emitActivityEventsDbUnavailableWarning(err); + + expect(console.warn).toHaveBeenCalledTimes(1); + expect(vi.mocked(console.warn).mock.calls[0]?.[0]).toContain( + "activity-events disabled: better-sqlite3 not compiled", + ); + }); + + it("suppresses non-events invocations unless AO_DEBUG=1", () => { + const err = new Error( + "Could not locate the bindings file. Tried:\n → /tmp/better_sqlite3.node", + ); + + process.argv = ["node", "ao", "spawn", "demo"]; + emitActivityEventsDbUnavailableWarning(err); + expect(console.warn).not.toHaveBeenCalled(); + + process.argv = ["node", "ao", "events", "stats"]; + emitActivityEventsDbUnavailableWarning(err); + expect(console.warn).toHaveBeenCalledTimes(1); + }); +}); diff --git a/packages/core/src/__tests__/metadata.test.ts b/packages/core/src/__tests__/metadata.test.ts index 03ac1c16b..47c66b0e0 100644 --- a/packages/core/src/__tests__/metadata.test.ts +++ b/packages/core/src/__tests__/metadata.test.ts @@ -1,5 +1,6 @@ -import { describe, it, expect, beforeEach, afterEach } from "vitest"; -import { mkdirSync, rmSync, readFileSync, existsSync, writeFileSync, readdirSync } from "node:fs"; +import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; +import { mkdirSync, rmSync, readFileSync, existsSync, writeFileSync, readdirSync, renameSync } from "node:fs"; +import type * as NodeFs from "node:fs"; import { join } from "node:path"; import { tmpdir } from "node:os"; import { randomUUID } from "node:crypto"; @@ -13,12 +14,29 @@ import { deleteMetadata, listMetadata, } from "../metadata.js"; +import { recordActivityEvent } from "../activity-events.js"; + +vi.mock("node:fs", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + renameSync: vi.fn((...args: Parameters) => + actual.renameSync(...args), + ), + }; +}); + +vi.mock("../activity-events.js", () => ({ + recordActivityEvent: vi.fn(), +})); let dataDir: string; beforeEach(() => { dataDir = join(tmpdir(), `ao-test-metadata-${randomUUID()}`); mkdirSync(dataDir, { recursive: true }); + vi.mocked(recordActivityEvent).mockClear(); + vi.mocked(renameSync).mockClear(); }); afterEach(() => { @@ -390,6 +408,123 @@ describe("mutateMetadata corrupt-file handling", () => { const corruptCopies = readdirSync(dataDir).filter((f) => f.includes(".corrupt-")); expect(corruptCopies).toHaveLength(0); }); + + it("emits metadata.corrupt_detected when JSON parse fails and file is renamed", () => { + const sessionPath = join(dataDir, "ao-3.json"); + writeFileSync(sessionPath, "{ broken json", "utf-8"); + + const result = mutateMetadata( + dataDir, + "ao-3", + (existing) => ({ ...existing, branch: "feat/x" }), + { createIfMissing: true }, + ); + + expect(result).not.toBeNull(); + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + sessionId: "ao-3", + source: "session-manager", + kind: "metadata.corrupt_detected", + level: "error", + summary: expect.stringContaining("renamed to"), + data: expect.objectContaining({ + renamedTo: expect.stringContaining(`${sessionPath}.corrupt-`), + renameSucceeded: true, + contentSample: "{ broken json", + path: sessionPath, + }), + }), + ); + }); + + it("emits a rename-failed summary when corrupt metadata cannot be renamed", () => { + const sessionPath = join(dataDir, "ao-rename-failed.json"); + writeFileSync(sessionPath, "{ broken json", "utf-8"); + vi.mocked(renameSync).mockImplementationOnce(() => { + throw new Error("rename denied"); + }); + + const result = mutateMetadata( + dataDir, + "ao-rename-failed", + (existing) => ({ ...existing, branch: "feat/x" }), + { createIfMissing: true }, + ); + + expect(result).not.toBeNull(); + const call = vi + .mocked(recordActivityEvent) + .mock.calls.find((c) => c[0].kind === "metadata.corrupt_detected"); + expect(call).toBeDefined(); + expect(call![0]).toMatchObject({ + sessionId: "ao-rename-failed", + summary: expect.stringContaining("failed to rename"), + data: expect.objectContaining({ + renamedTo: null, + renameSucceeded: false, + path: sessionPath, + }), + }); + expect(call![0].summary).not.toContain("renamed to"); + }); + + it("uses the provided source for metadata.corrupt_detected", () => { + const sessionPath = join(dataDir, "ao-api-source.json"); + writeFileSync(sessionPath, "{ broken json", "utf-8"); + + mutateMetadata( + dataDir, + "ao-api-source", + (existing) => ({ ...existing, branch: "feat/api" }), + { createIfMissing: true, activityEventSource: "api" }, + ); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + sessionId: "ao-api-source", + source: "api", + kind: "metadata.corrupt_detected", + }), + ); + }); + + it("truncates contentSample to 200 chars in metadata.corrupt_detected", () => { + const sessionPath = join(dataDir, "ao-4.json"); + // 250 char garbage payload — sanitizer cap is 16KB but invariant B11 caps + // forensic sample at 200 chars. + const huge = "x".repeat(250); + writeFileSync(sessionPath, huge, "utf-8"); + + mutateMetadata( + dataDir, + "ao-4", + (existing) => ({ ...existing, branch: "feat/y" }), + { createIfMissing: true }, + ); + + const call = vi + .mocked(recordActivityEvent) + .mock.calls.find((c) => c[0].kind === "metadata.corrupt_detected"); + expect(call).toBeDefined(); + const sample = (call![0].data as Record)["contentSample"] as string; + expect(sample.length).toBe(200); + expect((call![0].data as Record)["contentLength"]).toBe(250); + }); + + it("does not emit metadata.corrupt_detected for healthy JSON", () => { + writeMetadata(dataDir, "ao-5", { + worktree: "/tmp/w", + branch: "main", + status: "working", + }); + mutateMetadata(dataDir, "ao-5", (existing) => ({ ...existing, summary: "hi" })); + + const corruptCalls = vi + .mocked(recordActivityEvent) + .mock.calls.filter((c) => c[0].kind === "metadata.corrupt_detected"); + expect(corruptCalls).toHaveLength(0); + }); }); describe("readCanonicalLifecycle", () => { diff --git a/packages/core/src/__tests__/recovery-manager.test.ts b/packages/core/src/__tests__/recovery-manager.test.ts new file mode 100644 index 000000000..80647dc29 --- /dev/null +++ b/packages/core/src/__tests__/recovery-manager.test.ts @@ -0,0 +1,269 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { existsSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { join } from "node:path"; +import { tmpdir } from "node:os"; +import { randomUUID } from "node:crypto"; +import { recoverSessionById, runRecovery } from "../recovery/manager.js"; +import { recordActivityEvent } from "../activity-events.js"; +import { getProjectDir, getProjectSessionsDir } from "../paths.js"; +import { + DEFAULT_RECOVERY_CONFIG, + type RecoveryAssessment, + type RecoveryResult, +} from "../recovery/types.js"; +import * as actionsModule from "../recovery/actions.js"; +import * as validatorModule from "../recovery/validator.js"; +import type { OrchestratorConfig, PluginRegistry } from "../types.js"; + +vi.mock("../activity-events.js", () => ({ + recordActivityEvent: vi.fn(), +})); + +const PROJECT_ID = "app"; + +function makeConfig(rootDir: string): OrchestratorConfig { + return { + configPath: join(rootDir, "agent-orchestrator.yaml"), + port: 3000, + readyThresholdMs: 300_000, + power: { preventIdleSleep: false }, + defaults: { + runtime: "tmux", + agent: "claude-code", + workspace: "worktree", + notifiers: ["desktop"], + }, + projects: { + app: { + name: "app", + repo: "org/repo", + path: join(rootDir, "project"), + defaultBranch: "main", + sessionPrefix: "app", + }, + }, + notifiers: {}, + notificationRouting: { + urgent: ["desktop"], + action: ["desktop"], + warning: ["desktop"], + info: ["desktop"], + }, + reactions: {}, + }; +} + +function makeRegistry(): PluginRegistry { + return { + register: vi.fn(), + get: vi.fn().mockReturnValue(null), + list: vi.fn().mockReturnValue([]), + loadBuiltins: vi.fn().mockResolvedValue(undefined), + loadFromConfig: vi.fn().mockResolvedValue(undefined), + }; +} + +function makeAssessment(sessionId: string): RecoveryAssessment { + return { + sessionId, + projectId: PROJECT_ID, + classification: "live", + action: "recover", + reason: "needs recovery", + runtimeProbeSucceeded: true, + processProbeSucceeded: true, + signalDisagreement: false, + recoveryRule: "auto", + runtimeAlive: true, + runtimeHandle: null, + workspaceExists: true, + workspacePath: "/tmp/worktree", + agentProcessRunning: true, + agentActivity: "active", + metadataValid: true, + metadataStatus: "working", + rawMetadata: { project: PROJECT_ID, status: "working" }, + }; +} + +describe("runRecovery activity events", () => { + let rootDir: string; + let previousHome: string | undefined; + + beforeEach(() => { + rootDir = join(tmpdir(), `ao-recovery-events-${randomUUID()}`); + mkdirSync(rootDir, { recursive: true }); + mkdirSync(join(rootDir, "project"), { recursive: true }); + writeFileSync(join(rootDir, "agent-orchestrator.yaml"), "projects: {}\n", "utf-8"); + previousHome = process.env["HOME"]; + process.env["HOME"] = rootDir; + + const sessionsDir = getProjectSessionsDir(PROJECT_ID); + mkdirSync(sessionsDir, { recursive: true }); + writeFileSync( + join(sessionsDir, "app-1.json"), + JSON.stringify({ project: PROJECT_ID, status: "working" }) + "\n", + "utf-8", + ); + writeFileSync( + join(sessionsDir, "app-2.json"), + JSON.stringify({ project: PROJECT_ID, status: "working" }) + "\n", + "utf-8", + ); + + vi.mocked(recordActivityEvent).mockClear(); + vi.restoreAllMocks(); + }); + + afterEach(() => { + if (previousHome === undefined) { + delete process.env["HOME"]; + } else { + process.env["HOME"] = previousHome; + } + if (rootDir) { + const projectBaseDir = getProjectDir(PROJECT_ID); + if (existsSync(projectBaseDir)) { + rmSync(projectBaseDir, { recursive: true, force: true }); + } + rmSync(rootDir, { recursive: true, force: true }); + } + vi.restoreAllMocks(); + }); + + it("emits recovery.session_failed for each session that recovery couldn't fix", async () => { + vi.spyOn(validatorModule, "validateSession").mockImplementation(async (scanned) => + makeAssessment(scanned.sessionId), + ); + + const successResult: RecoveryResult = { + success: true, + sessionId: "app-1", + action: "recover", + }; + const failedResult: RecoveryResult = { + success: false, + sessionId: "app-2", + action: "recover", + error: "worktree missing", + }; + + vi.spyOn(actionsModule, "executeAction") + .mockResolvedValueOnce(successResult) + .mockResolvedValueOnce(failedResult); + + const config = makeConfig(rootDir); + const registry = makeRegistry(); + + const { report } = await runRecovery({ + config, + registry, + recoveryConfig: { + ...DEFAULT_RECOVERY_CONFIG, + logPath: join(rootDir, "recovery.log"), + }, + }); + + expect(report.errors).toHaveLength(1); + expect(report.errors[0]?.sessionId).toBe("app-2"); + + const emitCalls = vi + .mocked(recordActivityEvent) + .mock.calls.map((c) => c[0]) + .filter((e) => e.kind === "recovery.session_failed"); + + expect(emitCalls).toHaveLength(1); + expect(emitCalls[0]).toEqual( + expect.objectContaining({ + sessionId: "app-2", + projectId: PROJECT_ID, + source: "recovery", + kind: "recovery.session_failed", + level: "error", + data: expect.objectContaining({ + action: "recover", + errorMessage: "worktree missing", + }), + }), + ); + }); + + it("emits recovery.session_failed when single-session recovery fails", async () => { + vi.spyOn(validatorModule, "validateSession").mockImplementation(async (scanned) => + makeAssessment(scanned.sessionId), + ); + + const failedResult: RecoveryResult = { + success: false, + sessionId: "app-2", + action: "recover", + error: "agent process missing", + }; + + vi.spyOn(actionsModule, "executeAction").mockResolvedValueOnce(failedResult); + + const config = makeConfig(rootDir); + const registry = makeRegistry(); + + const result = await recoverSessionById("app-2", { + config, + registry, + recoveryConfig: { + ...DEFAULT_RECOVERY_CONFIG, + logPath: join(rootDir, "recovery.log"), + }, + }); + + expect(result).toEqual(failedResult); + + const emitCalls = vi + .mocked(recordActivityEvent) + .mock.calls.map((c) => c[0]) + .filter((e) => e.kind === "recovery.session_failed"); + + expect(emitCalls).toHaveLength(1); + expect(emitCalls[0]).toEqual( + expect.objectContaining({ + sessionId: "app-2", + projectId: PROJECT_ID, + source: "recovery", + kind: "recovery.session_failed", + level: "error", + data: expect.objectContaining({ + action: "recover", + errorMessage: "agent process missing", + }), + }), + ); + }); + + it("does not emit recovery.session_failed when every session recovers cleanly", async () => { + vi.spyOn(validatorModule, "validateSession").mockImplementation(async (scanned) => + makeAssessment(scanned.sessionId), + ); + + vi.spyOn(actionsModule, "executeAction").mockImplementation(async (assessment) => ({ + success: true, + sessionId: assessment.sessionId, + action: "recover", + })); + + const config = makeConfig(rootDir); + const registry = makeRegistry(); + + await runRecovery({ + config, + registry, + recoveryConfig: { + ...DEFAULT_RECOVERY_CONFIG, + logPath: join(rootDir, "recovery.log"), + }, + }); + + const failedEmits = vi + .mocked(recordActivityEvent) + .mock.calls.map((c) => c[0]) + .filter((e) => e.kind === "recovery.session_failed"); + expect(failedEmits).toHaveLength(0); + }); +}); diff --git a/packages/core/src/__tests__/session-manager/query.test.ts b/packages/core/src/__tests__/session-manager/query.test.ts index e8acb988d..17300fca6 100644 --- a/packages/core/src/__tests__/session-manager/query.test.ts +++ b/packages/core/src/__tests__/session-manager/query.test.ts @@ -8,6 +8,7 @@ import { createSessionManager } from "../../session-manager.js"; import { writeMetadata, readMetadataRaw, + updateMetadata, } from "../../metadata.js"; import { createInitialCanonicalLifecycle } from "../../lifecycle-state.js"; import type { @@ -63,6 +64,47 @@ describe("list", () => { expect(sessions.map((s) => s.id).sort()).toEqual(["app-1", "app-2"]); }); + it("skips dead-runtime agent metadata discovery when native restore metadata is already persisted", async () => { + writeMetadata(sessionsDir, "app-1", { + worktree: config.projects["my-app"]!.path, + branch: "feat/a", + status: "killed", + project: "my-app", + runtimeHandle: makeHandle("rt-old"), + }); + updateMetadata(sessionsDir, "app-1", { codexThreadId: "thread-1" }); + + const deadRuntime: Runtime = { + ...mockRuntime, + isAlive: vi.fn().mockResolvedValue(false), + }; + const agentWithSessionInfo: Agent = { + ...mockAgent, + name: "codex", + getSessionInfo: vi.fn().mockResolvedValue({ + summary: null, + agentSessionId: "rollout-1", + metadata: { codexThreadId: "thread-1" }, + }), + }; + const registryWithDeadRuntime: PluginRegistry = { + ...mockRegistry, + get: vi.fn().mockImplementation((slot: string) => { + if (slot === "runtime") return deadRuntime; + if (slot === "agent") return agentWithSessionInfo; + if (slot === "workspace") return mockWorkspace; + return null; + }), + }; + + const sm = createSessionManager({ config, registry: registryWithDeadRuntime }); + await sm.list("my-app"); + await sm.list("my-app"); + + expect(agentWithSessionInfo.getSessionInfo).not.toHaveBeenCalled(); + expect(readMetadataRaw(sessionsDir, "app-1")!["codexThreadId"]).toBe("thread-1"); + }); + it("does not backfill role onto foreign bare-id orchestrator records (issue #1048)", async () => { // Regression guard for PR #1075 review comment: a legacy record whose id // is `{projectId}-orchestrator` (pre-numbered scheme, wrong prefix) must @@ -220,7 +262,9 @@ describe("list", () => { const sm = createSessionManager({ config, registry: registryWithDead }); const sessions = await sm.list(); - expect(sessions[0].status).toBe("killed"); + // sm.list() persists "detecting" (not "terminated") so the lifecycle + // manager's probe pipeline makes the final terminal decision (#1735). + expect(sessions[0].status).toBe("detecting"); expect(sessions[0].activity).toBe("exited"); }); @@ -336,7 +380,8 @@ describe("list", () => { expect(sessions).toHaveLength(1); expect(sessions[0].runtimeHandle?.id).toBe(expectedTmuxName); - expect(sessions[0].status).toBe("killed"); + // sm.list() persists "detecting" so the lifecycle manager decides (#1735). + expect(sessions[0].status).toBe("detecting"); expect(sessions[0].activity).toBe("exited"); expect(agentWithSpy.getActivityState).not.toHaveBeenCalled(); }); diff --git a/packages/core/src/__tests__/session-manager/restore.test.ts b/packages/core/src/__tests__/session-manager/restore.test.ts index 9445d16f3..3ba13d03c 100644 --- a/packages/core/src/__tests__/session-manager/restore.test.ts +++ b/packages/core/src/__tests__/session-manager/restore.test.ts @@ -575,7 +575,7 @@ describe("restore", () => { expect(meta!["restoreFallbackReason"]).toBe("mock-agent.getRestoreCommand returned null"); }); - it("does not launch a fresh chat when a native-restore agent cannot build restore command", async () => { + it("falls back to a fresh launch when a native-restore agent cannot build restore command", async () => { const wsPath = join(tmpDir, "ws-app-native-restore-missing"); mkdirSync(wsPath, { recursive: true }); @@ -605,12 +605,119 @@ describe("restore", () => { const sm = createSessionManager({ config, registry: registryWithNativeRestoreAgent }); - await expect(sm.restore("app-1")).rejects.toThrow(SessionNotRestorableError); - expect(mockRuntime.create).not.toHaveBeenCalled(); + await sm.restore("app-1"); + + expect(mockRuntime.create).toHaveBeenCalled(); + const createCall = (mockRuntime.create as ReturnType).mock.calls[0][0]; + expect(createCall.launchCommand).toBe("mock-agent --start"); const meta = readMetadataRaw(sessionsDir, "app-1"); expect(meta!["restoreFallbackReason"]).toBe("codex.getRestoreCommand returned null"); }); + it("persists native restore metadata even when runtime is already dead", async () => { + const wsPath = join(tmpDir, "ws-app-dead-runtime-metadata"); + mkdirSync(wsPath, { recursive: true }); + + const deadRuntime: Runtime = { + ...mockRuntime, + isAlive: vi.fn().mockResolvedValue(false), + }; + const agentWithDiscoverableThread: Agent = { + ...mockAgent, + name: "codex", + getSessionInfo: vi.fn().mockResolvedValue({ + summary: null, + agentSessionId: "rollout-1", + metadata: { codexThreadId: "thread-1" }, + }), + getRestoreCommand: vi + .fn() + .mockImplementation(async (session) => + session.metadata?.codexThreadId ? `codex resume ${session.metadata.codexThreadId}` : null, + ), + }; + + const registryWithDeadRuntime: PluginRegistry = { + ...mockRegistry, + get: vi.fn().mockImplementation((slot: string) => { + if (slot === "runtime") return deadRuntime; + if (slot === "agent") return agentWithDiscoverableThread; + if (slot === "workspace") return mockWorkspace; + return null; + }), + }; + + writeMetadata(sessionsDir, "app-1", { + worktree: wsPath, + branch: "feat/TEST-1", + status: "killed", + project: "my-app", + runtimeHandle: makeHandle("rt-old"), + }); + + const sm = createSessionManager({ config, registry: registryWithDeadRuntime }); + const restored = await sm.restore("app-1"); + + expect(agentWithDiscoverableThread.getSessionInfo).toHaveBeenCalled(); + expect(agentWithDiscoverableThread.getRestoreCommand).toHaveBeenCalledWith( + expect.objectContaining({ + metadata: expect.objectContaining({ codexThreadId: "thread-1" }), + }), + expect.any(Object), + ); + expect(restored.metadata["codexThreadId"]).toBe("thread-1"); + const createCall = (deadRuntime.create as ReturnType).mock.calls[0][0]; + expect(createCall.launchCommand).toBe("codex resume thread-1"); + const meta = readMetadataRaw(sessionsDir, "app-1"); + expect(meta!["codexThreadId"]).toBe("thread-1"); + }); + + it("uses project path as restore workspace when worktree metadata is missing", async () => { + mkdirSync(config.projects["my-app"]!.path, { recursive: true }); + + const agentWithWorkspaceAssertion: Agent = { + ...mockAgent, + name: "codex", + getRestoreCommand: vi + .fn() + .mockImplementation(async (session) => + session.workspacePath === config.projects["my-app"]!.path + ? "codex resume thread-1" + : null, + ), + }; + + const registryWithWorkspaceAssertion: PluginRegistry = { + ...mockRegistry, + get: vi.fn().mockImplementation((slot: string) => { + if (slot === "runtime") return mockRuntime; + if (slot === "agent") return agentWithWorkspaceAssertion; + if (slot === "workspace") return mockWorkspace; + return null; + }), + }; + + writeMetadata(sessionsDir, "app-1", { + worktree: "", + branch: "feat/TEST-1", + status: "killed", + project: "my-app", + runtimeHandle: makeHandle("rt-old"), + }); + + const sm = createSessionManager({ config, registry: registryWithWorkspaceAssertion }); + const restored = await sm.restore("app-1"); + + expect(restored.workspacePath).toBe(config.projects["my-app"]!.path); + expect(agentWithWorkspaceAssertion.getRestoreCommand).toHaveBeenCalledWith( + expect.objectContaining({ workspacePath: config.projects["my-app"]!.path }), + expect.any(Object), + ); + const createCall = (mockRuntime.create as ReturnType).mock.calls[0][0]; + expect(createCall.workspacePath).toBe(config.projects["my-app"]!.path); + expect(createCall.launchCommand).toBe("codex resume thread-1"); + }); + it("clears restore fallback reason when getRestoreCommand succeeds", async () => { const wsPath = join(tmpDir, "ws-app-restore-clears-fallback"); mkdirSync(wsPath, { recursive: true }); diff --git a/packages/core/src/__tests__/session-manager/spawn.test.ts b/packages/core/src/__tests__/session-manager/spawn.test.ts index 8f2057e27..d92c1a4f1 100644 --- a/packages/core/src/__tests__/session-manager/spawn.test.ts +++ b/packages/core/src/__tests__/session-manager/spawn.test.ts @@ -13,7 +13,7 @@ import { readMetadata, readMetadataRaw, } from "../../metadata.js"; -import { getProjectWorktreesDir } from "../../paths.js"; +import { getProjectDir, getProjectWorktreesDir } from "../../paths.js"; import type { OrchestratorConfig, PluginRegistry, @@ -2489,4 +2489,183 @@ describe("spawn", () => { }); }); + + describe("relaunchOrchestrator", () => { + it("spawns a fresh orchestrator when none exists", async () => { + const sm = createSessionManager({ config, registry: mockRegistry }); + + const session = await sm.relaunchOrchestrator({ projectId: "my-app" }); + + expect(session.id).toBe("app-orchestrator"); + expect(session.status).toBe("working"); + expect(mockWorkspace.create).toHaveBeenCalledTimes(1); + expect(mockRuntime.create).toHaveBeenCalledTimes(1); + }); + + it("kills and replaces an existing orchestrator regardless of strategy", async () => { + writeMetadata(sessionsDir, "app-orchestrator", { + role: "orchestrator", + project: "my-app", + status: "working", + branch: "orchestrator/app-orchestrator", + worktree: join(tmpDir, "old-orchestrator-ws"), + runtimeHandle: makeHandle("old-rt"), + }); + const sm = createSessionManager({ config, registry: mockRegistry }); + + const session = await sm.relaunchOrchestrator({ projectId: "my-app" }); + + expect(session.id).toBe("app-orchestrator"); + expect(mockRuntime.destroy).toHaveBeenCalledWith(makeHandle("old-rt")); + expect(mockWorkspace.create).toHaveBeenCalledWith( + expect.objectContaining({ sessionId: "app-orchestrator" }), + ); + const meta = readMetadataRaw(sessionsDir, "app-orchestrator"); + expect(meta?.["status"]).toBe("working"); + expect(meta?.["runtimeHandle"]).not.toContain("old-rt"); + }); + + it("ignores project orchestratorSessionStrategy: ignore and still replaces", async () => { + const configWithIgnore: OrchestratorConfig = { + ...config, + projects: { + ...config.projects, + "my-app": { + ...config.projects["my-app"]!, + orchestratorSessionStrategy: "ignore", + }, + }, + }; + writeMetadata(sessionsDir, "app-orchestrator", { + role: "orchestrator", + project: "my-app", + status: "working", + branch: "orchestrator/app-orchestrator", + worktree: join(tmpDir, "old-orchestrator-ws"), + runtimeHandle: makeHandle("old-rt"), + }); + const sm = createSessionManager({ config: configWithIgnore, registry: mockRegistry }); + + await sm.relaunchOrchestrator({ projectId: "my-app" }); + + expect(mockRuntime.destroy).toHaveBeenCalledWith(makeHandle("old-rt")); + expect(mockWorkspace.create).toHaveBeenCalled(); + }); + + it("coalesces concurrent relaunch calls into a single replacement", async () => { + writeMetadata(sessionsDir, "app-orchestrator", { + role: "orchestrator", + project: "my-app", + status: "working", + branch: "orchestrator/app-orchestrator", + worktree: join(tmpDir, "old-orchestrator-ws"), + runtimeHandle: makeHandle("old-rt"), + }); + const sm = createSessionManager({ config, registry: mockRegistry }); + + const [s1, s2] = await Promise.all([ + sm.relaunchOrchestrator({ projectId: "my-app" }), + sm.relaunchOrchestrator({ projectId: "my-app" }), + ]); + + expect(s1.id).toBe("app-orchestrator"); + expect(s2.id).toBe("app-orchestrator"); + expect(mockRuntime.destroy).toHaveBeenCalledTimes(1); + expect(mockWorkspace.create).toHaveBeenCalledTimes(1); + expect(mockRuntime.create).toHaveBeenCalledTimes(1); + }); + + it("ensureOrchestrator waits for an in-flight relaunch before returning", async () => { + writeMetadata(sessionsDir, "app-orchestrator", { + role: "orchestrator", + project: "my-app", + status: "working", + branch: "orchestrator/app-orchestrator", + worktree: join(tmpDir, "old-orchestrator-ws"), + runtimeHandle: makeHandle("old-rt"), + }); + let releaseWorkspace: () => void = () => {}; + const blockingWorkspace = new Promise((resolve) => { + releaseWorkspace = resolve; + }); + (mockWorkspace.create as ReturnType).mockImplementationOnce(async (cfg) => { + await blockingWorkspace; + return { + path: join(tmpDir, "ws-relaunch"), + branch: cfg.branch, + sessionId: cfg.sessionId, + projectId: cfg.projectId, + }; + }); + const sm = createSessionManager({ config, registry: mockRegistry }); + + const relaunchPromise = sm.relaunchOrchestrator({ projectId: "my-app" }); + await Promise.resolve(); + const ensurePromise = sm.ensureOrchestrator({ projectId: "my-app" }); + + releaseWorkspace(); + + const [relaunched, ensured] = await Promise.all([relaunchPromise, ensurePromise]); + expect(relaunched.id).toBe("app-orchestrator"); + expect(ensured.id).toBe("app-orchestrator"); + // Both should resolve to the *post-relaunch* session, not the killed one. + expect(mockRuntime.destroy).toHaveBeenCalledWith(makeHandle("old-rt")); + }); + + it("waits for an in-flight ensureOrchestrator before replacing", async () => { + let releaseWorkspace: () => void = () => {}; + const blockingWorkspace = new Promise((resolve) => { + releaseWorkspace = resolve; + }); + const ensureWorkspacePath = join(tmpDir, "ws-ensure"); + (mockWorkspace.create as ReturnType).mockImplementationOnce(async (cfg) => { + await blockingWorkspace; + return { + path: ensureWorkspacePath, + branch: cfg.branch, + sessionId: cfg.sessionId, + projectId: cfg.projectId, + }; + }); + const sm = createSessionManager({ config, registry: mockRegistry }); + + const ensurePromise = sm.ensureOrchestrator({ projectId: "my-app" }); + // Yield so ensure can register itself in the in-flight map. + await Promise.resolve(); + const relaunchPromise = sm.relaunchOrchestrator({ projectId: "my-app" }); + + releaseWorkspace(); + + const [ensured, relaunched] = await Promise.all([ensurePromise, relaunchPromise]); + + expect(ensured.id).toBe("app-orchestrator"); + expect(relaunched.id).toBe("app-orchestrator"); + // Relaunch's kill must run, destroying the runtime ensure just spawned. + expect(mockRuntime.destroy).toHaveBeenCalled(); + }); + + it("rewrites the orchestrator-prompt file with the new system prompt", async () => { + writeMetadata(sessionsDir, "app-orchestrator", { + role: "orchestrator", + project: "my-app", + status: "working", + branch: "orchestrator/app-orchestrator", + worktree: join(tmpDir, "old-orchestrator-ws"), + runtimeHandle: makeHandle("old-rt"), + }); + const sm = createSessionManager({ config, registry: mockRegistry }); + + await sm.relaunchOrchestrator({ + projectId: "my-app", + systemPrompt: "FRESH ORCHESTRATOR PROMPT", + }); + + const promptPath = join( + getProjectDir("my-app"), + "orchestrator-prompt-app-orchestrator.md", + ); + expect(existsSync(promptPath)).toBe(true); + expect(readFileSync(promptPath, "utf-8")).toBe("FRESH ORCHESTRATOR PROMPT"); + }); + }); }); diff --git a/packages/core/src/__tests__/test-utils.ts b/packages/core/src/__tests__/test-utils.ts index e7a51e9df..b20f05ffa 100644 --- a/packages/core/src/__tests__/test-utils.ts +++ b/packages/core/src/__tests__/test-utils.ts @@ -499,6 +499,11 @@ export function createMockSessionManager(): OpenCodeSessionManager { .mockResolvedValue( makeSession({ id: "app-orchestrator", metadata: { role: "orchestrator" } }), ), + relaunchOrchestrator: vi + .fn() + .mockResolvedValue( + makeSession({ id: "app-orchestrator", metadata: { role: "orchestrator" } }), + ), restore: vi.fn().mockResolvedValue(makeSession()), list: vi.fn().mockResolvedValue([]), listCached: vi.fn().mockResolvedValue([]), diff --git a/packages/core/src/activity-events.ts b/packages/core/src/activity-events.ts index 3b5c5ea04..87b6c9175 100644 --- a/packages/core/src/activity-events.ts +++ b/packages/core/src/activity-events.ts @@ -20,11 +20,15 @@ export type ActivityEventSource = | "scm" | "runtime" | "agent" + | "tracker" + | "workspace" + | "notifier" | "reaction" | "report-watcher" | "config" | "plugin-registry" - | "migration"; + | "migration" + | "recovery"; export type ActivityEventKind = | "session.spawn_started" @@ -44,6 +48,20 @@ export type ActivityEventKind = | "runtime.probe_failed" | "agent.process_probe_failed" | "agent.activity_probe_failed" + // Plugin-internal failure shapes (issue #1659) + | "scm.gh_unavailable" + | "scm.batch_enrich_pr_failed" + | "scm.ci_summary_failclosed" + | "workspace.post_create_failed" + | "workspace.branch_collision" + | "workspace.destroy_fell_back" + | "workspace.corrupt_clone_skipped" + | "tracker.dep_missing" + | "tracker.api_timeout" + | "notifier.auth_failed" + | "notifier.unreachable" + | "notifier.rate_limited" + | "notifier.dep_missing" // Reaction lifecycle | "reaction.escalated" | "reaction.send_to_agent_failed" @@ -56,6 +74,7 @@ export type ActivityEventKind = | "detecting.escalated" // Report watcher | "report_watcher.triggered" + // Config/plugin-registry/storage migration | "config.project_resolve_failed" | "config.project_malformed" | "config.project_invalid" @@ -68,6 +87,23 @@ export type ActivityEventKind = | "migration.rename_failed" | "migration.completed" | "migration.rollback_skipped" + // Webhook ingress (api source) + | "api.webhook_unverified" + | "api.webhook_rejected" + | "api.webhook_received" + | "api.webhook_failed" + // WebSocket terminal mux (ui source — Node-side server only) + | "ui.terminal_connected" + | "ui.terminal_disconnected" + | "ui.terminal_heartbeat_lost" + | "ui.terminal_pty_lost" + | "ui.terminal_protocol_error" + | "ui.session_broadcast_failed" + // Recovery/forensic instrumentation + | "recovery.session_failed" + | "recovery.action_failed" + | "metadata.corrupt_detected" + | "api.agent_report.session_not_found" | "api.agent_report.transition_rejected" | "api.agent_report.apply_failed"; @@ -109,14 +145,12 @@ export function droppedEventCount(): number { } function pruneOldEvents(db: ReturnType, cutoff: number): void { - db - ?.prepare( - `DELETE FROM activity_events + db?.prepare( + `DELETE FROM activity_events WHERE rowid IN ( SELECT rowid FROM activity_events WHERE ts_epoch < ? LIMIT ? )`, - ) - .run(cutoff, PRUNE_BATCH_SIZE); + ).run(cutoff, PRUNE_BATCH_SIZE); } // Patterns that indicate sensitive field names @@ -133,7 +167,10 @@ function redactCredentialUrls(input: string): string { const proto = result.indexOf("://", offset); if (proto === -1) break; // Only match http:// or https:// (case-insensitive, matching old /gi flag) - if (proto < 4) { offset = proto + 3; continue; } + if (proto < 4) { + offset = proto + 3; + continue; + } const schemeEnd = result.slice(Math.max(0, proto - 5), proto).toLowerCase(); if (!schemeEnd.endsWith("http") && !schemeEnd.endsWith("https")) { offset = proto + 3; @@ -144,7 +181,7 @@ function redactCredentialUrls(input: string): string { while (cursor < result.length) { const ch = result.charCodeAt(cursor); // Space/control chars or '/' mean no '@' is coming in userinfo - if (ch <= 0x20 || ch === 0x2F) break; + if (ch <= 0x20 || ch === 0x2f) break; if (ch === 0x40) { // '@' found — redact everything between :// and @ // Lowercase the scheme to match the old /gi regex behavior @@ -157,7 +194,11 @@ function redactCredentialUrls(input: string): string { cursor++; } // No '@' found — not a credential URL, move past this :// - if (cursor >= result.length || result.charCodeAt(cursor) <= 0x20 || result.charCodeAt(cursor) === 0x2F) { + if ( + cursor >= result.length || + result.charCodeAt(cursor) <= 0x20 || + result.charCodeAt(cursor) === 0x2f + ) { offset = proto + 3; } } diff --git a/packages/core/src/activity-log.ts b/packages/core/src/activity-log.ts index 1334bfb2e..56a1e02c3 100644 --- a/packages/core/src/activity-log.ts +++ b/packages/core/src/activity-log.ts @@ -15,10 +15,8 @@ import { join, dirname } from "node:path"; import type { ActivityState, ActivityLogEntry, ActivityDetection } from "./types.js"; /** - * Maximum age (ms) for `waiting_input`/`blocked` entries before they're - * considered stale. If no new terminal output overwrites the entry within - * this window, the state falls through to downstream fallbacks instead of - * keeping the session stuck in `needs_input` on the dashboard forever. + * @deprecated Actionable states no longer decay on wallclock. Retained until + * the activity-reducer cleanup removes the old activity-log module. */ export const ACTIVITY_INPUT_STALENESS_MS = 5 * 60 * 1000; // 5 minutes @@ -129,7 +127,7 @@ export async function readLastActivityEntry( /** * Check the AO activity JSONL for actionable states only. * - * Only returns `waiting_input`/`blocked` (with a staleness cap). + * Only returns `waiting_input`/`blocked`. * Non-critical states (`active`, `ready`, `idle`) always return `null` so * callers fall through to their native signals (git commits, chat history, * API queries, native JSONL). This prevents the lifecycle manager's @@ -144,18 +142,12 @@ export function checkActivityLogState( const { entry } = activityResult; if (entry.state === "waiting_input" || entry.state === "blocked") { - // Use the entry's own timestamp for staleness — not file mtime, which - // gets refreshed every poll cycle by recordActivity and would prevent - // stale entries from ever being detected. const entryTs = new Date(entry.ts); if (Number.isNaN(entryTs.getTime())) return null; - const ageMs = Date.now() - entryTs.getTime(); - if (ageMs <= ACTIVITY_INPUT_STALENESS_MS) { - return { state: entry.state, timestamp: entryTs }; - } + return { state: entry.state, timestamp: entryTs }; } - // Non-critical states and stale entries — fall through to native signals + // Non-critical states fall through to native signals return null; } @@ -178,16 +170,8 @@ export function getActivityFallbackState( const entryTs = new Date(entry.ts); if (Number.isNaN(entryTs.getTime())) return null; - // Actionable states use the same staleness cap as checkActivityLogState. - // If the entry is stale, fall through to age-based decay instead of - // re-surfacing a waiting_input/blocked that checkActivityLogState already filtered. if (entry.state === "waiting_input" || entry.state === "blocked") { - const ageMs = Date.now() - entryTs.getTime(); - if (ageMs <= ACTIVITY_INPUT_STALENESS_MS) { - return { state: entry.state, timestamp: entryTs }; - } - // Stale actionable entry — treat as idle - return { state: "idle", timestamp: entryTs }; + return { state: entry.state, timestamp: entryTs }; } // Age-based decay: active→ready→idle, but never promote past the diff --git a/packages/core/src/agent-report.ts b/packages/core/src/agent-report.ts index 3c39d2c8a..091f56a3d 100644 --- a/packages/core/src/agent-report.ts +++ b/packages/core/src/agent-report.ts @@ -18,7 +18,7 @@ import { appendFileSync, existsSync, mkdirSync, readFileSync, writeFileSync } from "node:fs"; import { readFile } from "node:fs/promises"; -import { join } from "node:path"; +import { basename, dirname, join } from "node:path"; import type { CanonicalSessionLifecycle, CanonicalSessionReason, @@ -26,12 +26,17 @@ import type { SessionId, SessionStatus, } from "./types.js"; +import { recordActivityEvent } from "./activity-events.js"; import { mutateMetadata, readMetadataRaw } from "./metadata.js"; -import { buildLifecycleMetadataPatch, cloneLifecycle, deriveLegacyStatus, parseCanonicalLifecycle } from "./lifecycle-state.js"; +import { + buildLifecycleMetadataPatch, + cloneLifecycle, + deriveLegacyStatus, + parseCanonicalLifecycle, +} from "./lifecycle-state.js"; import { parsePrFromUrl } from "./utils/pr.js"; import { assertValidSessionIdComponent } from "./utils/session-id.js"; import { validateStatus } from "./utils/validation.js"; -import { recordActivityEvent } from "./activity-events.js"; /** * Canonical set of states an agent can self-declare. @@ -255,6 +260,11 @@ function buildAuditDir(dataDir: string): string { return join(dataDir, ".agent-report-audit"); } +function inferProjectIdFromDataDir(dataDir: string): string | undefined { + // dataDir is `.../projects/{projectId}/sessions`; recover projectId for filtering. + return basename(dirname(dataDir)) || undefined; +} + const AGENT_REPORT_AUDIT_MAX_BYTES = 256 * 1024; const AGENT_REPORT_AUDIT_MAX_ENTRIES = 200; @@ -384,8 +394,22 @@ export function applyAgentReport( sessionId: SessionId, input: ApplyAgentReportInput, ): ApplyAgentReportResult { + const projectId = inferProjectIdFromDataDir(dataDir); const raw = readMetadataRaw(dataDir, sessionId); if (!raw) { + recordActivityEvent({ + projectId, + sessionId, + source: "api", + kind: "api.agent_report.session_not_found", + level: "warn", + summary: `applyAgentReport: session not found: ${sessionId}`, + data: { + reportState: input.state, + actor: input.actor, + source: input.source, + }, + }); throw new Error(`Session not found: ${sessionId}`); } @@ -429,109 +453,113 @@ export function applyAgentReport( let legacyStatus: SessionStatus | null = null; let previousLegacyStatus: SessionStatus | null = null; - const nextMetadata = mutateMetadata(dataDir, sessionId, (existing) => { - const current = cloneLifecycle( - parseCanonicalLifecycle(existing, { - sessionId, - status: validateStatus(existing["status"]), - }), - ); - previousLegacyStatus = deriveLegacyStatus(current); - before = buildAuditSnapshot(current, previousLegacyStatus); - const validation = validateAgentReportTransition(current, input.state); - if (!validation.ok) { - appendAgentReportAuditEntry(dataDir, sessionId, { - timestamp: now, - actor, - source, - reportState: input.state, - note: trimmedNote, - prNumber, - prUrl: trimmedPrUrl, - prIsDraft, - accepted: false, - rejectionReason: validation.reason ?? "transition rejected", - before, - after: before, - }); - recordActivityEvent({ - sessionId, - source: "api", - kind: "api.agent_report.transition_rejected", - level: "warn", - summary: `agent report ${input.state} rejected: ${validation.reason ?? "transition rejected"}`, - data: { - reportState: input.state, - reason: validation.reason ?? "transition rejected", - previousSessionState: current.session.state, - previousLegacyStatus, + const nextMetadata = mutateMetadata( + dataDir, + sessionId, + (existing) => { + const current = cloneLifecycle( + parseCanonicalLifecycle(existing, { + sessionId, + status: validateStatus(existing["status"]), + }), + ); + previousLegacyStatus = deriveLegacyStatus(current); + before = buildAuditSnapshot(current, previousLegacyStatus); + const validation = validateAgentReportTransition(current, input.state); + if (!validation.ok) { + const rejectionReason = validation.reason ?? "transition rejected"; + appendAgentReportAuditEntry(dataDir, sessionId, { + timestamp: now, actor, source, - }, - }); - throw new Error(validation.reason ?? "transition rejected"); - } - const mapped = mapAgentReportToLifecycle(input.state); - previousState = current.session.state; - nextState = mapped.sessionState; - current.session.state = mapped.sessionState; - current.session.reason = mapped.sessionReason; - current.session.lastTransitionAt = now; - if (isPRWorkflowReport(input.state)) { - const effectivePrUrl = trimmedPrUrl ?? current.pr.url ?? existingPrUrl; - const effectivePrNumber = - prNumber ?? current.pr.number ?? existingPrNumber ?? parsedPrFromUrl?.number; - const canAdvancePrState = - effectivePrUrl !== undefined || - effectivePrNumber !== undefined || - current.pr.state !== "none"; - if (canAdvancePrState) { - current.pr.state = "open"; - current.pr.reason = - input.state === "ready_for_review" ? "review_pending" : "in_progress"; - current.pr.lastObservedAt = now; + reportState: input.state, + note: trimmedNote, + prNumber, + prUrl: trimmedPrUrl, + prIsDraft, + accepted: false, + rejectionReason, + before, + after: before, + }); + recordActivityEvent({ + projectId, + sessionId, + source: "api", + kind: "api.agent_report.transition_rejected", + level: "warn", + summary: `applyAgentReport rejected ${input.state} for ${sessionId}: ${rejectionReason}`, + data: { + reportState: input.state, + rejectionReason, + actor, + reportSource: source, + fromState: current.session.state, + fromReason: current.session.reason, + legacyStatus: previousLegacyStatus, + }, + }); + throw new Error(rejectionReason); } - if (effectivePrUrl) { - current.pr.url = effectivePrUrl; + const mapped = mapAgentReportToLifecycle(input.state); + previousState = current.session.state; + nextState = mapped.sessionState; + current.session.state = mapped.sessionState; + current.session.reason = mapped.sessionReason; + current.session.lastTransitionAt = now; + if (isPRWorkflowReport(input.state)) { + const effectivePrUrl = trimmedPrUrl ?? current.pr.url ?? existingPrUrl; + const effectivePrNumber = + prNumber ?? current.pr.number ?? existingPrNumber ?? parsedPrFromUrl?.number; + const canAdvancePrState = + effectivePrUrl !== undefined || + effectivePrNumber !== undefined || + current.pr.state !== "none"; + if (canAdvancePrState) { + current.pr.state = "open"; + current.pr.reason = input.state === "ready_for_review" ? "review_pending" : "in_progress"; + current.pr.lastObservedAt = now; + } + if (effectivePrUrl) { + current.pr.url = effectivePrUrl; + } + if (effectivePrNumber !== undefined) { + current.pr.number = effectivePrNumber; + } } - if (effectivePrNumber !== undefined) { - current.pr.number = effectivePrNumber; + if (mapped.sessionState === "working" && current.session.startedAt === null) { + current.session.startedAt = now; } - } - if (mapped.sessionState === "working" && current.session.startedAt === null) { - current.session.startedAt = now; - } - legacyStatus = deriveLegacyStatus(current); - const next = { ...existing }; - Object.assign( - next, - buildLifecycleMetadataPatch(current), - { + legacyStatus = deriveLegacyStatus(current); + const next = { ...existing }; + Object.assign(next, buildLifecycleMetadataPatch(current), { [AGENT_REPORT_METADATA_KEYS.STATE]: input.state, [AGENT_REPORT_METADATA_KEYS.AT]: now, - }, - ); - if (trimmedNote) { - next[AGENT_REPORT_METADATA_KEYS.NOTE] = trimmedNote; - } else { - next[AGENT_REPORT_METADATA_KEYS.NOTE] = ""; - } - if (isPRWorkflowReport(input.state)) { - if (trimmedPrUrl) { - next[AGENT_REPORT_METADATA_KEYS.PR_URL] = trimmedPrUrl; + }); + if (trimmedNote) { + next[AGENT_REPORT_METADATA_KEYS.NOTE] = trimmedNote; + } else { + next[AGENT_REPORT_METADATA_KEYS.NOTE] = ""; } - if (prNumber !== undefined) { - next[AGENT_REPORT_METADATA_KEYS.PR_NUMBER] = String(prNumber); + if (isPRWorkflowReport(input.state)) { + if (trimmedPrUrl) { + next[AGENT_REPORT_METADATA_KEYS.PR_URL] = trimmedPrUrl; + } + if (prNumber !== undefined) { + next[AGENT_REPORT_METADATA_KEYS.PR_NUMBER] = String(prNumber); + } + if (prIsDraft !== undefined) { + next[AGENT_REPORT_METADATA_KEYS.PR_IS_DRAFT] = prIsDraft ? "true" : "false"; + } } - if (prIsDraft !== undefined) { - next[AGENT_REPORT_METADATA_KEYS.PR_IS_DRAFT] = prIsDraft ? "true" : "false"; - } - } - return next; - }); + return next; + }, + { activityEventSource: "api" }, + ); if (!nextMetadata || !before || !previousState || !nextState || !legacyStatus) { recordActivityEvent({ + projectId, sessionId, source: "api", kind: "api.agent_report.apply_failed", diff --git a/packages/core/src/events-db.ts b/packages/core/src/events-db.ts index 71444a9a3..ca04f0062 100644 --- a/packages/core/src/events-db.ts +++ b/packages/core/src/events-db.ts @@ -24,6 +24,7 @@ type BetterSqlite3Database = { let _db: BetterSqlite3Database | null = null; let _dbFailed = false; let _ftsEnabled = false; +let _dbUnavailableWarningEmitted = false; const PRUNE_BATCH_SIZE = 1000; function getEventsDbPath(): string { @@ -90,14 +91,12 @@ function initFts(db: BetterSqlite3Database): void { } function pruneOldEvents(db: BetterSqlite3Database, cutoff: number): void { - db - .prepare( - `DELETE FROM activity_events + db.prepare( + `DELETE FROM activity_events WHERE rowid IN ( SELECT rowid FROM activity_events WHERE ts_epoch < ? LIMIT ? )`, - ) - .run(cutoff, PRUNE_BATCH_SIZE); + ).run(cutoff, PRUNE_BATCH_SIZE); } function openDb(): BetterSqlite3Database { @@ -161,6 +160,42 @@ export function closeDb(): void { _ftsEnabled = false; } +function isAoEventsInvocation(argv = process.argv): boolean { + return argv.slice(2).includes("events"); +} + +function isMissingBetterSqlite3Binding(err: unknown): boolean { + const message = err instanceof Error ? err.message : String(err); + return ( + message.includes("Could not locate the bindings file") || + message.includes("better_sqlite3.node") || + message.includes("Cannot find module 'better-sqlite3'") + ); +} + +function firstErrorLine(err: unknown): string { + return (err instanceof Error ? err.message : String(err)).split(/\r?\n/, 1)[0] ?? "unknown error"; +} + +export function formatActivityEventsDbUnavailableWarning(err: unknown): string { + if (isMissingBetterSqlite3Binding(err)) { + return `[ao] activity-events disabled: better-sqlite3 not compiled for Node ${process.version} (ABI v${process.versions.modules}). Run \`pnpm rebuild better-sqlite3\` or use a supported Node version.`; + } + return `[ao] activity-events disabled: better-sqlite3 failed to load: ${firstErrorLine(err)}`; +} + +export function emitActivityEventsDbUnavailableWarning(err: unknown): void { + if (_dbUnavailableWarningEmitted) return; + if (process.env["AO_DEBUG"] !== "1" && !isAoEventsInvocation()) return; + _dbUnavailableWarningEmitted = true; + // eslint-disable-next-line no-console + console.warn(formatActivityEventsDbUnavailableWarning(err)); +} + +export function __resetActivityEventsDbWarningForTests(): void { + _dbUnavailableWarningEmitted = false; +} + /** * Get the lazily-initialized DB connection. * Returns null if better-sqlite3 failed to load or init — callers should treat null as no-op. @@ -173,8 +208,7 @@ export function getDb(): BetterSqlite3Database | null { return _db; } catch (err) { _dbFailed = true; - // Log once so operators know events are being dropped; subsequent calls return null silently. - console.warn("[ao] activity-events DB unavailable — events will be dropped:", err instanceof Error ? err.message : String(err)); + emitActivityEventsDbUnavailableWarning(err); return null; } } diff --git a/packages/core/src/metadata.ts b/packages/core/src/metadata.ts index a4f3a19c5..451c127b5 100644 --- a/packages/core/src/metadata.ts +++ b/packages/core/src/metadata.ts @@ -22,9 +22,10 @@ import { closeSync, constants, } from "node:fs"; -import { join, dirname } from "node:path"; +import { basename, join, dirname } from "node:path"; import type { CanonicalSessionLifecycle, RuntimeHandle, SessionId, SessionMetadata } from "./types.js"; import { atomicWriteFileSync } from "./atomic-write.js"; +import { recordActivityEvent, type ActivityEventSource } from "./activity-events.js"; import { buildLifecycleMetadataPatch, cloneLifecycle, @@ -330,6 +331,11 @@ export function applyMetadataUpdates( return next; } +interface MutateMetadataOptions { + createIfMissing?: boolean; + activityEventSource?: ActivityEventSource; +} + function normalizeMetadataRecord(data: Record): Record { return Object.fromEntries( Object.entries(data).filter(([, value]) => value !== undefined && value !== ""), @@ -340,7 +346,7 @@ export function mutateMetadata( dataDir: string, sessionId: SessionId, updater: (existing: Record) => Record, - options: { createIfMissing?: boolean } = {}, + options: MutateMetadataOptions = {}, ): Record | null { const path = metadataPath(dataDir, sessionId); const lockPath = `${path}.lock`; @@ -368,8 +374,10 @@ export function mutateMetadata( // that anything was wrong — the file just becomes "not // corrupt anymore — and missing fields". const corruptPath = `${path}.corrupt-${Date.now()}`; + let renamed = false; try { renameSync(path, corruptPath); + renamed = true; // eslint-disable-next-line no-console console.warn( `[metadata] corrupt JSON at ${path}; preserved as ${corruptPath} before rewriting`, @@ -377,6 +385,31 @@ export function mutateMetadata( } catch { // best effort — proceed even if the rename fails (e.g. EACCES) } + // Forensic activity event so RCA can find every silent overwrite. + // Truncate the bad-JSON sample to 200 chars (B11 invariant — full file + // could be 16KB+ and would be dropped by the sanitizer cap). + const contentSample = + content.length > 200 ? content.slice(0, 200) : content; + // dataDir is `.../projects/{projectId}/sessions`; recover projectId for filtering. + const inferredProjectId = basename(dirname(dataDir)); + const summary = renamed + ? `Corrupt metadata for session ${sessionId} renamed to ${basename(corruptPath)}` + : `Corrupt metadata detected for session ${sessionId}; failed to rename forensic copy before rewrite`; + recordActivityEvent({ + projectId: inferredProjectId || undefined, + sessionId, + source: options.activityEventSource ?? "session-manager", + kind: "metadata.corrupt_detected", + level: "error", + summary, + data: { + path, + renamedTo: renamed ? corruptPath : null, + renameSucceeded: renamed, + contentSample, + contentLength: content.length, + }, + }); } } } else if (!options.createIfMissing) { diff --git a/packages/core/src/portfolio-session-service.ts b/packages/core/src/portfolio-session-service.ts index 0d1ce5472..4be551226 100644 --- a/packages/core/src/portfolio-session-service.ts +++ b/packages/core/src/portfolio-session-service.ts @@ -141,6 +141,7 @@ function metadataToSession(sessionId: string, project: PortfolioProject, metadat return sessionFromMetadata(sessionId, metadataToRecord(metadata), { projectId: project.id, + workspacePathFallback: project.repoPath, status: (metadata.status as Session["status"]) || "spawning", activity: null, runtimeHandle: metadata.runtimeHandle ?? null, diff --git a/packages/core/src/recovery/actions.ts b/packages/core/src/recovery/actions.ts index 14e315b08..f7b57980a 100644 --- a/packages/core/src/recovery/actions.ts +++ b/packages/core/src/recovery/actions.ts @@ -5,6 +5,7 @@ import type { Runtime, Workspace, } from "../types.js"; +import { recordActivityEvent } from "../activity-events.js"; import { updateMetadata } from "../metadata.js"; import { getProjectSessionsDir } from "../paths.js"; import { validateStatus } from "../utils/validation.js"; @@ -132,6 +133,7 @@ export async function recoverSession( const session = sessionFromMetadata(sessionId, updatedMetadata, { projectId: assessment.projectId, + workspacePathFallback: assessment.workspacePath ?? undefined, status: preservedStatus, runtimeHandle: assessment.runtimeHandle, lastActivityAt: new Date(), @@ -145,11 +147,25 @@ export async function recoverSession( session, }; } catch (error) { + const errorMessage = error instanceof Error ? error.message : String(error); + recordActivityEvent({ + projectId, + sessionId, + source: "recovery", + kind: "recovery.action_failed", + level: "error", + summary: `recoverSession threw for ${sessionId}: ${errorMessage}`, + data: { + action: "recover", + recoveryCount, + errorMessage, + }, + }); return { success: false, sessionId, action: "recover", - error: error instanceof Error ? error.message : String(error), + error: errorMessage, }; } } diff --git a/packages/core/src/recovery/manager.ts b/packages/core/src/recovery/manager.ts index 1c90e007f..330922e7f 100644 --- a/packages/core/src/recovery/manager.ts +++ b/packages/core/src/recovery/manager.ts @@ -1,4 +1,5 @@ import type { OrchestratorConfig, PluginRegistry, Session } from "../types.js"; +import { recordActivityEvent } from "../activity-events.js"; import { scanAllSessions, getRecoveryLogPath } from "./scanner.js"; import { validateSession } from "./validator.js"; import { executeAction } from "./actions.js"; @@ -106,9 +107,10 @@ export async function runRecovery(options: RecoveryManagerOptions): Promise, - projectId: string, - sessionPrefix?: string, - createdAt?: Date, - modifiedAt?: Date, + options: MetadataToSessionOptions, ): Session { const sessionKind = meta["role"] === "orchestrator" || - (sessionPrefix - ? new RegExp(`^${escapeRegex(sessionPrefix)}-orchestrator-\\d+$`).test(sessionId) + (options.sessionPrefix + ? new RegExp(`^${escapeRegex(options.sessionPrefix)}-orchestrator-\\d+$`).test(sessionId) : false) ? "orchestrator" : "worker"; return sessionFromMetadata(sessionId, meta, { - projectId, + projectId: options.projectId, + workspacePathFallback: options.workspacePathFallback, sessionKind, - createdAt, - lastActivityAt: modifiedAt ?? new Date(), + createdAt: options.createdAt, + lastActivityAt: options.modifiedAt ?? new Date(), }); } @@ -453,18 +459,6 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM return isOrchestratorSessionRecord(sessionId, metadata ?? {}, project.sessionPrefix); } - function requiresNativeRestore(agentName: string): boolean { - // kimicode is intentionally excluded: kimi's session dir only exists if the - // previous launch ran far enough to write to ~/.kimi/sessions. A failed - // launch (e.g. the --agent-file YAML crash) leaves no session to resume, - // and falling back to a fresh getLaunchCommand is the only sensible choice. - return ( - agentName === "claude-code" || - agentName === "codex" || - agentName === "opencode" - ); - } - function applyMetadataUpdatesToRaw( raw: Record, updates: Partial>, @@ -538,6 +532,7 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM expiresAt: number; } | null = null; const ensureOrchestratorPromises = new Map>(); + const relaunchOrchestratorPromises = new Map>(); function invalidateCache(): void { sessionCache = null; @@ -1019,12 +1014,68 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM */ const TERMINAL_SESSION_STATUSES = new Set(["killed", "done", "merged", "terminated", "cleanup"]); + function hasPersistedNativeRestoreMetadata(session: Session, agent: Agent): boolean { + const metadata = session.metadata ?? {}; + + switch (agent.name) { + case "claude-code": + return typeof metadata["claudeSessionUuid"] === "string" && metadata["claudeSessionUuid"].trim().length > 0; + case "codex": + return typeof metadata["codexThreadId"] === "string" && metadata["codexThreadId"].trim().length > 0; + case "opencode": + return asValidOpenCodeSessionId(metadata["opencodeSessionId"]) !== null; + default: + return false; + } + } + + function canDiscoverSessionInfoAfterRuntimeExit(agent: Agent): boolean { + return agent.name === "claude-code" || agent.name === "codex"; + } + async function enrichSessionWithRuntimeState( session: Session, plugins: ReturnType, handleFromMetadata: boolean, sessionsDir: string, ): Promise { + async function persistAgentSessionInfo(options?: { skipIfNativeRestoreMetadataPresent?: boolean }): Promise { + if (!plugins.agent) return; + if ( + options?.skipIfNativeRestoreMetadataPresent && + hasPersistedNativeRestoreMetadata(session, plugins.agent) + ) { + return; + } + + let info: Awaited>; + try { + info = await plugins.agent.getSessionInfo(session); + } catch { + // Can't get session info — keep existing values + info = null; + } + + if (!info) return; + + session.agentInfo = info; + const metadataUpdates = info.metadata ?? {}; + const allAlreadyPersisted = Object.keys(metadataUpdates).every( + (key) => session.metadata?.[key] === metadataUpdates[key], + ); + if (allAlreadyPersisted) return; + + if (Object.keys(metadataUpdates).length > 0) { + try { + updateMetadata(sessionsDir, session.id, metadataUpdates); + session.metadata = applyMetadataUpdates(session.metadata, metadataUpdates); + invalidateCache(); + } catch { + // Persisting agent metadata is best-effort; keep live agent info. + } + } + } + // Check runtime liveness first — for all statuses except "spawning". // Skip spawning sessions because tmux may not be fully initialized yet, // and a false-negative from isAlive() would permanently mark the session @@ -1065,6 +1116,11 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM activity: "exited", source: "runtime", }); + // Dead-runtime session info discovery is intentionally limited to + // agents that recover restore metadata from persisted local files. + if (plugins.agent && canDiscoverSessionInfoAfterRuntimeExit(plugins.agent)) { + await persistAgentSessionInfo({ skipIfNativeRestoreMetadataPresent: true }); + } return; } } catch { @@ -1101,28 +1157,8 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM session.activitySignal = createActivitySignal("probe_failure", { source: "native" }); } - // Enrich with live agent session info (summary, cost). - let info: Awaited>; - try { - info = await plugins.agent.getSessionInfo(session); - } catch { - // Can't get session info — keep existing values - info = null; - } - - if (info) { - session.agentInfo = info; - const metadataUpdates = info.metadata ?? {}; - if (Object.keys(metadataUpdates).length > 0) { - try { - updateMetadata(sessionsDir, session.id, metadataUpdates); - session.metadata = applyMetadataUpdates(session.metadata, metadataUpdates); - invalidateCache(); - } catch { - // Persisting agent metadata is best-effort; keep live agent info. - } - } - } + // Enrich with agent session info (summary, cost, native restore metadata). + await persistAgentSessionInfo(); } } @@ -1815,6 +1851,20 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM } const sessionId = getOrchestratorSessionId(project); + + // If a relaunch is mid-flight for this sessionId, wait it out — otherwise + // we could return a session that relaunch is about to kill, or race the + // relaunch's spawnOrchestrator on the same reservation. + const pendingRelaunch = relaunchOrchestratorPromises.get(sessionId); + if (pendingRelaunch) { + await pendingRelaunch.catch((err) => { + console.warn( + `[ensureOrchestrator] in-flight relaunch for ${sessionId} failed before ensure proceeded:`, + err, + ); + }); + } + const existing = await get(sessionId); if (existing) { const orchestratorSessionStrategy = normalizeOrchestratorSessionStrategy( @@ -1876,6 +1926,61 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM return promise; } + async function relaunchOrchestratorInternal( + orchestratorConfig: OrchestratorSpawnConfig, + ): Promise { + const project = config.projects[orchestratorConfig.projectId]; + if (!project) { + throw new Error(`Unknown project: ${orchestratorConfig.projectId}`); + } + const sessionId = getOrchestratorSessionId(project); + const sessionsDir = getProjectSessionsDir(orchestratorConfig.projectId); + + // If ensureOrchestrator is mid-flight for this sessionId, wait it out. + // Otherwise get() would return null (metadata not yet written) and we'd + // skip the kill, then race the in-flight spawnOrchestrator on the same + // reservation — surfacing "session already exists" instead of replacing. + const pendingEnsure = ensureOrchestratorPromises.get(sessionId); + if (pendingEnsure) { + await pendingEnsure.catch((err) => { + console.warn( + `[relaunchOrchestrator] in-flight ensure for ${sessionId} failed before relaunch proceeded:`, + err, + ); + }); + } + + const existing = await get(sessionId); + if (existing) { + const existingAgent = resolveSelectionForSession( + project, + sessionId, + readMetadataRaw(sessionsDir, sessionId) ?? {}, + ).agentName; + await kill(sessionId, { purgeOpenCode: existingAgent === "opencode" }); + deleteMetadata(sessionsDir, sessionId); + } + return spawnOrchestrator(orchestratorConfig); + } + + async function relaunchOrchestrator( + orchestratorConfig: OrchestratorSpawnConfig, + ): Promise { + const project = config.projects[orchestratorConfig.projectId]; + if (!project) { + throw new Error(`Unknown project: ${orchestratorConfig.projectId}`); + } + const sessionId = getOrchestratorSessionId(project); + const existingPromise = relaunchOrchestratorPromises.get(sessionId); + if (existingPromise) return existingPromise; + + const promise = relaunchOrchestratorInternal(orchestratorConfig).finally(() => { + relaunchOrchestratorPromises.delete(sessionId); + }); + relaunchOrchestratorPromises.set(sessionId, promise); + return promise; + } + async function list(projectId?: string): Promise { const allSessions = Object.entries(config.projects).flatMap(([entryProjectId, project]) => { if (projectId && entryProjectId !== projectId) return []; @@ -1907,10 +2012,13 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM const session = metadataToSession( sessionName, raw, - sessionProjectId, - project.sessionPrefix, - createdAt, - modifiedAt, + { + projectId: sessionProjectId, + sessionPrefix: project.sessionPrefix, + createdAt, + modifiedAt, + workspacePathFallback: project.path, + }, ); const selection = resolveSelectionForSession(project, sessionName, raw); const effectiveAgentName = selection.agentName; @@ -1941,23 +2049,30 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM } } - // Persist lifecycle to disk when enrichment detected a dead runtime. - // enrichSessionWithRuntimeState updates the in-memory lifecycle but - // doesn't write to disk — without this, the stale "alive" state persists - // and the dashboard shows terminated sessions on the active sidebar. + // Persist runtime probe result to disk so the lifecycle manager sees it + // on next poll. We only persist the runtime signal and detecting state — + // the lifecycle manager's resolveProbeDecision pipeline is the single + // authority on terminal decisions (terminated/done). See #1735. + // Check the on-disk state (raw) to avoid re-writing when already + // detecting — enrichment sets detecting in-memory, but we only need + // to persist the transition once to avoid resetting lastTransitionAt. + const onDiskLifecycle = parseCanonicalLifecycle(raw, { + sessionId: sessionName, + status: validateStatus(raw["status"]), + }); if ( session.lifecycle && (session.lifecycle.runtime.state === "missing" || session.lifecycle.runtime.state === "exited") && - session.lifecycle.session.state !== "terminated" && - session.lifecycle.session.state !== "done" + onDiskLifecycle.session.state !== "terminated" && + onDiskLifecycle.session.state !== "done" && + onDiskLifecycle.session.state !== "detecting" ) { try { const persisted = buildUpdatedLifecycle(sessionName, raw, (next) => { - next.session.state = "terminated"; + next.session.state = "detecting"; next.session.reason = "runtime_lost"; - next.session.terminatedAt = new Date().toISOString(); - next.session.lastTransitionAt = next.session.terminatedAt; + next.session.lastTransitionAt = new Date().toISOString(); next.runtime.state = session.lifecycle!.runtime.state; next.runtime.reason = session.lifecycle!.runtime.reason; next.runtime.lastObservedAt = new Date().toISOString(); @@ -2021,10 +2136,13 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM const session = metadataToSession( sessionId, repaired.raw, - projectId, - project.sessionPrefix, - createdAt, - modifiedAt, + { + projectId, + sessionPrefix: project.sessionPrefix, + createdAt, + modifiedAt, + workspacePathFallback: project.path, + }, ); const selection = resolveSelectionForSession(project, sessionId, repaired.raw); @@ -2795,7 +2913,15 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM // metadataToSession sets activity: null, so without enrichment a crashed // session (status "working", agent exited) would not be detected as terminal // and isRestorable would reject it. - const session = metadataToSession(sessionId, raw, projectId, project.sessionPrefix); + const session = metadataToSession( + sessionId, + raw, + { + projectId, + sessionPrefix: project.sessionPrefix, + workspacePathFallback: project.path, + }, + ); const plugins = resolvePlugins(project, selection.agentName); await enrichSessionWithRuntimeState(session, plugins, true, sessionsDir); @@ -2931,13 +3057,12 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM launchCommand = restoreCmd; updateMetadata(sessionsDir, sessionId, { restoreFallbackReason: "" }); } else { + // Agents with native restore can still launch fresh when no resumable + // session metadata exists; this keeps restore from becoming a hard stop. const reason = `${plugins.agent.name}.getRestoreCommand returned null`; updateMetadata(sessionsDir, sessionId, { restoreFallbackReason: reason, }); - if (requiresNativeRestore(plugins.agent.name)) { - throw new SessionNotRestorableError(sessionId, reason); - } launchCommand = plugins.agent.getLaunchCommand(agentLaunchConfig); } } else { @@ -3047,6 +3172,7 @@ export function createSessionManager(deps: SessionManagerDeps): OpenCodeSessionM spawn, spawnOrchestrator, ensureOrchestrator, + relaunchOrchestrator, restore, list, listCached, diff --git a/packages/core/src/types.ts b/packages/core/src/types.ts index 4ddc7aad0..b68deaf2a 100644 --- a/packages/core/src/types.ts +++ b/packages/core/src/types.ts @@ -1845,6 +1845,13 @@ export interface SessionManager { spawn(config: SessionSpawnConfig): Promise; spawnOrchestrator(config: OrchestratorSpawnConfig): Promise; ensureOrchestrator(config: OrchestratorSpawnConfig): Promise; + /** + * Replace the canonical orchestrator with a fresh one. If an orchestrator + * already exists for the project, it is killed, its metadata deleted, and a + * new orchestrator spawned with no carryover state. Ignores + * `orchestratorSessionStrategy` — replacement is the whole point. + */ + relaunchOrchestrator(config: OrchestratorSpawnConfig): Promise; restore(sessionId: SessionId): Promise; list(projectId?: string): Promise; get(sessionId: SessionId): Promise; diff --git a/packages/core/src/utils/session-from-metadata.ts b/packages/core/src/utils/session-from-metadata.ts index aeda52d79..9230ff18b 100644 --- a/packages/core/src/utils/session-from-metadata.ts +++ b/packages/core/src/utils/session-from-metadata.ts @@ -14,6 +14,7 @@ import { safeJsonParse, validateStatus } from "./validation.js"; interface SessionFromMetadataOptions { projectId?: string; + workspacePathFallback?: string; status?: SessionStatus; sessionKind?: SessionKind; activity?: Session["activity"]; @@ -86,7 +87,7 @@ export function sessionFromMetadata( }; })() : null, - workspacePath: meta["worktree"] || null, + workspacePath: meta["worktree"] || options.workspacePathFallback || null, runtimeHandle: lifecycle.runtime.handle ?? runtimeHandle, agentInfo: meta["summary"] ? { summary: meta["summary"], agentSessionId: null } : null, createdAt: meta["createdAt"] ? new Date(meta["createdAt"]) : (options.createdAt ?? new Date()), diff --git a/packages/integration-tests/src/tracker-linear.integration.test.ts b/packages/integration-tests/src/tracker-linear.integration.test.ts index c7406ce8c..6749761ea 100644 --- a/packages/integration-tests/src/tracker-linear.integration.test.ts +++ b/packages/integration-tests/src/tracker-linear.integration.test.ts @@ -252,10 +252,10 @@ describe.skipIf(!canRun)("tracker-linear (integration)", () => { // Linear API has eventual consistency — poll until the issue appears in list results const found = await pollUntil( async () => { - const issues = await tracker.listIssues!({ state: "open", limit: 50 }, project); + const issues = await tracker.listIssues!({ state: "open", limit: 100 }, project); return issues.find((i: { id: string }) => i.id === issueIdentifier); }, - { timeoutMs: 5_000, intervalMs: 500 }, + { timeoutMs: 15_000, intervalMs: 1_000 }, ); expect(found).toBeDefined(); diff --git a/packages/plugins/agent-claude-code/src/__tests__/activity-detection.test.ts b/packages/plugins/agent-claude-code/src/__tests__/activity-detection.test.ts index ff9a78bcb..6199b3b19 100644 --- a/packages/plugins/agent-claude-code/src/__tests__/activity-detection.test.ts +++ b/packages/plugins/agent-claude-code/src/__tests__/activity-detection.test.ts @@ -1,9 +1,15 @@ import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; -import { mkdtempSync, mkdirSync, writeFileSync, rmSync, utimesSync } from "node:fs"; +import { existsSync, mkdtempSync, mkdirSync, writeFileSync, rmSync, utimesSync } from "node:fs"; import { join } from "node:path"; import { tmpdir } from "node:os"; import { toClaudeProjectPath, create } from "../index.js"; -import { createActivitySignal, type Session, type RuntimeHandle } from "@aoagents/ao-core"; +import { + createActivitySignal, + readLastActivityEntry, + type ActivityState, + type Session, + type RuntimeHandle, +} from "@aoagents/ao-core"; // Mock homedir() so getActivityState looks in our temp dir vi.mock("node:os", async (importOriginal) => { @@ -57,6 +63,16 @@ function writeJsonl( } } +function writeActivityLog(state: ActivityState, ageMs = 0): void { + const ts = new Date(Date.now() - ageMs).toISOString(); + const aoDir = join(workspacePath, ".ao"); + mkdirSync(aoDir, { recursive: true }); + writeFileSync( + join(aoDir, "activity.jsonl"), + JSON.stringify({ ts, state, source: "terminal" }) + "\n", + ); +} + // ============================================================================= // toClaudeProjectPath // ============================================================================= @@ -141,23 +157,74 @@ describe("Claude Code Activity Detection", () => { // Fallback cases (no JSONL data available) // ----------------------------------------------------------------------- - it("returns 'idle' when no session file exists yet", async () => { - // projectDir exists but is empty — no .jsonl files yet (freshly spawned session) - const session = makeSession(); - const result = await agent.getActivityState(session); - expect(result?.state).toBe("idle"); - // timestamp must be session.createdAt so stuck-detection can fire eventually - expect(result?.timestamp).toBe(session.createdAt); + it("returns null when no session file or AO activity entry exists yet", async () => { + // projectDir exists but is empty, and the AO safety-net log is absent. + expect(await agent.getActivityState(makeSession())).toBeNull(); }); it("returns null when no workspacePath", async () => { expect(await agent.getActivityState(makeSession({ workspacePath: null }))).toBeNull(); }); - it("returns 'idle' when project directory does not exist", async () => { - // Process is running but no Claude project dir yet — treat as idle + it("returns null when project directory does not exist and AO activity is unavailable", async () => { const badPath = join(fakeHome, "nonexistent-workspace"); - expect((await agent.getActivityState(makeSession({ workspacePath: badPath })))?.state).toBe("idle"); + expect(await agent.getActivityState(makeSession({ workspacePath: badPath }))).toBeNull(); + }); + + it("recordActivity writes to .ao/activity.jsonl when workspacePath is set", async () => { + await agent.recordActivity?.(makeSession(), "Do you want to proceed?\n(Y)es / (N)o"); + + const result = await readLastActivityEntry(workspacePath); + expect(result?.entry.state).toBe("waiting_input"); + expect(result?.entry.source).toBe("terminal"); + expect(result?.entry.trigger).toContain("Do you want to proceed?"); + }); + + it("recordActivity is a no-op when workspacePath is null", async () => { + await agent.recordActivity?.( + makeSession({ workspacePath: null }), + "Do you want to proceed?\n(Y)es / (N)o", + ); + + expect(existsSync(join(workspacePath, ".ao", "activity.jsonl"))).toBe(false); + }); + + it("keeps native JSONL as primary when AO activity JSONL also exists", async () => { + writeJsonl([{ type: "assistant", message: { content: "Done!" } }]); + writeActivityLog("waiting_input"); + + expect((await agent.getActivityState(makeSession()))?.state).toBe("ready"); + }); + + it("falls back to AO JSONL waiting_input when native session lookup is unavailable", async () => { + await agent.recordActivity?.(makeSession(), "Do you want to proceed?\n(Y)es / (N)o"); + + expect((await agent.getActivityState(makeSession()))?.state).toBe("waiting_input"); + }); + + it("falls back to AO JSONL waiting_input when native session entry predates this session", async () => { + writeJsonl([{ type: "assistant", message: { content: "Previous session done" } }], 120_000); + const session = makeSession({ createdAt: new Date() }); + + await agent.recordActivity?.(session, "Do you want to proceed?\n(Y)es / (N)o"); + + expect((await agent.getActivityState(session))?.state).toBe("waiting_input"); + }); + + it("returns idle for stale native session entry when AO JSONL is unavailable", async () => { + writeJsonl([{ type: "assistant", message: { content: "Previous session done" } }], 120_000); + const session = makeSession({ createdAt: new Date() }); + + const result = await agent.getActivityState(session); + + expect(result?.state).toBe("idle"); + expect(result?.timestamp).toBe(session.createdAt); + }); + + it("falls back to AO JSONL age-decay when native session lookup is unavailable", async () => { + writeActivityLog("active", 400_000); + + expect((await agent.getActivityState(makeSession()))?.state).toBe("idle"); }); // ----------------------------------------------------------------------- @@ -306,8 +373,8 @@ describe("Claude Code Activity Detection", () => { it("ignores agent- prefixed JSONL files", async () => { writeJsonl([{ type: "user" }], 0, "agent-toolkit.jsonl"); - // No real session file → process is running, treat as idle - expect((await agent.getActivityState(makeSession()))?.state).toBe("idle"); + // No real session file and no AO activity fallback. + expect(await agent.getActivityState(makeSession())).toBeNull(); }); it("reads last entry from multi-entry JSONL (not first)", async () => { diff --git a/packages/plugins/agent-claude-code/src/index.ts b/packages/plugins/agent-claude-code/src/index.ts index 1e5a933ac..7235b05b4 100644 --- a/packages/plugins/agent-claude-code/src/index.ts +++ b/packages/plugins/agent-claude-code/src/index.ts @@ -6,6 +6,10 @@ import { PROCESS_PROBE_INDETERMINATE, DEFAULT_READY_THRESHOLD_MS, DEFAULT_ACTIVE_WINDOW_MS, + readLastActivityEntry, + checkActivityLogState, + getActivityFallbackState, + recordTerminalActivity, type Agent, type AgentSessionInfo, type AgentLaunchConfig, @@ -947,6 +951,13 @@ function createClaudeCodeAgent(): Agent { return classifyTerminalOutput(terminalOutput); }, + async recordActivity(session: Session, terminalOutput: string): Promise { + if (!session.workspacePath) return; + await recordTerminalActivity(session.workspacePath, terminalOutput, (output) => + this.detectActivity(output), + ); + }, + async isProcessRunning(handle: RuntimeHandle): Promise { const pid = await findClaudeProcess(handle); if (pid === PROCESS_PROBE_INDETERMINATE) return PROCESS_PROBE_INDETERMINATE; @@ -976,51 +987,67 @@ function createClaudeCodeAgent(): Agent { const projectDir = join(homedir(), ".claude", "projects", projectPath); const sessionFile = await findLatestSessionFile(projectDir); - if (!sessionFile) { - // No session file yet — process is running but no conversation started. - // Treat as idle (waiting for first task). - return { state: "idle", timestamp: session.createdAt }; + let staleNativeState: ActivityDetection | null = null; + if (sessionFile) { + const entry = await readLastJsonlEntry(sessionFile); + if (entry) { + // If the JSONL entry predates this session, it's from a previous session + // in the same worktree. Fall through to the AO safety net first: the + // terminal may have already surfaced waiting_input/blocked before + // Claude writes this session's first native JSONL entry. + if (session.createdAt && entry.modifiedAt < session.createdAt) { + staleNativeState = { state: "idle", timestamp: session.createdAt }; + } else { + const ageMs = Date.now() - entry.modifiedAt.getTime(); + const timestamp = entry.modifiedAt; + + const activeWindowMs = Math.min(DEFAULT_ACTIVE_WINDOW_MS, threshold); + switch (entry.lastType) { + case "user": + case "tool_use": + case "progress": + if (ageMs <= activeWindowMs) return { state: "active", timestamp }; + return { state: ageMs > threshold ? "idle" : "ready", timestamp }; + + case "assistant": + case "system": + case "summary": + case "result": + return { state: ageMs > threshold ? "idle" : "ready", timestamp }; + + case "permission_request": + return { state: "waiting_input", timestamp }; + + case "error": + return { state: "blocked", timestamp }; + + default: + if (ageMs <= activeWindowMs) return { state: "active", timestamp }; + return { state: ageMs > threshold ? "idle" : "ready", timestamp }; + } + } + } + + // Session file exists but no parseable entry — fall through to AO JSONL + // checks below instead of returning early, so terminal-derived + // waiting_input/blocked can still be detected. } - const entry = await readLastJsonlEntry(sessionFile); - if (!entry) { - // Empty file or read error — cannot determine activity - return null; - } - - // If the JSONL entry predates this session, it's from a previous session - // in the same worktree. Treat as no data (agent hasn't written yet). - if (session.createdAt && entry.modifiedAt < session.createdAt) { - return { state: "idle", timestamp: session.createdAt }; - } - - const ageMs = Date.now() - entry.modifiedAt.getTime(); - const timestamp = entry.modifiedAt; + // Fallback: check AO activity JSONL (terminal-derived) for + // waiting_input/blocked when Claude's native JSONL is unavailable. + const activityResult = await readLastActivityEntry(session.workspacePath); + const activityState = checkActivityLogState(activityResult); + if (activityState) return activityState; + // Last fallback: use the AO entry with age-based decay when native + // session lookup is missing or unparseable (e.g. Claude project slug drift). const activeWindowMs = Math.min(DEFAULT_ACTIVE_WINDOW_MS, threshold); - switch (entry.lastType) { - case "user": - case "tool_use": - case "progress": - if (ageMs <= activeWindowMs) return { state: "active", timestamp }; - return { state: ageMs > threshold ? "idle" : "ready", timestamp }; + const fallback = getActivityFallbackState(activityResult, activeWindowMs, threshold); + if (fallback) return fallback; - case "assistant": - case "system": - case "summary": - case "result": - return { state: ageMs > threshold ? "idle" : "ready", timestamp }; + if (staleNativeState) return staleNativeState; - case "permission_request": - return { state: "waiting_input", timestamp }; - - case "error": - return { state: "blocked", timestamp }; - - default: - if (ageMs <= activeWindowMs) return { state: "active", timestamp }; - return { state: ageMs > threshold ? "idle" : "ready", timestamp }; - } + return null; }, async getSessionInfo(session: Session): Promise { diff --git a/packages/plugins/notifier-composio/src/index.ts b/packages/plugins/notifier-composio/src/index.ts index 08bb21c53..d3dd456ab 100644 --- a/packages/plugins/notifier-composio/src/index.ts +++ b/packages/plugins/notifier-composio/src/index.ts @@ -1,12 +1,21 @@ -import type { - PluginModule, - Notifier, - OrchestratorEvent, - NotifyAction, - NotifyContext, - EventPriority, +import { + recordActivityEvent, + type EventPriority, + type Notifier, + type NotifyAction, + type NotifyContext, + type OrchestratorEvent, + type PluginModule, } from "@aoagents/ao-core"; +// Module-level guard so we only emit notifier.dep_missing once per process. +let depMissingEmitted = false; + +/** Test-only: reset the once-per-process dep_missing guard. */ +export function _resetDepMissingEmittedForTesting(): void { + depMissingEmitted = false; +} + export const manifest = { name: "composio", slot: "notifier" as const, @@ -71,6 +80,22 @@ async function loadComposioSDK(apiKey: string): Promise message.includes("MODULE_NOT_FOUND") || code === "ERR_MODULE_NOT_FOUND" ) { + // User-actionable. Emit once per process so RCA can answer + // "why is the composio notifier silent?" without spamming on every notify call. + if (!depMissingEmitted) { + depMissingEmitted = true; + recordActivityEvent({ + source: "notifier", + kind: "notifier.dep_missing", + level: "error", + summary: "Composio SDK (composio-core) is not installed", + data: { + plugin: "notifier-composio", + package: "composio-core", + installHint: "pnpm add composio-core", + }, + }); + } return null; } throw err; diff --git a/packages/plugins/notifier-discord/src/index.ts b/packages/plugins/notifier-discord/src/index.ts index 7134fa2e2..00191239e 100644 --- a/packages/plugins/notifier-discord/src/index.ts +++ b/packages/plugins/notifier-discord/src/index.ts @@ -1,4 +1,5 @@ import { + recordActivityEvent, validateUrl, type PluginModule, type Notifier, @@ -129,6 +130,17 @@ async function postWithRetry( // Rate-limit budget exhausted — fail immediately rather than falling through // to the error retry path (which would compound the two counters). const body = await response.text().catch(() => ""); + recordActivityEvent({ + source: "notifier", + kind: "notifier.rate_limited", + level: "warn", + summary: `Discord webhook rate-limit retry budget exhausted`, + data: { + plugin: "notifier-discord", + status: 429, + rateLimitRetries, + }, + }); lastError = new Error(`Discord webhook rate-limited (HTTP 429)${body ? `: ${body.trim()}` : ""}`); throw lastError; } diff --git a/packages/plugins/notifier-openclaw/src/activity-events.test.ts b/packages/plugins/notifier-openclaw/src/activity-events.test.ts new file mode 100644 index 000000000..da89f5e2f --- /dev/null +++ b/packages/plugins/notifier-openclaw/src/activity-events.test.ts @@ -0,0 +1,174 @@ +/** + * Regression tests for plugin-internal activity events (issue #1659). + * + * Covers notifier.auth_failed (MUST) and notifier.unreachable (SHOULD) — + * the two failure shapes RCA needs to distinguish. + */ + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import type { OrchestratorEvent } from "@aoagents/ao-core"; + +const { recordActivityEventMock } = vi.hoisted(() => ({ + recordActivityEventMock: vi.fn(), +})); + +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + recordActivityEvent: recordActivityEventMock, + }; +}); + +import { create } from "./index.js"; + +function makeEvent(overrides: Partial = {}): OrchestratorEvent { + return { + id: "evt-1", + type: "reaction.escalated", + priority: "urgent", + sessionId: "ao-5", + projectId: "ao", + timestamp: new Date("2026-03-08T12:00:00Z"), + message: "Reaction escalated", + data: {}, + ...overrides, + }; +} + +beforeEach(() => { + vi.clearAllMocks(); + vi.restoreAllMocks(); + delete process.env.OPENCLAW_HOOKS_TOKEN; +}); + +afterEach(() => { + vi.unstubAllGlobals(); +}); + +describe("notifier.auth_failed (MUST emit)", () => { + it("emits on 401 (distinct from notifier.unreachable on ECONNREFUSED)", async () => { + const fetchMock = vi + .fn() + .mockResolvedValue({ ok: false, status: 401, text: () => Promise.resolve("unauthorized") }); + vi.stubGlobal("fetch", fetchMock); + + const notifier = create({ token: "tok", retries: 0 }); + await expect(notifier.notify(makeEvent())).rejects.toThrow(/OpenClaw rejected the auth token/); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "notifier", + kind: "notifier.auth_failed", + level: "error", + sessionId: "ao-5", + data: expect.objectContaining({ + plugin: "notifier-openclaw", + status: 401, + }), + }), + ); + }); + + it("emits on 403", async () => { + const fetchMock = vi + .fn() + .mockResolvedValue({ ok: false, status: 403, text: () => Promise.resolve("forbidden") }); + vi.stubGlobal("fetch", fetchMock); + + const notifier = create({ token: "tok", retries: 0 }); + await expect(notifier.notify(makeEvent())).rejects.toThrow(); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + kind: "notifier.auth_failed", + data: expect.objectContaining({ status: 403 }), + }), + ); + }); +}); + +describe("notifier.unreachable (SHOULD emit)", () => { + it.each(["ECONNREFUSED", "ETIMEDOUT", "ENOTFOUND", "ENETUNREACH"])( + "emits on %s (distinct from notifier.auth_failed)", + async (code) => { + const fetchMock = vi.fn().mockRejectedValue(new Error(`fetch failed: ${code}`)); + vi.stubGlobal("fetch", fetchMock); + + const notifier = create({ token: "tok", retries: 0 }); + await expect(notifier.notify(makeEvent())).rejects.toThrow(/Can't reach OpenClaw gateway/); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "notifier", + kind: "notifier.unreachable", + level: "warn", + sessionId: "ao-5", + data: expect.objectContaining({ + plugin: "notifier-openclaw", + errorMessage: expect.stringContaining(code), + }), + }), + ); + + // Critically: should NOT also fire auth_failed — distinct shapes. + const authFailedCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "notifier.auth_failed", + ); + expect(authFailedCalls).toHaveLength(0); + }, + ); + + it.each(["ETIMEDOUT", "ENOTFOUND", "ENETUNREACH"])( + "does not emit on transient %s when a retry succeeds", + async (code) => { + const fetchMock = vi + .fn() + .mockRejectedValueOnce(new Error(`fetch failed: ${code}`)) + .mockResolvedValueOnce({ ok: true }); + vi.stubGlobal("fetch", fetchMock); + + const notifier = create({ token: "tok", retries: 1, retryDelayMs: 0 }); + await notifier.notify(makeEvent()); + + expect(fetchMock).toHaveBeenCalledTimes(2); + const unreachableCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "notifier.unreachable", + ); + expect(unreachableCalls).toHaveLength(0); + }, + ); + + it.each(["ETIMEDOUT", "ENOTFOUND", "ENETUNREACH"])( + "emits on transient %s only after retry budget is exhausted", + async (code) => { + const fetchMock = vi.fn().mockRejectedValue(new Error(`fetch failed: ${code}`)); + vi.stubGlobal("fetch", fetchMock); + + const notifier = create({ token: "tok", retries: 1, retryDelayMs: 0 }); + await expect(notifier.notify(makeEvent())).rejects.toThrow(/Can't reach OpenClaw gateway/); + + expect(fetchMock).toHaveBeenCalledTimes(2); + const unreachableCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "notifier.unreachable", + ); + expect(unreachableCalls).toHaveLength(1); + expect(unreachableCalls[0]?.[0].data).toMatchObject({ + errorMessage: expect.stringContaining(code), + }); + }, + ); + + it("does not emit unreachable for unrelated network errors", async () => { + const fetchMock = vi.fn().mockRejectedValue(new Error("fetch failed: certificate expired")); + vi.stubGlobal("fetch", fetchMock); + + const notifier = create({ token: "tok", retries: 0 }); + await expect(notifier.notify(makeEvent())).rejects.toThrow(/fetch failed: certificate expired/); + + const unreachableCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "notifier.unreachable", + ); + expect(unreachableCalls).toHaveLength(0); + }); +}); diff --git a/packages/plugins/notifier-openclaw/src/index.ts b/packages/plugins/notifier-openclaw/src/index.ts index fcd5ae6cf..ced31e7cf 100644 --- a/packages/plugins/notifier-openclaw/src/index.ts +++ b/packages/plugins/notifier-openclaw/src/index.ts @@ -9,6 +9,7 @@ import { type OrchestratorEvent, type PluginModule, getObservabilityBaseDir, + recordActivityEvent, } from "@aoagents/ao-core"; import { isRetryableHttpStatus, normalizeRetryConfig, validateUrl } from "@aoagents/ao-core/utils"; @@ -39,6 +40,13 @@ export const manifest = { }; const DEFAULT_TIMEOUT_MS = 10_000; +const UNREACHABLE_NETWORK_ERROR_CODES = [ + "ECONNREFUSED", + "ETIMEDOUT", + "ENOTFOUND", + "ENETUNREACH", +] as const; +type UnreachableNetworkErrorCode = (typeof UNREACHABLE_NETWORK_ERROR_CODES)[number]; type WakeMode = "now" | "next-heartbeat"; @@ -117,6 +125,10 @@ function recordHealthFailure(path: string | null, error: unknown): void { writeHealthSummary(path, summary); } +function getUnreachableNetworkErrorCode(error: Error): UnreachableNetworkErrorCode | undefined { + return UNREACHABLE_NETWORK_ERROR_CODES.find((code) => error.message.includes(code)); +} + async function postWithRetry( url: string, payload: OpenClawWebhookPayload, @@ -130,6 +142,7 @@ async function postWithRetry( for (let attempt = 0; attempt <= retries; attempt++) { const controller = new AbortController(); const timer = setTimeout(() => controller.abort(), DEFAULT_TIMEOUT_MS); + let shouldRethrowResponseError = false; try { const response = await fetch(url, { method: "POST", @@ -143,17 +156,33 @@ async function postWithRetry( const body = await response.text(); if (response.status === 401 || response.status === 403) { + // User-actionable: distinct from generic 5xx — token expired or wrong. + recordActivityEvent({ + sessionId: context.sessionId, + source: "notifier", + kind: "notifier.auth_failed", + level: "error", + summary: `OpenClaw rejected auth token (HTTP ${response.status})`, + data: { + plugin: "notifier-openclaw", + status: response.status, + url, + fixHint: "ao setup openclaw", + }, + }); lastError = new Error( `OpenClaw rejected the auth token (HTTP ${response.status}).\n` + ` Check that hooks.token in your OpenClaw config matches the token configured for AO.\n` + ` Reconfigure: ao setup openclaw`, ); + shouldRethrowResponseError = true; throw lastError; } lastError = new Error(`OpenClaw webhook failed (${response.status}): ${body}`); if (!isRetryableHttpStatus(response.status)) { + shouldRethrowResponseError = true; throw lastError; } @@ -163,10 +192,24 @@ async function postWithRetry( ); } } catch (err) { - if (err === lastError) throw err; + if (shouldRethrowResponseError && err === lastError) throw err; lastError = err instanceof Error ? err : new Error(String(err)); - if (lastError.message.includes("ECONNREFUSED")) { + const unreachableCode = getUnreachableNetworkErrorCode(lastError); + if (unreachableCode && (unreachableCode === "ECONNREFUSED" || attempt >= retries)) { + recordActivityEvent({ + sessionId: context.sessionId, + source: "notifier", + kind: "notifier.unreachable", + level: "warn", + summary: `OpenClaw gateway unreachable at ${url}`, + data: { + plugin: "notifier-openclaw", + url, + errorMessage: lastError.message, + fixHint: "openclaw status", + }, + }); throw new Error( `Can't reach OpenClaw gateway at ${url}.\n` + ` Is OpenClaw running? Check: openclaw status\n` + diff --git a/packages/plugins/scm-github/src/graphql-batch.ts b/packages/plugins/scm-github/src/graphql-batch.ts index 108d47168..cf92add29 100644 --- a/packages/plugins/scm-github/src/graphql-batch.ts +++ b/packages/plugins/scm-github/src/graphql-batch.ts @@ -9,6 +9,7 @@ import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { execGhObserved, + recordActivityEvent, type BatchObserver, type CICheck, type CIStatus, @@ -56,10 +57,10 @@ export function setExecGhAsync( * Configuration constants for cache sizing. * LRU cache automatically evicts oldest entries when these limits are reached. */ -const MAX_PR_LIST_ETAGS = 100; // Number of repos to cache -const MAX_COMMIT_STATUS_ETAGS = 500; // Number of commits to cache -const MAX_REVIEW_COMMENTS_ETAGS = 500; // Number of PRs to cache review ETags -const MAX_PR_METADATA = 200; // Number of PRs to cache full data +const MAX_PR_LIST_ETAGS = 100; // Number of repos to cache +const MAX_COMMIT_STATUS_ETAGS = 500; // Number of commits to cache +const MAX_REVIEW_COMMENTS_ETAGS = 500; // Number of PRs to cache review ETags +const MAX_PR_METADATA = 200; // Number of PRs to cache full data /** * ETag cache for REST API endpoints. @@ -125,11 +126,7 @@ export function getPRListETag(owner: string, repo: string): string | undefined { /** * Get commit status ETag for a specific commit. */ -export function getCommitStatusETag( - owner: string, - repo: string, - sha: string, -): string | undefined { +export function getCommitStatusETag(owner: string, repo: string, sha: string): string | undefined { return etagCache.commitStatus.get(`${owner}/${repo}#${sha}`); } @@ -145,12 +142,7 @@ export function setPRListETag(owner: string, repo: string, etag: string): void { * Set commit status ETag for a specific commit. * Exported for testing. */ -export function setCommitStatusETag( - owner: string, - repo: string, - sha: string, - etag: string, -): void { +export function setCommitStatusETag(owner: string, repo: string, sha: string, etag: string): void { etagCache.commitStatus.set(`${owner}/${repo}#${sha}`, etag); } @@ -161,10 +153,9 @@ export function setCommitStatusETag( * * Uses LRU eviction to ensure bounded memory usage. */ -const prMetadataCache = new LRUCache< - string, - { headSha: string | null; ciStatus: CIStatus } ->(MAX_PR_METADATA); +const prMetadataCache = new LRUCache( + MAX_PR_METADATA, +); /** * Cache for full PR enrichment data. @@ -241,7 +232,11 @@ export async function shouldRefreshPREnrichment( } if (repos.size === 0) { - return { shouldRefresh: false, details: ["No repos to check"], prListUnchangedRepos: new Set() }; + return { + shouldRefresh: false, + details: ["No repos to check"], + prListUnchangedRepos: new Set(), + }; } // Guard 1: Check PR list ETag for each repository @@ -297,9 +292,7 @@ export async function shouldRefreshPREnrichment( ); if (statusChanged) { shouldRefresh = true; - details.push( - `CI status changed for ${pr.owner}/${pr.repo}#${pr.number} (Guard 2)`, - ); + details.push(`CI status changed for ${pr.owner}/${pr.repo}#${pr.number} (Guard 2)`); } } } @@ -310,10 +303,7 @@ export async function shouldRefreshPREnrichment( /** * Get cached PR metadata for testing. */ -export function getPRMetadataCache(): Map< - string, - { headSha: string | null; ciStatus: CIStatus } -> { +export function getPRMetadataCache(): Map { return prMetadataCache.toMap(); } @@ -350,6 +340,11 @@ interface ErrorWithCause extends Error { cause?: unknown; } +// Module-level guard so we only emit gh_unavailable once per process. +// The error is system-wide (gh missing globally), not session-specific. +let ghUnavailableEmitted = false; +const batchEnrichPRFailedEmitted = new Set(); + /** * Pre-flight check to verify gh CLI is available and authenticated. * This prevents silent failures during GraphQL batch queries. @@ -357,7 +352,21 @@ interface ErrorWithCause extends Error { async function verifyGhCLI(): Promise { try { await execFileAsync("gh", ["--version"], { timeout: 5000 }); - } catch { + } catch (err) { + if (!ghUnavailableEmitted) { + ghUnavailableEmitted = true; + const errorMessage = err instanceof Error ? err.message : String(err); + recordActivityEvent({ + source: "scm", + kind: "scm.gh_unavailable", + level: "error", + summary: "gh CLI not available or not authenticated", + data: { + plugin: "scm-github", + errorMessage, + }, + }); + } const error = new Error( "gh CLI not available or not authenticated. GraphQL batch enrichment requires gh CLI to be installed and configured.", ) as ErrorWithCause; @@ -366,6 +375,16 @@ async function verifyGhCLI(): Promise { } } +/** Test-only: reset the once-per-process gh_unavailable guard. */ +export function _resetGhUnavailableEmittedForTesting(): void { + ghUnavailableEmitted = false; +} + +/** Test-only: reset the once-per-PR batch extraction failure guard. */ +export function _resetBatchEnrichPRFailedEmittedForTesting(): void { + batchEnrichPRFailedEmitted.clear(); +} + /** * Maximum number of PRs to query in a single GraphQL batch. * GitHub has limits on query complexity and we stay well under this limit. @@ -533,7 +552,10 @@ async function checkCommitStatusETag( if (is304(errorMsg)) { return false; } - observer?.log("warn", `[ETag Guard 2] Commit status check failed for ${commitKey}: ${errorMsg}`); + observer?.log( + "warn", + `[ETag Guard 2] Commit status check failed for ${commitKey}: ${errorMsg}`, + ); return true; // Assume changed to be safe } } @@ -596,7 +618,10 @@ export async function checkReviewCommentsETag( if (is304(errorMsg)) { return false; } - observer?.log("warn", `[ETag Guard 3] Review comments check failed for ${cacheKey}: ${errorMsg}`); + observer?.log( + "warn", + `[ETag Guard 3] Review comments check failed for ${cacheKey}: ${errorMsg}`, + ); return true; // Assume changed to be safe } } @@ -704,9 +729,7 @@ export function generateBatchQuery(prs: PRInfo[]): { * * @throws Error if the query fails with GraphQL errors or parsing issues. */ -async function executeBatchQuery( - prs: PRInfo[], -): Promise> { +async function executeBatchQuery(prs: PRInfo[]): Promise> { const { query, variables } = generateBatchQuery(prs); // Handle empty array - no query needed @@ -866,9 +889,7 @@ function parseCheckContexts(contexts: unknown): CICheck[] { * Uses only the top-level aggregate state to determine overall CI status. * Individual check details are parsed separately via parseCheckContexts(). */ -function parseCIState( - statusCheckRollup: unknown, -): CIStatus { +function parseCIState(statusCheckRollup: unknown): CIStatus { if (!statusCheckRollup || typeof statusCheckRollup !== "object") { return "none"; } @@ -885,8 +906,7 @@ function parseCIState( if (state === "PENDING" || state === "EXPECTED") return "pending"; if (state === "TIMED_OUT" || state === "CANCELLED" || state === "ACTION_REQUIRED") return "failing"; - if (state === "QUEUED" || state === "IN_PROGRESS" || state === "WAITING") - return "pending"; + if (state === "QUEUED" || state === "IN_PROGRESS" || state === "WAITING") return "pending"; return "none"; } @@ -927,11 +947,7 @@ function extractPREnrichment( const pr = pullRequest as Record; // Check for at least one required field to validate this is a valid PR object - if ( - pr["state"] === undefined && - pr["title"] === undefined && - pr["commits"] === undefined - ) { + if (pr["state"] === undefined && pr["title"] === undefined && pr["commits"] === undefined) { return null; } @@ -954,9 +970,7 @@ function extractPREnrichment( // Extract merge info const mergeable = pr["mergeable"]; const mergeStateStatus = - typeof pr["mergeStateStatus"] === "string" - ? pr["mergeStateStatus"].toUpperCase() - : ""; + typeof pr["mergeStateStatus"] === "string" ? pr["mergeStateStatus"].toUpperCase() : ""; const hasConflicts = mergeable === "CONFLICTING"; const isBehind = mergeStateStatus === "BEHIND"; @@ -974,9 +988,7 @@ function extractPREnrichment( // contexts(first: 20) silently truncates PRs with >20 checks — when truncated, // the failing check may be missing, so we set ciChecks to undefined to force // the getCIChecks() REST fallback in maybeDispatchCIFailureDetails. - const contextsField = statusCheckRollup?.["contexts"] as - | Record - | undefined; + const contextsField = statusCheckRollup?.["contexts"] as Record | undefined; const pageInfo = contextsField?.["pageInfo"]; const contextsHasNextPage = pageInfo !== null && @@ -984,15 +996,12 @@ function extractPREnrichment( typeof pageInfo === "object" && (pageInfo as Record)["hasNextPage"] === true; const ciChecks = - contextsField && !contextsHasNextPage - ? parseCheckContexts(contextsField) - : undefined; + contextsField && !contextsHasNextPage ? parseCheckContexts(contextsField) : undefined; // Build blockers list const blockers: string[] = []; if (ciStatus === "failing") blockers.push("CI is failing"); - if (reviewDecision === "changes_requested") - blockers.push("Changes requested in review"); + if (reviewDecision === "changes_requested") blockers.push("Changes requested in review"); if (reviewDecision === "pending") blockers.push("Review required"); if (hasConflicts) blockers.push("Merge conflicts"); if (isBehind) blockers.push("Branch is behind base branch"); @@ -1126,6 +1135,25 @@ export async function enrichSessionsPRBatch( result.set(prKey, enrichment); // Update PR metadata cache for future ETag checks updatePRMetadataCache(prKey, enrichment, headSha); + } else { + // GraphQL returned a PR object but extractPREnrichment couldn't + // parse it (missing fields, schema drift). Distinct from the + // whole-batch failure D02 catches further down. + if (!batchEnrichPRFailedEmitted.has(prKey)) { + batchEnrichPRFailedEmitted.add(prKey); + recordActivityEvent({ + source: "scm", + kind: "scm.batch_enrich_pr_failed", + level: "warn", + summary: `batch enrich extraction failed for PR #${pr.number}`, + data: { + plugin: "scm-github", + prNumber: pr.number, + prOwner: pr.owner, + prRepo: pr.repo, + }, + }); + } } } else { // PR not found (deleted/closed/permission issue) @@ -1147,7 +1175,10 @@ export async function enrichSessionsPRBatch( durationMs: batchDuration, }; observer?.recordSuccess(successData); - observer?.log("info", `[GraphQL Batch Success] Batch ${batchIndex + 1}/${batches.length} succeeded: added ${prCountAfter - prCountBefore} PRs to cache (${batchDuration}ms)`); + observer?.log( + "info", + `[GraphQL Batch Success] Batch ${batchIndex + 1}/${batches.length} succeeded: added ${prCountAfter - prCountBefore} PRs to cache (${batchDuration}ms)`, + ); } } catch (err) { // Calculate duration even on failure diff --git a/packages/plugins/scm-github/src/index.ts b/packages/plugins/scm-github/src/index.ts index 8d065db01..fa59cecdc 100644 --- a/packages/plugins/scm-github/src/index.ts +++ b/packages/plugins/scm-github/src/index.ts @@ -11,6 +11,7 @@ import { CI_STATUS, execGhObserved, memoizeAsync, + recordActivityEvent, type PluginModule, type PreflightContext, type SCM, @@ -62,6 +63,12 @@ const BOT_AUTHORS = new Set([ ]); const CI_FAILURE_LOG_TAIL_LINES = 120; +const ciSummaryFailClosedEmitted = new Set(); + +/** Test-only: reset once-per-PR activity event guards. */ +export function _resetGitHubActivityEventDedupeForTesting(): void { + ciSummaryFailClosedEmitted.clear(); +} // --------------------------------------------------------------------------- // Helpers @@ -237,10 +244,7 @@ async function getFailedJobLog( ]); } catch (err) { if (!runReference.jobId) throw err; - return gh([ - "api", - `repos/${pr.owner}/${pr.repo}/actions/jobs/${runReference.jobId}/logs`, - ]); + return gh(["api", `repos/${pr.owner}/${pr.repo}/actions/jobs/${runReference.jobId}/logs`]); } } @@ -522,6 +526,10 @@ function repoFlag(pr: PRInfo): string { return `${pr.owner}/${pr.repo}`; } +function prEventKey(pr: PRInfo): string { + return `${repoFlag(pr)}#${pr.number}`; +} + function parseDate(val: string | undefined | null): Date { if (!val) return new Date(0); const d = new Date(val); @@ -932,7 +940,7 @@ function createGitHubSCM(): SCM { let checks: CICheck[]; try { checks = await this.getCIChecks(pr); - } catch { + } catch (err) { // Before fail-closing, check if the PR is merged/closed — // GitHub may not return check data for those, and reporting // "failing" for a merged PR is wrong. @@ -943,7 +951,26 @@ function createGitHubSCM(): SCM { // Can't determine state either; fall through to fail-closed. } // Fail closed for open PRs: report as failing rather than - // "none" (which getMergeability treats as passing). + // "none" (which getMergeability treats as passing). Emit so RCA + // can distinguish "really failing" from "we couldn't tell". + const eventKey = prEventKey(pr); + if (!ciSummaryFailClosedEmitted.has(eventKey)) { + ciSummaryFailClosedEmitted.add(eventKey); + const errorMessage = err instanceof Error ? err.message : String(err); + recordActivityEvent({ + source: "scm", + kind: "scm.ci_summary_failclosed", + level: "warn", + summary: `getCISummary failed-closed for PR #${pr.number}`, + data: { + plugin: "scm-github", + prNumber: pr.number, + prOwner: pr.owner, + prRepo: pr.repo, + errorMessage, + }, + }); + } return "failing"; } if (checks.length === 0) return "none"; @@ -1035,16 +1062,16 @@ function createGitHubSCM(): SCM { try { // Use GraphQL with variables to get review threads with actual isResolved status const raw = await gh([ - "api", - "graphql", - "-f", - `owner=${pr.owner}`, - "-f", - `name=${pr.repo}`, - "-F", - `number=${pr.number}`, - "-f", - `query=query($owner: String!, $name: String!, $number: Int!) { + "api", + "graphql", + "-f", + `owner=${pr.owner}`, + "-f", + `name=${pr.repo}`, + "-F", + `number=${pr.number}`, + "-f", + `query=query($owner: String!, $name: String!, $number: Int!) { repository(owner: $owner, name: $name) { pullRequest(number: $number) { reviewThreads(first: 100) { @@ -1067,58 +1094,58 @@ function createGitHubSCM(): SCM { } } }`, - ]); + ]); - const data: { - data: { - repository: { - pullRequest: { - reviewThreads: { - nodes: Array<{ - id: string; - isResolved: boolean; - comments: { - nodes: Array<{ - id: string; - author: { login: string } | null; - body: string; - path: string | null; - line: number | null; - url: string; - createdAt: string; - }>; - }; - }>; + const data: { + data: { + repository: { + pullRequest: { + reviewThreads: { + nodes: Array<{ + id: string; + isResolved: boolean; + comments: { + nodes: Array<{ + id: string; + author: { login: string } | null; + body: string; + path: string | null; + line: number | null; + url: string; + createdAt: string; + }>; + }; + }>; + }; }; }; }; - }; - } = JSON.parse(raw); + } = JSON.parse(raw); - const threads = data.data.repository.pullRequest.reviewThreads.nodes; + const threads = data.data.repository.pullRequest.reviewThreads.nodes; - return threads - .filter((t) => { - if (t.isResolved) return false; // only pending (unresolved) threads - const c = t.comments.nodes[0]; - if (!c) return false; // skip threads with no comments - const author = c.author?.login ?? ""; - return !BOT_AUTHORS.has(author); - }) - .map((t) => { - const c = t.comments.nodes[0]; - return { - id: c.id, - threadId: t.id, - author: c.author?.login ?? "unknown", - body: c.body, - path: c.path || undefined, - line: c.line ?? undefined, - isResolved: t.isResolved, - createdAt: parseDate(c.createdAt), - url: c.url, - }; - }); + return threads + .filter((t) => { + if (t.isResolved) return false; // only pending (unresolved) threads + const c = t.comments.nodes[0]; + if (!c) return false; // skip threads with no comments + const author = c.author?.login ?? ""; + return !BOT_AUTHORS.has(author); + }) + .map((t) => { + const c = t.comments.nodes[0]; + return { + id: c.id, + threadId: t.id, + author: c.author?.login ?? "unknown", + body: c.body, + path: c.path || undefined, + line: c.line ?? undefined, + isResolved: t.isResolved, + createdAt: parseDate(c.createdAt), + url: c.url, + }; + }); } catch (err) { throw new Error("Failed to fetch pending comments", { cause: err }); } @@ -1129,7 +1156,12 @@ function createGitHubSCM(): SCM { const cacheKey = `${pr.owner}/${pr.repo}#${pr.number}`; // Guard 3: check if review comments changed via REST ETag - const reviewsChanged = await checkReviewCommentsETag(pr.owner, pr.repo, pr.number, instanceObserver); + const reviewsChanged = await checkReviewCommentsETag( + pr.owner, + pr.repo, + pr.number, + instanceObserver, + ); if (!reviewsChanged) { const cached = reviewThreadsCache.get(cacheKey); if (cached) return cached; diff --git a/packages/plugins/scm-github/test/activity-events.test.ts b/packages/plugins/scm-github/test/activity-events.test.ts new file mode 100644 index 000000000..ad54976df --- /dev/null +++ b/packages/plugins/scm-github/test/activity-events.test.ts @@ -0,0 +1,152 @@ +/** + * Regression tests for plugin-internal activity events (issue #1659). + * + * Covers scm.gh_unavailable (MUST emit, deduped once-per-process). + */ + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +const { recordActivityEventMock } = vi.hoisted(() => ({ + recordActivityEventMock: vi.fn(), +})); + +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + recordActivityEvent: recordActivityEventMock, + }; +}); + +import { + enrichSessionsPRBatch, + setExecFileAsync, + setExecGhAsync, + clearETagCache, + clearPRMetadataCache, + _resetGhUnavailableEmittedForTesting, + _resetBatchEnrichPRFailedEmittedForTesting, +} from "../src/graphql-batch.js"; +import type { PRInfo } from "@aoagents/ao-core"; + +const samplePRs: PRInfo[] = [ + { + owner: "octocat", + repo: "hello-world", + number: 42, + url: "https://github.com/octocat/hello-world/pull/42", + title: "Add new feature", + branch: "feature/new", + baseBranch: "main", + isDraft: false, + }, +]; + +beforeEach(() => { + vi.clearAllMocks(); + clearETagCache(); + clearPRMetadataCache(); + _resetGhUnavailableEmittedForTesting(); + _resetBatchEnrichPRFailedEmittedForTesting(); +}); + +afterEach(() => { + vi.restoreAllMocks(); +}); + +describe("scm.gh_unavailable (MUST emit)", () => { + it("emits when verifyGhCLI fails because gh is missing/unauthenticated", async () => { + const execFileMock = vi.fn().mockImplementation((file: string) => { + if (file === "gh") { + const err = new Error("spawn gh ENOENT") as Error & { code?: string }; + err.code = "ENOENT"; + return Promise.reject(err); + } + return Promise.resolve({ stdout: "", stderr: "" }); + }); + setExecFileAsync(execFileMock as unknown as Parameters[0]); + + // The batch-level try/catch swallows verifyGhCLI's throw — but the event + // fires before the throw, which is what we care about for RCA. + const result = await enrichSessionsPRBatch(samplePRs); + expect(result.enrichment.size).toBe(0); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "scm", + kind: "scm.gh_unavailable", + level: "error", + data: expect.objectContaining({ + plugin: "scm-github", + errorMessage: expect.any(String), + }), + }), + ); + }); + + it("emits exactly once across multiple gh-missing failures (deduped per-process)", async () => { + const execFileMock = vi.fn().mockImplementation((file: string) => { + if (file === "gh") { + return Promise.reject(new Error("gh ENOENT")); + } + return Promise.resolve({ stdout: "", stderr: "" }); + }); + setExecFileAsync(execFileMock as unknown as Parameters[0]); + + await enrichSessionsPRBatch(samplePRs); + await enrichSessionsPRBatch(samplePRs); + await enrichSessionsPRBatch(samplePRs); + + const ghUnavailableCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "scm.gh_unavailable", + ); + expect(ghUnavailableCalls).toHaveLength(1); + }); +}); + +describe("scm.batch_enrich_pr_failed (poll-path emit)", () => { + it("emits exactly once per PR across repeated extraction failures", async () => { + const execFileMock = vi.fn().mockResolvedValue({ stdout: "gh version", stderr: "" }); + setExecFileAsync(execFileMock as unknown as Parameters[0]); + + const execGhMock = vi.fn( + async (_args: string[], _timeout: number, operation: string): Promise => { + if (operation === "gh.api.guard-pr-list") { + return 'HTTP/2 200 OK\netag: W/"pr-list"\n\n[]'; + } + if (operation === "gh.api.graphql-batch") { + return `HTTP/2 200 OK\n\n${JSON.stringify({ + data: { + pr0: { + pullRequest: { unexpectedShape: true }, + }, + }, + })}`; + } + throw new Error(`Unexpected gh operation: ${operation}`); + }, + ); + setExecGhAsync(execGhMock); + + await enrichSessionsPRBatch(samplePRs); + await enrichSessionsPRBatch(samplePRs); + + const extractionFailureCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "scm.batch_enrich_pr_failed", + ); + expect(extractionFailureCalls).toHaveLength(1); + expect(extractionFailureCalls[0]?.[0]).toEqual( + expect.objectContaining({ + source: "scm", + kind: "scm.batch_enrich_pr_failed", + level: "warn", + data: expect.objectContaining({ + plugin: "scm-github", + prNumber: 42, + prOwner: "octocat", + prRepo: "hello-world", + }), + }), + ); + }); +}); diff --git a/packages/plugins/scm-github/test/index.test.ts b/packages/plugins/scm-github/test/index.test.ts index 501ff4e09..7fee69578 100644 --- a/packages/plugins/scm-github/test/index.test.ts +++ b/packages/plugins/scm-github/test/index.test.ts @@ -4,7 +4,10 @@ import { describe, it, expect, beforeEach, vi } from "vitest"; // Mock node:child_process — gh CLI calls go through execFileAsync = promisify(execFile) // vi.hoisted ensures the mock fn is available when vi.mock factory runs (hoisted above imports) // --------------------------------------------------------------------------- -const { ghMock } = vi.hoisted(() => ({ ghMock: vi.fn() })); +const { ghMock, recordActivityEventMock } = vi.hoisted(() => ({ + ghMock: vi.fn(), + recordActivityEventMock: vi.fn(), +})); vi.mock("node:child_process", () => { // Attach the custom promisify symbol so `promisify(execFile)` returns ghMock @@ -14,8 +17,24 @@ vi.mock("node:child_process", () => { return { execFile }; }); -import { create, manifest } from "../src/index.js"; -import { _clearProcessCacheForTests, createActivitySignal, type PreflightContext, type PRInfo, type SCMWebhookRequest, type Session, type ProjectConfig } from "@aoagents/ao-core"; +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + recordActivityEvent: recordActivityEventMock, + }; +}); + +import { create, manifest, _resetGitHubActivityEventDedupeForTesting } from "../src/index.js"; +import { + _clearProcessCacheForTests, + createActivitySignal, + type PreflightContext, + type PRInfo, + type SCMWebhookRequest, + type Session, + type ProjectConfig, +} from "@aoagents/ao-core"; // --------------------------------------------------------------------------- // Fixtures @@ -101,6 +120,7 @@ describe("scm-github plugin", () => { beforeEach(() => { vi.clearAllMocks(); + _resetGitHubActivityEventDedupeForTesting(); scm = create(); delete process.env["GITHUB_WEBHOOK_SECRET"]; }); @@ -665,13 +685,10 @@ describe("scm-github plugin", () => { url: "https://github.com/acme/repo/actions/runs/124/job/457", }, ]; - const logLines = Array.from( - { length: 125 }, - (_, index) => { - const step = index < 100 ? "Install dependencies" : "Run pnpm test"; - return `build\t${step}\t2026-05-12T00:00:00Z line ${index + 1}`; - }, - ); + const logLines = Array.from({ length: 125 }, (_, index) => { + const step = index < 100 ? "Install dependencies" : "Run pnpm test"; + return `build\t${step}\t2026-05-12T00:00:00Z line ${index + 1}`; + }); mockGhRaw(logLines.join("\n")); const summary = await scm.getCIFailureSummary?.(pr, checks); @@ -774,6 +791,34 @@ describe("scm-github plugin", () => { expect(await scm.getCISummary(pr)).toBe("failing"); }); + it("dedupes fail-closed activity events per PR", async () => { + mockGhError("checks failed"); + mockGhError("state failed"); + mockGhError("checks failed again"); + mockGhError("state failed again"); + + expect(await scm.getCISummary(pr)).toBe("failing"); + expect(await scm.getCISummary(pr)).toBe("failing"); + + const failClosedCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "scm.ci_summary_failclosed", + ); + expect(failClosedCalls).toHaveLength(1); + expect(failClosedCalls[0]?.[0]).toEqual( + expect.objectContaining({ + source: "scm", + kind: "scm.ci_summary_failclosed", + level: "warn", + data: expect.objectContaining({ + plugin: "scm-github", + prNumber: 42, + prOwner: "acme", + prRepo: "repo", + }), + }), + ); + }); + it('returns "none" when all checks are skipped', async () => { mockGh([ { name: "a", state: "SKIPPED" }, diff --git a/packages/plugins/scm-gitlab/src/index.ts b/packages/plugins/scm-gitlab/src/index.ts index df9511244..31402653a 100644 --- a/packages/plugins/scm-gitlab/src/index.ts +++ b/packages/plugins/scm-gitlab/src/index.ts @@ -7,6 +7,7 @@ import { createHash, timingSafeEqual } from "node:crypto"; import { CI_STATUS, + recordActivityEvent, type PluginModule, type SCM, type SCMWebhookEvent, @@ -45,6 +46,14 @@ const BOT_AUTHORS = new Set([ "sonarcloud[bot]", "snyk-bot", ]); +const ciSummaryFailClosedEmitted = new Set(); +const reviewFetchFailedEmitted = new Set(); + +/** Test-only: reset once-per-PR activity event guards. */ +export function _resetGitLabActivityEventDedupeForTesting(): void { + ciSummaryFailClosedEmitted.clear(); + reviewFetchFailedEmitted.clear(); +} function isBot(username: string): boolean { return ( @@ -60,6 +69,10 @@ function repoFlag(pr: PRInfo): string { return `${pr.owner}/${pr.repo}`; } +function prEventKey(pr: PRInfo): string { + return `${repoFlag(pr)}#${pr.number}`; +} + function parseDate(val: string | undefined | null): Date { if (!val) return new Date(0); const d = new Date(val); @@ -106,7 +119,6 @@ function mapPRState(state: string): PRState { return "open"; } - function getGitLabWebhookConfig(project: ProjectConfig) { const webhook = project.scm?.webhook; return { @@ -585,6 +597,24 @@ function createGitLabSCM(config?: Record): SCM { `getCISummary: PR state fallback also failed for MR !${pr.number}: ${(innerErr as Error).message}`, ); } + const eventKey = prEventKey(pr); + if (!ciSummaryFailClosedEmitted.has(eventKey)) { + ciSummaryFailClosedEmitted.add(eventKey); + const errorMessage = err instanceof Error ? err.message : String(err); + recordActivityEvent({ + source: "scm", + kind: "scm.ci_summary_failclosed", + level: "warn", + summary: `getCISummary failed-closed for MR !${pr.number}`, + data: { + plugin: "scm-gitlab", + prNumber: pr.number, + prOwner: pr.owner, + prRepo: pr.repo, + errorMessage, + }, + }); + } return "failing"; } if (checks.length === 0) return "none"; @@ -642,6 +672,24 @@ function createGitLabSCM(config?: Record): SCM { console.warn( `getReviews: discussions fetch failed for MR !${pr.number}: ${(err as Error).message}`, ); + const eventKey = prEventKey(pr); + if (!reviewFetchFailedEmitted.has(eventKey)) { + reviewFetchFailedEmitted.add(eventKey); + const errorMessage = err instanceof Error ? err.message : String(err); + recordActivityEvent({ + source: "scm", + kind: "scm.review_fetch_failed", + level: "warn", + summary: `getReviews discussions fetch failed for MR !${pr.number}`, + data: { + plugin: "scm-gitlab", + prNumber: pr.number, + prOwner: pr.owner, + prRepo: pr.repo, + errorMessage, + }, + }); + } } return reviews; diff --git a/packages/plugins/scm-gitlab/test/index.test.ts b/packages/plugins/scm-gitlab/test/index.test.ts index 9dad4d0eb..c72d93ee0 100644 --- a/packages/plugins/scm-gitlab/test/index.test.ts +++ b/packages/plugins/scm-gitlab/test/index.test.ts @@ -3,7 +3,10 @@ import { describe, it, expect, beforeEach, afterEach, vi } from "vitest"; // --------------------------------------------------------------------------- // Mock node:child_process — glab CLI calls go through execFileAsync // --------------------------------------------------------------------------- -const { glabMock } = vi.hoisted(() => ({ glabMock: vi.fn() })); +const { glabMock, recordActivityEventMock } = vi.hoisted(() => ({ + glabMock: vi.fn(), + recordActivityEventMock: vi.fn(), +})); vi.mock("node:child_process", () => { const execFile = Object.assign(vi.fn(), { @@ -12,8 +15,22 @@ vi.mock("node:child_process", () => { return { execFile }; }); -import { create, manifest } from "../src/index.js"; -import { createActivitySignal, type PRInfo, type Session, type ProjectConfig, type SCMWebhookRequest } from "@aoagents/ao-core"; +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + recordActivityEvent: recordActivityEventMock, + }; +}); + +import { create, manifest, _resetGitLabActivityEventDedupeForTesting } from "../src/index.js"; +import { + createActivitySignal, + type PRInfo, + type Session, + type ProjectConfig, + type SCMWebhookRequest, +} from "@aoagents/ao-core"; // --------------------------------------------------------------------------- // Fixtures @@ -104,6 +121,7 @@ describe("scm-gitlab plugin", () => { beforeEach(() => { vi.clearAllMocks(); + _resetGitLabActivityEventDedupeForTesting(); warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {}); scm = create(); delete process.env["GITLAB_WEBHOOK_SECRET"]; @@ -701,6 +719,34 @@ describe("scm-gitlab plugin", () => { expect(await scm.getCISummary(pr)).toBe("failing"); }); + it("dedupes fail-closed activity events per MR", async () => { + mockGlabError("pipeline error"); + mockGlabError("network error"); + mockGlabError("pipeline error again"); + mockGlabError("network error again"); + + expect(await scm.getCISummary(pr)).toBe("failing"); + expect(await scm.getCISummary(pr)).toBe("failing"); + + const failClosedCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "scm.ci_summary_failclosed", + ); + expect(failClosedCalls).toHaveLength(1); + expect(failClosedCalls[0]?.[0]).toEqual( + expect.objectContaining({ + source: "scm", + kind: "scm.ci_summary_failclosed", + level: "warn", + data: expect.objectContaining({ + plugin: "scm-gitlab", + prNumber: 42, + prOwner: "acme", + prRepo: "repo", + }), + }), + ); + }); + it('returns "none" when all jobs are skipped', async () => { mockGlab([{ id: 1 }]); mockGlab([ @@ -807,6 +853,34 @@ describe("scm-gitlab plugin", () => { ); }); + it("dedupes review fetch failed activity events per MR", async () => { + mockGlab({ approved_by: [] }); + mockGlabError("discussions fetch failed"); + mockGlab({ approved_by: [] }); + mockGlabError("discussions fetch failed again"); + + expect(await scm.getReviews(pr)).toEqual([]); + expect(await scm.getReviews(pr)).toEqual([]); + + const reviewFetchCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "scm.review_fetch_failed", + ); + expect(reviewFetchCalls).toHaveLength(1); + expect(reviewFetchCalls[0]?.[0]).toEqual( + expect.objectContaining({ + source: "scm", + kind: "scm.review_fetch_failed", + level: "warn", + data: expect.objectContaining({ + plugin: "scm-gitlab", + prNumber: 42, + prOwner: "acme", + prRepo: "repo", + }), + }), + ); + }); + it("filters bot authors from discussions", async () => { mockGlab({ approved_by: [] }); mockGlab([ diff --git a/packages/plugins/tracker-linear/src/activity-events.test.ts b/packages/plugins/tracker-linear/src/activity-events.test.ts new file mode 100644 index 000000000..06d552ee4 --- /dev/null +++ b/packages/plugins/tracker-linear/src/activity-events.test.ts @@ -0,0 +1,143 @@ +/** + * Regression tests for plugin-internal activity events (issue #1659). + * + * Covers tracker.dep_missing (MUST emit, deduped once-per-process). + */ + +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; + +const { recordActivityEventMock, requestMock } = vi.hoisted(() => ({ + recordActivityEventMock: vi.fn(), + requestMock: vi.fn(), +})); + +vi.mock("node:https", () => ({ + request: requestMock, +})); + +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + recordActivityEvent: recordActivityEventMock, + }; +}); + +// @composio/core is intentionally not installed — the real dynamic import +// will fail with ERR_MODULE_NOT_FOUND, which is exactly the dep_missing +// shape we want to exercise. + +import { create, _resetDepMissingEmittedForTesting } from "./index.js"; +import type { ProjectConfig } from "@aoagents/ao-core"; + +beforeEach(() => { + vi.clearAllMocks(); + recordActivityEventMock.mockReset(); + requestMock.mockReset(); + _resetDepMissingEmittedForTesting(); + process.env.COMPOSIO_API_KEY = "test-key"; + process.env.COMPOSIO_ENTITY_ID = "test-entity"; + delete process.env.LINEAR_API_KEY; +}); + +afterEach(() => { + delete process.env.COMPOSIO_API_KEY; + delete process.env.COMPOSIO_ENTITY_ID; + delete process.env.LINEAR_API_KEY; + vi.useRealTimers(); +}); + +function makeProject(): ProjectConfig { + return { + name: "test-project", + repo: "test/repo", + path: "/repo/path", + defaultBranch: "main", + sessionPrefix: "test", + tracker: { teamId: "TEAM-1" }, + }; +} + +describe("tracker.dep_missing (MUST emit)", () => { + it("emits when Composio SDK is not installed", async () => { + const tracker = create(); + + // Any tracker call routes through the composio transport, which will + // fail to load the missing SDK on first use. + await expect(tracker.getIssue("TEST-1", makeProject())).rejects.toThrow( + /Composio SDK.*not installed/, + ); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "tracker", + kind: "tracker.dep_missing", + level: "error", + data: expect.objectContaining({ + plugin: "tracker-linear", + package: "@composio/core", + }), + }), + ); + }); + + it("emits exactly once across multiple calls (deduped per-process)", async () => { + const tracker = create(); + + await expect(tracker.getIssue("TEST-1", makeProject())).rejects.toThrow(); + await expect(tracker.getIssue("TEST-2", makeProject())).rejects.toThrow(); + await expect(tracker.getIssue("TEST-3", makeProject())).rejects.toThrow(); + + const depMissingCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "tracker.dep_missing", + ); + expect(depMissingCalls).toHaveLength(1); + }); +}); + +describe("tracker.api_timeout", () => { + it("rejects direct transport timeouts even when activity logging throws", async () => { + delete process.env.COMPOSIO_API_KEY; + delete process.env.COMPOSIO_ENTITY_ID; + process.env.LINEAR_API_KEY = "linear-key"; + vi.useFakeTimers(); + + const req = { + setTimeout: vi.fn(), + on: vi.fn(), + write: vi.fn(), + end: vi.fn(), + destroy: vi.fn(), + }; + req.setTimeout.mockImplementation((_timeoutMs: number, cb: () => void) => { + setTimeout(cb, 0); + return req; + }); + req.on.mockReturnValue(req); + requestMock.mockReturnValue(req); + recordActivityEventMock.mockImplementationOnce(() => { + throw new Error("activity sink failed"); + }); + + const tracker = create(); + const timeoutExpectation = expect(tracker.getIssue("TEST-1", makeProject())).rejects.toThrow( + "Linear API request timed out after 30s", + ); + + await vi.runAllTimersAsync(); + await timeoutExpectation; + expect(req.destroy).toHaveBeenCalled(); + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "tracker", + kind: "tracker.api_timeout", + level: "warn", + data: expect.objectContaining({ + plugin: "tracker-linear", + transport: "direct", + timeoutMs: 30_000, + }), + }), + ); + }); +}); diff --git a/packages/plugins/tracker-linear/src/index.ts b/packages/plugins/tracker-linear/src/index.ts index fd93e3eb8..6101ac1c3 100644 --- a/packages/plugins/tracker-linear/src/index.ts +++ b/packages/plugins/tracker-linear/src/index.ts @@ -9,17 +9,35 @@ */ import { request } from "node:https"; -import type { - PluginModule, - Tracker, - Issue, - IssueFilters, - IssueUpdate, - CreateIssueInput, - ProjectConfig, +import { + recordActivityEvent, + type CreateIssueInput, + type Issue, + type IssueFilters, + type IssueUpdate, + type PluginModule, + type ProjectConfig, + type Tracker, } from "@aoagents/ao-core"; import type { Composio } from "@composio/core"; +// Module-level guard so we only emit tracker.dep_missing once per process +// even if multiple sessions trigger the missing-SDK path. +let depMissingEmitted = false; + +/** Test-only: reset the once-per-process dep_missing guard. */ +export function _resetDepMissingEmittedForTesting(): void { + depMissingEmitted = false; +} + +function recordTransportActivityEvent(event: Parameters[0]): void { + try { + recordActivityEvent(event); + } catch { + // Activity logging must never prevent timeout promises from settling. + } +} + // --------------------------------------------------------------------------- // Transport abstraction // --------------------------------------------------------------------------- @@ -111,6 +129,17 @@ function createDirectTransport(): GraphQLTransport { req.setTimeout(30_000, () => { settle(() => { req.destroy(); + recordTransportActivityEvent({ + source: "tracker", + kind: "tracker.api_timeout", + level: "warn", + summary: "Linear API request timed out after 30s", + data: { + plugin: "tracker-linear", + transport: "direct", + timeoutMs: 30_000, + }, + }); reject(new Error("Linear API request timed out after 30s")); }); }); @@ -147,6 +176,21 @@ function createComposioTransport(apiKey: string, entityId: string): GraphQLTrans msg.includes("Cannot find package") || msg.includes("ERR_MODULE_NOT_FOUND") ) { + // User-actionable, system-wide. Emit once per process. + if (!depMissingEmitted) { + depMissingEmitted = true; + recordActivityEvent({ + source: "tracker", + kind: "tracker.dep_missing", + level: "error", + summary: "Composio SDK (@composio/core) is not installed", + data: { + plugin: "tracker-linear", + package: "@composio/core", + installHint: "pnpm add @composio/core", + }, + }); + } throw new Error( "Composio SDK (@composio/core) is not installed. " + "Install it with: pnpm add @composio/core", @@ -175,6 +219,17 @@ function createComposioTransport(apiKey: string, entityId: string): GraphQLTrans let timer: ReturnType | undefined; const timeoutPromise = new Promise((_resolve, reject) => { timer = setTimeout(() => { + recordTransportActivityEvent({ + source: "tracker", + kind: "tracker.api_timeout", + level: "warn", + summary: "Composio Linear API request timed out after 30s", + data: { + plugin: "tracker-linear", + transport: "composio", + timeoutMs: 30_000, + }, + }); reject(new Error("Composio Linear API request timed out after 30s")); }, 30_000); }); diff --git a/packages/plugins/workspace-clone/src/__tests__/index.test.ts b/packages/plugins/workspace-clone/src/__tests__/index.test.ts index 2b194ccd3..0c2b1da45 100644 --- a/packages/plugins/workspace-clone/src/__tests__/index.test.ts +++ b/packages/plugins/workspace-clone/src/__tests__/index.test.ts @@ -3,6 +3,20 @@ import * as childProcess from "node:child_process"; import * as fs from "node:fs"; import type { ProjectConfig } from "@aoagents/ao-core"; +const { getShellMock, recordActivityEventMock } = vi.hoisted(() => ({ + getShellMock: vi.fn(() => ({ cmd: "sh", args: (c: string) => ["-c", c] })), + recordActivityEventMock: vi.fn(), +})); + +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + getShell: getShellMock, + recordActivityEvent: recordActivityEventMock, + }; +}); + // Mock node:child_process with custom promisify support vi.mock("node:child_process", () => { const mockExecFile = vi.fn(); @@ -10,10 +24,6 @@ vi.mock("node:child_process", () => { return { execFile: mockExecFile }; }); -vi.mock("@aoagents/ao-core", () => ({ - getShell: vi.fn(() => ({ cmd: "sh", args: (c: string) => ["-c", c] })), -})); - // Mock node:fs vi.mock("node:fs", () => ({ existsSync: vi.fn(), @@ -267,9 +277,48 @@ describe("workspace.create()", () => { cwd: "/mock-home/.ao-clones/proj/sess", }); + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "workspace", + kind: "workspace.branch_collision", + level: "warn", + projectId: "proj", + sessionId: "sess", + data: expect.objectContaining({ + plugin: "workspace-clone", + branch: "feat/existing", + errorMessage: expect.stringContaining("already exists"), + }), + }), + ); expect(info.branch).toBe("feat/existing"); }); + it("does not emit branch_collision when checkout -b fails for a non-collision reason", async () => { + const workspace = create(); + + mockGitSuccess("https://github.com/test/repo.git"); + (fs.existsSync as ReturnType).mockReturnValue(false); + mockGitSuccess(""); + // git checkout -b fails for a non-collision reason + mockGitError("fatal: cannot lock ref 'refs/heads/feat/locked': Permission denied"); + // git checkout (plain) succeeds + mockGitSuccess(""); + + const info = await workspace.create({ + projectId: "proj", + sessionId: "sess", + branch: "feat/locked", + project: makeProject(), + }); + + expect(info.branch).toBe("feat/locked"); + const branchCollisionCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "workspace.branch_collision", + ); + expect(branchCollisionCalls).toHaveLength(0); + }); + it("cleans up partial clone on clone failure", async () => { const workspace = create(); @@ -588,6 +637,41 @@ describe("workspace.list()", () => { warnSpy.mockRestore(); }); + it("emits corrupt_clone_skipped only once per clone path", async () => { + const workspace = create(); + (fs.existsSync as ReturnType).mockReturnValue(true); + + const warnSpy = vi.spyOn(console, "warn").mockImplementation(() => {}); + + (fs.readdirSync as ReturnType).mockReturnValue([ + { name: "corrupt-repeat", isDirectory: () => true }, + ]); + + mockGitError("fatal: not a git repository"); + mockGitError("fatal: not a git repository"); + + await expect(workspace.list("myproject")).resolves.toEqual([]); + await expect(workspace.list("myproject")).resolves.toEqual([]); + + const corruptCloneCalls = recordActivityEventMock.mock.calls.filter( + ([event]) => event.kind === "workspace.corrupt_clone_skipped", + ); + expect(corruptCloneCalls).toHaveLength(1); + expect(corruptCloneCalls[0][0]).toEqual( + expect.objectContaining({ + projectId: "myproject", + sessionId: "corrupt-repeat", + source: "workspace", + kind: "workspace.corrupt_clone_skipped", + data: expect.objectContaining({ + clonePath: "/mock-home/.ao-clones/myproject/corrupt-repeat", + }), + }), + ); + + warnSpy.mockRestore(); + }); + it("rejects invalid projectId with special characters", async () => { const workspace = create(); diff --git a/packages/plugins/workspace-clone/src/index.ts b/packages/plugins/workspace-clone/src/index.ts index 075fc3119..71100644a 100644 --- a/packages/plugins/workspace-clone/src/index.ts +++ b/packages/plugins/workspace-clone/src/index.ts @@ -3,7 +3,15 @@ import { promisify } from "node:util"; import { existsSync, rmSync, mkdirSync, readdirSync } from "node:fs"; import { join } from "node:path"; import { homedir } from "node:os"; -import { getShell, type PluginModule, type Workspace, type WorkspaceCreateConfig, type WorkspaceInfo, type ProjectConfig } from "@aoagents/ao-core"; +import { + getShell, + recordActivityEvent, + type PluginModule, + type ProjectConfig, + type Workspace, + type WorkspaceCreateConfig, + type WorkspaceInfo, +} from "@aoagents/ao-core"; const execFileAsync = promisify(execFile); @@ -37,6 +45,16 @@ function expandPath(p: string): string { return p; } +function getErrorMessage(err: unknown): string { + return err instanceof Error ? err.message : String(err); +} + +function isBranchAlreadyExistsError(err: unknown): boolean { + return getErrorMessage(err).toLowerCase().includes("already exists"); +} + +const emittedCorruptClonePaths = new Set(); + export function create(config?: Record): Workspace { const cloneBaseDir = config?.cloneDir ? expandPath(config.cloneDir as string) @@ -97,8 +115,24 @@ export function create(config?: Record): Workspace { // Create and checkout the feature branch try { await git(clonePath, "checkout", "-b", cfg.branch); - } catch { - // Branch may exist on remote — try plain checkout + } catch (branchErr: unknown) { + // Branch may exist on remote — try plain checkout, but only label it + // as a branch_collision when git reported the distinct collision shape. + if (isBranchAlreadyExistsError(branchErr)) { + recordActivityEvent({ + projectId: cfg.projectId, + sessionId: cfg.sessionId, + source: "workspace", + kind: "workspace.branch_collision", + level: "warn", + summary: `branch "${cfg.branch}" already exists; falling back to checkout`, + data: { + plugin: "workspace-clone", + branch: cfg.branch, + errorMessage: getErrorMessage(branchErr), + }, + }); + } try { await git(clonePath, "checkout", cfg.branch); } catch (checkoutErr: unknown) { @@ -142,10 +176,27 @@ export function create(config?: Record): Workspace { try { branch = await git(clonePath, "branch", "--show-current"); } catch (err: unknown) { - // Warn about corrupted clones instead of silently skipping + // Warn about corrupted clones instead of silently skipping. + // RCA: "session shows up on disk but isn't returned by list()". const msg = err instanceof Error ? err.message : String(err); // eslint-disable-next-line no-console -- expected diagnostic for corrupted clones console.warn(`[workspace-clone] Skipping "${entry.name}": not a valid git repo (${msg})`); + if (!emittedCorruptClonePaths.has(clonePath)) { + emittedCorruptClonePaths.add(clonePath); + recordActivityEvent({ + projectId, + sessionId: entry.name, + source: "workspace", + kind: "workspace.corrupt_clone_skipped", + level: "warn", + summary: `skipped corrupt clone "${entry.name}"`, + data: { + plugin: "workspace-clone", + clonePath, + errorMessage: msg, + }, + }); + } continue; } diff --git a/packages/plugins/workspace-worktree/src/__tests__/activity-events.test.ts b/packages/plugins/workspace-worktree/src/__tests__/activity-events.test.ts new file mode 100644 index 000000000..3a2f6039c --- /dev/null +++ b/packages/plugins/workspace-worktree/src/__tests__/activity-events.test.ts @@ -0,0 +1,192 @@ +/** + * Regression tests for plugin-internal activity events (issue #1659). + * + * Covers the MUST emit: workspace.post_create_failed, plus the SHOULDs + * workspace.branch_collision and workspace.destroy_fell_back. + */ + +import { describe, it, expect, beforeEach, vi } from "vitest"; + +// --------------------------------------------------------------------------- +// Mocks — declared before any import that uses the mocked modules +// --------------------------------------------------------------------------- + +const { recordActivityEventMock } = vi.hoisted(() => ({ + recordActivityEventMock: vi.fn(), +})); + +vi.mock("@aoagents/ao-core", async () => { + const actual = (await vi.importActual("@aoagents/ao-core")) as Record; + return { + ...actual, + recordActivityEvent: recordActivityEventMock, + }; +}); + +vi.mock("node:child_process", () => { + const mockExecFile = vi.fn(); + (mockExecFile as unknown as Record)[Symbol.for("nodejs.util.promisify.custom")] = + vi.fn(); + return { execFile: mockExecFile }; +}); + +vi.mock("node:fs", () => ({ + cpSync: vi.fn(), + existsSync: vi.fn(() => false), + linkSync: vi.fn(), + lstatSync: vi.fn(), + statSync: vi.fn(), + symlinkSync: vi.fn(), + rmSync: vi.fn(), + mkdirSync: vi.fn(), + readdirSync: vi.fn(), +})); + +vi.mock("node:os", () => ({ homedir: () => "/mock-home" })); + +import * as childProcess from "node:child_process"; +import { create } from "../index.js"; +import type { ProjectConfig, WorkspaceCreateConfig, WorkspaceInfo } from "@aoagents/ao-core/types"; + +const mockExecFileAsync = (childProcess.execFile as unknown as Record)[ + Symbol.for("nodejs.util.promisify.custom") +] as ReturnType; + +function makeProject(overrides?: Partial): ProjectConfig { + return { + name: "test-project", + repo: "test/repo", + path: "/repo/path", + defaultBranch: "main", + sessionPrefix: "test", + ...overrides, + }; +} + +function makeCreateConfig(overrides?: Partial): WorkspaceCreateConfig { + return { + projectId: "myproject", + project: makeProject(), + sessionId: "session-1", + branch: "feat/TEST-1", + ...overrides, + }; +} + +function makeWorkspaceInfo(): WorkspaceInfo { + return { + path: "/mock-home/.worktrees/myproject/session-1", + branch: "feat/TEST-1", + sessionId: "session-1", + projectId: "myproject", + }; +} + +beforeEach(() => { + vi.clearAllMocks(); +}); + +describe("workspace.post_create_failed (MUST emit)", () => { + it("emits when a postCreate command fails, then rethrows", async () => { + const ws = create(); + const project = makeProject({ postCreate: ["pnpm install"] }); + + mockExecFileAsync.mockRejectedValueOnce(new Error("Command failed: exit 127")); + + await expect(ws.postCreate!(makeWorkspaceInfo(), project)).rejects.toThrow( + "Command failed: exit 127", + ); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "workspace", + kind: "workspace.post_create_failed", + level: "error", + sessionId: "session-1", + projectId: "myproject", + data: expect.objectContaining({ + plugin: "workspace-worktree", + command: "pnpm install", + errorMessage: expect.stringContaining("exit 127"), + }), + }), + ); + }); + + it("does NOT emit when postCreate command succeeds", async () => { + const ws = create(); + const project = makeProject({ postCreate: ["echo hi"] }); + + mockExecFileAsync.mockResolvedValueOnce({ stdout: "hi\n", stderr: "" }); + + await ws.postCreate!(makeWorkspaceInfo(), project); + + expect(recordActivityEventMock).not.toHaveBeenCalled(); + }); +}); + +describe("workspace.branch_collision (SHOULD emit)", () => { + it("emits when worktree add -b fails because branch already exists", async () => { + const ws = create(); + + // git remote get-url origin + mockExecFileAsync.mockResolvedValueOnce({ stdout: "origin\n", stderr: "" }); + // git fetch origin --quiet + mockExecFileAsync.mockResolvedValueOnce({ stdout: "", stderr: "" }); + // resolveBaseRef -> rev-parse origin/main + mockExecFileAsync.mockResolvedValueOnce({ stdout: "abc\n", stderr: "" }); + // git worktree add -b ... → already exists + mockExecFileAsync.mockRejectedValueOnce( + new Error("fatal: a branch named 'feat/TEST-1' already exists"), + ); + // rev-parse baseRef for stale-branch comparison + mockExecFileAsync.mockResolvedValueOnce({ stdout: "abc\n", stderr: "" }); + // refExists(branchRef) -> true + mockExecFileAsync.mockResolvedValueOnce({ stdout: "refs/heads/feat/TEST-1\n", stderr: "" }); + // rev-parse existing branch -> same as base, so reuse it + mockExecFileAsync.mockResolvedValueOnce({ stdout: "abc\n", stderr: "" }); + // git worktree add (without -b) — succeeds + mockExecFileAsync.mockResolvedValueOnce({ stdout: "", stderr: "" }); + + await ws.create(makeCreateConfig()); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "workspace", + kind: "workspace.branch_collision", + level: "warn", + projectId: "myproject", + sessionId: "session-1", + data: expect.objectContaining({ + plugin: "workspace-worktree", + branch: "feat/TEST-1", + errorMessage: expect.stringContaining("already exists"), + }), + }), + ); + }); +}); + +describe("workspace.destroy_fell_back (SHOULD emit)", () => { + it("emits when destroy() falls back to rmSync after git failure", async () => { + const ws = create(); + + // git rev-parse --git-common-dir → fails + mockExecFileAsync.mockRejectedValueOnce(new Error("not a git repository")); + + await ws.destroy("/some/stale/path"); + + expect(recordActivityEventMock).toHaveBeenCalledWith( + expect.objectContaining({ + source: "workspace", + kind: "workspace.destroy_fell_back", + level: "warn", + data: expect.objectContaining({ + plugin: "workspace-worktree", + workspacePath: "/some/stale/path", + errorMessage: expect.stringContaining("not a git repository"), + }), + }), + ); + }); +}); diff --git a/packages/plugins/workspace-worktree/src/__tests__/index.test.ts b/packages/plugins/workspace-worktree/src/__tests__/index.test.ts index f4837fd62..c39afd6d9 100644 --- a/packages/plugins/workspace-worktree/src/__tests__/index.test.ts +++ b/packages/plugins/workspace-worktree/src/__tests__/index.test.ts @@ -5,6 +5,10 @@ import type { ProjectConfig, WorkspaceCreateConfig, WorkspaceInfo } from "@aoage // Mocks — must be declared before any import that uses the mocked modules // --------------------------------------------------------------------------- +const { recordActivityEventMock } = vi.hoisted(() => ({ + recordActivityEventMock: vi.fn(), +})); + vi.mock("node:child_process", () => { const mockExecFile = vi.fn(); // Set custom promisify so `promisify(execFile)` returns { stdout, stderr } @@ -27,6 +31,7 @@ vi.mock("node:fs", () => ({ vi.mock("@aoagents/ao-core", () => ({ getShell: vi.fn(() => ({ cmd: "sh", args: (c: string) => ["-c", c] })), isWindows: vi.fn(() => false), + recordActivityEvent: recordActivityEventMock, })); vi.mock("node:os", () => ({ diff --git a/packages/plugins/workspace-worktree/src/index.ts b/packages/plugins/workspace-worktree/src/index.ts index 7e0b6f61e..1f5c32840 100644 --- a/packages/plugins/workspace-worktree/src/index.ts +++ b/packages/plugins/workspace-worktree/src/index.ts @@ -16,6 +16,7 @@ import { homedir } from "node:os"; import { getShell, isWindows, + recordActivityEvent, type PluginModule, type Workspace, type WorkspaceCreateConfig, @@ -361,7 +362,21 @@ export function create(config?: Record): Workspace { // Branch already exists. It may be a stale session branch left behind // from an earlier spawn, so compare it with the freshly-resolved base - // before reusing it. + // before reusing it. Surface the collision shape for RCA before the + // recovery path decides whether to reuse or reset the local branch. + recordActivityEvent({ + projectId: cfg.projectId, + sessionId: cfg.sessionId, + source: "workspace", + kind: "workspace.branch_collision", + level: "warn", + summary: `branch "${cfg.branch}" already exists; falling back to worktree recovery`, + data: { + plugin: "workspace-worktree", + branch: cfg.branch, + errorMessage: msg, + }, + }); const baseSha = await git(repoPath, "rev-parse", baseRef); const branchRef = `refs/heads/${cfg.branch}`; const existingBranchSha = (await refExists(repoPath, branchRef)) @@ -453,8 +468,24 @@ export function create(config?: Record): Workspace { // pre-existing local branches unrelated to this workspace (any branch // containing "/" would have been deleted). Stale branches can be // cleaned up separately via `git branch --merged` or similar. - } catch { + } catch (err) { // If git commands fail, try to clean up the directory. + // The worktree metadata may be left stale in `git worktree list` + // because we couldn't run `worktree remove`. Surface so RCA can + // explain why a path was deleted but `git worktree list` still + // references it. + const errorMessage = err instanceof Error ? err.message : String(err); + recordActivityEvent({ + source: "workspace", + kind: "workspace.destroy_fell_back", + level: "warn", + summary: "destroy fell back to rmSync; git worktree metadata may be stale", + data: { + plugin: "workspace-worktree", + workspacePath, + errorMessage, + }, + }); // On Windows, retry with backoff for the file-handle drain race // (just-killed pty-host children still hold handles inside the worktree). if (existsSync(workspacePath)) { @@ -661,10 +692,31 @@ export function create(config?: Record): Workspace { if (project.postCreate) { const shell = getShell(); for (const command of project.postCreate) { - await execFileAsync(shell.cmd, shell.args(command), { - cwd: info.path, - windowsHide: true, - }); + try { + await execFileAsync(shell.cmd, shell.args(command), { + cwd: info.path, + windowsHide: true, + }); + } catch (err) { + // Surface which postCreate command failed. Lifecycle records + // a generic spawn_failed but loses the specific command and + // its sanitized error output. + const errorMessage = err instanceof Error ? err.message : String(err); + recordActivityEvent({ + projectId: info.projectId, + sessionId: info.sessionId, + source: "workspace", + kind: "workspace.post_create_failed", + level: "error", + summary: `postCreate command failed for session ${info.sessionId}`, + data: { + plugin: "workspace-worktree", + command, + errorMessage, + }, + }); + throw err; + } } } }, diff --git a/packages/web/server/__tests__/mux-websocket.test.ts b/packages/web/server/__tests__/mux-websocket.test.ts index 48f38e96f..3e9c41377 100644 --- a/packages/web/server/__tests__/mux-websocket.test.ts +++ b/packages/web/server/__tests__/mux-websocket.test.ts @@ -1,15 +1,26 @@ import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; import { EventEmitter } from "node:events"; +import type { Socket } from "node:net"; +import { WebSocket } from "ws"; import type { SessionBroadcaster as SessionBroadcasterType } from "../mux-websocket"; // vi.mock factories run before module-level statements. Hoist the mock // fns so the factories close over the same instances the tests use. -const { mockSpawn, mockPtySpawn, mockTmuxHasSession } = vi.hoisted(() => ({ +const { mockSpawn, mockPtySpawn, mockTmuxHasSession, recordActivityEvent } = vi.hoisted(() => ({ mockSpawn: vi.fn(), mockPtySpawn: vi.fn(), mockTmuxHasSession: vi.fn(), + recordActivityEvent: vi.fn(), })); +vi.mock("@aoagents/ao-core", async (importOriginal) => { + const actual = (await importOriginal()) as Record; + return { + ...actual, + recordActivityEvent: (event: unknown) => recordActivityEvent(event), + }; +}); + vi.mock("node:child_process", async (importOriginal) => { const actual = (await importOriginal()) as Record; const spawnFn = (...args: unknown[]) => mockSpawn(...args); @@ -34,21 +45,72 @@ vi.mock("../tmux-utils.js", () => ({ findTmux: () => "/usr/bin/tmux", validateSessionId: () => true, resolveTmuxSession: () => "ao-177", + resolvePipePath: () => null, tmuxHasSession: (...args: unknown[]) => mockTmuxHasSession(...args), })); -const { SessionBroadcaster, TerminalManager } = await import("../mux-websocket"); +const { SessionBroadcaster, TerminalManager, createMuxWebSocket, handleWindowsPipeMessage } = + await import("../mux-websocket"); // Mock global fetch const mockFetch = vi.fn(); global.fetch = mockFetch; +type MockPty = { + dataHandlers: Array<(data: string) => void>; + exitHandlers: Array<(event: { exitCode: number }) => void>; + onData: ReturnType; + onExit: ReturnType; + write: ReturnType; + resize: ReturnType; + kill: ReturnType; + emitData: (data: string) => void; + emitExit: (exitCode: number) => Promise; +}; + +const ptyInstances: MockPty[] = []; + +function createMockPty(): MockPty { + const pty = {} as MockPty; + pty.dataHandlers = []; + pty.exitHandlers = []; + pty.onData = vi.fn((handler: (data: string) => void) => { + pty.dataHandlers.push(handler); + }); + pty.onExit = vi.fn((handler: (event: { exitCode: number }) => void) => { + pty.exitHandlers.push(handler); + }); + pty.write = vi.fn(); + pty.resize = vi.fn(); + pty.kill = vi.fn(); + pty.emitData = (data: string) => { + for (const handler of pty.dataHandlers) handler(data); + }; + pty.emitExit = async (exitCode: number) => { + await Promise.all([...pty.exitHandlers].map((handler) => handler({ exitCode }))); + }; + ptyInstances.push(pty); + return pty; +} + +function resetPtyMock(): void { + ptyInstances.length = 0; + mockSpawn.mockReset(); + mockPtySpawn.mockReset(); + mockTmuxHasSession.mockReset(); + mockTmuxHasSession.mockResolvedValue(true); + mockSpawn.mockImplementation(() => new EventEmitter()); + mockPtySpawn.mockImplementation(createMockPty); +} + describe("SessionBroadcaster", () => { let broadcaster: SessionBroadcasterType; beforeEach(() => { vi.useFakeTimers(); mockFetch.mockReset(); + recordActivityEvent.mockClear(); + resetPtyMock(); broadcaster = new SessionBroadcaster("3000"); }); @@ -262,6 +324,451 @@ describe("SessionBroadcaster", () => { expect(mockFetch).toHaveBeenCalledTimes(1); }); }); + + describe("ui.session_broadcast_failed activity events", () => { + function failedKinds(): string[] { + return recordActivityEvent.mock.calls + .map(([e]) => (e as { kind: string }).kind) + .filter((k) => k === "ui.session_broadcast_failed"); + } + + it("emits exactly once on the healthy→failing transition", async () => { + // First fetch fails — triggers emission + mockFetch.mockRejectedValueOnce(new Error("ECONNREFUSED")); + // Second fetch (3s later) also fails — should NOT emit again + mockFetch.mockRejectedValueOnce(new Error("ECONNREFUSED")); + + broadcaster.subscribe(vi.fn()); + await vi.advanceTimersByTimeAsync(10); + await vi.advanceTimersByTimeAsync(3010); + + expect(failedKinds()).toEqual(["ui.session_broadcast_failed"]); + }); + + it("re-arms after recovery (success → failure emits again)", async () => { + // fail → succeed → fail + mockFetch.mockRejectedValueOnce(new Error("net down")); + mockFetch.mockResolvedValueOnce({ ok: true, json: async () => ({ sessions: [] }) }); + mockFetch.mockRejectedValueOnce(new Error("net down again")); + + broadcaster.subscribe(vi.fn()); + await vi.advanceTimersByTimeAsync(10); + await vi.advanceTimersByTimeAsync(3010); // poll #1 → success + await vi.advanceTimersByTimeAsync(3010); // poll #2 → failure + + expect(failedKinds().length).toBe(2); + }); + + it("emits with source=ui, level=warn, and the failure URL in data", async () => { + mockFetch.mockRejectedValueOnce(new Error("ETIMEDOUT")); + + broadcaster.subscribe(vi.fn()); + await vi.advanceTimersByTimeAsync(10); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "ui", + kind: "ui.session_broadcast_failed", + level: "warn", + }), + ); + const call = recordActivityEvent.mock.calls.find( + ([e]) => (e as { kind: string }).kind === "ui.session_broadcast_failed", + )![0] as { data: Record }; + expect(call.data["url"]).toContain("/api/sessions/patches"); + expect(call.data["errorMessage"]).toContain("ETIMEDOUT"); + }); + + it("includes httpStatus when fetch returns non-OK response", async () => { + mockFetch.mockResolvedValueOnce({ ok: false, status: 503 }); + + broadcaster.subscribe(vi.fn()); + await vi.advanceTimersByTimeAsync(10); + + const call = recordActivityEvent.mock.calls.find( + ([e]) => (e as { kind: string }).kind === "ui.session_broadcast_failed", + )![0] as { data: Record }; + expect(call.data["httpStatus"]).toBe(503); + }); + }); +}); + +// ── Connection-level activity events ────────────────────────────────── +// These verify ui.terminal_* events fire at the right WS lifecycle points. +// We exercise the connection handler directly by emitting "connection" on +// the WebSocketServer and feeding a fake ws + IncomingMessage stand-in. + +class FakeWS extends EventEmitter { + readyState: 0 | 1 | 2 | 3 = WebSocket.OPEN; + bufferedAmount = 0; + ping = vi.fn(); + terminate = vi.fn(() => { + this.readyState = WebSocket.CLOSED; + }); + send = vi.fn(); +} + +class FakePipeSocket extends EventEmitter { + write = vi.fn(); + end = vi.fn(() => { + this.emit("close"); + }); + destroy = vi.fn(() => { + this.emit("close"); + }); +} + +function makeFakeRequest(opts?: { remoteAddress?: string; xff?: string }) { + return { + headers: opts?.xff ? { "x-forwarded-for": opts.xff } : {}, + socket: { remoteAddress: opts?.remoteAddress ?? "127.0.0.1" }, + }; +} + +describe("mux WebSocket connection events", () => { + beforeEach(() => { + vi.useFakeTimers(); + recordActivityEvent.mockClear(); + resetPtyMock(); + }); + + afterEach(() => { + vi.clearAllTimers(); + vi.useRealTimers(); + }); + + function emitConnection(opts?: Parameters[0]) { + const wss = createMuxWebSocket(); + if (!wss) { + throw new Error("mux WS server not created — node-pty unavailable"); + } + const ws = new FakeWS(); + wss.emit("connection", ws as unknown as WebSocket, makeFakeRequest(opts)); + return { wss, ws }; + } + + function findEvent(kind: string): { data: Record } | undefined { + const found = recordActivityEvent.mock.calls.find( + ([e]) => (e as { kind: string }).kind === kind, + ); + return found?.[0] as { data: Record } | undefined; + } + + it("emits ui.terminal_connected on a new mux connection (with remoteAddr)", () => { + emitConnection({ xff: "198.51.100.5, 10.0.0.1" }); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ source: "ui", kind: "ui.terminal_connected" }), + ); + const evt = findEvent("ui.terminal_connected")!; + expect(evt.data["remoteAddr"]).toBe("198.51.100.5"); + }); + + it("emits ui.terminal_disconnected exactly once on close", () => { + const { ws } = emitConnection(); + recordActivityEvent.mockClear(); + + ws.emit("close", 1000, Buffer.from("normal")); + + const calls = recordActivityEvent.mock.calls.filter( + ([e]) => (e as { kind: string }).kind === "ui.terminal_disconnected", + ); + expect(calls.length).toBe(1); + const evt = findEvent("ui.terminal_disconnected")!; + expect(evt.data["code"]).toBe(1000); + expect(evt.data["reason"]).toBe("normal"); + }); + + it("emits ui.terminal_heartbeat_lost once on 3 missed pongs and terminates", () => { + const { ws } = emitConnection(); + recordActivityEvent.mockClear(); + + // Each 15s interval sends a ping and increments missedPongs by 1. + // After 3 ticks (45s) it should hit MAX_MISSED_PONGS=3 and terminate. + vi.advanceTimersByTime(15_000); + vi.advanceTimersByTime(15_000); + vi.advanceTimersByTime(15_000); + + const calls = recordActivityEvent.mock.calls.filter( + ([e]) => (e as { kind: string }).kind === "ui.terminal_heartbeat_lost", + ); + expect(calls.length).toBe(1); + expect(ws.terminate).toHaveBeenCalled(); + + // Issue invariant: at most one emit per state change — extra ticks must not + // produce another event. + vi.advanceTimersByTime(15_000); + expect( + recordActivityEvent.mock.calls.filter( + ([e]) => (e as { kind: string }).kind === "ui.terminal_heartbeat_lost", + ).length, + ).toBe(1); + }); + + it("does NOT emit heartbeat_lost when pong arrives before 3 missed pings", () => { + const { ws } = emitConnection(); + recordActivityEvent.mockClear(); + + vi.advanceTimersByTime(15_000); // missedPongs=1 + ws.emit("pong"); // resets to 0 + vi.advanceTimersByTime(15_000); // missedPongs=1 + vi.advanceTimersByTime(15_000); // missedPongs=2 + + expect( + recordActivityEvent.mock.calls.filter( + ([e]) => (e as { kind: string }).kind === "ui.terminal_heartbeat_lost", + ).length, + ).toBe(0); + expect(ws.terminate).not.toHaveBeenCalled(); + }); + + it("emits ui.terminal_protocol_error on malformed client message", () => { + const { ws } = emitConnection(); + recordActivityEvent.mockClear(); + + ws.emit("message", Buffer.from("not-json{{{")); + + const calls = recordActivityEvent.mock.calls.filter( + ([e]) => (e as { kind: string }).kind === "ui.terminal_protocol_error", + ); + expect(calls.length).toBe(1); + const evt = findEvent("ui.terminal_protocol_error")!; + expect(evt.data["errorMessage"]).toBeTruthy(); + }); +}); + +describe("Windows pipe ui.terminal_pty_lost activity events", () => { + beforeEach(() => { + recordActivityEvent.mockClear(); + }); + + function framedMessage(type: number, payload: unknown): Buffer { + const body = Buffer.from(JSON.stringify(payload), "utf-8"); + const header = Buffer.alloc(5); + header.writeUInt8(type, 0); + header.writeUInt32BE(body.length, 1); + return Buffer.concat([header, body]); + } + + function openPipe() { + const ws = new FakeWS(); + const pipe = new FakePipeSocket(); + const winPipes = new Map(); + const winPipeBuffers = new Map(); + const deps = { + connect: vi.fn(() => pipe as unknown as Socket), + resolvePipePath: vi.fn(() => "\\\\.\\pipe\\ao-pty-app-1"), + }; + + handleWindowsPipeMessage( + { id: "app-1", type: "open", projectId: "proj-1" }, + ws, + winPipes, + winPipeBuffers, + deps, + ); + pipe.emit("connect"); + recordActivityEvent.mockClear(); + ws.send.mockClear(); + + return { ws, pipe, winPipes, winPipeBuffers, deps }; + } + + function ptyLostEvents(): Array<{ data: Record; sessionId?: string }> { + return recordActivityEvent.mock.calls + .map(([e]) => e as { kind: string; data: Record; sessionId?: string }) + .filter((event) => event.kind === "ui.terminal_pty_lost"); + } + + it("emits ui.terminal_pty_lost when the PTY host pipe closes while the socket is open", () => { + const { ws, pipe } = openPipe(); + + pipe.emit("close"); + + expect(ptyLostEvents()).toEqual([ + expect.objectContaining({ + sessionId: "app-1", + data: expect.objectContaining({ + sessionId: "app-1", + transport: "windows_pipe", + reason: "pipe_closed", + }), + }), + ]); + expect(ws.send).toHaveBeenCalledWith( + JSON.stringify({ ch: "terminal", id: "app-1", type: "exited", code: 0, projectId: "proj-1" }), + ); + }); + + it("emits ui.terminal_pty_lost when the PTY host reports not alive", () => { + const { ws, pipe } = openPipe(); + + pipe.emit("data", framedMessage(0x07, { alive: false })); + pipe.emit("close"); + + const events = ptyLostEvents(); + expect(events).toHaveLength(1); + expect(events[0]).toEqual( + expect.objectContaining({ + sessionId: "app-1", + data: expect.objectContaining({ + sessionId: "app-1", + transport: "windows_pipe", + reason: "host_not_alive", + }), + }), + ); + expect(ws.send).toHaveBeenCalledWith( + JSON.stringify({ ch: "terminal", id: "app-1", type: "exited", code: 0, projectId: "proj-1" }), + ); + }); + + it("does not emit ui.terminal_pty_lost for an intentional client close", () => { + const { ws, winPipes, winPipeBuffers, deps } = openPipe(); + + handleWindowsPipeMessage( + { id: "app-1", type: "close", projectId: "proj-1" }, + ws, + winPipes, + winPipeBuffers, + deps, + ); + + expect(ptyLostEvents()).toEqual([]); + }); +}); + +describe("TerminalManager ui.terminal_pty_lost activity events", () => { + beforeEach(() => { + recordActivityEvent.mockClear(); + resetPtyMock(); + }); + + function ptyLostEvents(): Array<{ data: Record; sessionId?: string }> { + return recordActivityEvent.mock.calls + .map(([e]) => e as { kind: string; data: Record; sessionId?: string }) + .filter((event) => event.kind === "ui.terminal_pty_lost"); + } + + it("emits ui.terminal_pty_lost when a subscribed PTY exits and reattach fails", async () => { + const manager = new TerminalManager("/usr/bin/tmux"); + manager.open("app-1", "proj-1", "tmux-app-1"); + const pty = ptyInstances[0]; + expect(pty).toBeDefined(); + + manager.subscribe("app-1", "proj-1", vi.fn(), vi.fn()); + mockPtySpawn.mockImplementationOnce(() => { + throw new Error("reattach unavailable"); + }); + + await pty!.emitExit(9); + + const events = ptyLostEvents(); + expect(events).toHaveLength(1); + expect(events[0]).toEqual( + expect.objectContaining({ + sessionId: "app-1", + data: expect.objectContaining({ + sessionId: "app-1", + exitCode: 9, + subscriberCount: 1, + reattachError: "reattach unavailable", + }), + }), + ); + }); + + it("emits ui.terminal_pty_lost when a subscribed PTY exits and reattach succeeds", async () => { + const manager = new TerminalManager("/usr/bin/tmux"); + manager.open("app-1", "proj-1", "tmux-app-1"); + const pty = ptyInstances[0]; + expect(pty).toBeDefined(); + + manager.subscribe("app-1", "proj-1", vi.fn(), vi.fn()); + + await pty!.emitExit(9); + + const events = ptyLostEvents(); + expect(mockPtySpawn).toHaveBeenCalledTimes(2); + expect(events).toHaveLength(1); + expect(events[0]).toEqual( + expect.objectContaining({ + sessionId: "app-1", + data: expect.objectContaining({ + sessionId: "app-1", + exitCode: 9, + subscriberCount: 1, + reattachRecovered: true, + reattachExhausted: false, + }), + }), + ); + }); + + it("does not emit ui.terminal_pty_lost when the last subscriber already left", async () => { + const manager = new TerminalManager("/usr/bin/tmux"); + manager.open("app-1", "proj-1", "tmux-app-1"); + const pty = ptyInstances[0]; + expect(pty).toBeDefined(); + + const unsubscribe = manager.subscribe("app-1", "proj-1", vi.fn(), vi.fn()); + unsubscribe(); + + await pty!.emitExit(0); + + expect(ptyLostEvents()).toEqual([]); + }); + + it("emits ui.terminal_pty_lost at most once across reattach cycles", async () => { + const manager = new TerminalManager("/usr/bin/tmux"); + manager.open("app-1", "proj-1", "tmux-app-1"); + const firstPty = ptyInstances[0]; + expect(firstPty).toBeDefined(); + + manager.subscribe("app-1", "proj-1", vi.fn(), vi.fn()); + mockPtySpawn.mockImplementationOnce(() => { + throw new Error("first reattach unavailable"); + }); + await firstPty!.emitExit(7); + expect(ptyLostEvents()).toHaveLength(1); + + // A client may try to re-open the terminal after the first PTY loss. + // A second failed reattach should not produce another activity event for + // the same terminal entry. + manager.open("app-1", "proj-1", "tmux-app-1"); + const secondPty = ptyInstances[1]; + expect(secondPty).toBeDefined(); + mockPtySpawn.mockImplementationOnce(() => { + throw new Error("second reattach unavailable"); + }); + await secondPty!.emitExit(8); + + expect(ptyLostEvents()).toHaveLength(1); + }); + + it("re-arms ui.terminal_pty_lost after a successful reattach survives the grace period", async () => { + vi.useFakeTimers(); + try { + const manager = new TerminalManager("/usr/bin/tmux"); + manager.open("app-1", "proj-1", "tmux-app-1"); + const firstPty = ptyInstances[0]; + expect(firstPty).toBeDefined(); + + manager.subscribe("app-1", "proj-1", vi.fn(), vi.fn()); + await firstPty!.emitExit(7); + expect(ptyLostEvents()).toHaveLength(1); + + await vi.advanceTimersByTimeAsync(5_000); + + const secondPty = ptyInstances[1]; + expect(secondPty).toBeDefined(); + await secondPty!.emitExit(8); + + expect(ptyLostEvents()).toHaveLength(2); + } finally { + vi.useRealTimers(); + } + }); }); describe("TerminalManager.open — tmux target args (regression for #1714)", () => { @@ -325,6 +832,7 @@ describe("TerminalManager.open — re-attach skipped when tmux session is gone ( mockSpawn.mockReset(); mockPtySpawn.mockReset(); mockTmuxHasSession.mockReset(); + recordActivityEvent.mockClear(); capturedOnExit = undefined; mockSpawn.mockImplementation(() => new EventEmitter()); @@ -355,6 +863,20 @@ describe("TerminalManager.open — re-attach skipped when tmux session is gone ( // Subscribers were notified with the original exit code. expect(exitCb).toHaveBeenCalledTimes(1); expect(exitCb).toHaveBeenCalledWith(0); + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "ui", + kind: "ui.terminal_pty_lost", + level: "warn", + sessionId: "ao-177", + data: expect.objectContaining({ + sessionId: "ao-177", + exitCode: 0, + reattachSkipped: true, + tmuxSessionPresent: false, + }), + }), + ); }); it("still re-attaches when has-session reports the tmux session is alive", async () => { diff --git a/packages/web/server/mux-websocket.ts b/packages/web/server/mux-websocket.ts index 5515574df..5db3d4630 100644 --- a/packages/web/server/mux-websocket.ts +++ b/packages/web/server/mux-websocket.ts @@ -16,7 +16,7 @@ import { tmuxHasSession, validateSessionId, } from "./tmux-utils.js"; -import { getEnvDefaults, isWindows } from "@aoagents/ao-core"; +import { getEnvDefaults, isWindows, recordActivityEvent } from "@aoagents/ao-core"; // These types mirror src/lib/mux-protocol.ts exactly. // tsconfig.server.json constrains rootDir to "server/", so we cannot import @@ -63,6 +63,9 @@ export class SessionBroadcaster { private errorSubscribers = new Set<(error: string) => void>(); private intervalId: ReturnType | null = null; private polling = false; + // Tracks the last fetch outcome so we only emit ui.session_broadcast_failed on + // the healthy → failing transition (not every 3s during an outage). + private lastFetchOk = true; private readonly baseUrl: string; constructor(nextPort: string) { @@ -158,18 +161,42 @@ export class SessionBroadcaster { if (!res.ok) { const msg = `Session fetch failed: HTTP ${res.status}`; console.warn(`[SessionBroadcaster] ${msg}`); + this.recordFetchFailure(msg, { httpStatus: res.status }); return { sessions: null, error: msg }; } const data = (await res.json()) as { sessions?: SessionPatch[] }; + this.lastFetchOk = true; return { sessions: data.sessions ?? null, error: null }; } catch (err) { clearTimeout(timeoutId); const msg = err instanceof Error ? err.message : String(err); console.warn("[SessionBroadcaster] fetchSnapshot error:", msg); + this.recordFetchFailure(msg); return { sessions: null, error: msg }; } } + /** + * Emit ui.session_broadcast_failed once per healthy→failing transition. + * The broadcaster polls every 3s; emitting on every failure during a long + * outage would flood the events table (~20/min). Recovery resets the flag. + */ + private recordFetchFailure(message: string, extra?: Record): void { + if (!this.lastFetchOk) return; + this.lastFetchOk = false; + recordActivityEvent({ + source: "ui", + kind: "ui.session_broadcast_failed", + level: "warn", + summary: `session broadcaster fetch failed: ${message}`, + data: { + url: `${this.baseUrl}/api/sessions/patches`, + errorMessage: message, + ...extra, + }, + }); + } + private disconnect(): void { if (this.intervalId !== null) { clearInterval(this.intervalId); @@ -199,6 +226,7 @@ interface ManagedTerminal { buffer: string[]; bufferBytes: number; reattachAttempts: number; + ptyLostEmitted: boolean; /** * Pending grace-period timer that resets reattachAttempts when the * currently-attached PTY survives REATTACH_RESET_GRACE_MS. Tracked so @@ -276,6 +304,7 @@ export class TerminalManager { buffer: [], bufferBytes: 0, reattachAttempts: 0, + ptyLostEmitted: false, }; this.terminals.set(key, terminal); } @@ -346,6 +375,7 @@ export class TerminalManager { terminal.resetTimer = undefined; if (terminal.pty === pty) { terminal.reattachAttempts = 0; + terminal.ptyLostEmitted = false; } }, REATTACH_RESET_GRACE_MS); terminal.resetTimer.unref(); @@ -383,6 +413,7 @@ export class TerminalManager { pty.onExit(async ({ exitCode }) => { console.log(`[MuxServer] PTY exited for ${id} with code ${exitCode}`); terminal.pty = null; + let reattachError: string | undefined; // Skip the re-attach loop entirely when the underlying tmux session is // gone (e.g. user pressed Ctrl-C in the pane and the launch command @@ -392,15 +423,33 @@ export class TerminalManager { // clean user-initiated termination — see issue #1756. The // MAX_REATTACH_ATTEMPTS bound from #1640 still covers tmux server // hiccups where the session does still exist. - if ( - terminal.subscribers.size > 0 && - !(await tmuxHasSession(this.TMUX, tmuxSessionId)) - ) { + if (terminal.subscribers.size > 0 && !(await tmuxHasSession(this.TMUX, tmuxSessionId))) { console.log(`[MuxServer] tmux session ${tmuxSessionId} is gone, not re-attaching`); if (terminal.resetTimer) { clearTimeout(terminal.resetTimer); terminal.resetTimer = undefined; } + if (!terminal.ptyLostEmitted) { + terminal.ptyLostEmitted = true; + recordActivityEvent({ + projectId, + sessionId: id, + source: "ui", + kind: "ui.terminal_pty_lost", + level: "warn", + summary: `terminal PTY exited (code ${exitCode}) — tmux session gone`, + data: { + sessionId: id, + exitCode, + reattachAttempts: terminal.reattachAttempts, + maxReattachAttempts: MAX_REATTACH_ATTEMPTS, + reattachExhausted: false, + reattachSkipped: true, + tmuxSessionPresent: false, + subscriberCount: terminal.subscribers.size, + }, + }); + } for (const cb of terminal.exitCallbacks) { cb(exitCode); } @@ -423,14 +472,63 @@ export class TerminalManager { ); try { this.open(id, projectId, tmuxSessionId); + if (!terminal.ptyLostEmitted) { + terminal.ptyLostEmitted = true; + recordActivityEvent({ + projectId, + sessionId: id, + source: "ui", + kind: "ui.terminal_pty_lost", + level: "warn", + summary: `terminal PTY exited (code ${exitCode}) — reattached`, + data: { + sessionId: id, + exitCode, + reattachAttempts: terminal.reattachAttempts, + maxReattachAttempts: MAX_REATTACH_ATTEMPTS, + reattachExhausted: false, + reattachRecovered: true, + subscriberCount: terminal.subscribers.size, + }, + }); + } return; // re-attached — don't notify exit } catch (err) { + reattachError = err instanceof Error ? err.message : String(err); console.error(`[MuxServer] Failed to re-attach ${id}:`, err); } } else if (terminal.reattachAttempts >= MAX_REATTACH_ATTEMPTS) { console.error(`[MuxServer] Max re-attach attempts reached for ${id}, giving up`); } + // PTY actually died (vs user closed browser): only emit when subscribers + // are still attached — otherwise the exit is just normal cleanup. + // Keep this event one-shot for the terminal entry. Clients may re-open + // the same terminal after a failed reattach; repeated PTY exits should + // not flood the activity log for the same loss condition. + if (terminal.subscribers.size > 0 && !terminal.ptyLostEmitted) { + terminal.ptyLostEmitted = true; + recordActivityEvent({ + projectId, + sessionId: id, + source: "ui", + kind: "ui.terminal_pty_lost", + level: "warn", + summary: `terminal PTY exited (code ${exitCode})${ + terminal.reattachAttempts >= MAX_REATTACH_ATTEMPTS ? " — reattach exhausted" : "" + }`, + data: { + sessionId: id, + exitCode, + reattachAttempts: terminal.reattachAttempts, + maxReattachAttempts: MAX_REATTACH_ATTEMPTS, + reattachExhausted: terminal.reattachAttempts >= MAX_REATTACH_ATTEMPTS, + subscriberCount: terminal.subscribers.size, + ...(reattachError ? { reattachError } : {}), + }, + }); + } + // Notify subscribers that the terminal has exited (re-attach failed or no subscribers) for (const cb of terminal.exitCallbacks) { cb(exitCode); @@ -515,6 +613,8 @@ export class TerminalManager { // ── Windows Pipe Relay (extracted for testability) ── +const intentionalWinPipeCloses = new WeakSet(); + /** Minimal WebSocket-like interface for the pipe relay handler */ export interface WsSink { send(data: string): void; @@ -586,8 +686,36 @@ export function handleWindowsPipeMessage( const pipeSocket = deps.connect(pipePath); winPipes.set(pipeKey, pipeSocket); winPipeBuffers.set(pipeKey, Buffer.alloc(0)); + let ptyLostEmitted = false; + const recordWindowsPtyLost = ( + reason: "pipe_closed" | "host_not_alive" | "pipe_error", + extra?: Record, + ): void => { + if (ptyLostEmitted || ws.readyState !== WS_OPEN) return; + ptyLostEmitted = true; + recordActivityEvent({ + projectId, + sessionId: id, + source: "ui", + kind: "ui.terminal_pty_lost", + level: "warn", + summary: + reason === "host_not_alive" + ? `terminal PTY host reported not alive for ${id}` + : reason === "pipe_error" + ? `terminal PTY host pipe errored for ${id}` + : `terminal PTY host pipe closed for ${id}`, + data: { + sessionId: id, + transport: "windows_pipe", + reason, + ...extra, + }, + }); + }; pipeSocket.on("error", (err) => { + recordWindowsPtyLost("pipe_error", { errorMessage: err.message }); winPipes.delete(pipeKey); winPipeBuffers.delete(pipeKey); pipeSocket.destroy(); @@ -637,9 +765,8 @@ export function handleWindowsPipeMessage( try { const status = JSON.parse(payload.toString("utf-8")) as { alive: boolean }; if (!status.alive && ws.readyState === WS_OPEN) { - ws.send( - JSON.stringify({ ch: "terminal", id, type: "exited", code: 0, ...echo }), - ); + recordWindowsPtyLost("host_not_alive"); + ws.send(JSON.stringify({ ch: "terminal", id, type: "exited", code: 0, ...echo })); } } catch { /* ignore parse errors */ @@ -651,7 +778,11 @@ export function handleWindowsPipeMessage( pipeSocket.on("close", () => { winPipes.delete(pipeKey); winPipeBuffers.delete(pipeKey); + const intentionalClose = intentionalWinPipeCloses.delete(pipeSocket); if (ws.readyState === WS_OPEN) { + if (!intentionalClose) { + recordWindowsPtyLost("pipe_closed"); + } ws.send(JSON.stringify({ ch: "terminal", id, type: "exited", code: 0, ...echo })); } }); @@ -678,6 +809,7 @@ export function handleWindowsPipeMessage( } else if (type === "close") { const pipeSocket = winPipes.get(pipeKey); if (pipeSocket) { + intentionalWinPipeCloses.add(pipeSocket); pipeSocket.end(); winPipes.delete(pipeKey); winPipeBuffers.delete(pipeKey); @@ -706,9 +838,26 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | const wss = new WebSocketServer({ noServer: true }); - wss.on("connection", (ws) => { + wss.on("connection", (ws, request) => { console.log("[MuxServer] New mux connection"); + const connectedAt = Date.now(); + // Best-effort remote addr — proxy headers if present, else socket peer. + const xff = request?.headers["x-forwarded-for"]; + const xffStr = Array.isArray(xff) ? xff[0] : xff; + const remoteAddr = + (typeof xffStr === "string" ? xffStr.split(",")[0]?.trim() : undefined) ?? + request?.socket?.remoteAddress ?? + undefined; + + recordActivityEvent({ + source: "ui", + kind: "ui.terminal_connected", + level: "info", + summary: "mux WebSocket connection opened", + data: { remoteAddr }, + }); + const subscriptions = new Map void>(); // Windows: named pipe sockets keyed by session ID const winPipes = new Map>(); @@ -716,6 +865,7 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | const winPipeBuffers = new Map(); let sessionUnsubscribe: (() => void) | null = null; let missedPongs = 0; + let heartbeatLostEmitted = false; const MAX_MISSED_PONGS = 3; // Heartbeat: send native WebSocket ping every 15s. @@ -728,6 +878,22 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | missedPongs += 1; if (missedPongs >= MAX_MISSED_PONGS) { console.log("[MuxServer] Too many missed pongs, terminating connection"); + if (!heartbeatLostEmitted) { + heartbeatLostEmitted = true; + recordActivityEvent({ + source: "ui", + kind: "ui.terminal_heartbeat_lost", + level: "warn", + summary: `mux WebSocket heartbeat lost (${missedPongs} missed pongs)`, + data: { + missedPongs, + maxMissedPongs: MAX_MISSED_PONGS, + connectionAgeMs: Date.now() - connectedAt, + remoteAddr, + subscriberCount: subscriptions.size, + }, + }); + } ws.terminate(); } } @@ -759,7 +925,14 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | if (type === "open") { if (isWindows()) { handleWindowsPipeMessage( - msg as { id: string; type: string; projectId?: string; data?: string; cols?: number; rows?: number }, + msg as { + id: string; + type: string; + projectId?: string; + data?: string; + cols?: number; + rows?: number; + }, ws, winPipes, winPipeBuffers, @@ -841,7 +1014,13 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | } else if (type === "resize" && "cols" in msg && "rows" in msg) { if (isWindows()) { handleWindowsPipeMessage( - msg as { id: string; type: string; projectId?: string; cols: number; rows: number }, + msg as { + id: string; + type: string; + projectId?: string; + cols: number; + rows: number; + }, ws, winPipes, winPipeBuffers, @@ -853,7 +1032,14 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | } else if (type === "close") { if (isWindows()) { handleWindowsPipeMessage( - msg as { id: string; type: string; projectId?: string; data?: string; cols?: number; rows?: number }, + msg as { + id: string; + type: string; + projectId?: string; + data?: string; + cols?: number; + rows?: number; + }, ws, winPipes, winPipeBuffers, @@ -903,6 +1089,17 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | } } catch (err) { console.error("[MuxServer] Failed to parse message:", err); + recordActivityEvent({ + source: "ui", + kind: "ui.terminal_protocol_error", + level: "warn", + summary: "invalid mux client message — parse failed", + data: { + errorMessage: err instanceof Error ? err.message : String(err), + remoteAddr, + subscriberCount: subscriptions.size, + }, + }); const errorMsg: ServerMessage = { ch: "system", type: "error", @@ -917,8 +1114,22 @@ export function createMuxWebSocket(tmuxPath?: string | null): WebSocketServer | /** * Handle connection close */ - ws.on("close", () => { + ws.on("close", (code, reason) => { console.log("[MuxServer] Mux connection closed"); + recordActivityEvent({ + source: "ui", + kind: "ui.terminal_disconnected", + level: "info", + summary: "mux WebSocket connection closed", + data: { + code, + reason: reason?.toString("utf8") || undefined, + connectionAgeMs: Date.now() - connectedAt, + subscriberCount: subscriptions.size, + heartbeatLost: heartbeatLostEmitted, + remoteAddr, + }, + }); clearInterval(heartbeatInterval); sessionUnsubscribe?.(); sessionUnsubscribe = null; diff --git a/packages/web/src/__tests__/activity-events-projects.test.ts b/packages/web/src/__tests__/activity-events-projects.test.ts new file mode 100644 index 000000000..2aad007da --- /dev/null +++ b/packages/web/src/__tests__/activity-events-projects.test.ts @@ -0,0 +1,164 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { mkdirSync, mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import path from "node:path"; +import { execSync } from "node:child_process"; +import { NextRequest } from "next/server"; +import { recordActivityEvent, registerProjectInGlobalConfig } from "@aoagents/ao-core"; + +vi.mock("@aoagents/ao-core", async () => { + const actual = await vi.importActual("@aoagents/ao-core"); + return { + ...(actual as Record), + recordActivityEvent: vi.fn(), + }; +}); + +const invalidatePortfolioServicesCache = vi.fn(); +const getServices = vi.fn(); + +vi.mock("@/lib/services", () => ({ + invalidatePortfolioServicesCache, + getServices, +})); + +function makeRequest(method: string, url: string, body?: Record): NextRequest { + return new NextRequest(url, { + method, + body: body ? JSON.stringify(body) : undefined, + headers: body ? { "Content-Type": "application/json" } : undefined, + }); +} + +const recorded = vi.mocked(recordActivityEvent); + +describe("Activity events — project mutation routes", () => { + let oldGlobalConfig: string | undefined; + let oldConfigPath: string | undefined; + let oldHome: string | undefined; + let tempRoot: string; + let configPath: string; + + beforeEach(() => { + vi.resetModules(); + recorded.mockClear(); + invalidatePortfolioServicesCache.mockReset(); + getServices.mockReset(); + getServices.mockResolvedValue({ + registry: { get: vi.fn().mockReturnValue(null) }, + sessionManager: { + list: vi.fn().mockResolvedValue([]), + kill: vi.fn().mockResolvedValue(undefined), + }, + }); + oldGlobalConfig = process.env["AO_GLOBAL_CONFIG"]; + oldConfigPath = process.env["AO_CONFIG_PATH"]; + oldHome = process.env["HOME"]; + tempRoot = mkdtempSync(path.join(tmpdir(), "ao-activity-projects-")); + configPath = path.join(tempRoot, "config.yaml"); + process.env["AO_GLOBAL_CONFIG"] = configPath; + process.env["AO_CONFIG_PATH"] = configPath; + process.env["HOME"] = tempRoot; + }); + + afterEach(() => { + if (oldGlobalConfig === undefined) delete process.env["AO_GLOBAL_CONFIG"]; + else process.env["AO_GLOBAL_CONFIG"] = oldGlobalConfig; + if (oldConfigPath === undefined) delete process.env["AO_CONFIG_PATH"]; + else process.env["AO_CONFIG_PATH"] = oldConfigPath; + if (oldHome === undefined) delete process.env["HOME"]; + else process.env["HOME"] = oldHome; + rmSync(tempRoot, { recursive: true, force: true }); + }); + + it("POST /api/projects emits api.project_added on success", async () => { + const repoDir = path.join(tempRoot, "demo-add"); + mkdirSync(repoDir, { recursive: true }); + execSync("git init -q", { cwd: repoDir }); + + const { POST } = await import("@/app/api/projects/route"); + const res = await POST( + makeRequest("POST", "http://localhost:3000/api/projects", { + path: repoDir, + projectId: "demo-add", + name: "Demo Add", + }), + ); + + expect(res.status).toBe(201); + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.project_added", + }), + ); + }); + + it("PATCH /api/projects/:id emits api.project_updated with changed keys (not values)", async () => { + const repoDir = path.join(tempRoot, "demo-patch"); + mkdirSync(repoDir, { recursive: true }); + const effectiveId = registerProjectInGlobalConfig("demo-patch", "Demo Patch", repoDir); + + const { PATCH } = await import("@/app/api/projects/[id]/route"); + const res = await PATCH( + makeRequest("PATCH", `http://localhost:3000/api/projects/${effectiveId}`, { + agent: "codex", + runtime: "tmux", + someUnknownField: "do-not-record", + }), + { params: Promise.resolve({ id: effectiveId }) }, + ); + + expect(res.status).toBe(200); + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.project_updated", + projectId: effectiveId, + data: expect.objectContaining({ + changedKeys: expect.arrayContaining(["agent", "runtime"]), + }), + }), + ); + + // Ensure no value content (e.g. "codex", "tmux") leaked into the event payload + const calls = recorded.mock.calls.filter( + (c) => (c[0] as { kind: string }).kind === "api.project_updated", + ); + expect(calls[0]?.[0]).toEqual( + expect.objectContaining({ + data: expect.objectContaining({ + changedKeys: ["agent", "runtime"], + }), + }), + ); + for (const [event] of calls) { + const json = JSON.stringify(event); + expect(json).not.toContain("codex"); + expect(json).not.toContain('"tmux"'); + expect(json).not.toContain("someUnknownField"); + expect(json).not.toContain("do-not-record"); + } + }); + + it("DELETE /api/projects/:id emits api.project_removed on success", async () => { + const repoDir = path.join(tempRoot, "demo-delete"); + mkdirSync(repoDir, { recursive: true }); + const effectiveId = registerProjectInGlobalConfig("demo-delete", "Demo Delete", repoDir); + + const { DELETE } = await import("@/app/api/projects/[id]/route"); + const res = await DELETE( + makeRequest("DELETE", `http://localhost:3000/api/projects/${effectiveId}`), + { params: Promise.resolve({ id: effectiveId }) }, + ); + + expect(res.status).toBe(200); + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.project_removed", + projectId: effectiveId, + }), + ); + }); +}); diff --git a/packages/web/src/__tests__/activity-events-routes.test.ts b/packages/web/src/__tests__/activity-events-routes.test.ts new file mode 100644 index 000000000..fbd2b474c --- /dev/null +++ b/packages/web/src/__tests__/activity-events-routes.test.ts @@ -0,0 +1,498 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { NextRequest } from "next/server"; +import { + SessionNotFoundError, + SessionNotRestorableError, + WorkspaceMissingError, + recordActivityEvent, + createInitialCanonicalLifecycle, + createActivitySignal, + type Session, + type SessionManager, + type OrchestratorConfig, + type PluginRegistry, + type SCM, +} from "@aoagents/ao-core"; + +// Partial mock so we replace recordActivityEvent but keep types/helpers +vi.mock("@aoagents/ao-core", async () => { + const actual = await vi.importActual("@aoagents/ao-core"); + return { + ...(actual as Record), + recordActivityEvent: vi.fn(), + }; +}); + +vi.mock("@/lib/observability", async () => { + const actual = await vi.importActual("@/lib/observability"); + return { + ...(actual as Record), + recordApiObservation: vi.fn(), + }; +}); + +function makeSession(overrides: Partial & { id: string }): Session { + const lifecycle = createInitialCanonicalLifecycle("worker", new Date()); + lifecycle.session.state = "working"; + lifecycle.session.reason = "task_in_progress"; + lifecycle.session.startedAt = lifecycle.session.lastTransitionAt; + lifecycle.runtime.state = "alive"; + lifecycle.runtime.reason = "process_running"; + return { + projectId: "my-app", + status: "working", + activity: "active", + activitySignal: createActivitySignal("valid", { + activity: "active", + timestamp: new Date(), + source: "native", + }), + lifecycle, + branch: null, + issueId: null, + pr: null, + workspacePath: null, + runtimeHandle: null, + agentInfo: null, + createdAt: new Date(), + lastActivityAt: new Date(), + metadata: {}, + ...overrides, + }; +} + +const baseSessions: Session[] = [ + makeSession({ id: "backend-3" }), + makeSession({ + id: "backend-7", + pr: { + number: 432, + url: "https://github.com/acme/my-app/pull/432", + title: "feat: health check", + owner: "acme", + repo: "my-app", + branch: "feat/health-check", + baseBranch: "main", + isDraft: false, + }, + }), + makeSession({ id: "frontend-1", status: "killed", activity: "exited" }), +]; + +const mockSessionManager: SessionManager = { + list: vi.fn(async () => baseSessions), + listCached: vi.fn(async () => baseSessions), + invalidateCache: vi.fn(), + get: vi.fn(async (id: string) => baseSessions.find((s) => s.id === id) ?? null), + spawn: vi.fn(async (cfg) => + makeSession({ + id: `session-${Date.now()}`, + projectId: cfg.projectId, + issueId: cfg.issueId ?? null, + status: "spawning", + }), + ), + kill: vi.fn(async (id: string) => { + if (!baseSessions.find((s) => s.id === id)) { + throw new SessionNotFoundError(id); + } + }), + send: vi.fn(async (id: string) => { + if (!baseSessions.find((s) => s.id === id)) { + throw new SessionNotFoundError(id); + } + }), + cleanup: vi.fn(async () => ({ killed: [], skipped: [], errors: [] })), + spawnOrchestrator: vi.fn(async () => + makeSession({ + id: "my-app-orchestrator", + projectId: "my-app", + metadata: { role: "orchestrator" }, + }), + ), + relaunchOrchestrator: vi.fn(async () => + makeSession({ + id: "my-app-orchestrator", + projectId: "my-app", + metadata: { role: "orchestrator" }, + }), + ), + ensureOrchestrator: vi.fn(), + remap: vi.fn(async () => "ses_mock"), + restore: vi.fn(async (id: string) => { + const session = baseSessions.find((s) => s.id === id); + if (!session) throw new SessionNotFoundError(id); + return { ...session, status: "spawning" as const, activity: "active" as const }; + }), +}; + +const mockSCM: SCM = { + name: "github", + detectPR: vi.fn(async () => null), + getPRState: vi.fn(async () => "open" as const), + mergePR: vi.fn(async () => {}), + closePR: vi.fn(async () => {}), + getCIChecks: vi.fn(async () => []), + getCISummary: vi.fn(async () => "passing" as const), + getReviews: vi.fn(async () => []), + getReviewDecision: vi.fn(async () => "approved" as const), + getPendingComments: vi.fn(async () => []), + getAutomatedComments: vi.fn(async () => []), + getMergeability: vi.fn(async () => ({ + mergeable: true, + ciPassing: true, + approved: true, + noConflicts: true, + blockers: [], + })), +}; + +const mockRegistry: PluginRegistry = { + register: vi.fn(), + get: vi.fn(() => mockSCM) as PluginRegistry["get"], + list: vi.fn(() => []), + loadBuiltins: vi.fn(async () => {}), + loadFromConfig: vi.fn(async () => {}), +}; + +const mockConfig: OrchestratorConfig = { + configPath: "/tmp/ao-test/agent-orchestrator.yaml", + port: 3000, + readyThresholdMs: 300_000, + defaults: { runtime: "tmux", agent: "claude-code", workspace: "worktree", notifiers: [] }, + projects: { + "my-app": { + name: "My App", + repo: "acme/my-app", + path: "/tmp/my-app", + defaultBranch: "main", + sessionPrefix: "my-app", + scm: { plugin: "github" }, + }, + }, + notifiers: {}, + notificationRouting: { urgent: [], action: [], warning: [], info: [] }, + reactions: {}, +}; + +vi.mock("@/lib/services", () => ({ + getServices: vi.fn(async () => ({ + config: mockConfig, + registry: mockRegistry, + sessionManager: mockSessionManager, + })), + getVerifyIssues: vi.fn(async () => []), + getSCM: vi.fn(() => mockSCM), + invalidatePortfolioServicesCache: vi.fn(), +})); + +import { getServices } from "@/lib/services"; +import { recordApiObservation } from "@/lib/observability"; +import { POST as spawnPOST } from "@/app/api/spawn/route"; +import { POST as killPOST } from "@/app/api/sessions/[id]/kill/route"; +import { POST as sendPOST } from "@/app/api/sessions/[id]/send/route"; +import { POST as messagePOST } from "@/app/api/sessions/[id]/message/route"; +import { POST as restorePOST } from "@/app/api/sessions/[id]/restore/route"; +import { POST as orchestratorsPOST } from "@/app/api/orchestrators/route"; +import { POST as mergePOST } from "@/app/api/prs/[id]/merge/route"; + +function makeRequest(url: string, init?: RequestInit): NextRequest { + return new NextRequest( + new URL(url, "http://localhost:3000"), + init as ConstructorParameters[1], + ); +} + +const recorded = vi.mocked(recordActivityEvent); + +beforeEach(() => { + recorded.mockClear(); + vi.mocked(recordApiObservation).mockClear(); + vi.mocked(getServices).mockClear(); +}); + +describe("API mutation routes emit activity events (api source)", () => { + describe("MUST emits — session mutations", () => { + it("POST /api/spawn emits api.session_spawn_requested on success", async () => { + const req = makeRequest("/api/spawn", { + method: "POST", + body: JSON.stringify({ projectId: "my-app", issueId: "INT-100" }), + headers: { "Content-Type": "application/json" }, + }); + await spawnPOST(req); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_spawn_requested", + projectId: "my-app", + }), + ); + }); + + it("POST /api/sessions/:id/kill emits api.session_kill_requested on success", async () => { + const req = makeRequest("/api/sessions/backend-3/kill", { method: "POST" }); + await killPOST(req, { params: Promise.resolve({ id: "backend-3" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_kill_requested", + sessionId: "backend-3", + }), + ); + }); + + it("POST /api/sessions/:id/send emits api.session_message_sent with messageLength", async () => { + const req = makeRequest("/api/sessions/backend-3/send", { + method: "POST", + body: JSON.stringify({ message: "Fix the tests" }), + headers: { "Content-Type": "application/json" }, + }); + await sendPOST(req, { params: Promise.resolve({ id: "backend-3" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_message_sent", + sessionId: "backend-3", + data: expect.objectContaining({ messageLength: "Fix the tests".length }), + }), + ); + }); + + it("POST /api/sessions/:id/send does NOT include the raw message in data", async () => { + const secret = "very-secret-PII content"; + const req = makeRequest("/api/sessions/backend-3/send", { + method: "POST", + body: JSON.stringify({ message: secret }), + headers: { "Content-Type": "application/json" }, + }); + await sendPOST(req, { params: Promise.resolve({ id: "backend-3" }) }); + + const calls = recorded.mock.calls.filter( + (c) => (c[0] as { kind: string }).kind === "api.session_message_sent", + ); + expect(calls.length).toBeGreaterThan(0); + for (const [event] of calls) { + const json = JSON.stringify(event); + expect(json).not.toContain(secret); + } + }); + + it("POST /api/sessions/:id/message emits api.session_message_sent with messageLength", async () => { + const req = makeRequest("/api/sessions/backend-3/message", { + method: "POST", + body: JSON.stringify({ message: "Hi" }), + headers: { "Content-Type": "application/json" }, + }); + await messagePOST(req, { params: Promise.resolve({ id: "backend-3" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_message_sent", + sessionId: "backend-3", + data: expect.objectContaining({ messageLength: 2 }), + }), + ); + }); + + it("POST /api/sessions/:id/restore emits api.session_restore_requested on success", async () => { + const req = makeRequest("/api/sessions/frontend-1/restore", { method: "POST" }); + await restorePOST(req, { params: Promise.resolve({ id: "frontend-1" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_restore_requested", + sessionId: "frontend-1", + }), + ); + }); + }); + + describe("MUST emits — orchestrator + PR mutations", () => { + it("POST /api/orchestrators emits api.orchestrator_spawn_requested on success", async () => { + const req = makeRequest("/api/orchestrators", { + method: "POST", + body: JSON.stringify({ projectId: "my-app" }), + headers: { "Content-Type": "application/json" }, + }); + await orchestratorsPOST(req); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.orchestrator_spawn_requested", + projectId: "my-app", + }), + ); + }); + + it("POST /api/prs/:id/merge emits api.pr_merge_requested on success", async () => { + const req = makeRequest("/api/prs/432/merge", { method: "POST" }); + await mergePOST(req, { params: Promise.resolve({ id: "432" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.pr_merge_requested", + data: expect.objectContaining({ prNumber: 432 }), + }), + ); + }); + }); + + describe("SHOULD emits — failure paths", () => { + it("POST /api/spawn emits api.session_spawn_rejected for unknown project", async () => { + const req = makeRequest("/api/spawn", { + method: "POST", + body: JSON.stringify({ projectId: "unknown-app" }), + headers: { "Content-Type": "application/json" }, + }); + await spawnPOST(req); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_spawn_rejected", + projectId: "unknown-app", + }), + ); + }); + + it("POST /api/spawn does not emit api.session_spawn_failed when core spawn throws", async () => { + (mockSessionManager.spawn as ReturnType).mockRejectedValueOnce( + new Error("runtime failed"), + ); + const req = makeRequest("/api/spawn", { + method: "POST", + body: JSON.stringify({ projectId: "my-app", issueId: "INT-101" }), + headers: { "Content-Type": "application/json" }, + }); + const res = await spawnPOST(req); + + expect(res.status).toBe(500); + expect( + recorded.mock.calls.some( + ([event]) => (event as { kind: string }).kind === "api.session_spawn_failed", + ), + ).toBe(false); + }); + + it.each([ + ["spawn", false, mockSessionManager.spawnOrchestrator], + ["clean relaunch", true, mockSessionManager.relaunchOrchestrator], + ])( + "POST /api/orchestrators does not emit api.orchestrator_spawn_failed when core %s throws", + async (_name, clean, method) => { + (method as ReturnType).mockRejectedValueOnce(new Error("runtime failed")); + const req = makeRequest("/api/orchestrators", { + method: "POST", + body: JSON.stringify({ projectId: "my-app", clean }), + headers: { "Content-Type": "application/json" }, + }); + const res = await orchestratorsPOST(req); + + expect(res.status).toBe(500); + expect( + recorded.mock.calls.some( + ([event]) => (event as { kind: string }).kind === "api.orchestrator_spawn_failed", + ), + ).toBe(false); + }, + ); + + it("POST /api/sessions/:id/send emits api.session_message_failed on unexpected error", async () => { + (mockSessionManager.send as ReturnType).mockRejectedValueOnce( + new Error("write failed"), + ); + const req = makeRequest("/api/sessions/backend-3/send", { + method: "POST", + body: JSON.stringify({ message: "hi" }), + headers: { "Content-Type": "application/json" }, + }); + await sendPOST(req, { params: Promise.resolve({ id: "backend-3" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_message_failed", + sessionId: "backend-3", + data: expect.objectContaining({ messageLength: 2 }), + }), + ); + }); + + it.each([ + ["non-restorable session", new SessionNotRestorableError("my-app-123", "still working"), 409], + ["missing workspace", new WorkspaceMissingError("/tmp/missing-workspace"), 422], + ["unexpected restore error", new Error("restore failed"), 500], + ])( + "POST /api/sessions/:id/restore emits attributed api.session_restore_failed for %s", + async (_name, error, statusCode) => { + (mockSessionManager.restore as ReturnType).mockRejectedValueOnce(error); + const req = makeRequest("/api/sessions/my-app-123/restore", { method: "POST" }); + const res = await restorePOST(req, { params: Promise.resolve({ id: "my-app-123" }) }); + + expect(res.status).toBe(statusCode); + expect(vi.mocked(getServices)).toHaveBeenCalledTimes(1); + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.session_restore_failed", + projectId: "my-app", + sessionId: "my-app-123", + data: expect.objectContaining({ statusCode }), + }), + ); + }, + ); + + it("POST /api/prs/:id/merge emits api.pr_merge_rejected for non-mergeable PR", async () => { + (mockSCM.getMergeability as ReturnType).mockResolvedValueOnce({ + mergeable: false, + ciPassing: false, + approved: false, + noConflicts: true, + blockers: ["CI checks failing"], + }); + const req = makeRequest("/api/prs/432/merge", { method: "POST" }); + await mergePOST(req, { params: Promise.resolve({ id: "432" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.pr_merge_rejected", + data: expect.objectContaining({ prNumber: 432 }), + }), + ); + }); + + it("POST /api/prs/:id/merge emits api.pr_merge_failed when mergePR throws", async () => { + (mockSCM.mergePR as ReturnType).mockRejectedValueOnce(new Error("github 500")); + const req = makeRequest("/api/prs/432/merge", { method: "POST" }); + await mergePOST(req, { params: Promise.resolve({ id: "432" }) }); + + expect(recorded).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.pr_merge_failed", + projectId: "my-app", + sessionId: "backend-7", + data: expect.objectContaining({ prNumber: 432, reason: "github 500" }), + }), + ); + expect(recordApiObservation).toHaveBeenCalledWith( + expect.objectContaining({ + outcome: "failure", + statusCode: 500, + projectId: "my-app", + sessionId: "backend-7", + data: expect.objectContaining({ prNumber: 432 }), + }), + ); + }); + }); +}); diff --git a/packages/web/src/__tests__/api-routes.test.ts b/packages/web/src/__tests__/api-routes.test.ts index 84f769e97..0a32bf0dc 100644 --- a/packages/web/src/__tests__/api-routes.test.ts +++ b/packages/web/src/__tests__/api-routes.test.ts @@ -129,6 +129,7 @@ const mockSessionManager: SessionManager = { cleanup: vi.fn(async () => ({ killed: [], skipped: [], errors: [] })), spawnOrchestrator: vi.fn(), ensureOrchestrator: vi.fn(), + relaunchOrchestrator: vi.fn(), remap: vi.fn(async () => "ses_mock"), restore: vi.fn(async (id: string) => { const session = testSessions.find((s) => s.id === id); @@ -232,7 +233,7 @@ import { GET as sessionDetailGET, PATCH as sessionDetailPATCH, } from "@/app/api/sessions/[id]/route"; -import { POST as orchestratorsPOST, GET as orchestratorsGET } from "@/app/api/orchestrators/route"; +import { POST as orchestratorsPOST } from "@/app/api/orchestrators/route"; import { POST as spawnPOST } from "@/app/api/spawn/route"; import { POST as sendPOST } from "@/app/api/sessions/[id]/send/route"; import { POST as messagePOST } from "@/app/api/sessions/[id]/message/route"; @@ -1173,53 +1174,49 @@ describe("API Routes", () => { expect(data.recovery).toBe("reuse-or-recreate-workspace"); expect(data.error).toContain('AO found an older orchestrator workspace for "my-app"'); }); - }); - describe("GET /api/orchestrators", () => { - it("returns orchestrators for a project", async () => { - const orchestrator = makeSession({ - id: "my-app-orchestrator", - projectId: "my-app", - metadata: { role: "orchestrator" }, + it("calls relaunchOrchestrator instead of spawnOrchestrator when clean is true", async () => { + (mockSessionManager.relaunchOrchestrator as ReturnType).mockResolvedValueOnce( + makeSession({ + id: "my-app-orchestrator", + projectId: "my-app", + metadata: { role: "orchestrator" }, + }), + ); + + const req = makeRequest("/api/orchestrators", { + method: "POST", + body: JSON.stringify({ projectId: "my-app", clean: true }), + headers: { "Content-Type": "application/json" }, }); - (mockSessionManager.list as ReturnType).mockResolvedValueOnce([orchestrator]); + const res = await orchestratorsPOST(req); - const res = await orchestratorsGET( - makeRequest("http://localhost:3000/api/orchestrators?project=my-app"), - ); - expect(res.status).toBe(200); - const data = await res.json(); - expect(data.orchestrators).toHaveLength(1); - expect(data.orchestrators[0].id).toBe("my-app-orchestrator"); - expect(data.projectName).toBe("My App"); + expect(res.status).toBe(201); + expect(mockSessionManager.relaunchOrchestrator).toHaveBeenCalledWith({ + projectId: "my-app", + systemPrompt: expect.stringContaining("# My App Orchestrator"), + }); + expect(mockSessionManager.spawnOrchestrator).not.toHaveBeenCalled(); }); - it("returns 400 when project parameter is missing", async () => { - const res = await orchestratorsGET(makeRequest("http://localhost:3000/api/orchestrators")); - expect(res.status).toBe(400); - const data = await res.json(); - expect(data.error).toMatch(/Missing project query parameter/); - }); + it("uses spawnOrchestrator when clean is false or omitted", async () => { + (mockSessionManager.spawnOrchestrator as ReturnType).mockResolvedValueOnce( + makeSession({ + id: "my-app-orchestrator", + projectId: "my-app", + metadata: { role: "orchestrator" }, + }), + ); - it("returns 404 for unknown project", async () => { - const res = await orchestratorsGET( - makeRequest("http://localhost:3000/api/orchestrators?project=unknown-app"), - ); - expect(res.status).toBe(404); - const data = await res.json(); - expect(data.error).toMatch(/Unknown project/); - }); + const req = makeRequest("/api/orchestrators", { + method: "POST", + body: JSON.stringify({ projectId: "my-app", clean: false }), + headers: { "Content-Type": "application/json" }, + }); + await orchestratorsPOST(req); - it("returns 500 when list fails", async () => { - (mockSessionManager.list as ReturnType).mockRejectedValueOnce( - new Error("boom"), - ); - const res = await orchestratorsGET( - makeRequest("http://localhost:3000/api/orchestrators?project=my-app"), - ); - expect(res.status).toBe(500); - const data = await res.json(); - expect(data.error).toBe("boom"); + expect(mockSessionManager.spawnOrchestrator).toHaveBeenCalled(); + expect(mockSessionManager.relaunchOrchestrator).not.toHaveBeenCalled(); }); }); diff --git a/packages/web/src/__tests__/orchestrators.test.tsx b/packages/web/src/__tests__/orchestrators.test.tsx deleted file mode 100644 index 8aaba6d00..000000000 --- a/packages/web/src/__tests__/orchestrators.test.tsx +++ /dev/null @@ -1,103 +0,0 @@ -import { describe, it, expect, vi, beforeEach } from "vitest"; -import { render, screen } from "@testing-library/react"; -import OrchestratorsRoute from "@/app/orchestrators/page"; -import { getServices } from "@/lib/services"; -import { getAllProjects } from "@/lib/project-name"; - -// ── Mocks ───────────────────────────────────────────────────────────── - -vi.mock("next/navigation", () => ({ - useRouter: () => ({ - push: vi.fn(), - }), -})); - -vi.mock("next/link", () => ({ - default: ({ children, href }: { children: React.ReactNode; href: string }) => ( - {children} - ), -})); - -vi.mock("@/lib/services", () => ({ - getServices: vi.fn(), -})); - -vi.mock("@/lib/project-name", () => ({ - getAllProjects: vi.fn(), -})); - -global.fetch = vi.fn(); - -// ── Tests ───────────────────────────────────────────────────────────── - -describe("Orchestrators Page (OrchestratorsRoute)", () => { - beforeEach(() => { - vi.clearAllMocks(); - }); - - it("renders the page with searchParams and listed orchestrators", async () => { - const mockSessionManager = { - list: vi.fn().mockResolvedValue([ - { - id: "app-orchestrator", - projectId: "my-app", - status: "working", - activity: "active", - metadata: { role: "orchestrator" }, - createdAt: new Date(), - lastActivityAt: new Date(), - }, - ]), - }; - - (getServices as any).mockResolvedValue({ - config: { - projects: { - "my-app": { name: "My App", sessionPrefix: "app" }, - }, - }, - sessionManager: mockSessionManager, - }); - - (getAllProjects as any).mockReturnValue([{ id: "my-app", name: "My App" }]); - - const searchParams = Promise.resolve({ project: "my-app" }); - const jsx = await OrchestratorsRoute({ searchParams }); - render(jsx); - - expect(screen.getByText("My App")).toBeInTheDocument(); - expect(screen.getByText("app-orchestrator")).toBeInTheDocument(); - }); - - it("shows error when project is missing in searchParams", async () => { - const searchParams = Promise.resolve({}); - const jsx = await OrchestratorsRoute({ searchParams }); - render(jsx); - - expect(screen.getByText("Missing Project")).toBeInTheDocument(); - }); - - it("shows error when project is not found in config", async () => { - (getServices as any).mockResolvedValue({ - config: { projects: {} }, - sessionManager: { list: vi.fn() }, - }); - (getAllProjects as any).mockReturnValue([]); - - const searchParams = Promise.resolve({ project: "ghost" }); - const jsx = await OrchestratorsRoute({ searchParams }); - render(jsx); - - expect(screen.getByText('Project "ghost" not found')).toBeInTheDocument(); - }); - - it("handles service errors gracefully", async () => { - (getServices as any).mockRejectedValue(new Error("Database down")); - - const searchParams = Promise.resolve({ project: "my-app" }); - const jsx = await OrchestratorsRoute({ searchParams }); - render(jsx); - - expect(screen.getByText("Database down")).toBeInTheDocument(); - }); -}); diff --git a/packages/web/src/__tests__/setup.ts b/packages/web/src/__tests__/setup.ts index b3258b10f..1ccc65a90 100644 --- a/packages/web/src/__tests__/setup.ts +++ b/packages/web/src/__tests__/setup.ts @@ -2,6 +2,22 @@ import { cleanup } from "@testing-library/react"; import * as matchers from "@testing-library/jest-dom/matchers"; import { afterEach, expect } from "vitest"; +// Node.js 25 exposes a native localStorage stub via --localstorage-file that +// lacks .clear()/.key()/.length. Replace it with a complete in-memory mock so +// tests that call window.localStorage.clear() work on any Node version. +const localStorageMock = (() => { + let store: Record = {}; + return { + getItem: (k: string) => store[k] ?? null, + setItem: (k: string, v: string) => { store[k] = String(v); }, + removeItem: (k: string) => { Reflect.deleteProperty(store, k); }, + clear: () => { store = {}; }, + get length() { return Object.keys(store).length; }, + key: (i: number) => Object.keys(store)[i] ?? null, + }; +})(); +Object.defineProperty(window, "localStorage", { value: localStorageMock, writable: true, configurable: true }); + expect.extend(matchers); afterEach(() => { cleanup(); diff --git a/packages/web/src/__tests__/webhook-route.test.ts b/packages/web/src/__tests__/webhook-route.test.ts new file mode 100644 index 000000000..0bea074e5 --- /dev/null +++ b/packages/web/src/__tests__/webhook-route.test.ts @@ -0,0 +1,328 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { + createInitialCanonicalLifecycle, + createActivitySignal, + type Session, + type SessionManager, + type OrchestratorConfig, + type PluginRegistry, + type SCM, + type LifecycleManager, +} from "@aoagents/ao-core"; + +// Activity event recording is mocked so we can assert what fires without +// touching the real SQLite layer. +const recordActivityEvent = vi.fn(); +vi.mock("@aoagents/ao-core", async (importOriginal) => { + const actual = (await importOriginal()) as Record; + return { + ...actual, + recordActivityEvent: (event: unknown) => recordActivityEvent(event), + }; +}); + +// ── Mock services + plugin registry ─────────────────────────────────── + +function makeSession(id: string, overrides: Partial = {}): Session { + const lifecycle = createInitialCanonicalLifecycle("worker", new Date()); + lifecycle.session.state = "working"; + lifecycle.session.reason = "task_in_progress"; + lifecycle.session.startedAt = lifecycle.session.lastTransitionAt; + lifecycle.runtime.state = "alive"; + lifecycle.runtime.reason = "process_running"; + return { + id, + projectId: "my-app", + status: "working", + activity: "active", + activitySignal: createActivitySignal("valid", { + activity: "active", + timestamp: new Date(), + source: "native", + }), + lifecycle, + branch: "feat/x", + issueId: null, + pr: { + number: 42, + url: "u", + title: "t", + owner: "acme", + repo: "my-app", + branch: "feat/x", + baseBranch: "main", + isDraft: false, + }, + workspacePath: null, + runtimeHandle: null, + agentInfo: null, + createdAt: new Date(), + lastActivityAt: new Date(), + metadata: {}, + ...overrides, + }; +} + +const verifyWebhook = vi.fn(); +const parseWebhook = vi.fn(); +const mockSCM: SCM = { + name: "github", + detectPR: vi.fn(), + getPRState: vi.fn(), + mergePR: vi.fn(), + closePR: vi.fn(), + getCIChecks: vi.fn(), + getCISummary: vi.fn(), + getReviews: vi.fn(), + getReviewDecision: vi.fn(), + getPendingComments: vi.fn(), + getAutomatedComments: vi.fn(), + getMergeability: vi.fn(), + verifyWebhook, + parseWebhook, +} as unknown as SCM; + +const mockRegistry: PluginRegistry = { + register: vi.fn(), + get: vi.fn(() => mockSCM) as PluginRegistry["get"], + list: vi.fn(() => []), + loadBuiltins: vi.fn(), + loadFromConfig: vi.fn(), +}; + +const mockConfig: OrchestratorConfig = { + configPath: "/tmp/agent-orchestrator.yaml", + port: 3000, + readyThresholdMs: 300_000, + defaults: { runtime: "tmux", agent: "claude-code", workspace: "worktree", notifiers: [] }, + projects: { + "my-app": { + name: "My App", + repo: "acme/my-app", + path: "/tmp/my-app", + defaultBranch: "main", + sessionPrefix: "my-app", + scm: { + plugin: "github", + webhook: { enabled: true, path: "/api/webhooks/github", maxBodyBytes: 1024 }, + }, + }, + }, + notifiers: {}, + notificationRouting: { urgent: [], action: [], warning: [], info: [] }, + reactions: {}, +}; + +const mockSessionManager = { + list: vi.fn(async () => [makeSession("s1")]), +} as unknown as SessionManager; + +const mockLifecycle = { + check: vi.fn(async () => {}), +} as unknown as LifecycleManager; + +vi.mock("@/lib/services", () => ({ + getServices: vi.fn(async () => ({ + config: mockConfig, + registry: mockRegistry, + sessionManager: mockSessionManager, + lifecycleManager: mockLifecycle, + })), +})); + +import { POST as webhookPOST } from "@/app/api/webhooks/[...slug]/route"; + +function makeWebhookRequest(opts?: { + body?: string; + contentLength?: number; + headers?: Record; +}): Request { + const body = opts?.body ?? JSON.stringify({ action: "synchronize", number: 42 }); + const headers: Record = { + "content-type": "application/json", + ...opts?.headers, + }; + if (opts?.contentLength !== undefined) { + headers["content-length"] = String(opts.contentLength); + } + return new Request("http://localhost:3000/api/webhooks/github", { + method: "POST", + headers, + body, + }); +} + +beforeEach(() => { + vi.clearAllMocks(); + recordActivityEvent.mockClear(); + verifyWebhook.mockResolvedValue({ ok: true }); + parseWebhook.mockResolvedValue({ + provider: "github", + kind: "pull_request", + action: "synchronize", + rawEventType: "pull_request", + repository: { owner: "acme", name: "my-app" }, + prNumber: 42, + branch: "feat/x", + data: {}, + }); +}); + +// ── Tests ───────────────────────────────────────────────────────────── + +describe("POST /api/webhooks/[...slug] — activity events", () => { + it("rejects unverified webhook with 401 and emits api.webhook_unverified", async () => { + verifyWebhook.mockResolvedValueOnce({ ok: false, reason: "bad signature" }); + const req = makeWebhookRequest({ + headers: { "x-hub-signature-256": "sha256=bogus", "x-forwarded-for": "203.0.113.7" }, + }); + + const res = await webhookPOST(req); + expect(res.status).toBe(401); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.webhook_unverified", + level: "warn", + }), + ); + const call = recordActivityEvent.mock.calls[0]![0] as { + summary: string; + data: Record; + }; + // Critical: signature value must NOT be in data (or anywhere) + expect(JSON.stringify(call)).not.toContain("bogus"); + expect(call.data["slug"]).toBe("github"); + expect(call.data["remoteAddr"]).toBe("203.0.113.7"); + expect(call.data["verificationSupported"]).toBe(true); + expect(call.data["reason"]).toBe("bad signature"); + }); + + it("keeps unsupported webhook verification as 404 while recording audit context", async () => { + vi.mocked(mockRegistry.get).mockReturnValueOnce({ + ...mockSCM, + verifyWebhook: undefined, + } as unknown as SCM); + + const res = await webhookPOST(makeWebhookRequest()); + expect(res.status).toBe(404); + + expect(verifyWebhook).not.toHaveBeenCalled(); + expect(parseWebhook).not.toHaveBeenCalled(); + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.webhook_unverified", + level: "warn", + summary: expect.stringContaining("verification unsupported"), + }), + ); + const call = recordActivityEvent.mock.calls[0]![0] as { + data: Record; + }; + expect(call.data["slug"]).toBe("github"); + expect(call.data["verificationSupported"]).toBe(false); + expect(call.data["unsupportedVerificationCount"]).toBe(1); + expect(call.data["reason"]).toBe("verification_unsupported"); + }); + + it("emits api.webhook_rejected when content-length exceeds maxBodyBytes (413)", async () => { + const req = makeWebhookRequest({ contentLength: 2048 }); // > 1024 max + + const res = await webhookPOST(req); + expect(res.status).toBe(413); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.webhook_rejected", + level: "warn", + }), + ); + const call = recordActivityEvent.mock.calls[0]![0] as { data: Record }; + expect(call.data["slug"]).toBe("github"); + expect(call.data["contentLength"]).toBe(2048); + expect(call.data["maxBodyBytes"]).toBe(1024); + // No body content captured + expect(JSON.stringify(call)).not.toContain("synchronize"); + }); + + it("emits api.webhook_received with counts (not body) on 202 success", async () => { + const req = makeWebhookRequest({ + headers: { "x-forwarded-for": "192.0.2.1" }, + }); + + const res = await webhookPOST(req); + expect(res.status).toBe(202); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.webhook_received", + }), + ); + const call = recordActivityEvent.mock.calls[0]![0] as { data: Record }; + expect(call.data["slug"]).toBe("github"); + expect(call.data["remoteAddr"]).toBe("192.0.2.1"); + expect(call.data["matchedSessions"]).toBe(1); + expect(call.data["parseErrorCount"]).toBe(0); + expect(call.data["lifecycleErrorCount"]).toBe(0); + expect(call.data["projectIds"]).toEqual(["my-app"]); + // Critical: payload body is NOT included + expect(JSON.stringify(call)).not.toContain("synchronize"); + }); + + it("emits api.webhook_received with elevated level when parse/lifecycle errors occurred", async () => { + parseWebhook.mockRejectedValueOnce(new Error("boom")); + // Verification passes so we get into the parse path + verifyWebhook.mockResolvedValueOnce({ ok: true }); + + const res = await webhookPOST(makeWebhookRequest()); + expect(res.status).toBe(202); + + const received = recordActivityEvent.mock.calls.find( + ([e]) => (e as { kind: string }).kind === "api.webhook_received", + ); + expect(received).toBeDefined(); + expect((received![0] as { level: string }).level).toBe("warn"); + expect((received![0] as { data: Record }).data["parseErrorCount"]).toBe(1); + }); + + it("emits api.webhook_failed on 500 outer crash", async () => { + // Force the list() call inside POST to throw, hitting the outer catch. + (mockSessionManager.list as ReturnType).mockRejectedValueOnce( + new Error("session manager exploded"), + ); + + const res = await webhookPOST(makeWebhookRequest()); + expect(res.status).toBe(500); + + expect(recordActivityEvent).toHaveBeenCalledWith( + expect.objectContaining({ + source: "api", + kind: "api.webhook_failed", + level: "error", + }), + ); + const call = recordActivityEvent.mock.calls[0]![0] as { data: Record }; + expect(call.data["slug"]).toBe("github"); + expect(call.data["errorMessage"]).toContain("session manager exploded"); + }); + + it("does not emit any event for 404 (unknown path)", async () => { + // Empty config so no candidates match + vi.mocked(mockRegistry.get).mockReturnValueOnce(undefined as unknown as SCM); + const req = new Request("http://localhost:3000/api/webhooks/unknown", { + method: "POST", + headers: { "content-type": "application/json" }, + body: "{}", + }); + + const res = await webhookPOST(req); + expect(res.status).toBe(404); + // No webhook events for 404 — it's a config issue, not an external signal + const kinds = recordActivityEvent.mock.calls.map(([e]) => (e as { kind: string }).kind); + expect(kinds.filter((k) => k.startsWith("api.webhook_"))).toEqual([]); + }); +}); diff --git a/packages/web/src/app/api/issues/route.ts b/packages/web/src/app/api/issues/route.ts index 6063a67f1..07b18847a 100644 --- a/packages/web/src/app/api/issues/route.ts +++ b/packages/web/src/app/api/issues/route.ts @@ -1,7 +1,7 @@ import { type NextRequest, NextResponse } from "next/server"; import { getServices } from "@/lib/services"; import { validateString, validateConfiguredProject } from "@/lib/validation"; -import type { Tracker } from "@aoagents/ao-core"; +import { recordActivityEvent, type Tracker } from "@aoagents/ao-core"; export const dynamic = "force-dynamic"; @@ -95,11 +95,25 @@ export async function POST(request: NextRequest) { project, ); + recordActivityEvent({ + projectId, + source: "api", + kind: "api.issue_created", + summary: `issue created: ${issue.id}`, + data: { issueId: issue.id, addToBacklog: Boolean(body.addToBacklog) }, + }); + return NextResponse.json({ issue: { projectId, ...issue } }, { status: 201 }); } catch (err) { - return NextResponse.json( - { error: err instanceof Error ? err.message : "Failed to create issue" }, - { status: 500 }, - ); + const reason = err instanceof Error ? err.message : "Failed to create issue"; + recordActivityEvent({ + projectId, + source: "api", + kind: "api.issue_create_failed", + level: "error", + summary: `issue create failed: ${reason}`, + data: { reason }, + }); + return NextResponse.json({ error: reason }, { status: 500 }); } } diff --git a/packages/web/src/app/api/orchestrators/route.ts b/packages/web/src/app/api/orchestrators/route.ts index 886761693..40e6c3972 100644 --- a/packages/web/src/app/api/orchestrators/route.ts +++ b/packages/web/src/app/api/orchestrators/route.ts @@ -1,10 +1,12 @@ import { type NextRequest, NextResponse } from "next/server"; -import { generateOrchestratorPrompt, generateSessionPrefix } from "@aoagents/ao-core"; +import { generateOrchestratorPrompt, recordActivityEvent } from "@aoagents/ao-core"; import { getServices } from "@/lib/services"; import { validateIdentifier, validateConfiguredProject } from "@/lib/validation"; -import { mapSessionsToOrchestrators } from "@/lib/orchestrator-utils"; -function classifySpawnError(projectId: string, error: unknown): { +function classifySpawnError( + projectId: string, + error: unknown, +): { status: number; payload: Record; } { @@ -35,46 +37,6 @@ function classifySpawnError(projectId: string, error: unknown): { }; } -/** - * GET /api/orchestrators?project= - * List existing orchestrator sessions for a project. - */ -export async function GET(request: NextRequest) { - const projectId = request.nextUrl.searchParams.get("project"); - - if (!projectId) { - return NextResponse.json({ error: "Missing project query parameter" }, { status: 400 }); - } - - const projectErr = validateIdentifier(projectId, "projectId"); - if (projectErr) { - return NextResponse.json({ error: projectErr }, { status: 400 }); - } - - try { - const { config, sessionManager } = await getServices(); - const configProjectErr = validateConfiguredProject(config.projects, projectId); - if (configProjectErr) { - return NextResponse.json({ error: configProjectErr }, { status: 404 }); - } - const project = config.projects[projectId]; - const sessionPrefix = project.sessionPrefix ?? projectId; - - const allSessions = await sessionManager.list(projectId); - const allSessionPrefixes = Object.entries(config.projects).map( - ([, p]) => p.sessionPrefix ?? generateSessionPrefix(p.name ?? ""), - ); - const orchestrators = mapSessionsToOrchestrators(allSessions, sessionPrefix, project.name, allSessionPrefixes); - - return NextResponse.json({ orchestrators, projectName: project.name }); - } catch (err) { - return NextResponse.json( - { error: err instanceof Error ? err.message : "Failed to list orchestrators" }, - { status: 500 }, - ); - } -} - export async function POST(request: NextRequest) { const body = (await request.json().catch(() => null)) as Record | null; if (!body) { @@ -86,6 +48,8 @@ export async function POST(request: NextRequest) { return NextResponse.json({ error: projectErr }, { status: 400 }); } + const clean = body.clean === true; + try { const { config, sessionManager } = await getServices(); const projectId = body.projectId as string; @@ -96,7 +60,17 @@ export async function POST(request: NextRequest) { const project = config.projects[projectId]; const systemPrompt = generateOrchestratorPrompt({ config, projectId, project }); - const session = await sessionManager.spawnOrchestrator({ projectId, systemPrompt }); + const session = clean + ? await sessionManager.relaunchOrchestrator({ projectId, systemPrompt }) + : await sessionManager.spawnOrchestrator({ projectId, systemPrompt }); + + recordActivityEvent({ + projectId, + sessionId: session.id, + source: "api", + kind: "api.orchestrator_spawn_requested", + summary: `orchestrator spawn requested for ${projectId}`, + }); return NextResponse.json( { diff --git a/packages/web/src/app/api/projects/[id]/route.ts b/packages/web/src/app/api/projects/[id]/route.ts index f940aefae..3ac591fb9 100644 --- a/packages/web/src/app/api/projects/[id]/route.ts +++ b/packages/web/src/app/api/projects/[id]/route.ts @@ -9,6 +9,7 @@ import { loadConfig, loadGlobalConfig, loadLocalProjectConfigDetailed, + recordActivityEvent, repairWrappedLocalProjectConfig, unregisterProject, writeLocalProjectConfig, @@ -23,6 +24,7 @@ import { stopStaleWindowsPtyHosts } from "@/lib/windows-pty-cleanup"; export const dynamic = "force-dynamic"; const IDENTITY_FIELDS = new Set(["projectId", "path", "repo", "defaultBranch"]); +const EDITABLE_CONFIG_FIELDS = new Set(["agent", "runtime", "tracker", "scm", "reactions"]); function sanitizeString(value: unknown): string | undefined { if (typeof value !== "string") return undefined; @@ -31,7 +33,7 @@ function sanitizeString(value: unknown): string | undefined { } function revalidateProjectPaths(projectId: string): void { - for (const route of ["/", "/orchestrators", "/prs", `/projects/${projectId}`]) { + for (const route of ["/", "/prs", `/projects/${projectId}`]) { try { revalidatePath(route); } catch { @@ -112,7 +114,10 @@ function getProjectState(projectId: string) { }; } -function degradedPayload(projectId: string, degradedProject: NonNullable["degradedProject"]>) { +function degradedPayload( + projectId: string, + degradedProject: NonNullable["degradedProject"]>, +) { return { error: degradedProject.resolveError, projectId, @@ -126,10 +131,7 @@ function degradedPayload(projectId: string, degradedProject: NonNullable }, -) { +export async function GET(_request: NextRequest, context: { params: Promise<{ id: string }> }) { try { const { id } = await context.params; const state = getProjectState(id); @@ -172,10 +174,7 @@ export async function GET( } } -export async function PATCH( - request: NextRequest, - context: { params: Promise<{ id: string }> }, -) { +export async function PATCH(request: NextRequest, context: { params: Promise<{ id: string }> }) { try { const { id } = await context.params; const body = (await request.json().catch(() => null)) as Record | null; @@ -200,7 +199,10 @@ export async function PATCH( } const projectPath = state.globalEntry?.path; if (!projectPath) { - return NextResponse.json({ error: `Project "${id}" is missing a registry path.` }, { status: 409 }); + return NextResponse.json( + { error: `Project "${id}" is missing a registry path.` }, + { status: 409 }, + ); } const localConfigResult = loadLocalProjectConfigDetailed(projectPath); @@ -211,7 +213,8 @@ export async function PATCH( return NextResponse.json({ error: localConfigResult.error }, { status: 400 }); } - const currentConfig: LocalProjectConfig = localConfigResult.kind === "loaded" ? { ...localConfigResult.config } : {}; + const currentConfig: LocalProjectConfig = + localConfigResult.kind === "loaded" ? { ...localConfigResult.config } : {}; const nextConfig: LocalProjectConfig = { ...currentConfig, }; @@ -231,8 +234,7 @@ export async function PATCH( ...(body["tracker"] as Record), } as LocalProjectConfig["tracker"]) : undefined; - nextConfig.tracker = - nextTracker; + nextConfig.tracker = nextTracker; } if (hasOwn("scm")) { const nextScm = @@ -242,8 +244,7 @@ export async function PATCH( ...(body["scm"] as Record), } as LocalProjectConfig["scm"]) : undefined; - nextConfig.scm = - nextScm; + nextConfig.scm = nextScm; } if (hasOwn("reactions")) { nextConfig.reactions = @@ -261,6 +262,16 @@ export async function PATCH( invalidatePortfolioServicesCache(); revalidateProjectPaths(id); + // Record only changed *keys*, never values — config can contain tokens. + const changedKeys = Object.keys(body).filter((k) => EDITABLE_CONFIG_FIELDS.has(k)); + recordActivityEvent({ + projectId: id, + source: "api", + kind: "api.project_updated", + summary: `project updated: ${id}`, + data: { changedKeys }, + }); + return NextResponse.json({ ok: true }); } catch (error) { return NextResponse.json( @@ -270,19 +281,14 @@ export async function PATCH( } } -export async function PUT( - request: NextRequest, - context: { params: Promise<{ id: string }> }, -) { +export async function PUT(request: NextRequest, context: { params: Promise<{ id: string }> }) { return PATCH(request, context); } -export async function DELETE( - _request: NextRequest, - context: { params: Promise<{ id: string }> }, -) { +export async function DELETE(_request: NextRequest, context: { params: Promise<{ id: string }> }) { + let id: string | undefined; try { - const { id } = await context.params; + ({ id } = await context.params); const state = getProjectState(id); if (!state.globalEntry && !state.project && !state.degradedProject) { return NextResponse.json({ error: `Unknown project: ${id}` }, { status: 404 }); @@ -299,7 +305,8 @@ export async function DELETE( return NextResponse.json({ error: `Invalid project ID: ${id}` }, { status: 400 }); } - const workspacePluginName = state.project?.workspace ?? state.config.defaults.workspace ?? "worktree"; + const workspacePluginName = + state.project?.workspace ?? state.config.defaults.workspace ?? "worktree"; const { registry, sessionManager } = await getServices(); await stopProjectSessions(id, sessionManager); await stopStaleWindowsPtyHosts(projectDir); @@ -310,23 +317,34 @@ export async function DELETE( invalidatePortfolioServicesCache(); revalidateProjectPaths(id); + recordActivityEvent({ + projectId: id, + source: "api", + kind: "api.project_removed", + summary: `project removed: ${id}`, + data: { removedStorageDir: hadStorageDir }, + }); + return NextResponse.json({ ok: true, projectId: id, removedStorageDir: hadStorageDir, }); } catch (error) { - return NextResponse.json( - { error: error instanceof Error ? error.message : "Failed to delete project" }, - { status: 500 }, - ); + const reason = error instanceof Error ? error.message : "Failed to delete project"; + recordActivityEvent({ + projectId: id, + source: "api", + kind: "api.project_remove_failed", + level: "error", + summary: `project remove failed: ${reason}`, + data: { reason }, + }); + return NextResponse.json({ error: reason }, { status: 500 }); } } -export async function POST( - _request: NextRequest, - context: { params: Promise<{ id: string }> }, -) { +export async function POST(_request: NextRequest, context: { params: Promise<{ id: string }> }) { try { const { id } = await context.params; const state = getProjectState(id); @@ -337,7 +355,9 @@ export async function POST( return NextResponse.json({ error: "Project does not need repair." }, { status: 400 }); } - const isWrappedConfigError = state.degradedProject.resolveError.includes("wrapped projects: format"); + const isWrappedConfigError = state.degradedProject.resolveError.includes( + "wrapped projects: format", + ); if (!isWrappedConfigError) { return NextResponse.json( { error: "Automatic repair is not available for this degraded config." }, @@ -349,6 +369,13 @@ export async function POST( invalidatePortfolioServicesCache(); revalidateProjectPaths(id); + recordActivityEvent({ + projectId: id, + source: "api", + kind: "api.project_repaired", + summary: `project repaired: ${id}`, + }); + return NextResponse.json({ ok: true, repaired: true, projectId: id }); } catch (error) { return NextResponse.json( diff --git a/packages/web/src/app/api/projects/reload/route.ts b/packages/web/src/app/api/projects/reload/route.ts index e1a13f333..ecacc9dde 100644 --- a/packages/web/src/app/api/projects/reload/route.ts +++ b/packages/web/src/app/api/projects/reload/route.ts @@ -1,5 +1,10 @@ import { NextResponse } from "next/server"; -import { ConfigNotFoundError, getGlobalConfigPath, loadConfig } from "@aoagents/ao-core"; +import { + ConfigNotFoundError, + getGlobalConfigPath, + loadConfig, + recordActivityEvent, +} from "@aoagents/ao-core"; import { invalidatePortfolioServicesCache } from "@/lib/services"; export const dynamic = "force-dynamic"; @@ -25,10 +30,19 @@ export async function POST() { invalidatePortfolioServicesCache(); const config = loadReloadConfig(); + const projectCount = Object.keys(config.projects).length; + const degradedCount = Object.keys(config.degradedProjects).length; + recordActivityEvent({ + source: "api", + kind: "api.config_reloaded", + summary: `config reloaded: ${projectCount} projects, ${degradedCount} degraded`, + data: { projectCount, degradedCount }, + }); + return NextResponse.json({ reloaded: true, - projectCount: Object.keys(config.projects).length, - degradedCount: Object.keys(config.degradedProjects).length, + projectCount, + degradedCount, }); } catch (error) { return NextResponse.json( diff --git a/packages/web/src/app/api/projects/route.ts b/packages/web/src/app/api/projects/route.ts index 0947558a6..ca9c1c9ba 100644 --- a/packages/web/src/app/api/projects/route.ts +++ b/packages/web/src/app/api/projects/route.ts @@ -8,6 +8,7 @@ import { getGlobalConfigPath, loadConfig, migrateToGlobalConfig, + recordActivityEvent, registerProjectInGlobalConfig, } from "@aoagents/ao-core"; import { revalidatePath } from "next/cache"; @@ -33,7 +34,7 @@ function isGitRepository(projectPath: string): boolean { } function revalidatePortfolioPaths(projectId: string): void { - for (const route of ["/", "/orchestrators", "/prs", `/projects/${projectId}`]) { + for (const route of ["/", "/prs", `/projects/${projectId}`]) { try { revalidatePath(route); } catch { @@ -108,6 +109,12 @@ export async function POST(request: NextRequest) { ); invalidatePortfolioServicesCache(); revalidatePortfolioPaths(registeredProjectId); + recordActivityEvent({ + projectId: registeredProjectId, + source: "api", + kind: "api.project_added", + summary: `project added: ${registeredProjectId}`, + }); return NextResponse.json({ ok: true, projectId: registeredProjectId }, { status: 201 }); } catch (err) { const message = err instanceof Error ? err.message : "Failed to add project"; @@ -124,6 +131,14 @@ export async function POST(request: NextRequest) { if (pathAlreadyRegistered) { const existingProjectId = pathAlreadyRegistered[1]; const suggestedProjectId = generateExternalId(resolvedPath); + recordActivityEvent({ + projectId, + source: "api", + kind: "api.project_add_rejected", + level: "warn", + summary: `project add rejected: path already registered`, + data: { reason: "path_already_registered", existingProjectId, statusCode: 409 }, + }); return NextResponse.json( { error: message, @@ -138,6 +153,14 @@ export async function POST(request: NextRequest) { if (idAlreadyRegistered) { const existingProjectId = idAlreadyRegistered[1]; const suggestedProjectId = generateExternalId(resolvedPath); + recordActivityEvent({ + projectId, + source: "api", + kind: "api.project_add_rejected", + level: "warn", + summary: `project add rejected: id already registered`, + data: { reason: "id_already_registered", existingProjectId, statusCode: 409 }, + }); return NextResponse.json( { error: message, diff --git a/packages/web/src/app/api/prs/[id]/merge/route.ts b/packages/web/src/app/api/prs/[id]/merge/route.ts index 9ab5aed1a..c996a60f9 100644 --- a/packages/web/src/app/api/prs/[id]/merge/route.ts +++ b/packages/web/src/app/api/prs/[id]/merge/route.ts @@ -1,4 +1,5 @@ import { type NextRequest } from "next/server"; +import { recordActivityEvent, type OrchestratorConfig } from "@aoagents/ao-core"; import { getServices, getSCM } from "@/lib/services"; import { getCorrelationId, jsonWithCorrelation, recordApiObservation } from "@/lib/observability"; @@ -11,15 +12,21 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< return jsonWithCorrelation({ error: "Invalid PR number" }, { status: 400 }, correlationId); } const prNumber = Number(id); + let configForObservation: OrchestratorConfig | undefined; + let projectId: string | undefined; + let sessionId: string | undefined; try { const { config, registry, sessionManager } = await getServices(); + configForObservation = config; const sessions = await sessionManager.list(); const session = sessions.find((s) => s.pr?.number === prNumber); if (!session?.pr) { return jsonWithCorrelation({ error: "PR not found" }, { status: 404 }, correlationId); } + projectId = session.projectId; + sessionId = session.id; const project = config.projects[session.projectId]; const scm = getSCM(registry, project); @@ -34,6 +41,15 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< // Validate PR is in a mergeable state const state = await scm.getPRState(session.pr); if (state !== "open") { + recordActivityEvent({ + projectId: session.projectId, + sessionId: session.id, + source: "api", + kind: "api.pr_merge_rejected", + level: "warn", + summary: `PR ${prNumber} merge rejected: state is ${state}`, + data: { prNumber, prState: state, statusCode: 409 }, + }); return jsonWithCorrelation( { error: `PR is ${state}, not open` }, { status: 409 }, @@ -43,6 +59,15 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< const mergeability = await scm.getMergeability(session.pr); if (!mergeability.mergeable) { + recordActivityEvent({ + projectId: session.projectId, + sessionId: session.id, + source: "api", + kind: "api.pr_merge_rejected", + level: "warn", + summary: `PR ${prNumber} merge rejected: not mergeable`, + data: { prNumber, blockers: mergeability.blockers, statusCode: 422 }, + }); return jsonWithCorrelation( { error: "PR is not mergeable", blockers: mergeability.blockers }, { status: 422 }, @@ -63,13 +88,22 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< sessionId: session.id, data: { prNumber }, }); + recordActivityEvent({ + projectId: session.projectId, + sessionId: session.id, + source: "api", + kind: "api.pr_merge_requested", + summary: `PR ${prNumber} merge requested`, + data: { prNumber, method: "squash" }, + }); return jsonWithCorrelation( { ok: true, prNumber, method: "squash" }, { status: 200 }, correlationId, ); } catch (err) { - const { config } = await getServices().catch(() => ({ config: undefined })); + const config = + configForObservation ?? (await getServices().catch(() => ({ config: undefined }))).config; if (config) { recordApiObservation({ config, @@ -79,10 +113,22 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< startedAt, outcome: "failure", statusCode: 500, + projectId, + sessionId, reason: err instanceof Error ? err.message : "Failed to merge PR", data: { prNumber }, }); } + const reason = err instanceof Error ? err.message : "Failed to merge PR"; + recordActivityEvent({ + projectId, + sessionId, + source: "api", + kind: "api.pr_merge_failed", + level: "error", + summary: `PR ${prNumber} merge failed: ${reason}`, + data: { prNumber, reason }, + }); return jsonWithCorrelation( { error: err instanceof Error ? err.message : "Failed to merge PR" }, { status: 500 }, diff --git a/packages/web/src/app/api/sessions/[id]/kill/route.ts b/packages/web/src/app/api/sessions/[id]/kill/route.ts index cf4a6b117..a71bac6e9 100644 --- a/packages/web/src/app/api/sessions/[id]/kill/route.ts +++ b/packages/web/src/app/api/sessions/[id]/kill/route.ts @@ -1,7 +1,7 @@ import { type NextRequest } from "next/server"; import { validateIdentifier } from "@/lib/validation"; import { getServices } from "@/lib/services"; -import { SessionNotFoundError } from "@aoagents/ao-core"; +import { SessionNotFoundError, recordActivityEvent } from "@aoagents/ao-core"; import { getCorrelationId, jsonWithCorrelation, @@ -34,6 +34,13 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< projectId, sessionId: id, }); + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_kill_requested", + summary: `session kill requested: ${id}`, + }); return jsonWithCorrelation({ ok: true, sessionId: id }, { status: 200 }, correlationId); } catch (err) { if (err instanceof SessionNotFoundError) { @@ -56,6 +63,15 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< }); } const msg = err instanceof Error ? err.message : "Failed to kill session"; + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_kill_failed", + level: "error", + summary: `session kill failed: ${msg}`, + data: { reason: msg }, + }); return jsonWithCorrelation({ error: msg }, { status: 500 }, correlationId); } } diff --git a/packages/web/src/app/api/sessions/[id]/message/route.ts b/packages/web/src/app/api/sessions/[id]/message/route.ts index 463927af5..1df3e4bae 100644 --- a/packages/web/src/app/api/sessions/[id]/message/route.ts +++ b/packages/web/src/app/api/sessions/[id]/message/route.ts @@ -1,7 +1,7 @@ import { type NextRequest } from "next/server"; import { getServices } from "@/lib/services"; import { stripControlChars, validateIdentifier, validateString } from "@/lib/validation"; -import { SessionNotFoundError } from "@aoagents/ao-core"; +import { SessionNotFoundError, recordActivityEvent } from "@aoagents/ao-core"; import { getCorrelationId, jsonWithCorrelation, @@ -79,6 +79,14 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ sessionId: id, data: { messageLength: message.length }, }); + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_message_sent", + summary: `message sent to session ${id}`, + data: { messageLength: message.length }, + }); return jsonWithCorrelation({ success: true }, { status: 200 }, correlationId); } catch (err) { const errorMsg = err instanceof Error ? err.message : String(err); @@ -98,6 +106,15 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ if (err instanceof SessionNotFoundError) { return jsonWithCorrelation({ error: err.message }, { status: 404 }, correlationId); } + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_message_failed", + level: "error", + summary: `session message failed: ${errorMsg}`, + data: { messageLength: message.length, reason: errorMsg }, + }); console.error("Failed to send message:", errorMsg); return jsonWithCorrelation( { error: `Failed to send message: ${errorMsg}` }, diff --git a/packages/web/src/app/api/sessions/[id]/remap/route.ts b/packages/web/src/app/api/sessions/[id]/remap/route.ts index d71ea32f9..5c5f0bb83 100644 --- a/packages/web/src/app/api/sessions/[id]/remap/route.ts +++ b/packages/web/src/app/api/sessions/[id]/remap/route.ts @@ -1,7 +1,7 @@ import { type NextRequest } from "next/server"; import { validateIdentifier } from "@/lib/validation"; import { getServices } from "@/lib/services"; -import { SessionNotFoundError } from "@aoagents/ao-core"; +import { SessionNotFoundError, recordActivityEvent } from "@aoagents/ao-core"; import { getCorrelationId, jsonWithCorrelation, @@ -33,6 +33,13 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ projectId, sessionId: id, }); + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_remap_requested", + summary: `session remap requested: ${id}`, + }); return jsonWithCorrelation( { ok: true, sessionId: id, opencodeSessionId }, { status: 200 }, @@ -74,6 +81,15 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ reason: msg, }); } + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_remap_failed", + level: "warn", + summary: `session remap failed: ${msg}`, + data: { reason: msg, statusCode: 422 }, + }); return jsonWithCorrelation({ error: msg }, { status: 422 }, correlationId); } if (config) { @@ -90,6 +106,15 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ reason: msg, }); } + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_remap_failed", + level: "error", + summary: `session remap failed: ${msg}`, + data: { reason: msg, statusCode: 500 }, + }); return jsonWithCorrelation({ error: msg }, { status: 500 }, correlationId); } } diff --git a/packages/web/src/app/api/sessions/[id]/restore/route.ts b/packages/web/src/app/api/sessions/[id]/restore/route.ts index 5372b9012..7c1bea560 100644 --- a/packages/web/src/app/api/sessions/[id]/restore/route.ts +++ b/packages/web/src/app/api/sessions/[id]/restore/route.ts @@ -6,6 +6,8 @@ import { SessionNotRestorableError, WorkspaceMissingError, SessionNotFoundError, + recordActivityEvent, + type OrchestratorConfig, } from "@aoagents/ao-core"; import { getCorrelationId, @@ -24,9 +26,13 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< return jsonWithCorrelation({ error: idErr }, { status: 400 }, correlationId); } + let configForAttribution: OrchestratorConfig | undefined; + let projectIdForAttribution: string | undefined; + try { const { config, sessionManager } = await getServices(); - const projectId = resolveProjectIdForSessionId(config, id); + configForAttribution = config; + projectIdForAttribution = resolveProjectIdForSessionId(config, id); const restored = await sessionManager.restore(id); recordApiObservation({ @@ -37,9 +43,16 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< startedAt, outcome: "success", statusCode: 200, - projectId: restored.projectId ?? projectId, + projectId: restored.projectId ?? projectIdForAttribution, sessionId: id, }); + recordActivityEvent({ + projectId: restored.projectId ?? projectIdForAttribution, + sessionId: id, + source: "api", + kind: "api.session_restore_requested", + summary: `session restore requested: ${id}`, + }); return jsonWithCorrelation( { @@ -54,29 +67,61 @@ export async function POST(_request: NextRequest, { params }: { params: Promise< if (err instanceof SessionNotFoundError) { return jsonWithCorrelation({ error: err.message }, { status: 404 }, correlationId); } + if (!configForAttribution) { + const serviceContext = await getServices().catch(() => undefined); + configForAttribution = serviceContext?.config; + projectIdForAttribution = configForAttribution + ? resolveProjectIdForSessionId(configForAttribution, id) + : undefined; + } if (err instanceof SessionNotRestorableError) { + recordActivityEvent({ + projectId: projectIdForAttribution, + sessionId: id, + source: "api", + kind: "api.session_restore_failed", + level: "warn", + summary: `session restore failed: ${err.message}`, + data: { reason: err.message, statusCode: 409 }, + }); return jsonWithCorrelation({ error: err.message }, { status: 409 }, correlationId); } if (err instanceof WorkspaceMissingError) { + recordActivityEvent({ + projectId: projectIdForAttribution, + sessionId: id, + source: "api", + kind: "api.session_restore_failed", + level: "warn", + summary: `session restore failed: ${err.message}`, + data: { reason: err.message, statusCode: 422 }, + }); return jsonWithCorrelation({ error: err.message }, { status: 422 }, correlationId); } - const { config } = await getServices().catch(() => ({ config: undefined })); - const projectId = config ? resolveProjectIdForSessionId(config, id) : undefined; - if (config) { + if (configForAttribution) { recordApiObservation({ - config, + config: configForAttribution, method: "POST", path: "/api/sessions/[id]/restore", correlationId, startedAt, outcome: "failure", statusCode: 500, - projectId, + projectId: projectIdForAttribution, sessionId: id, reason: err instanceof Error ? err.message : "Failed to restore session", }); } const msg = err instanceof Error ? err.message : "Failed to restore session"; + recordActivityEvent({ + projectId: projectIdForAttribution, + sessionId: id, + source: "api", + kind: "api.session_restore_failed", + level: "error", + summary: `session restore failed: ${msg}`, + data: { reason: msg, statusCode: 500 }, + }); return jsonWithCorrelation({ error: msg }, { status: 500 }, correlationId); } } diff --git a/packages/web/src/app/api/sessions/[id]/send/route.ts b/packages/web/src/app/api/sessions/[id]/send/route.ts index 44a8198df..8c0b67cd2 100644 --- a/packages/web/src/app/api/sessions/[id]/send/route.ts +++ b/packages/web/src/app/api/sessions/[id]/send/route.ts @@ -1,7 +1,7 @@ import { type NextRequest } from "next/server"; import { validateIdentifier, validateString, stripControlChars } from "@/lib/validation"; import { getServices } from "@/lib/services"; -import { SessionNotFoundError } from "@aoagents/ao-core"; +import { SessionNotFoundError, recordActivityEvent } from "@aoagents/ao-core"; import { getCorrelationId, jsonWithCorrelation, @@ -55,6 +55,14 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ sessionId: id, data: { messageLength: message.length }, }); + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_message_sent", + summary: `message sent to session ${id}`, + data: { messageLength: message.length }, + }); return jsonWithCorrelation( { ok: true, sessionId: id, message }, { status: 200 }, @@ -82,6 +90,15 @@ export async function POST(request: NextRequest, { params }: { params: Promise<{ }); } const msg = err instanceof Error ? err.message : "Failed to send message"; + recordActivityEvent({ + projectId, + sessionId: id, + source: "api", + kind: "api.session_message_failed", + level: "error", + summary: `session message failed: ${msg}`, + data: { messageLength: message.length, reason: msg }, + }); return jsonWithCorrelation({ error: msg }, { status: 500 }, correlationId); } } diff --git a/packages/web/src/app/api/sessions/patches/route.ts b/packages/web/src/app/api/sessions/patches/route.ts index 1ba9fa9de..0aa9df61b 100644 --- a/packages/web/src/app/api/sessions/patches/route.ts +++ b/packages/web/src/app/api/sessions/patches/route.ts @@ -16,7 +16,7 @@ export async function GET(request: Request) { ? projectFilter : undefined; - const coreSessions = await sessionManager.list(requestedProjectId); + const coreSessions = await sessionManager.listCached(requestedProjectId); const visibleSessions = filterWorkerSessions(coreSessions, projectFilter, config.projects); // Convert to dashboard format diff --git a/packages/web/src/app/api/setup-labels/route.ts b/packages/web/src/app/api/setup-labels/route.ts index 7b0c32465..75a3b597e 100644 --- a/packages/web/src/app/api/setup-labels/route.ts +++ b/packages/web/src/app/api/setup-labels/route.ts @@ -1,4 +1,5 @@ import { NextResponse } from "next/server"; +import { recordActivityEvent } from "@aoagents/ao-core"; import { getServices } from "@/lib/services"; import { execFile } from "node:child_process"; import { promisify } from "node:util"; @@ -42,6 +43,15 @@ export async function POST() { } } + const created = results.filter((r) => r.status === "created").length; + const exists = results.filter((r) => r.status === "exists").length; + recordActivityEvent({ + source: "api", + kind: "api.labels_setup", + summary: `labels setup complete: ${created} created, ${exists} exists`, + data: { created, exists, total: results.length }, + }); + return NextResponse.json({ results }); } catch (err) { return NextResponse.json( diff --git a/packages/web/src/app/api/spawn/route.ts b/packages/web/src/app/api/spawn/route.ts index 192a4a557..8fcc37fe5 100644 --- a/packages/web/src/app/api/spawn/route.ts +++ b/packages/web/src/app/api/spawn/route.ts @@ -1,4 +1,5 @@ import { type NextRequest } from "next/server"; +import { recordActivityEvent } from "@aoagents/ao-core"; import { validateIdentifier, validateString, validateConfiguredProject } from "@/lib/validation"; import { getServices } from "@/lib/services"; import { sessionToDashboard } from "@/lib/serialize"; @@ -50,6 +51,14 @@ export async function POST(request: NextRequest) { reason: projectErr, data: { issueId: body.issueId }, }); + recordActivityEvent({ + projectId, + source: "api", + kind: "api.session_spawn_rejected", + level: "warn", + summary: `session spawn rejected: ${projectErr}`, + data: { reason: "project_not_configured" }, + }); return jsonWithCorrelation({ error: projectErr }, { status: 404 }, correlationId); } @@ -75,6 +84,17 @@ export async function POST(request: NextRequest) { sessionId: session.id, data: { issueId: session.issueId }, }); + recordActivityEvent({ + projectId: session.projectId, + sessionId: session.id, + source: "api", + kind: "api.session_spawn_requested", + summary: `session spawn requested for ${session.projectId}`, + data: { + issueId: session.issueId ?? undefined, + hasPrompt: Boolean(prompt), + }, + }); return jsonWithCorrelation( { session: sessionToDashboard(session) }, diff --git a/packages/web/src/app/api/verify/route.ts b/packages/web/src/app/api/verify/route.ts index ef2ab0661..b713aaf52 100644 --- a/packages/web/src/app/api/verify/route.ts +++ b/packages/web/src/app/api/verify/route.ts @@ -1,7 +1,7 @@ import { type NextRequest, NextResponse } from "next/server"; import { getVerifyIssues, getServices } from "@/lib/services"; import { validateConfiguredProject } from "@/lib/validation"; -import type { Tracker } from "@aoagents/ao-core"; +import { recordActivityEvent, type Tracker } from "@aoagents/ao-core"; export const dynamic = "force-dynamic"; @@ -25,6 +25,9 @@ export async function GET() { * Body: { issueId: string, projectId: string, action: "verify" | "fail", comment?: string } */ export async function POST(req: NextRequest) { + let issueId: string | undefined; + let projectId: string | undefined; + let action: "verify" | "fail" | undefined; try { const body = (await req.json().catch(() => null)) as | { @@ -37,12 +40,13 @@ export async function POST(req: NextRequest) { if (!body) { return NextResponse.json({ error: "Invalid JSON body" }, { status: 400 }); } - const { issueId, projectId, action, comment } = body as { + let comment: string | undefined; + ({ issueId, projectId, action, comment } = body as { issueId: string; projectId: string; action: "verify" | "fail"; comment?: string; - }; + }); if (!issueId || !projectId || !action) { return NextResponse.json( @@ -96,11 +100,25 @@ export async function POST(req: NextRequest) { ); } + recordActivityEvent({ + projectId, + source: "api", + kind: "api.issue_verified", + summary: `issue ${issueId} ${action}`, + data: { issueId, action }, + }); + return NextResponse.json({ ok: true }); } catch (err) { - return NextResponse.json( - { error: err instanceof Error ? err.message : "Failed to update issue" }, - { status: 500 }, - ); + const reason = err instanceof Error ? err.message : "Failed to update issue"; + recordActivityEvent({ + projectId, + source: "api", + kind: "api.issue_verify_failed", + level: "error", + summary: `issue verify failed: ${reason}`, + data: { issueId, action, reason }, + }); + return NextResponse.json({ error: reason }, { status: 500 }); } } diff --git a/packages/web/src/app/api/webhooks/[...slug]/route.ts b/packages/web/src/app/api/webhooks/[...slug]/route.ts index 9ccf878ad..08660dac8 100644 --- a/packages/web/src/app/api/webhooks/[...slug]/route.ts +++ b/packages/web/src/app/api/webhooks/[...slug]/route.ts @@ -1,4 +1,5 @@ import { NextResponse } from "next/server"; +import { recordActivityEvent } from "@aoagents/ao-core"; import { getServices } from "@/lib/services"; import { buildWebhookRequest, @@ -9,11 +10,35 @@ import { export const dynamic = "force-dynamic"; +const WEBHOOK_PATH_PREFIX = "/api/webhooks/"; + +function deriveSlug(pathname: string): string { + return pathname.startsWith(WEBHOOK_PATH_PREFIX) + ? pathname.slice(WEBHOOK_PATH_PREFIX.length) + : pathname; +} + +function deriveRemoteAddr(request: Request): string | undefined { + // Next.js does not expose the socket peer address on Request. The standard + // proxy headers are the only signal — first hop in x-forwarded-for is the + // original client. Sanitizer in recordActivityEvent does not redact IPs; + // they are intentionally retained for security audit (per issue #1656). + const xff = request.headers.get("x-forwarded-for"); + if (xff) { + const first = xff.split(",")[0]?.trim(); + if (first) return first; + } + return request.headers.get("x-real-ip") ?? undefined; +} + export async function POST(request: Request): Promise { + const pathname = new URL(request.url).pathname; + const slug = deriveSlug(pathname); + const remoteAddr = deriveRemoteAddr(request); + try { const services = await getServices(); - const path = new URL(request.url).pathname; - const candidates = findWebhookProjects(services.config, services.registry, path); + const candidates = findWebhookProjects(services.config, services.registry, pathname); if (candidates.length === 0) { return NextResponse.json( @@ -36,6 +61,19 @@ export async function POST(request: Request): Promise { Number.isFinite(contentLength) && contentLength > maxBodyBytes ) { + recordActivityEvent({ + source: "api", + kind: "api.webhook_rejected", + level: "warn", + summary: `webhook payload exceeded ${maxBodyBytes} bytes for ${slug}`, + data: { + slug, + remoteAddr, + contentLength, + maxBodyBytes, + reason: "payload_too_large", + }, + }); return NextResponse.json( { error: "Webhook payload exceeds configured maxBodyBytes" }, { status: 413 }, @@ -50,12 +88,21 @@ export async function POST(request: Request): Promise { const sessionIds = new Set(); const projectIds = new Set(); let verified = false; + let verificationSupported = false; + let unsupportedVerificationCount = 0; const errors: string[] = []; const parseErrors: string[] = []; const lifecycleErrors: string[] = []; for (const candidate of candidates) { - const verification = await candidate.scm.verifyWebhook?.(webhookRequest, candidate.project); + if (!candidate.scm.verifyWebhook) { + unsupportedVerificationCount += 1; + errors.push("Webhook verification not supported by SCM plugin"); + continue; + } + + verificationSupported = true; + const verification = await candidate.scm.verifyWebhook(webhookRequest, candidate.project); if (!verification?.ok) { if (verification?.reason) errors.push(verification.reason); continue; @@ -93,12 +140,52 @@ export async function POST(request: Request): Promise { } if (!verified) { + const unsupportedOnly = !verificationSupported && unsupportedVerificationCount > 0; + recordActivityEvent({ + source: "api", + kind: "api.webhook_unverified", + level: "warn", + summary: unsupportedOnly + ? `webhook verification unsupported for ${slug}` + : `webhook signature verification failed for ${slug}`, + data: { + slug, + remoteAddr, + candidateCount: candidates.length, + verificationSupported, + unsupportedVerificationCount, + reason: unsupportedOnly + ? "verification_unsupported" + : (errors[0] ?? "verification_failed"), + }, + }); return NextResponse.json( - { error: errors[0] ?? "Webhook verification failed", ok: false }, - { status: 401 }, + { + error: unsupportedOnly + ? "No SCM webhook configured for this path" + : (errors[0] ?? "Webhook verification failed"), + ok: false, + verificationSupported, + }, + { status: unsupportedOnly ? 404 : 401 }, ); } + recordActivityEvent({ + source: "api", + kind: "api.webhook_received", + level: parseErrors.length > 0 || lifecycleErrors.length > 0 ? "warn" : "info", + summary: `webhook accepted for ${slug}: ${sessionIds.size} session(s) matched`, + data: { + slug, + remoteAddr, + projectIds: [...projectIds], + matchedSessions: sessionIds.size, + parseErrorCount: parseErrors.length, + lifecycleErrorCount: lifecycleErrors.length, + }, + }); + return NextResponse.json( { ok: true, @@ -111,6 +198,17 @@ export async function POST(request: Request): Promise { { status: 202 }, ); } catch (err) { + recordActivityEvent({ + source: "api", + kind: "api.webhook_failed", + level: "error", + summary: `webhook pipeline crashed for ${slug}`, + data: { + slug, + remoteAddr, + errorMessage: err instanceof Error ? err.message : String(err), + }, + }); return NextResponse.json( { error: err instanceof Error ? err.message : "Failed to process SCM webhook" }, { status: 500 }, diff --git a/packages/web/src/app/globals.css b/packages/web/src/app/globals.css index 8bb4b4016..14f6fcf83 100644 --- a/packages/web/src/app/globals.css +++ b/packages/web/src/app/globals.css @@ -1230,6 +1230,11 @@ html.light .xterm .xterm-viewport:hover::-webkit-scrollbar-thumb:active { color 100ms ease, transform 120ms cubic-bezier(0.23, 1, 0.32, 1); } +.dashboard-app-btn--icon { + width: 27px; + padding: 0; + justify-content: center; +} .dashboard-app-btn:hover { background: var(--color-bg-subtle); @@ -1328,6 +1333,12 @@ html.light .xterm .xterm-viewport:hover::-webkit-scrollbar-thumb:active { .topbar-status-pill__label { color: inherit; } +.topbar-status-pill__dot--working { + background: var(--color-status-working); +} +.topbar-status-pill__dot--attention { + background: var(--color-status-attention); +} /* Branch pill — compact topbar variant */ .topbar-branch-pill { @@ -3846,7 +3857,8 @@ html.light .xterm .xterm-viewport:hover::-webkit-scrollbar-thumb:active { display: flex; align-items: center; justify-content: space-between; - padding: 10px 12px 8px; + height: 48px; + padding: 0 12px; border-bottom: 1px solid var(--sidebar-border); flex-shrink: 0; } @@ -4483,6 +4495,8 @@ html.light .xterm .xterm-viewport:hover::-webkit-scrollbar-thumb:active { .project-sidebar__sess-row--active { background: transparent; + border-left: 2px solid var(--color-accent-amber); + padding-left: 24px; } /* Session label */ @@ -4627,6 +4641,13 @@ html.light .xterm .xterm-viewport:hover::-webkit-scrollbar-thumb:active { font-size: 11px; } +.project-sidebar__empty-hint { + display: block; + font-size: 10px; + margin-top: 3px; + color: var(--color-text-muted); +} + .project-sidebar__footer { margin-top: auto; } diff --git a/packages/web/src/app/orchestrators/page.tsx b/packages/web/src/app/orchestrators/page.tsx deleted file mode 100644 index 63444c915..000000000 --- a/packages/web/src/app/orchestrators/page.tsx +++ /dev/null @@ -1,83 +0,0 @@ -import type { Metadata } from "next"; -import { OrchestratorSelector, type Orchestrator } from "@/components/OrchestratorSelector"; -import { getServices } from "@/lib/services"; -import { getAllProjects } from "@/lib/project-name"; -import { generateSessionPrefix } from "@aoagents/ao-core"; -import { mapSessionsToOrchestrators } from "@/lib/orchestrator-utils"; - -export const dynamic = "force-dynamic"; - -export async function generateMetadata(props: { - searchParams: Promise<{ project?: string }>; -}): Promise { - const searchParams = await props.searchParams; - const projectId = searchParams.project; - let projectName = "Orchestrator"; - if (projectId) { - const projects = getAllProjects(); - const project = projects.find((p) => p.id === projectId); - if (project) { - projectName = project.name; - } - } - return { title: { absolute: `ao | ${projectName} - Orchestrator` } }; -} - -export default async function OrchestratorsRoute(props: { - searchParams: Promise<{ project?: string }>; -}) { - const searchParams = await props.searchParams; - const projectId = searchParams.project; - - if (!projectId) { - return ( -
-
-

- Missing Project -

-

- No project specified. Please provide a project parameter. -

-
-
- ); - } - - let orchestrators: Orchestrator[] = []; - let projectName = projectId; - let error: string | null = null; - - try { - const { config, sessionManager } = await getServices(); - const project = config.projects[projectId]; - - if (!project) { - error = `Project "${projectId}" not found`; - } else { - projectName = project.name; - const sessionPrefix = project.sessionPrefix ?? projectId; - const allSessions = await sessionManager.list(projectId); - const allSessionPrefixes = Object.entries(config.projects).map( - ([, p]) => p.sessionPrefix ?? generateSessionPrefix(p.name ?? ""), - ); - orchestrators = mapSessionsToOrchestrators( - allSessions, - sessionPrefix, - project.name, - allSessionPrefixes, - ); - } - } catch (err) { - error = err instanceof Error ? err.message : "Failed to load orchestrators"; - } - - return ( - - ); -} diff --git a/packages/web/src/app/projects/[projectId]/__tests__/project-layout-client.test.tsx b/packages/web/src/app/projects/[projectId]/__tests__/project-layout-client.test.tsx new file mode 100644 index 000000000..e94256838 --- /dev/null +++ b/packages/web/src/app/projects/[projectId]/__tests__/project-layout-client.test.tsx @@ -0,0 +1,122 @@ +import { render, screen, act } from "@testing-library/react"; +import { beforeEach, describe, expect, it, vi } from "vitest"; + +let mockPathname = "/projects/proj-1"; +let mockParams: Record = { projectId: "proj-1" }; + +vi.mock("next/navigation", () => ({ + useParams: () => mockParams, + usePathname: () => mockPathname, +})); + +vi.mock("next-themes", () => ({ + useTheme: () => ({ resolvedTheme: "light", setTheme: vi.fn() }), +})); + +vi.mock("@/providers/MuxProvider", () => ({ + useMuxOptional: () => ({ status: "connecting", sessions: [], lastError: null }), +})); + +vi.mock("@/hooks/useSessionEvents", () => ({ + useSessionEvents: ({ initialSessions }: { initialSessions: unknown[] }) => ({ + sessions: initialSessions, + liveSessionsResolved: true, + attentionLevels: {}, + loadError: null, + }), +})); + +vi.mock("@/components/ProjectSidebar", () => ({ + ProjectSidebar: (props: { activeProjectId?: string; orchestrators?: unknown[] }) => ( +
+ ), +})); + +import { ProjectLayoutClient } from "../project-layout-client"; + +const projects = [{ id: "proj-1", name: "Project One", sessionPrefix: "proj-1" }]; +const orchestrators = [{ id: "proj-1-orchestrator", projectId: "proj-1" }]; + +beforeEach(() => { + mockPathname = "/projects/proj-1"; + mockParams = { projectId: "proj-1" }; +}); + +describe("ProjectLayoutClient", () => { + it("renders children and sidebar", () => { + render( + +
Page
+
, + ); + + expect(screen.getByTestId("sidebar")).toBeInTheDocument(); + expect(screen.getByTestId("page-content")).toBeInTheDocument(); + }); + + it("passes activeProjectId from route params to sidebar", () => { + mockParams = { projectId: "proj-1" }; + + render( + +
+ , + ); + + expect(screen.getByTestId("sidebar").dataset.project).toBe("proj-1"); + }); + + it("passes initialOrchestrators directly to sidebar", () => { + render( + +
+ , + ); + + const sidebar = screen.getByTestId("sidebar"); + expect(JSON.parse(sidebar.dataset.orchestrators ?? "[]")).toEqual(orchestrators); + }); + + it("resets mobile sidebar when pathname changes", async () => { + const { rerender } = render( + +
+ , + ); + + // Simulate pathname change + mockPathname = "/projects/proj-1/sessions/sess-1"; + + await act(async () => { + rerender( + +
+ , + ); + }); + + // Sidebar wrapper should not have the mobile-open class + const wrapper = document.querySelector(".sidebar-wrapper"); + expect(wrapper?.classList.contains("sidebar-wrapper--mobile-open")).toBe(false); + }); +}); diff --git a/packages/web/src/app/projects/[projectId]/layout.tsx b/packages/web/src/app/projects/[projectId]/layout.tsx new file mode 100644 index 000000000..330afda9c --- /dev/null +++ b/packages/web/src/app/projects/[projectId]/layout.tsx @@ -0,0 +1,5 @@ +import type { ReactNode } from "react"; + +export default function ProjectIdLayout({ children }: { children: ReactNode }) { + return <>{children}; +} diff --git a/packages/web/src/app/projects/[projectId]/loading.test.tsx b/packages/web/src/app/projects/[projectId]/loading.test.tsx index 54d978a5b..61726031e 100644 --- a/packages/web/src/app/projects/[projectId]/loading.test.tsx +++ b/packages/web/src/app/projects/[projectId]/loading.test.tsx @@ -8,7 +8,8 @@ describe("ProjectRouteLoading", () => { expect(screen.getByText("Agent Orchestrator")).toBeInTheDocument(); expect(screen.getByText("Loading project…")).toBeInTheDocument(); - expect(screen.getByText("Projects")).toBeInTheDocument(); - expect(screen.getByLabelText("Loading project dashboard")).toBeInTheDocument(); + // Sidebar is owned by ProjectLayoutClient — no duplicate skeleton sidebar here + expect(screen.queryByText("Projects")).not.toBeInTheDocument(); + expect(screen.getByText("Working")).toBeInTheDocument(); }); }); diff --git a/packages/web/src/app/projects/[projectId]/loading.tsx b/packages/web/src/app/projects/[projectId]/loading.tsx index cc385e2fd..ffeee105c 100644 --- a/packages/web/src/app/projects/[projectId]/loading.tsx +++ b/packages/web/src/app/projects/[projectId]/loading.tsx @@ -1,101 +1,40 @@ -function ProjectLoadingSidebar() { - return ( - - ); -} - export default function ProjectRouteLoading() { return ( -
-
- - -
- +
+ -
-
-
-
-
- -
); } diff --git a/packages/web/src/app/projects/[projectId]/page.tsx b/packages/web/src/app/projects/[projectId]/page.tsx index 2c677fb1f..39d317391 100644 --- a/packages/web/src/app/projects/[projectId]/page.tsx +++ b/packages/web/src/app/projects/[projectId]/page.tsx @@ -29,7 +29,7 @@ export default async function ProjectPage(props: { const pageData = await getDashboardPageData(projectId); return ( -
+
{ + setMobileSidebarOpen(false); + }, [pathname]); + + const mux = useMuxOptional(); + const { sessions, liveSessionsResolved } = useSessionEvents({ + initialSessions, + muxSessions: mux?.status === "connected" ? mux.sessions : undefined, + muxLastError: mux?.lastError, + attentionZones: "simple", + }); + + const handleToggleSidebar = useCallback(() => { + if (isMobile) { + setMobileSidebarOpen((v) => !v); + } else { + setSidebarCollapsed((v) => !v); + } + }, [isMobile]); + + return ( + +
+
+
+ setSidebarCollapsed((v) => !v)} + onMobileClose={() => setMobileSidebarOpen(false)} + /> +
+ {mobileSidebarOpen && ( +
setMobileSidebarOpen(false)} /> + )} + {children} +
+
+ + ); +} diff --git a/packages/web/src/app/projects/[projectId]/sessions/[id]/page.tsx b/packages/web/src/app/projects/[projectId]/sessions/[id]/page.tsx index 3e7be8541..39c70fcab 100644 --- a/packages/web/src/app/projects/[projectId]/sessions/[id]/page.tsx +++ b/packages/web/src/app/projects/[projectId]/sessions/[id]/page.tsx @@ -1 +1,459 @@ -export { default } from "../../../../sessions/[id]/page"; +"use client"; + +import { useEffect, useState, useCallback, useRef } from "react"; +import { useParams, usePathname, useRouter } from "next/navigation"; +import { isOrchestratorSession } from "@aoagents/ao-core/types"; +import { SessionDetail } from "@/components/SessionDetail"; +import { ErrorDisplay } from "@/components/ErrorDisplay"; +import { + type DashboardSession, + type ActivityState, + getAttentionLevel, +} from "@/lib/types"; +import { activityIcon } from "@/lib/activity-icons"; +import type { ProjectInfo } from "@/lib/project-name"; +import { getSessionTitle } from "@/lib/format"; +import { useMuxSessionActivity } from "@/hooks/useMuxSessionActivity"; +import { projectSessionPath } from "@/lib/routes"; +import { fetchJsonWithTimeout } from "@/lib/client-fetch"; + +function truncate(s: string, max: number): string { + const codePoints = Array.from(s); + return codePoints.length > max ? codePoints.slice(0, max).join("") + "..." : s; +} + +function buildSessionTitle( + session: DashboardSession, + prefixByProject: Map, + activityOverride?: ActivityState | null, +): string { + const id = session.id; + const activity = activityOverride !== undefined ? activityOverride : session.activity; + const emoji = activity ? (activityIcon[activity] ?? "") : ""; + const allPrefixes = [...prefixByProject.values()]; + const isOrchestrator = isOrchestratorSession( + session, + prefixByProject.get(session.projectId), + allPrefixes, + ); + const detail = isOrchestrator ? "Orchestrator Terminal" : truncate(getSessionTitle(session), 40); + return emoji ? `${emoji} ${id} | ${detail}` : `${id} | ${detail}`; +} + +interface ZoneCounts { + merge: number; + respond: number; + review: number; + pending: number; + working: number; + done: number; +} + +interface ProjectSessionsBody { + sessions?: DashboardSession[]; + orchestratorId?: string | null; + orchestrators?: Array<{ id: string; projectId: string; projectName: string }>; +} + +let cachedProjects: ProjectInfo[] | null = null; + +const SESSION_PAGE_REFRESH_INTERVAL_MS = 2000; +const SESSION_FETCH_TIMEOUT_MS = 8000; +const PROJECT_SESSIONS_FETCH_TIMEOUT_MS = 5000; +const PROJECTS_FETCH_TIMEOUT_MS = 5000; +function areProjectsEqual(previous: ProjectInfo[] | null, next: ProjectInfo[]): boolean { + if (!previous || previous.length !== next.length) return false; + return previous.every((p, i) => JSON.stringify(p) === JSON.stringify(next[i])); +} + +function isAbortLikeError(error: unknown): boolean { + if (error instanceof DOMException) return error.name === "AbortError"; + if (error instanceof Error) { + const msg = error.message.toLowerCase(); + return msg.includes("aborted") || msg.includes("aborterror"); + } + return false; +} + +function getSessionLoadErrorMessage(error: Error): string { + const normalized = error.message.toLowerCase(); + if (normalized.includes("timed out")) + return "The session request is taking too long. You can retry, or return to the project and reopen a different session."; + if (normalized.includes("network")) + return "The session request failed before the dashboard got a response. Check the local server connection and try again."; + if (normalized.includes("404")) + return "This session is no longer available. It may have been removed while the page was open."; + if (normalized.includes("500")) + return "The server returned an internal error while loading this session. Try again to re-fetch the latest state."; + return "The dashboard could not load this session cleanly. Try again to re-fetch the latest state."; +} + +function LoadingContent() { + return ( +
+
+ + + +
Loading session…
+
+
+ ); +} + +export default function ProjectSessionPage() { + const params = useParams(); + const pathname = usePathname(); + const router = useRouter(); + const id = params.id as string; + const expectedProjectId = typeof params.projectId === "string" ? params.projectId : undefined; + + // Read optimistic session data written by sidebar navigation + const cachedSession = (() => { + if (typeof sessionStorage === "undefined") return null; + try { + const raw = sessionStorage.getItem(`ao-session-nav:${id}`); + if (raw) { + sessionStorage.removeItem(`ao-session-nav:${id}`); + return JSON.parse(raw) as DashboardSession; + } + } catch { + /* ignore */ + } + return null; + })(); + + const [session, setSession] = useState(cachedSession); + const [zoneCounts, setZoneCounts] = useState(null); + const [projectOrchestratorId, setProjectOrchestratorId] = useState( + undefined, + ); + const [projects, setProjects] = useState([]); + const [loading, setLoading] = useState(cachedSession === null); + const [routeError, setRouteError] = useState(null); + const [sessionMissing, setSessionMissing] = useState(false); + const [prefixByProject, setPrefixByProject] = useState>(new Map()); + + const sessionProjectId = session?.projectId ?? null; + const allPrefixes = [...prefixByProject.values()]; + const sessionIsOrchestrator = session + ? isOrchestratorSession(session, prefixByProject.get(session.projectId), allPrefixes) + : false; + + const sessionProjectIdRef = useRef(null); + const sessionIsOrchestratorRef = useRef(false); + const resolvedProjectSessionsKeyRef = useRef(null); + const prefixByProjectRef = useRef>(new Map()); + const hasLoadedSessionRef = useRef(cachedSession !== null); + const fetchingSessionRef = useRef(false); + const fetchingProjectSessionsRef = useRef(false); + const sessionFetchControllerRef = useRef(null); + const projectSessionsFetchControllerRef = useRef(null); + const pageUnloadingRef = useRef(false); + const mountedRef = useRef(true); + + const sseActivity = useMuxSessionActivity(id); + + useEffect(() => { + prefixByProjectRef.current = prefixByProject; + }, [prefixByProject]); + + useEffect(() => { + if (session) { + document.title = buildSessionTitle(session, prefixByProject, sseActivity?.activity); + } else { + document.title = `${id} | Session Detail`; + } + }, [session, id, prefixByProject, sseActivity]); + + useEffect(() => { + sessionProjectIdRef.current = sessionProjectId; + }, [sessionProjectId]); + + useEffect(() => { + sessionIsOrchestratorRef.current = sessionIsOrchestrator; + }, [sessionIsOrchestrator]); + + useEffect(() => { + if (!session || !projects.some((p) => p.id === session.projectId)) return; + if ( + pathname?.startsWith("/projects/") && + expectedProjectId && + session.projectId !== expectedProjectId + ) { + router.replace(projectSessionPath(session.projectId, session.id)); + } + }, [expectedProjectId, pathname, projects, router, session]); + + useEffect(() => { + if (!sessionIsOrchestrator) setZoneCounts(null); + }, [sessionIsOrchestrator]); + + const fetchProjects = useCallback(async () => { + if (cachedProjects) { + setProjects(cachedProjects); + setPrefixByProject(new Map(cachedProjects.map((p) => [p.id, p.sessionPrefix ?? p.id]))); + } + try { + const data = await fetchJsonWithTimeout<{ projects?: ProjectInfo[] } | null>( + "/api/projects", + { + timeoutMs: PROJECTS_FETCH_TIMEOUT_MS, + timeoutMessage: `Projects request timed out after ${PROJECTS_FETCH_TIMEOUT_MS}ms`, + }, + ); + if (!data?.projects) return; + if (!areProjectsEqual(cachedProjects, data.projects)) { + cachedProjects = data.projects; + setProjects(data.projects); + setPrefixByProject(new Map(data.projects.map((p) => [p.id, p.sessionPrefix ?? p.id]))); + } + } catch (err) { + console.error("Failed to fetch projects:", err); + } + }, []); + + const fetchSession = useCallback(async () => { + if (fetchingSessionRef.current) return; + fetchingSessionRef.current = true; + const controller = new AbortController(); + sessionFetchControllerRef.current = controller; + try { + const data = await fetchJsonWithTimeout( + `/api/sessions/${encodeURIComponent(id)}`, + { + signal: controller.signal, + timeoutMs: SESSION_FETCH_TIMEOUT_MS, + timeoutMessage: `Session request timed out after ${SESSION_FETCH_TIMEOUT_MS}ms`, + }, + ); + setSession(data as DashboardSession); + setRouteError(null); + setSessionMissing(false); + hasLoadedSessionRef.current = true; + } catch (err) { + if (pageUnloadingRef.current || controller.signal.aborted || isAbortLikeError(err)) return; + const message = err instanceof Error ? err.message : "Failed to load session"; + const normalized = message.toLowerCase(); + if (normalized.includes("session not found") || normalized.includes("http 404")) { + if (!hasLoadedSessionRef.current) setSessionMissing(true); + setLoading(false); + return; + } + console.error("Failed to fetch session:", err); + if (!hasLoadedSessionRef.current) { + setRouteError(err instanceof Error ? err : new Error("Failed to load session")); + } + } finally { + fetchingSessionRef.current = false; + if (sessionFetchControllerRef.current === controller) + sessionFetchControllerRef.current = null; + if (!controller.signal.aborted || hasLoadedSessionRef.current) { + setLoading(false); + } else if (mountedRef.current) { + // Aborted before any session was loaded and the component is still + // mounted — React Strict Mode fired the cleanup between mount 1 and + // mount 2, aborting the first fetch. Mount 2's fetchSession() was + // blocked by fetchingSessionRef (not yet reset). Retry immediately + // now that the ref is clear. mountedRef guards against the navigation- + // away case where the component is genuinely unmounted and we should + // not start a new request. + void fetchSession(); + } + } + }, [id]); + + const fetchProjectSessions = useCallback(async () => { + if (fetchingProjectSessionsRef.current) return; + const projectId = sessionProjectIdRef.current; + if (!projectId) return; + const isOrchestrator = sessionIsOrchestratorRef.current; + const projectSessionsKey = `${projectId}:${isOrchestrator ? "orchestrator" : "worker"}`; + if (!isOrchestrator && resolvedProjectSessionsKeyRef.current === projectSessionsKey) return; + fetchingProjectSessionsRef.current = true; + const controller = new AbortController(); + projectSessionsFetchControllerRef.current = controller; + try { + const query = isOrchestrator + ? `/api/sessions?project=${encodeURIComponent(projectId)}&fresh=true` + : `/api/sessions?project=${encodeURIComponent(projectId)}&orchestratorOnly=true&fresh=true`; + const body = await fetchJsonWithTimeout(query, { + signal: controller.signal, + timeoutMs: PROJECT_SESSIONS_FETCH_TIMEOUT_MS, + timeoutMessage: `Project sessions request timed out after ${PROJECT_SESSIONS_FETCH_TIMEOUT_MS}ms`, + }); + const sessions = body.sessions ?? []; + const orchestratorId = + body.orchestratorId ?? + body.orchestrators?.find((o) => o.projectId === projectId)?.id ?? + null; + setProjectOrchestratorId((current) => + current === orchestratorId ? current : orchestratorId, + ); + + if (!isOrchestrator) { + resolvedProjectSessionsKeyRef.current = projectSessionsKey; + return; + } + + const counts: ZoneCounts = { + merge: 0, + respond: 0, + review: 0, + pending: 0, + working: 0, + done: 0, + }; + const allPfxs = [...prefixByProjectRef.current.values()]; + for (const s of sessions) { + if (!isOrchestratorSession(s, prefixByProjectRef.current.get(s.projectId), allPfxs)) { + const level = getAttentionLevel(s); + if (level === "action") continue; + counts[level]++; + } + } + setZoneCounts(counts); + } catch (err) { + if (pageUnloadingRef.current || controller.signal.aborted || isAbortLikeError(err)) return; + console.error("Failed to fetch project sessions:", err); + } finally { + fetchingProjectSessionsRef.current = false; + if (projectSessionsFetchControllerRef.current === controller) + projectSessionsFetchControllerRef.current = null; + } + }, []); + + useEffect(() => { + void Promise.all([fetchProjects(), fetchSession()]); + }, [fetchProjects, fetchSession]); + + useEffect(() => { + if (!sessionProjectId) return; + void fetchProjectSessions(); + }, [fetchProjectSessions, sessionIsOrchestrator, sessionProjectId]); + + useEffect(() => { + const interval = setInterval(() => { + void fetchSession(); + void fetchProjectSessions(); + }, SESSION_PAGE_REFRESH_INTERVAL_MS); + return () => clearInterval(interval); + }, [fetchSession, fetchProjectSessions]); + + useEffect(() => { + pageUnloadingRef.current = false; + const mark = () => { + pageUnloadingRef.current = true; + }; + window.addEventListener("pagehide", mark); + window.addEventListener("beforeunload", mark); + return () => { + window.removeEventListener("pagehide", mark); + window.removeEventListener("beforeunload", mark); + }; + }, []); + + useEffect(() => { + mountedRef.current = true; + return () => { + mountedRef.current = false; + }; + }, []); + + useEffect(() => { + return () => { + sessionFetchControllerRef.current?.abort(); + projectSessionsFetchControllerRef.current?.abort(); + }; + }, []); + + if (loading) return
; + + if (sessionMissing) { + return ( +
+
+ void fetchSession() }} + compact + chrome="card" + /> +
+
+ ); + } + + if (routeError) { + return ( +
+
+ { + setRouteError(null); + setSessionMissing(false); + setLoading(true); + void Promise.all([fetchProjects(), fetchSession()]); + }, + }} + secondaryAction={{ + label: "Back to dashboard", + href: session?.projectId ? `/projects/${session.projectId}` : "/", + }} + error={routeError} + compact + chrome="card" + /> +
+
+ ); + } + + if (!session) { + return ( +
+
+ void fetchSession() }} + secondaryAction={{ + label: "Back to dashboard", + href: expectedProjectId ? `/projects/${expectedProjectId}` : "/", + }} + compact + chrome="card" + /> +
+
+ ); + } + + return ( + + ); +} diff --git a/packages/web/src/app/projects/layout.tsx b/packages/web/src/app/projects/layout.tsx new file mode 100644 index 000000000..91140bcd1 --- /dev/null +++ b/packages/web/src/app/projects/layout.tsx @@ -0,0 +1,19 @@ +import { getDashboardPageData } from "@/lib/dashboard-page-data"; +import { ProjectLayoutClient } from "./[projectId]/project-layout-client"; +import type { ReactNode } from "react"; + +export const dynamic = "force-dynamic"; + +export default async function ProjectLayout({ children }: { children: ReactNode }) { + const pageData = await getDashboardPageData("all"); + + return ( + + {children} + + ); +} diff --git a/packages/web/src/app/sessions/[id]/page.test.tsx b/packages/web/src/app/sessions/[id]/page.test.tsx index 1f3158e8d..6fa1990e9 100644 --- a/packages/web/src/app/sessions/[id]/page.test.tsx +++ b/packages/web/src/app/sessions/[id]/page.test.tsx @@ -454,77 +454,6 @@ describe("SessionPage project polling", () => { expect(screen.getAllByText("Failed to load session").length).toBeGreaterThan(0); }); - it("marks sidebar data as loading until the sessions list resolves", async () => { - const workerSession = makeWorkerSession(); - let resolveSidebarSessions: ((value: Response) => void) | null = null; - - global.fetch = vi.fn((input: RequestInfo | URL) => { - const url = String(input); - if (url === "/api/projects") { - return Promise.resolve({ - ok: true, - status: 200, - json: async () => ({ - projects: [{ id: "my-app", name: "My App", sessionPrefix: "my-app" }], - }), - } as Response); - } - - if (url === "/api/sessions/worker-1") { - return Promise.resolve({ - ok: true, - status: 200, - json: async () => workerSession, - } as Response); - } - - if (url === "/api/sessions?fresh=true") { - return new Promise((resolve) => { - resolveSidebarSessions = resolve; - }); - } - - if (url === "/api/sessions?project=my-app&orchestratorOnly=true&fresh=true") { - return Promise.resolve({ - ok: true, - status: 200, - json: async () => ({ orchestratorId: "my-app-orchestrator" }), - } as Response); - } - - return Promise.reject(new Error(`Unexpected fetch: ${url}`)); - }) as typeof fetch; - - const { default: SessionPage } = await import("./page"); - - render(); - await flushAsyncWork(); - - const latestBeforeSidebarResolve = sessionDetailSpy.mock.lastCall?.[0] as { - sidebarLoading?: boolean; - sidebarSessions?: DashboardSession[] | null; - }; - - expect(latestBeforeSidebarResolve.sidebarLoading).toBe(true); - expect(latestBeforeSidebarResolve.sidebarSessions).toBeNull(); - - await act(async () => { - resolveSidebarSessions?.({ - ok: true, - status: 200, - json: async () => ({ sessions: [workerSession] }), - } as Response); - await Promise.resolve(); - }); - - const latestAfterSidebarResolve = sessionDetailSpy.mock.lastCall?.[0] as { - sidebarLoading?: boolean; - sidebarSessions?: DashboardSession[] | null; - }; - - expect(latestAfterSidebarResolve.sidebarLoading).toBe(false); - expect(latestAfterSidebarResolve.sidebarSessions).toEqual([workerSession]); - }); it("revalidates projects and sidebar sessions on remount even when cache exists", async () => { const workerSession = makeWorkerSession(); @@ -642,145 +571,7 @@ describe("SessionPage project polling", () => { ); }); - it("surfaces sidebar fetch failures instead of leaving the loading skeleton active", async () => { - const workerSession = makeWorkerSession(); - global.fetch = vi.fn(async (input: RequestInfo | URL) => { - const url = String(input); - if (url === "/api/projects") { - return { - ok: true, - status: 200, - json: async () => ({ - projects: [{ id: "my-app", name: "My App", sessionPrefix: "my-app" }], - }), - } as Response; - } - - if (url === "/api/sessions/worker-1") { - return { - ok: true, - status: 200, - json: async () => workerSession, - } as Response; - } - - if (url === "/api/sessions?fresh=true") { - return { - ok: false, - status: 500, - json: async () => ({}), - } as Response; - } - - if (url === "/api/sessions?project=my-app&orchestratorOnly=true&fresh=true") { - return { - ok: true, - status: 200, - json: async () => ({ orchestratorId: "my-app-orchestrator" }), - } as Response; - } - - throw new Error(`Unexpected fetch: ${url}`); - }) as typeof fetch; - - const { default: SessionPage } = await import("./page"); - - render(); - await flushAsyncWork(); - - const latestProps = sessionDetailSpy.mock.lastCall?.[0] as { - sidebarError?: boolean; - sidebarLoading?: boolean; - sidebarSessions?: DashboardSession[] | null; - }; - - expect(latestProps.sidebarLoading).toBe(false); - expect(latestProps.sidebarError).toBe(true); - expect(latestProps.sidebarSessions).toEqual([]); - }); - - it("applies mux snapshots that arrive before the initial sidebar fetch resolves", async () => { - const workerSession = makeWorkerSession(); - const muxPatchedLastActivityAt = "2026-04-14T12:00:00.000Z"; - let resolveSidebarSessions: ((value: Response) => void) | null = null; - - mockMuxState.current = { - status: "connected", - sessions: [ - { - id: "worker-1", - status: "working", - activity: "ready", - attentionLevel: "pending", - lastActivityAt: muxPatchedLastActivityAt, - }, - ], - }; - - global.fetch = vi.fn((input: RequestInfo | URL) => { - const url = String(input); - if (url === "/api/projects") { - return Promise.resolve({ - ok: true, - status: 200, - json: async () => ({ - projects: [{ id: "my-app", name: "My App", sessionPrefix: "my-app" }], - }), - } as Response); - } - - if (url === "/api/sessions/worker-1") { - return Promise.resolve({ - ok: true, - status: 200, - json: async () => workerSession, - } as Response); - } - - if (url === "/api/sessions?fresh=true") { - return new Promise((resolve) => { - resolveSidebarSessions = resolve; - }); - } - - if (url === "/api/sessions?project=my-app&orchestratorOnly=true&fresh=true") { - return Promise.resolve({ - ok: true, - status: 200, - json: async () => ({ orchestratorId: "my-app-orchestrator" }), - } as Response); - } - - return Promise.reject(new Error(`Unexpected fetch: ${url}`)); - }) as typeof fetch; - - const { default: SessionPage } = await import("./page"); - - render(); - await flushAsyncWork(); - - await act(async () => { - resolveSidebarSessions?.({ - ok: true, - status: 200, - json: async () => ({ sessions: [workerSession] }), - } as Response); - await Promise.resolve(); - }); - - const latestProps = sessionDetailSpy.mock.lastCall?.[0] as { - sidebarSessions?: DashboardSession[] | null; - }; - - expect(latestProps.sidebarSessions).toEqual([ - { - ...workerSession, - activity: "ready", - lastActivityAt: muxPatchedLastActivityAt, - }, - ]); - }); it("redirects the legacy session URL to the project-scoped route for clean projects", async () => { mockPathname = "/sessions/worker-1"; diff --git a/packages/web/src/app/sessions/[id]/page.tsx b/packages/web/src/app/sessions/[id]/page.tsx index f16c5c365..a69d80e9c 100644 --- a/packages/web/src/app/sessions/[id]/page.tsx +++ b/packages/web/src/app/sessions/[id]/page.tsx @@ -5,10 +5,7 @@ import { useParams, usePathname, useRouter } from "next/navigation"; import { ACTIVITY_STATE, SESSION_STATUS, isOrchestratorSession } from "@aoagents/ao-core/types"; import { SessionDetail } from "@/components/SessionDetail"; import { ErrorDisplay } from "@/components/ErrorDisplay"; -import { - ProjectSidebar, - type ProjectSidebarOrchestrator, -} from "@/components/ProjectSidebar"; +import { ProjectSidebar, type ProjectSidebarOrchestrator } from "@/components/ProjectSidebar"; import { useMediaQuery, MOBILE_BREAKPOINT } from "@/hooks/useMediaQuery"; import { type DashboardSession, @@ -938,11 +935,6 @@ export default function SessionPage() { orchestratorZones={zoneCounts ?? undefined} projectOrchestratorId={projectOrchestratorId} projects={projects} - sidebarSessions={sidebarSessions} - sidebarOrchestrators={sidebarOrchestrators} - sidebarLoading={sidebarSessions === null} - sidebarError={sidebarError} - onRetrySidebar={fetchSidebarSessions} /> ); } diff --git a/packages/web/src/components/CopyDebugBundleButton.tsx b/packages/web/src/components/CopyDebugBundleButton.tsx index a58cc9607..847494c67 100644 --- a/packages/web/src/components/CopyDebugBundleButton.tsx +++ b/packages/web/src/components/CopyDebugBundleButton.tsx @@ -108,9 +108,10 @@ export function CopyDebugBundleButton({ projectId }: CopyDebugBundleButtonProps) return ( ); } diff --git a/packages/web/src/components/Dashboard.tsx b/packages/web/src/components/Dashboard.tsx index 3b790b182..c270831f1 100644 --- a/packages/web/src/components/Dashboard.tsx +++ b/packages/web/src/components/Dashboard.tsx @@ -18,14 +18,15 @@ import { AttentionZone } from "./AttentionZone"; import { DynamicFavicon, countNeedingAttention } from "./DynamicFavicon"; import { useSessionEvents } from "@/hooks/useSessionEvents"; import { useMuxOptional } from "@/providers/MuxProvider"; -import { ProjectSidebar } from "./ProjectSidebar"; import type { ProjectInfo } from "@/lib/project-name"; import { EmptyState } from "./Skeleton"; import { ToastProvider, useToast } from "./Toast"; import { ConnectionBar } from "./ConnectionBar"; import { UpdateBanner } from "./UpdateBanner"; import { CopyDebugBundleButton } from "./CopyDebugBundleButton"; -import { SidebarContext } from "./workspace/SidebarContext"; +import { SidebarContext, useSidebarContext } from "./workspace/SidebarContext"; +import { ProjectSidebar } from "./ProjectSidebar"; +import { isOrchestratorSession } from "@aoagents/ao-core/types"; import { projectDashboardPath, projectSessionPath } from "@/lib/routes"; import { BottomSheet } from "./BottomSheet"; @@ -175,6 +176,25 @@ function DashboardInner({ if (!projectId) return sessions; return sessions.filter((s) => s.projectId === projectId); }, [sessions, projectId]); + + const allSessionPrefixes = useMemo( + () => projects.map((p) => p.sessionPrefix ?? p.id), + [projects], + ); + + const sidebarOrchestrators = useMemo( + () => + sessions + .filter((s) => + isOrchestratorSession( + s, + projects.find((p) => p.id === s.projectId)?.sessionPrefix ?? s.projectId, + allSessionPrefixes, + ), + ) + .map((s) => ({ id: s.id, projectId: s.projectId })), + [sessions, projects, allSessionPrefixes], + ); const connectionStatus: "connected" | "reconnecting" | "disconnected" = mux?.status === "disconnected" ? "disconnected" @@ -195,9 +215,20 @@ function DashboardInner({ useState(orchestratorLinks); const [spawningProjectIds, setSpawningProjectIds] = useState([]); const [spawnErrors, setSpawnErrors] = useState>({}); - const [sidebarCollapsed, setSidebarCollapsed] = useState(false); - const [mobileMenuOpen, setMobileMenuOpen] = useState(false); const isMobile = useMediaQuery(MOBILE_BREAKPOINT); + // Detect if a parent layout already owns the sidebar — if so, skip rendering our own. + const parentCtx = useSidebarContext(); + const isInsideLayout = parentCtx !== null; + const [sidebarCollapsed, setSidebarCollapsed] = useState(false); + const [mobileSidebarOpen, setMobileSidebarOpen] = useState(false); + const handleToggleSidebar = useCallback(() => { + if (isInsideLayout && parentCtx) { parentCtx.onToggleSidebar(); return; } + if (isMobile) { + setMobileSidebarOpen((v) => !v); + } else { + setSidebarCollapsed((v) => !v); + } + }, [isMobile, isInsideLayout, parentCtx]); const [collapsedZones, setCollapsedZones] = useState>( () => new Set(["done", "working"]), ); @@ -250,10 +281,6 @@ function DashboardInner({ document.title = needsAttention > 0 ? `${label} (${needsAttention} need attention)` : label; }, [attentionLevels, projectName]); - useEffect(() => { - setMobileMenuOpen(false); - }, [searchParams]); - const grouped = useMemo(() => { const zones: Record = { merge: [], @@ -516,292 +543,336 @@ function DashboardInner({ return next; }); }; - const handleToggleSidebar = () => { - if (typeof window !== "undefined" && window.innerWidth < 768) { - setMobileMenuOpen((current) => !current); - } else { - setSidebarCollapsed((current) => !current); - } - }; - return ( - - <> - - -
-
- +
+ Agent Orchestrator +
+ {showHeaderProjectLabel ? ( + <> +
+ +
+ +
+

Dashboard

+

+ Live agent sessions, pull requests, and merge status. +

+
+ +
+ {loadErrorBanner} + {anyRateLimited && !rateLimitDismissed && ( +
+ - ) : ( - - )} - -
-
- {showHeaderProjectLabel ? ( - <> -
+
+ ); + + const bottomSheet = ( + + ); + + if (isInsideLayout) { + return ( + <> + + + {mainPanel} + {bottomSheet} + ); + } + + return ( + + + +
+
+
+ setSidebarCollapsed((v) => !v)} + onMobileClose={() => setMobileSidebarOpen(false)} + /> +
+ {mobileSidebarOpen && ( +
setMobileSidebarOpen(false)} + /> + )} + {mainPanel} +
+
+ {bottomSheet} ); } diff --git a/packages/web/src/components/OrchestratorSelector.tsx b/packages/web/src/components/OrchestratorSelector.tsx deleted file mode 100644 index a178a7a6a..000000000 --- a/packages/web/src/components/OrchestratorSelector.tsx +++ /dev/null @@ -1,273 +0,0 @@ -"use client"; - -import { useState, useRef } from "react"; -import { useRouter } from "next/navigation"; -import Link from "next/link"; -import { cn } from "@/lib/cn"; -import { projectDashboardPath, projectSessionPath } from "@/lib/routes"; - -export interface Orchestrator { - id: string; - projectId: string; - projectName: string; - status: string; - activity: string | null; - createdAt: string | null; - lastActivityAt: string | null; -} - -interface OrchestratorSelectorProps { - orchestrators: Orchestrator[]; - projectId: string; - projectName: string; - error: string | null; -} - -function formatRelativeTime(isoDate: string | null): string { - if (!isoDate) return "Unknown"; - const date = new Date(isoDate); - const timestamp = date.getTime(); - // Guard against invalid dates (NaN) and future timestamps - if (!Number.isFinite(timestamp)) return "Unknown"; - const now = new Date(); - const diffMs = now.getTime() - timestamp; - // Handle future timestamps - if (diffMs < 0) return "Just now"; - const diffMins = Math.floor(diffMs / 60000); - const diffHours = Math.floor(diffMins / 60); - const diffDays = Math.floor(diffHours / 24); - - if (diffMins < 1) return "Just now"; - if (diffMins < 60) return `${diffMins}m ago`; - if (diffHours < 24) return `${diffHours}h ago`; - return `${diffDays}d ago`; -} - -function getStatusColor(status: string): string { - switch (status) { - case "working": - return "var(--color-status-working)"; - case "spawning": - return "var(--color-status-attention)"; - case "pr_open": - case "review_pending": - case "approved": - case "mergeable": - return "var(--color-status-ready)"; - case "ci_failed": - case "changes_requested": - return "var(--color-status-error)"; - case "merged": - case "done": - case "killed": - case "terminated": - return "var(--color-text-tertiary)"; - default: - return "var(--color-text-secondary)"; - } -} - -function getActivityLabel(activity: string | null): string { - if (!activity) return ""; - switch (activity) { - case "active": - return "Active"; - case "ready": - return "Ready"; - case "idle": - return "Idle"; - case "waiting_input": - return "Waiting"; - case "blocked": - return "Blocked"; - case "exited": - return "Exited"; - default: - return activity; - } -} - -export function OrchestratorSelector({ - orchestrators, - projectId, - projectName, - error, -}: OrchestratorSelectorProps) { - const router = useRouter(); - const [isSpawning, setIsSpawning] = useState(false); - const [spawnError, setSpawnError] = useState(null); - const spawnLockRef = useRef(false); - - const handleSpawnNew = async () => { - // Synchronous re-entrancy guard: React state updates are async, - // so two clicks before rerender would fire two POSTs without this. - if (spawnLockRef.current) return; - spawnLockRef.current = true; - setIsSpawning(true); - setSpawnError(null); - - try { - const response = await fetch("/api/orchestrators", { - method: "POST", - headers: { "Content-Type": "application/json" }, - body: JSON.stringify({ projectId }), - }); - - if (!response.ok) { - const data = await response.json(); - throw new Error(data.error || "Failed to spawn orchestrator"); - } - - const data = await response.json(); - router.push(projectSessionPath(projectId, data.orchestrator.id)); - } catch (err) { - setSpawnError(err instanceof Error ? err.message : "Failed to spawn orchestrator"); - } finally { - setIsSpawning(false); - spawnLockRef.current = false; - } - }; - - if (error) { - return ( -
-
-

Error

-

{error}

- - Go to Dashboard - -
-
- ); - } - - return ( -
- {/* Header */} -
-
-
-

- {projectName} -

-

Project orchestrator

-
- - Dashboard - -
-
- - {/* Content */} -
-
- {/* Info banner */} -
-

- AO keeps one main orchestrator per project. Opening or spawning here reuses that - canonical session instead of creating another duplicate. -

-
- - {/* Existing orchestrators */} -
-

- Main Session -

-
- {orchestrators.map((orch) => ( - -
-
-
-
{orch.id}
-
- {orch.status.replace(/_/g, " ")} - {orch.activity && ( - <> - · - {getActivityLabel(orch.activity)} - - )} -
-
-
-
-
Created {formatRelativeTime(orch.createdAt)}
- {orch.lastActivityAt && ( -
Active {formatRelativeTime(orch.lastActivityAt)}
- )} -
- - ))} -
-
- - {/* Start new section */} -
-

- Open Orchestrator -

- - {spawnError && ( -

{spawnError}

- )} -
-
-
-
- ); -} diff --git a/packages/web/src/components/ProjectSidebar.tsx b/packages/web/src/components/ProjectSidebar.tsx index 2b6b78e1c..c6b29a7f6 100644 --- a/packages/web/src/components/ProjectSidebar.tsx +++ b/packages/web/src/components/ProjectSidebar.tsx @@ -1,11 +1,11 @@ "use client"; import Link from "next/link"; -import { useState, useEffect, useMemo, useRef } from "react"; +import { useState, useEffect, useMemo, useRef, useCallback, memo } from "react"; import { useRouter } from "next/navigation"; import { cn } from "@/lib/cn"; import type { ProjectInfo } from "@/lib/project-name"; -import { getAttentionLevel, type DashboardSession, type AttentionLevel } from "@/lib/types"; +import { getAttentionLevel, type DashboardSession } from "@/lib/types"; import { isOrchestratorSession } from "@aoagents/ao-core/types"; import { getSessionTitle, humanizeBranch } from "@/lib/format"; import { usePopoverClamp } from "@/hooks/usePopoverClamp"; @@ -42,7 +42,7 @@ interface ProjectSidebarProps { type SessionDotLevel = "respond" | "review" | "action" | "pending" | "working" | "merge" | "done"; -function SessionDot({ level }: { level: SessionDotLevel }) { +const SessionDot = memo(function SessionDot({ level }: { level: SessionDotLevel }) { return (
); -} +}); // ProjectSidebar consumes `getAttentionLevel()` without passing a mode, // so the function defaults to "detailed" and `action` never appears here @@ -70,15 +70,41 @@ function loadShowSessionId(): boolean { } } -const LEVEL_LABELS: Record = { - working: "working", - pending: "pending", - review: "review", - respond: "respond", - action: "action", - merge: "merge", - done: "done", -}; +const SHOW_KILLED_KEY = "ao:sidebar:show-killed"; +const SHOW_DONE_KEY = "ao:sidebar:show-done"; +const EXPANDED_PROJECTS_KEY = "ao:sidebar:expanded-projects"; + +function loadShowKilled(): boolean { + if (typeof window === "undefined") return false; + try { + return window.sessionStorage.getItem(SHOW_KILLED_KEY) === "true"; + } catch { + return false; + } +} + +function loadShowDone(): boolean { + if (typeof window === "undefined") return false; + try { + return window.sessionStorage.getItem(SHOW_DONE_KEY) === "true"; + } catch { + return false; + } +} + +function loadExpandedProjects(): Set | null { + if (typeof window === "undefined") return null; + try { + const raw = window.sessionStorage.getItem(EXPANDED_PROJECTS_KEY); + if (!raw) return null; + const parsed = JSON.parse(raw); + if (Array.isArray(parsed)) return new Set(parsed); + return null; + } catch { + return null; + } +} + export function ProjectSidebar(props: ProjectSidebarProps) { if (props.projects.length === 0) { @@ -87,6 +113,102 @@ export function ProjectSidebar(props: ProjectSidebarProps) { return ; } +interface SessionRowProps { + session: DashboardSession; + level: SessionDotLevel; + isActive: boolean; + showSessionId: boolean; + pendingRename: string | undefined; + onNavigate: (href: string, session: DashboardSession) => void; + onStartRename: (session: DashboardSession, title: string) => void; +} + +const SessionRow = memo(function SessionRow({ + session, + level, + isActive, + showSessionId, + pendingRename, + onNavigate, + onStartRename, +}: SessionRowProps) { + const effectiveDisplayName = + pendingRename !== undefined + ? pendingRename + : session.displayNameUserSet + ? (session.displayName ?? "") + : ""; + const title = + effectiveDisplayName !== "" + ? effectiveDisplayName + : (session.branch ?? getSessionTitle(session)); + const sessionHref = projectSessionPath(session.projectId, session.id); + + return ( +
+ { + if (e.metaKey || e.ctrlKey || e.shiftKey || e.button === 1) return; + e.preventDefault(); + onNavigate(sessionHref, session); + }} + className="project-sidebar__sess-link flex flex-1 min-w-0 items-center gap-[7px]" + aria-current={isActive ? "page" : undefined} + aria-label={`Open ${title}`} + > + +
+ + {title} + + {showSessionId ? ( +
+ {session.id} +
+ ) : null} +
+
+ +
+ ); +}); + function ProjectSidebarEmpty({ collapsed = false }: { collapsed?: boolean }) { const [addProjectOpen, setAddProjectOpen] = useState(false); @@ -162,13 +284,15 @@ function ProjectSidebarInner({ onMobileClose, }: ProjectSidebarProps) { const router = useRouter(); - const isLoading = loading || sessions === null; + const _isLoading = loading || sessions === null; const [expandedProjects, setExpandedProjects] = useState>( - () => new Set(activeProjectId && activeProjectId !== "all" ? [activeProjectId] : []), + () => + loadExpandedProjects() ?? + new Set(activeProjectId && activeProjectId !== "all" ? [activeProjectId] : []), ); - const [showKilled, setShowKilled] = useState(false); - const [showDone, setShowDone] = useState(false); + const [showKilled, setShowKilled] = useState(loadShowKilled); + const [showDone, setShowDone] = useState(loadShowDone); const [showSessionId, setShowSessionId] = useState(loadShowSessionId); // Inline session-rename state. Only one row is edited at a time. `pendingRenames` // mirrors the in-flight / just-saved value so the new label appears immediately @@ -198,6 +322,30 @@ function ProjectSidebarInner({ } }, [showSessionId]); + useEffect(() => { + try { + window.sessionStorage.setItem(SHOW_KILLED_KEY, String(showKilled)); + } catch { + // sessionStorage unavailable — accept in-memory state. + } + }, [showKilled]); + + useEffect(() => { + try { + window.sessionStorage.setItem(SHOW_DONE_KEY, String(showDone)); + } catch { + // sessionStorage unavailable — accept in-memory state. + } + }, [showDone]); + + useEffect(() => { + try { + window.sessionStorage.setItem(EXPANDED_PROJECTS_KEY, JSON.stringify([...expandedProjects])); + } catch { + // sessionStorage unavailable — accept in-memory state. + } + }, [expandedProjects]); + // Close the settings popover on outside click or Escape. useEffect(() => { if (!settingsOpen) return; @@ -266,20 +414,37 @@ function ProjectSidebarInner({ [visibleProjects], ); - // The API (selectPreferredOrchestratorId) sends at most one entry per - // project, so collapsing into a Map keyed by projectId is lossless. If a - // future API change starts emitting multiples, the last one wins here. const orchestratorByProject = useMemo( () => new Map((orchestrators ?? []).map((o) => [o.projectId, o])), [orchestrators], ); + // Stable ref so sessionsByProject can read latest sessions without depending + // on the array reference (which changes every SSE tick even when content is unchanged). + const sessionsRef = useRef(sessions); + sessionsRef.current = sessions; + + // Content-based key — only changes when session IDs, statuses, or projects change. + // Used as the sole sessions-related dependency of sessionsByProject below. + const sessionsKey = useMemo( + () => + (sessions ?? []) + .map( + (s) => + `${s.id}:${s.status}:${s.activity ?? ""}:${s.projectId}:${s.displayName ?? ""}:${s.displayNameUserSet ? "1" : "0"}:${s.branch ?? ""}:${s.issueTitle ?? ""}:${s.pr?.title ?? ""}:${s.summary ?? ""}`, + ) + .join("|"), + [sessions], + ); + const sessionsByProject = useMemo(() => { const map = new Map(); // Build a set of valid project IDs to filter sessions strictly const validProjectIds = new Set(visibleProjects.map((p) => p.id)); - for (const s of sessions ?? []) { + // Read via ref so this memo only reruns when sessionsKey changes (content + // changed), not when sessions gets a new array reference with identical data. + for (const s of sessionsRef.current ?? []) { // Only include sessions whose projectId matches a configured project if (!validProjectIds.has(s.projectId)) continue; if (isOrchestratorSession(s, prefixByProject.get(s.projectId), allPrefixes)) continue; @@ -295,7 +460,8 @@ function ProjectSidebarInner({ map.set(s.projectId, list); } return map; - }, [sessions, prefixByProject, allPrefixes, visibleProjects, showKilled, showDone]); + }, [sessionsKey, prefixByProject, allPrefixes, visibleProjects, showKilled, showDone]); + // Clear an optimistic rename once the prop session.displayName catches up. // Without this, we'd keep masking the server value forever after a save. @@ -313,18 +479,23 @@ function ProjectSidebarInner({ if (changed) setPendingRenames(next); }, [sessions, pendingRenames]); - const startRename = (session: DashboardSession, currentTitle: string) => { - // Prefer the in-flight optimistic value over the prop — if the user opens - // rename while a previous PATCH is still propagating, the prop still shows - // the pre-rename value but we want the input to start from the latest. - // Auto-derived displayName isn't pre-filled (user-set flag absent) — start - // from the live title so the user types over the visible label. - const pending = pendingRenames.get(session.id); - const initial = - pending ?? (session.displayNameUserSet ? (session.displayName ?? "") : ""); - setEditingSessionId(session.id); - setEditingValue(initial || currentTitle); - }; + const pendingRenamesRef = useRef(pendingRenames); + pendingRenamesRef.current = pendingRenames; + + const startRename = useCallback( + (session: DashboardSession, currentTitle: string) => { + // Prefer the in-flight optimistic value over the prop — if the user opens + // rename while a previous PATCH is still propagating, the prop still shows + // the pre-rename value but we want the input to start from the latest. + // Auto-derived displayName isn't pre-filled (user-set flag absent) — start + // from the live title so the user types over the visible label. + const pending = pendingRenamesRef.current.get(session.id); + const initial = pending ?? (session.displayNameUserSet ? (session.displayName ?? "") : ""); + setEditingSessionId(session.id); + setEditingValue(initial || currentTitle); + }, + [], + ); const cancelRename = () => { setEditingSessionId(null); @@ -367,17 +538,20 @@ function ProjectSidebarInner({ } }; - const navigate = (url: string, session?: DashboardSession) => { - if (session) { - try { - sessionStorage.setItem(`ao-session-nav:${session.id}`, JSON.stringify(session)); - } catch { - // sessionStorage unavailable — silent fallback + const navigate = useCallback( + (url: string, session?: DashboardSession) => { + if (session) { + try { + sessionStorage.setItem(`ao-session-nav:${session.id}`, JSON.stringify(session)); + } catch { + // sessionStorage unavailable — silent fallback + } } - } - router.push(url); - onMobileClose?.(); - }; + router.push(url); + onMobileClose?.(); + }, + [router, onMobileClose], + ); const toggleExpand = (projectId: string) => { setExpandedProjects((prev) => { @@ -544,6 +718,16 @@ function ProjectSidebarInner({ {/* Project tree */}
+ {sessions === null ? ( +
+ {Array.from({ length: 4 }, (_, i) => ( +
+
+
+
+ ))} +
+ ) : null} {visibleProjects.map((project) => { const workerSessions = sessionsByProject.get(project.id) ?? []; const isExpanded = expandedProjects.has(project.id); @@ -553,8 +737,14 @@ function ProjectSidebarInner({ // sessionsByProject already applies the showDone filter consistently. const visibleSessions = workerSessions; const hasActiveSessions = visibleSessions.length > 0; - const orchestratorLink = orchestratorByProject.get(project.id) ?? null; + // Look up the full session object so navigate() can cache it in + // sessionStorage — prevents the "Session unavailable" flash on + // first load. Orchestrators are filtered out of sessionsByProject + // but still present in the raw sessions prop. + const orchestratorSession = orchestratorLink + ? (sessions?.find((s) => s.id === orchestratorLink.id) ?? null) + : null; return (
@@ -625,7 +815,7 @@ function ProjectSidebarInner({ hasActiveSessions && "project-sidebar__proj-badge--active", )} > - {workerSessions.length} + {sessionsByProject.get(project.id)?.length ?? 0} )} @@ -656,14 +846,17 @@ function ProjectSidebarInner({ ) : null} - {/* Orchestrator button */} {!isDegraded && orchestratorLink && ( - { + if (e.metaKey || e.ctrlKey || e.shiftKey || e.button === 1) return; + e.preventDefault(); e.stopPropagation(); - onMobileClose?.(); + navigate( + projectSessionPath(project.id, orchestratorLink.id), + orchestratorSession ?? undefined, + ); }} className="project-sidebar__proj-action" aria-label={`Open ${project.name} orchestrator`} @@ -683,7 +876,7 @@ function ProjectSidebarInner({ - + )}
{ setProjectMenuOpenId(null); - navigate(projectSessionPath(project.id, orchestratorLink.id)); + navigate( + projectSessionPath(project.id, orchestratorLink.id), + orchestratorSession ?? undefined, + ); }} > Open orchestrator @@ -768,7 +964,7 @@ function ProjectSidebarInner({ {/* Sessions */} {!isDegraded && isExpanded && (
- {isLoading ? ( + {sessions === null ? (
{Array.from({ length: 3 }, (_, index) => (
{ const level = getAttentionLevel(session); const isSessionActive = activeSessionId === session.id; - // Display precedence: optimistic rename (just-saved value) - // → user-set displayName → branch → fallback chain. - // Auto-derived displayName (displayNameUserSet=false) is - // intentionally skipped here so PR/issue titles surfaced - // by getSessionTitle aren't shadowed — mirrors the gate in - // format.ts:getSessionTitle. - const pending = pendingRenames.get(session.id); - const effectiveDisplayName = - pending !== undefined - ? pending - : session.displayNameUserSet - ? (session.displayName ?? "") - : ""; - const title = - effectiveDisplayName !== "" - ? effectiveDisplayName - : (session.branch ?? getSessionTitle(session)); - const sessionHref = projectSessionPath(project.id, session.id); const isEditing = editingSessionId === session.id; if (isEditing) { return ( @@ -839,76 +1017,16 @@ function ProjectSidebarInner({ ); } return ( - + session={session} + level={level} + isActive={isSessionActive} + showSessionId={showSessionId} + pendingRename={pendingRenames.get(session.id)} + onNavigate={navigate} + onStartRename={startRename} + /> ); }) ) : error ? ( @@ -923,7 +1041,9 @@ function ProjectSidebarInner({
) : ( -
No sessions shown
+
+ No active sessions +
)}
)} diff --git a/packages/web/src/components/SessionDetail.tsx b/packages/web/src/components/SessionDetail.tsx index d904d559d..e94d2d972 100644 --- a/packages/web/src/components/SessionDetail.tsx +++ b/packages/web/src/components/SessionDetail.tsx @@ -11,10 +11,9 @@ import { import dynamic from "next/dynamic"; import { getSessionTitle } from "@/lib/format"; import type { ProjectInfo } from "@/lib/project-name"; -import { SidebarContext } from "./workspace/SidebarContext"; +import { useSidebarContext } from "./workspace/SidebarContext"; import { projectDashboardPath, projectSessionPath } from "@/lib/routes"; -import { ProjectSidebar, type ProjectSidebarOrchestrator } from "./ProjectSidebar"; import { MobileBottomNav } from "./MobileBottomNav"; import { SessionDetailHeader, type OrchestratorZones } from "./SessionDetailHeader"; import { SessionEndedSummary } from "./SessionEndedSummary"; @@ -40,11 +39,6 @@ interface SessionDetailProps { orchestratorZones?: OrchestratorZones; projectOrchestratorId?: string | null; projects?: ProjectInfo[]; - sidebarSessions?: DashboardSession[] | null; - sidebarOrchestrators?: ProjectSidebarOrchestrator[]; - sidebarLoading?: boolean; - sidebarError?: boolean; - onRetrySidebar?: () => void; } export function SessionDetail({ @@ -53,19 +47,14 @@ export function SessionDetail({ orchestratorZones, projectOrchestratorId = null, projects = [], - sidebarSessions = [], - sidebarOrchestrators, - sidebarLoading = false, - sidebarError = false, - onRetrySidebar, }: SessionDetailProps) { const router = useRouter(); const searchParams = useSearchParams(); const isMobile = useMediaQuery(MOBILE_BREAKPOINT); + const sidebarCtx = useSidebarContext(); const startFullscreen = searchParams.get("fullscreen") === "true"; - const [sidebarCollapsed, setSidebarCollapsed] = useState(false); - const [mobileSidebarOpen, setMobileSidebarOpen] = useState(false); const [showTerminal, setShowTerminal] = useState(false); + const [relaunchError, setRelaunchError] = useState(null); const pr = session.pr; const terminalEnded = isDashboardSessionTerminal(session); const isRestorable = isDashboardSessionRestorable(session); @@ -119,11 +108,52 @@ export function SessionDetail({ } }, [session.id]); + const handleRelaunchClean = useCallback(async () => { + const confirmed = window.confirm( + "This will discard the current orchestrator's conversation and state. Continue?", + ); + if (!confirmed) return; + setRelaunchError(null); + try { + const res = await fetch("/api/orchestrators", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ projectId: session.projectId, clean: true }), + }); + if (!res.ok) { + // Surface server-side errors. Note: a failure here may indicate the + // existing orchestrator was killed but respawn failed — the banner + // tells the user the previous session was terminated so they don't + // assume the page they're looking at is still live. + let message = ""; + try { + const data = (await res.json()) as { error?: string }; + message = data.error ?? ""; + } catch { + message = await res.text().catch(() => ""); + } + throw new Error(message || `HTTP ${res.status}`); + } + // Hard-navigate to the freshly spawned orchestrator's session page. + // Orchestrator session IDs are fixed per project, so this is the same + // path in practice — but reading from the response keeps us correct if + // the contract ever changes (and a hard nav forces the terminal + // WebSocket to reconnect against the new tmux session). + const data = (await res.json()) as { orchestrator?: { id: string } }; + const newId = data.orchestrator?.id ?? session.id; + window.location.href = projectSessionPath(session.projectId, newId); + } catch (err) { + const message = err instanceof Error ? err.message : "Failed to relaunch orchestrator"; + console.error("Failed to relaunch orchestrator:", err); + setRelaunchError(message); + } + }, [session.id, session.projectId]); + const orchestratorHref = useMemo(() => { - if (isOrchestrator) return projectSessionPath(session.projectId, session.id); + if (isOrchestrator) return null; if (projectOrchestratorId) return projectSessionPath(session.projectId, projectOrchestratorId); return null; - }, [isOrchestrator, projectOrchestratorId, session.id, session.projectId]); + }, [isOrchestrator, projectOrchestratorId, session.projectId]); useEffect(() => { const frame = window.requestAnimationFrame(() => setShowTerminal(true)); @@ -133,104 +163,86 @@ export function SessionDetail({ }; }, [session.id]); - const handleToggleSidebar = useCallback(() => { - if (isMobile) { - setMobileSidebarOpen((v) => !v); - } else { - setSidebarCollapsed((v) => !v); - } - }, [isMobile]); - return ( - -
- - -
- {projects.length > 0 ? ( -
- setSidebarCollapsed((current) => !current)} - onMobileClose={() => setMobileSidebarOpen(false)} - /> -
- ) : null} - {mobileSidebarOpen && ( -
setMobileSidebarOpen(false)} /> - )} - -
-
-
- {!showTerminal ? ( -
- ) : terminalEnded ? ( - - ) : ( - - )} +
+ {})} + onRestore={handleRestore} + onKill={handleKill} + onRelaunchClean={isOrchestrator ? handleRelaunchClean : undefined} + /> +
+ {relaunchError ? ( +
+
+
+ Relaunch failed: {relaunchError} +
+ The previous orchestrator may already be terminated. Try again from the + project dashboard. +
-
+ +
+ ) : null} +
+ {!showTerminal ? ( +
+ ) : terminalEnded ? ( + + ) : ( + + )}
- -
- +
+ +
); } diff --git a/packages/web/src/components/SessionDetailHeader.tsx b/packages/web/src/components/SessionDetailHeader.tsx index f6090efcd..25432df01 100644 --- a/packages/web/src/components/SessionDetailHeader.tsx +++ b/packages/web/src/components/SessionDetailHeader.tsx @@ -36,6 +36,7 @@ interface SessionDetailHeaderProps { onToggleSidebar: () => void; onRestore: () => void; onKill: () => void; + onRelaunchClean?: () => void; } function normalizeActivityLabelForClass(activityLabel: string): string { @@ -80,6 +81,7 @@ export function SessionDetailHeader({ onToggleSidebar, onRestore, onKill, + onRelaunchClean, }: SessionDetailHeaderProps) { const pr = session.pr; const allGreen = pr ? isPRMergeReady(pr) : false; @@ -302,6 +304,30 @@ export function SessionDetailHeader({ ) : null} + {isOrchestrator && onRelaunchClean ? ( + + ) : null} + {orchestratorHref ? ( PR #{pr.number}: {pr.title} - - +{pr.additions}{" "} - -{pr.deletions} - + {showDiffStats ? ( + + +{pr.additions}{" "} + -{pr.deletions} + + ) : null} {fileCount > 0 ? ( {fileCount} file{fileCount !== 1 ? "s" : ""} diff --git a/packages/web/src/components/__tests__/Dashboard.emptyState.test.tsx b/packages/web/src/components/__tests__/Dashboard.emptyState.test.tsx index 418789a74..8055feced 100644 --- a/packages/web/src/components/__tests__/Dashboard.emptyState.test.tsx +++ b/packages/web/src/components/__tests__/Dashboard.emptyState.test.tsx @@ -165,6 +165,7 @@ describe("Dashboard empty state", () => { expect(screen.queryByRole("alert")).not.toBeInTheDocument(); }); + it("mounts the sidebar empty state on a fresh install with zero projects", () => { render(); diff --git a/packages/web/src/components/__tests__/OrchestratorSelector.test.tsx b/packages/web/src/components/__tests__/OrchestratorSelector.test.tsx deleted file mode 100644 index 28aefde42..000000000 --- a/packages/web/src/components/__tests__/OrchestratorSelector.test.tsx +++ /dev/null @@ -1,241 +0,0 @@ -import { render, screen, fireEvent, waitFor } from "@testing-library/react"; -import { beforeEach, describe, expect, it, vi } from "vitest"; -import { OrchestratorSelector } from "../OrchestratorSelector"; - -const mockPush = vi.fn(); -vi.mock("next/navigation", () => ({ - useRouter: () => ({ push: mockPush }), -})); - -const mockOrchestrators = [ - { - id: "app-orchestrator", - projectId: "my-project", - projectName: "My Project", - status: "working", - activity: "active", - createdAt: new Date(Date.now() - 3600000).toISOString(), // 1 hour ago - lastActivityAt: new Date(Date.now() - 300000).toISOString(), // 5 min ago - }, -]; - -const defaultProps = { - orchestrators: mockOrchestrators, - projectId: "my-project", - projectName: "My Project", - error: null, -}; - -describe("OrchestratorSelector", () => { - beforeEach(() => { - vi.clearAllMocks(); - mockPush.mockClear(); - global.fetch = vi.fn(); - }); - - it("renders orchestrator list", () => { - render(); - - expect(screen.getByText("app-orchestrator")).toBeInTheDocument(); - }); - - it("displays project name in header", () => { - render(); - - expect(screen.getByText("My Project")).toBeInTheDocument(); - expect(screen.getByText("Project orchestrator")).toBeInTheDocument(); - expect(screen.getByRole("link", { name: "Dashboard" })).toHaveAttribute( - "href", - "/projects/my-project", - ); - }); - - it("explains that orchestrator opening reuses the canonical session", () => { - render(); - - expect(screen.getByText(/one main orchestrator per project/i)).toBeInTheDocument(); - }); - - it("shows error state", () => { - render(); - - expect(screen.getByText("Error")).toBeInTheDocument(); - expect(screen.getByText("Project not found")).toBeInTheDocument(); - }); - - it("shows open orchestrator button", () => { - render(); - - expect(screen.getByRole("button", { name: /open orchestrator/i })).toBeInTheDocument(); - }); - - it("spawns new orchestrator on button click and navigates", async () => { - const mockFetch = vi.fn().mockResolvedValue({ - ok: true, - json: () => - Promise.resolve({ - orchestrator: { id: "app-orchestrator" }, - }), - }); - global.fetch = mockFetch; - - render(); - - const button = screen.getByRole("button", { name: /open orchestrator/i }); - fireEvent.click(button); - - await waitFor(() => { - expect(mockFetch).toHaveBeenCalledWith("/api/orchestrators", expect.any(Object)); - }); - - await waitFor(() => { - expect(mockPush).toHaveBeenCalledWith("/projects/my-project/sessions/app-orchestrator"); - }); - }); - - it("shows loading state while spawning", async () => { - const mockFetch = vi.fn().mockImplementation( - () => new Promise(() => {}), // Never resolves - ); - global.fetch = mockFetch; - - render(); - - const button = screen.getByRole("button", { name: /open orchestrator/i }); - fireEvent.click(button); - - await waitFor(() => { - expect(screen.getByText(/opening orchestrator/i)).toBeInTheDocument(); - }); - }); - - it("shows error when spawn fails", async () => { - const mockFetch = vi.fn().mockResolvedValue({ - ok: false, - json: () => Promise.resolve({ error: "Failed to spawn" }), - }); - global.fetch = mockFetch; - - render(); - - const button = screen.getByRole("button", { name: /open orchestrator/i }); - fireEvent.click(button); - - await waitFor(() => { - expect(screen.getByText("Failed to spawn")).toBeInTheDocument(); - }); - }); - - it("links to orchestrator session page", () => { - render(); - - const link = screen.getByRole("link", { name: /app-orchestrator/i }); - expect(link).toHaveAttribute("href", "/projects/my-project/sessions/app-orchestrator"); - }); - - it("displays status and activity for each orchestrator", () => { - render(); - - expect(screen.getByText("working")).toBeInTheDocument(); - expect(screen.getByText("Active")).toBeInTheDocument(); - }); - - it("covers relative time for days and status colors/labels", () => { - const wideOrchestrators = [ - { - id: "orch-2", - projectId: "my-project", - projectName: "My Project", - status: "ci_failed", - activity: "waiting_input", - createdAt: new Date(Date.now() - 3600000 * 50).toISOString(), // 2d ago - lastActivityAt: null, - }, - { - id: "orch-3", - projectId: "my-project", - projectName: "My Project", - status: "killed", - activity: "ready", - createdAt: new Date(Date.now() - 1000).toISOString(), // Just now - lastActivityAt: null, - }, - { - id: "orch-4", - projectId: "my-project", - projectName: "My Project", - status: "unknown", - activity: "blocked", - createdAt: new Date().toISOString(), - lastActivityAt: null, - }, - { - id: "orch-5", - projectId: "my-project", - projectName: "My Project", - status: "mergeable", - activity: "exited", - createdAt: new Date().toISOString(), - lastActivityAt: null, - }, - ]; - - render(); - - expect(screen.getByText(/2d ago/)).toBeInTheDocument(); - expect(screen.getAllByText(/Just now/i).length).toBeGreaterThan(0); - expect(screen.getByText(/Waiting/)).toBeInTheDocument(); - expect(screen.getByText(/Ready/)).toBeInTheDocument(); - expect(screen.getByText(/Blocked/)).toBeInTheDocument(); - expect(screen.getByText(/Exited/)).toBeInTheDocument(); - expect(screen.getByText(/ci failed/i)).toBeInTheDocument(); - }); - - describe("formatRelativeTime edge cases", () => { - it("shows Unknown for invalid date strings", () => { - const orchestratorsWithInvalidDate = [ - { - ...mockOrchestrators[0], - createdAt: "not-a-valid-date", - lastActivityAt: null, - }, - ]; - render( - , - ); - - // The "Created Unknown" text should appear for invalid dates - expect(screen.getByText(/Created Unknown/)).toBeInTheDocument(); - }); - - it("shows Just now for future timestamps", () => { - const futureDate = new Date(Date.now() + 60000).toISOString(); // 1 minute in future - const orchestratorsWithFutureDate = [ - { - ...mockOrchestrators[0], - createdAt: futureDate, - lastActivityAt: null, - }, - ]; - render( - , - ); - - // Future timestamps should show "Just now" instead of negative values - expect(screen.getByText(/Created Just now/)).toBeInTheDocument(); - }); - - it("shows Unknown for null dates", () => { - const orchestratorsWithNullDate = [ - { - ...mockOrchestrators[0], - createdAt: null, - lastActivityAt: null, - }, - ]; - render(); - - expect(screen.getByText(/Created Unknown/)).toBeInTheDocument(); - }); - }); -}); diff --git a/packages/web/src/components/__tests__/ProjectSidebar.test.tsx b/packages/web/src/components/__tests__/ProjectSidebar.test.tsx index 13536ee04..6f7150c37 100644 --- a/packages/web/src/components/__tests__/ProjectSidebar.test.tsx +++ b/packages/web/src/components/__tests__/ProjectSidebar.test.tsx @@ -399,6 +399,8 @@ describe("ProjectSidebar", () => { />, ); + // Only the killed-but-still-needs-attention session is counted; the merged + // session is filtered out by sessionsByProject (showDone = false by default). expect(screen.getByRole("button", { name: /^Project One 1$/ })).toBeInTheDocument(); expect( screen.getByRole("link", { name: "Open Runtime missing but needs review" }), diff --git a/packages/web/src/components/__tests__/SessionDetail.desktop.test.tsx b/packages/web/src/components/__tests__/SessionDetail.desktop.test.tsx index d63ad18f8..07a77e48f 100644 --- a/packages/web/src/components/__tests__/SessionDetail.desktop.test.tsx +++ b/packages/web/src/components/__tests__/SessionDetail.desktop.test.tsx @@ -304,6 +304,161 @@ describe("SessionDetail desktop layout", () => { ).not.toBeInTheDocument(); }); + it("renders Relaunch (clean) on live orchestrator sessions and navigates to the new session", async () => { + const confirmSpy = vi.spyOn(window, "confirm").mockReturnValue(true); + const hrefSetter = vi.fn(); + Object.defineProperty(window, "location", { + value: { + ...window.location, + set href(value: string) { + hrefSetter(value); + }, + }, + writable: true, + }); + vi.mocked(global.fetch).mockImplementation(async (input: RequestInfo | URL) => { + const url = typeof input === "string" ? input : input.toString(); + if (url === "/api/orchestrators") { + return { + ok: true, + json: async () => ({ + orchestrator: { id: "my-app-orchestrator", projectId: "my-app" }, + }), + } as Response; + } + return { ok: true, json: async () => ({}), text: async () => "" } as Response; + }); + + render( + , + ); + + const relaunchBtn = within(screen.getByRole("banner")).getByRole("button", { + name: /launch orchestrator \(clean context\)/i, + }); + fireEvent.click(relaunchBtn); + + expect(confirmSpy).toHaveBeenCalled(); + await act(async () => {}); + + expect(global.fetch).toHaveBeenCalledWith("/api/orchestrators", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify({ projectId: "my-app", clean: true }), + }); + expect(hrefSetter).toHaveBeenCalledWith("/projects/my-app/sessions/my-app-orchestrator"); + + confirmSpy.mockRestore(); + }); + + it("keeps Relaunch (clean) visible on terminated orchestrator sessions", () => { + render( + , + ); + + expect( + within(screen.getByRole("banner")).getByRole("button", { + name: /launch orchestrator \(clean context\)/i, + }), + ).toBeInTheDocument(); + }); + + it("surfaces a relaunch error banner when POST fails after confirm", async () => { + const confirmSpy = vi.spyOn(window, "confirm").mockReturnValue(true); + vi.mocked(global.fetch).mockImplementation(async (input: RequestInfo | URL) => { + const url = typeof input === "string" ? input : input.toString(); + if (url === "/api/orchestrators") { + return { + ok: false, + status: 500, + json: async () => ({ error: "kill+respawn failed" }), + text: async () => "kill+respawn failed", + } as Response; + } + return { ok: true, json: async () => ({}), text: async () => "" } as Response; + }); + + render( + , + ); + + fireEvent.click( + within(screen.getByRole("banner")).getByRole("button", { + name: /launch orchestrator \(clean context\)/i, + }), + ); + + const alert = await screen.findByRole("alert"); + expect(alert).toHaveTextContent(/kill\+respawn failed/i); + expect(alert).toHaveTextContent(/previous orchestrator may already be terminated/i); + + fireEvent.click(within(alert).getByRole("button", { name: "Dismiss" })); + expect(screen.queryByRole("alert")).not.toBeInTheDocument(); + + confirmSpy.mockRestore(); + }); + + it("does not render Relaunch (clean) on worker sessions", () => { + render( + , + ); + + expect( + screen.queryByRole("button", { name: /launch orchestrator \(clean context\)/i }), + ).not.toBeInTheDocument(); + }); + it("restores without using router refresh on the client-only session page", async () => { render( { expect(routerRefreshMock).not.toHaveBeenCalled(); }); - it("keeps the desktop orchestrator button on orchestrator session pages", () => { + it("hides the desktop orchestrator button on orchestrator session pages", () => { render( { ); expect( - within(screen.getByRole("banner")).getByRole("link", { name: "Orchestrator" }), - ).toHaveAttribute("href", "/projects/my-app/sessions/my-app-orchestrator"); + within(screen.getByRole("banner")).queryByRole("link", { name: "Orchestrator" }), + ).not.toBeInTheDocument(); expect(screen.getByText("orchestrator")).toBeInTheDocument(); }); @@ -402,8 +557,8 @@ describe("SessionDetail desktop layout", () => { ); expect( - within(screen.getByRole("banner")).getByRole("link", { name: "Orchestrator" }), - ).toHaveAttribute("href", "/projects/my-app/sessions/my-app-orchestrator"); + within(screen.getByRole("banner")).queryByRole("link", { name: "Orchestrator" }), + ).not.toBeInTheDocument(); }); it("routes to the project orchestrator after killing a worker session", async () => { diff --git a/packages/web/src/components/__tests__/SessionDetailPRCard.diffStats.test.tsx b/packages/web/src/components/__tests__/SessionDetailPRCard.diffStats.test.tsx new file mode 100644 index 000000000..0f57e6b70 --- /dev/null +++ b/packages/web/src/components/__tests__/SessionDetailPRCard.diffStats.test.tsx @@ -0,0 +1,40 @@ +import { render, screen } from "@testing-library/react"; +import { describe, expect, it, vi } from "vitest"; +import { SessionDetailPRCard } from "../SessionDetailPRCard"; +import { makePR } from "../../__tests__/helpers"; + +describe("SessionDetailPRCard diff stats", () => { + it("shows diff stats for enriched PRs", () => { + render( + , + ); + + expect(screen.getByText("+629")).toBeInTheDocument(); + expect(screen.getByText("-44")).toBeInTheDocument(); + }); + + it("hides diff stats for unenriched PRs", () => { + render( + , + ); + + expect(screen.queryByText("+629")).not.toBeInTheDocument(); + expect(screen.queryByText("-44")).not.toBeInTheDocument(); + }); +}); diff --git a/packages/web/src/lib/orchestrator-utils.ts b/packages/web/src/lib/orchestrator-utils.ts deleted file mode 100644 index b057654c9..000000000 --- a/packages/web/src/lib/orchestrator-utils.ts +++ /dev/null @@ -1,39 +0,0 @@ -import { - isOrchestratorSession, - isTerminalSession, - type Session, -} from "@aoagents/ao-core/types"; -import type { Orchestrator } from "@/components/OrchestratorSelector"; - -export function getOrchestratorSessionId(project: { sessionPrefix: string }): string { - return `${project.sessionPrefix}-orchestrator`; -} - -/** - * Filter and map sessions to orchestrator DTOs. - * Shared between page.tsx and API route to ensure consistent orchestrator listing. - */ -export function mapSessionsToOrchestrators( - sessions: Session[], - sessionPrefix: string, - projectName: string, - allSessionPrefixes?: string[], -): Orchestrator[] { - const canonicalId = getOrchestratorSessionId({ sessionPrefix }); - return sessions - .filter((s) => isOrchestratorSession(s, sessionPrefix, allSessionPrefixes) && !isTerminalSession(s)) - .sort((a, b) => { - if (a.id === canonicalId) return -1; - if (b.id === canonicalId) return 1; - return (b.lastActivityAt?.getTime() ?? 0) - (a.lastActivityAt?.getTime() ?? 0); - }) - .map((s) => ({ - id: s.id, - projectId: s.projectId, - projectName, - status: s.status, - activity: s.activity, - createdAt: s.createdAt?.toISOString() ?? null, - lastActivityAt: s.lastActivityAt?.toISOString() ?? null, - })); -} diff --git a/packages/web/src/lib/scm-webhooks.ts b/packages/web/src/lib/scm-webhooks.ts index 63f8d301a..360a8ff6c 100644 --- a/packages/web/src/lib/scm-webhooks.ts +++ b/packages/web/src/lib/scm-webhooks.ts @@ -40,7 +40,7 @@ export function findWebhookProjects( const webhookPath = getProjectWebhookPath(project); if (!webhookPath || webhookPath !== pathname) return []; const scm = registry.get("scm", project.scm.plugin); - if (!scm?.parseWebhook || !scm.verifyWebhook) return []; + if (!scm?.parseWebhook) return []; return [{ projectId, project, scm }]; }); } diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml index 632a7cb12..0b7d3e9a0 100644 --- a/pnpm-lock.yaml +++ b/pnpm-lock.yaml @@ -163,8 +163,8 @@ importers: version: 3.25.76 optionalDependencies: better-sqlite3: - specifier: ^11.0.0 - version: 11.10.0 + specifier: ^12.10.0 + version: 12.10.0 devDependencies: '@rollup/plugin-typescript': specifier: ^12.3.0 @@ -2352,8 +2352,9 @@ packages: resolution: {integrity: sha512-pbnl5XzGBdrFU/wT4jqmJVPn2B6UHPBOhzMQkY/SPUPB6QtUXtmBHBIwCbXJol93mOpGMnQyP/+BB19q04xj7g==} engines: {node: '>=4'} - better-sqlite3@11.10.0: - resolution: {integrity: sha512-EwhOpyXiOEL/lKzHz9AW1msWFNzGc/z+LzeB3/jnFJpxu+th2yqvzsSWas1v9jgs9+xiXJcD5A8CJxAG2TaghQ==} + better-sqlite3@12.10.0: + resolution: {integrity: sha512-CyzaZRQKyHkB2ZInfTTl2nvT33EbDpjkLEbE8/Zck3Ll6O0qqvuGdrJ45HgtH+HykRg88ITY3AdreBGN70aBSQ==} + engines: {node: 20.x || 22.x || 23.x || 24.x || 25.x || 26.x} bindings@1.5.0: resolution: {integrity: sha512-p2q/t/mhvuOj/UeLlV6566GD/guowlr0hHxClI0W9m7MWYkL1F0hLo+0Aexs9HSPCtR1SXQ0TD3MMKrXZajbiQ==} @@ -5917,7 +5918,7 @@ snapshots: dependencies: is-windows: 1.0.2 - better-sqlite3@11.10.0: + better-sqlite3@12.10.0: dependencies: bindings: 1.5.0 prebuild-install: 7.1.3 diff --git a/website/content/docs/architecture.mdx b/website/content/docs/architecture.mdx index 421d7d935..dacbe9b96 100644 --- a/website/content/docs/architecture.mdx +++ b/website/content/docs/architecture.mdx @@ -253,7 +253,7 @@ Step 4 (the JSONL entry fallback) is mandatory. Skipping it means `getActivitySt |---|---|---| | `DEFAULT_ACTIVE_WINDOW_MS` | 30 seconds | Activity newer than this is `active`; older is `ready` | | `DEFAULT_READY_THRESHOLD_MS` | 5 minutes | `ready` sessions older than this become `idle` | -| `ACTIVITY_INPUT_STALENESS_MS` | 5 minutes | `waiting_input` / `blocked` JSONL entries expire after this duration | +| `ACTIVITY_INPUT_STALENESS_MS` | 5 minutes | Deprecated compatibility export. `waiting_input` / `blocked` entries no longer expire by wallclock; they persist until process death or a newer entry overrides them. | --- diff --git a/website/content/docs/plugins/authoring.mdx b/website/content/docs/plugins/authoring.mdx index 57a10415e..cd8798eb7 100644 --- a/website/content/docs/plugins/authoring.mdx +++ b/website/content/docs/plugins/authoring.mdx @@ -468,7 +468,7 @@ All utilities are exported from `@aoagents/ao-core`. The source lives in `packag |--------|-------| | `DEFAULT_READY_THRESHOLD_MS` | `300_000` (5 min) — ready → idle threshold | | `DEFAULT_ACTIVE_WINDOW_MS` | `30_000` (30 s) — active → ready window | -| `ACTIVITY_INPUT_STALENESS_MS` | `300_000` (5 min) — waiting_input/blocked expiry | +| `ACTIVITY_INPUT_STALENESS_MS` | Deprecated compatibility export (`300_000`); waiting_input/blocked no longer expire by wallclock | | `PREFERRED_GH_PATH` | `/usr/local/bin/gh` | | `CI_STATUS` | `{ PENDING, PASSING, FAILING, NONE }` | | `ACTIVITY_STATE` | `{ ACTIVE, READY, IDLE, WAITING_INPUT, BLOCKED, EXITED }` |