diff --git a/docs/design-npm-global-install-fixes.html b/docs/design-npm-global-install-fixes.html new file mode 100644 index 000000000..f1d216e7e --- /dev/null +++ b/docs/design-npm-global-install-fixes.html @@ -0,0 +1,718 @@ + + + + + + Design: True Two-Step Setup Fixes — Agent Orchestrator + + + +
+ +

True Two-Step Setup Fixes

+
+ PR #638 + Issues: #619, #624 + Status: Open + Date: 2026-03-24 +
+ + +

1. Problem Statement

+ +

+ The target onboarding experience for Agent Orchestrator is truly two commands + from a fresh machine: +

+ +
npm install -g @composio/ao && ao start
+ +

+ This does not work. Multiple bugs cause ao start to fail after + a global npm install. Beyond the bugs, there is an architectural issue: ao start + hardcodes runtime: "tmux" in the generated config, making tmux a hidden + prerequisite that breaks the "two commands" promise. +

+ +

The failure chain on a fresh machine:

+ +
    +
  1. "Dependencies not installed"checkBuilt() hardcoded path fails for npm hoisted layout
  2. +
  3. "posix_spawnp: Operation not permitted" — node-pty spawn-helper missing execute bit
  4. +
  5. tmux is a hidden prerequisite — config always sets runtime: "tmux", orchestrator creation fails hard if tmux is missing, kills the dashboard
  6. +
  7. Terminal server crash is permanent — no auto-restart after crash
  8. +
+ +
+ + +

2. Bug Analysis

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
#BugSeverityIssueAffects
1checkBuilt() hardcoded single-level pathP0#619All npm/yarn global installs
2node-pty spawn-helper missing execute bitP0#624macOS + Linux npm installs
3tmux is a hidden prerequisite — hardcoded runtime: "tmux", no auto-install, no fallbackP0Any fresh machine without tmux
4Terminal server crash is permanent (no restart), bookkeeping bug on restartP1Related to #489linux-arm64 (AWS Graviton)
+ +
+ + +

3. Fix 1 — checkBuilt() Walk-Up Resolution

+ +

Root Cause

+

+ preflight.ts verifies @composio/ao-core exists using a single hardcoded path: +

+ +
// BEFORE: only checks one level deep
+const nodeModules = resolve(webDir, "node_modules", "@composio", "ao-core");
+ +

+ npm hoists dependencies to a parent node_modules/. + The check always fails for global installs because ao-core lives one or more + levels up from ao-web. +

+ +
+
+

pnpm layout (works)

+
packages/web/
+  node_modules/
+    @composio/ao-core -> ../../core
+
+
+

npm global layout (broken)

+
/usr/lib/node_modules/@composio/
+  ao-web/          # webDir
+  ao-core/         # sibling, not child
+
+
+ +

Fix

+

+ Replace with findPackageUp() — walks up directories checking + node_modules/@composio/ao-core at each level, mirroring Node's own resolution: +

+ +
function findPackageUp(startDir: string, ...segments: string[]): string | null {
+  let dir = resolve(startDir);
+  while (true) {
+    const candidate = resolve(dir, "node_modules", ...segments);
+    if (existsSync(candidate)) return candidate;
+    const parent = dirname(dir);
+    if (parent === dir) break;
+    dir = parent;
+  }
+  return null;
+}
+ +

Error messages distinguish install method:

+ + +
+ + +

4. Fix 2 — node-pty spawn-helper Postinstall

+ +

Root Cause

+

+ node-pty@1.1.0 ships spawn-helper without the execute bit. + The monorepo's scripts/rebuild-node-pty.js fixes this via node-gyp rebuild, + but it hardcodes pnpm paths and never runs for npm global installs. + Upstream fix (microsoft/node-pty#866) + is only in 1.2.0-beta. +

+ +

Fix

+

+ New packages/ao/bin/postinstall.js runs after npm install -g @composio/ao. + Uses the same findPackageUp pattern to locate node-pty, then chmod 0o755 + on spawn-helper. Skips on Windows, silent on failure. +

+ +
+ Temporary workaround. Remove when node-pty ships stable release with the upstream fix. +
+ +
+ + +

5. Fix 3 — tmux Auto-Install + Process Runtime Fallback

+ +

Root Cause

+

+ This was the biggest gap in the "two-step" promise. Three compounding issues: +

+ + +

On a fresh machine without tmux, the user saw:

+
$ ao start
+✓ Config created: agent-orchestrator.yaml
+⚠ tmux not found — install with: brew install tmux
+...
+✗ Failed to setup orchestrator: tmux: command not found
+ +

Fix: Three-Layer Strategy

+ +
ao start + ↓ + detectEnvironment() → hasTmux = false + ↓ + [1] Try auto-install tmux + macOS: brew install tmux + Linux: sudo apt-get install -y tmux || sudo dnf install -y tmux + ↓ + Success? → hasTmux = true, use runtime: "tmux" + ↓ + Fail? → continue + ↓ + [2] Fallback: generate config with runtime: "process" + (agents run as child processes instead of tmux sessions) + ↓ + [3] If auto-install impossible (e.g. no sudo, Windows): + Show platform-specific instructions as last resort
+ +

Layer 1: Auto-Install

+

checkTmux() now attempts installation before throwing:

+ +
async function autoInstallTmux(): Promise<boolean> {
+  const attempts =
+    process.platform === "darwin"
+      ? [{ cmd: "brew", args: ["install", "tmux"] }]
+      : process.platform === "linux"
+        ? [
+            { cmd: "sudo", args: ["apt-get", "install", "-y", "tmux"] },
+            { cmd: "sudo", args: ["dnf", "install", "-y", "tmux"] },
+          ]
+        : [];
+
+  for (const { cmd, args } of attempts) {
+    try {
+      await exec(cmd, args);
+      await exec("tmux", ["-V"]); // verify
+      return true;
+    } catch { /* try next */ }
+  }
+  return false;
+}
+ +

Layer 2: Process Runtime Fallback

+

+ start.ts config generation now checks env.hasTmux: +

+ +
const runtime = env.hasTmux ? "tmux" : "process";
+ +

+ The process runtime (@composio/ao-plugin-runtime-process) runs agents as + plain child processes. No tmux dependency. Users get a working AO immediately — they + can install tmux later and update the config for the full experience. +

+ +

Layer 3: Platform-Specific Error Messages

+

If auto-install fails and the user needs manual action:

+ + +
+ Process runtime is a degraded experience (no tmux session attach, no terminal panel in dashboard), + but it unblocks the two-step setup completely. The user is informed and can upgrade to tmux later. +
+ +
+ + +

6. Fix 4 — Terminal Server Crash Recovery

+ +

Root Cause

+

+ start-all.ts spawns three child processes. If a terminal server crashes, + it logs the exit and never restarts. +

+ +

Fix

+

+ Auto-restart logic with correct bookkeeping: +

+ + + +

Bookkeeping: Slot-Based Child Tracking

+

+ The initial version had a bug: launch() always called children.push(child), + creating duplicate entries on restart. The cleanup() function counts + children.length and waits for exits, so stale handles would skew the alive count + and trigger the 5-second force-shutdown timer. +

+ +

Fix: each process gets a slot index assigned on first launch. Restarts replace in-place:

+ +
let slotIndex = -1;
+
+function launch(): ChildProcess {
+  const child = spawn(command, args, { ... });
+
+  child.on("exit", (code) => {
+    if (!shuttingDown && opts?.restart && code !== 0 && restarts < maxRestarts) {
+      restarts++;
+      const replacement = launch();
+      children[slotIndex] = replacement; // replace in-place
+    }
+  });
+
+  // Only push on first launch; restarts replace the existing slot
+  if (slotIndex === -1) {
+    slotIndex = children.length;
+    children.push(child);
+  }
+
+  return child;
+}
+ +
+ + +

7. Files Changed

+ + + +
+ + +

8. What Was Already Working

+ +

Investigated but not broken:

+ + + + + + + + + + + + + + + + + + + + + + + +
Suspected BlockerActual State
Port 3000 conflictAlready fixedfindFreePort(port + 1) auto-increments up to 100 ports
node-pty crash kills dashboardAlready handleddirect-terminal-ws.ts catches import failure, gracefully rejects connections
findWebDir() in global installAlready handled — uses require.resolve() with proper walk-up
concurrently at runtimeAlready fixed — production uses node dist-server/start-all.js
+ +
+ + +

9. Post-Merge: True Two-Step Matrix

+ +

After this PR, npm install -g @composio/ao && ao start works on:

+ + + + + + + + + + + + + + + + + + + + + + + + + + + +
Platformtmux Availabletmux Not Available
macOS (Intel + Apple Silicon)Full experience
runtime: tmux
Auto-installs via brew
Falls back to process if brew missing
Linux x64Full experience
runtime: tmux
Auto-installs via apt/dnf
Falls back to process if sudo unavailable
Linux arm64 (Graviton)Works
Direct terminal unavailable (no node-pty prebuild)
Falls back to process
Functional but no tmux attach
WindowsN/AFalls back to process
Error message guides to WSL for full experience
+ +

+ Key improvement: No platform requires manual prerequisite installation before + ao start. The worst case is runtime: "process" — a degraded + but functional experience. +

+ +
+ + +

10. Testing

+ + + + + + + + + + + +
CheckResult
TypeScript typecheckPasses (CLI + web server)
ESLintNo new errors on changed files
Preflight tests15/15 pass (expanded from 13)
CLI tests222/223 pass (1 = pre-existing flaky doctor-script.test.ts)
+ +

New/Updated Test Cases

+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
TestWhat It Verifies
pnpm layout (direct match)findPackageUp finds ao-core in webDir/node_modules on first check
npm hoisted layout (walk-up)findPackageUp misses first level, finds ao-core one level up
npm hint on missing depsGlobal install path shows npm install -g hint
pnpm hint on missing depsMonorepo path shows pnpm install && pnpm build hint
Missing distao-core found but dist/index.js missing shows build hint
tmux auto-install successWhen tmux -V fails initially but install succeeds, checkTmux passes
tmux auto-install failureWhen all install attempts fail, throws with platform-specific instructions
+ +
+ + +

11. Design Decisions

+ +
+ Auto-install tmux rather than just messaging. + Showing "install tmux" defeats the two-step promise. We try brew (macOS) and apt/dnf (Linux) + automatically. If sudo is unavailable or the package manager fails, we fall back to process + runtime instead of blocking. +
+ +
+ Process runtime as fallback, not an error. + The process runtime runs agents as child processes. It lacks tmux session attach + and the dashboard terminal panel, but agents work correctly. Users see + "Using process runtime (tmux not available)" and can upgrade later. +
+ +
+ Slot-based restart bookkeeping. + Each child process gets a fixed slot index in the children array on first launch. + Restarts replace in-place (children[slotIndex] = replacement) instead of pushing, + preventing duplicate entries that would break the cleanup countdown. +
+ +
+ Postinstall is temporary. + The spawn-helper chmod workaround will be removed once node-pty ships + a stable release with the upstream fix (microsoft/node-pty#866). +
+ +
+ + diff --git a/packages/cli/__tests__/lib/preflight.test.ts b/packages/cli/__tests__/lib/preflight.test.ts index c9a249599..36446ea46 100644 --- a/packages/cli/__tests__/lib/preflight.test.ts +++ b/packages/cli/__tests__/lib/preflight.test.ts @@ -91,22 +91,26 @@ describe("preflight.checkBuilt", () => { }); describe("preflight.checkTmux", () => { - it("passes when tmux is installed", async () => { + it("passes when tmux is already installed", async () => { mockExec.mockResolvedValue({ stdout: "tmux 3.3a", stderr: "" }); await expect(preflight.checkTmux()).resolves.toBeUndefined(); expect(mockExec).toHaveBeenCalledWith("tmux", ["-V"]); }); - it("throws when tmux is not installed", async () => { - mockExec.mockRejectedValue(new Error("ENOENT")); - await expect(preflight.checkTmux()).rejects.toThrow( - "tmux is not installed", - ); + it("attempts auto-install when tmux is missing", async () => { + // First call: tmux -V fails (not installed) + // Second call: auto-install attempt (brew/apt/dnf) + // Third call: tmux -V succeeds (installed now) + mockExec + .mockRejectedValueOnce(new Error("ENOENT")) // tmux -V + .mockResolvedValueOnce({ stdout: "", stderr: "" }) // install command + .mockResolvedValueOnce({ stdout: "tmux 3.3a", stderr: "" }); // verify + await expect(preflight.checkTmux()).resolves.toBeUndefined(); }); - it("includes platform-appropriate install instruction in error", async () => { + it("throws with install instructions when auto-install fails", async () => { + // All attempts fail mockExec.mockRejectedValue(new Error("ENOENT")); - // On macOS: brew install tmux, on Linux: apt/dnf, on Windows: WSL const err = await preflight.checkTmux().catch((e: Error) => e); expect(err).toBeInstanceOf(Error); expect(err.message).toContain("tmux is not installed"); diff --git a/packages/cli/src/commands/start.ts b/packages/cli/src/commands/start.ts index c0d363409..cc08a7e97 100644 --- a/packages/cli/src/commands/start.ts +++ b/packages/cli/src/commands/start.ts @@ -294,10 +294,15 @@ async function autoCreateConfig(workingDir: string): Promise console.log(chalk.yellow(` ⚠ Port ${DEFAULT_PORT} is busy — using ${port} instead.`)); } + const runtime = env.hasTmux ? "tmux" : "process"; + if (runtime === "process") { + console.log(chalk.dim(" Using process runtime (tmux not available)")); + } + const config: Record = { port: port ?? DEFAULT_PORT, defaults: { - runtime: "tmux", + runtime, agent, workspace: "worktree", notifiers: ["desktop"], @@ -331,13 +336,14 @@ async function autoCreateConfig(workingDir: string): Promise } if (!env.hasTmux) { - const tmuxHint = - process.platform === "darwin" - ? "brew install tmux" - : process.platform === "win32" - ? "Use WSL: wsl --install, then: sudo apt install tmux" - : "sudo apt install tmux (Debian/Ubuntu) or sudo dnf install tmux (Fedora)"; - console.log(chalk.yellow(`⚠ tmux not found — install with: ${tmuxHint}`)); + console.log(chalk.yellow("⚠ tmux not found — attempting auto-install...")); + try { + await preflight.checkTmux(); // attempts auto-install + env.hasTmux = true; + console.log(chalk.green(" ✓ tmux installed successfully")); + } catch { + console.log(chalk.yellow(" ⚠ Could not auto-install tmux — using process runtime instead")); + } } if (!env.ghAuthed && env.hasGh) { console.log(chalk.yellow("⚠ GitHub CLI not authenticated — run: gh auth login")); diff --git a/packages/cli/src/lib/preflight.ts b/packages/cli/src/lib/preflight.ts index f8c6b4135..fdb7cbd29 100644 --- a/packages/cli/src/lib/preflight.ts +++ b/packages/cli/src/lib/preflight.ts @@ -66,21 +66,57 @@ function findPackageUp(startDir: string, ...segments: string[]): string | null { } /** - * Check that tmux is installed (required for the default runtime). - * Throws with platform-appropriate install instructions. + * Check that tmux is installed. If missing, attempt auto-install. + * Falls back to a clear error with platform-appropriate instructions + * if auto-install is not possible. */ async function checkTmux(): Promise { try { await exec("tmux", ["-V"]); + return; } catch { - const hint = - process.platform === "darwin" - ? "brew install tmux" - : process.platform === "win32" - ? "tmux is not available on Windows. Use WSL: wsl --install, then: sudo apt install tmux" - : "sudo apt install tmux (Debian/Ubuntu) or sudo dnf install tmux (Fedora)"; - throw new Error(`tmux is not installed. Install it: ${hint}`); + // tmux not found — try to install it } + + // Attempt auto-install based on platform + const installed = await autoInstallTmux(); + if (installed) return; + + const hint = + process.platform === "darwin" + ? "brew install tmux" + : process.platform === "win32" + ? "tmux is not available on Windows. Use WSL: wsl --install, then: sudo apt install tmux" + : "sudo apt install tmux (Debian/Ubuntu) or sudo dnf install tmux (Fedora)"; + throw new Error(`tmux is not installed. Install it: ${hint}`); +} + +/** + * Try to auto-install tmux. Returns true if successful. + * Tries brew (macOS), apt-get, then dnf (Linux). Silent on failure. + */ +async function autoInstallTmux(): Promise { + const attempts: Array<{ cmd: string; args: string[] }> = + process.platform === "darwin" + ? [{ cmd: "brew", args: ["install", "tmux"] }] + : process.platform === "linux" + ? [ + { cmd: "sudo", args: ["apt-get", "install", "-y", "tmux"] }, + { cmd: "sudo", args: ["dnf", "install", "-y", "tmux"] }, + ] + : []; + + for (const { cmd, args } of attempts) { + try { + await exec(cmd, args); + // Verify it actually worked + await exec("tmux", ["-V"]); + return true; + } catch { + // Try next method + } + } + return false; } /** diff --git a/packages/web/server/start-all.ts b/packages/web/server/start-all.ts index e914b6218..560ee3e3e 100644 --- a/packages/web/server/start-all.ts +++ b/packages/web/server/start-all.ts @@ -30,6 +30,7 @@ function spawnProcess( ): ChildProcess { let restarts = 0; const maxRestarts = opts?.maxRestarts ?? 3; + let slotIndex = -1; function launch(): ChildProcess { const child = spawn(command, args, { @@ -56,13 +57,17 @@ function spawnProcess( restarts++; log(label, `restarting (attempt ${restarts}/${maxRestarts})`); const replacement = launch(); - const idx = children.indexOf(child); - if (idx !== -1) children[idx] = replacement; - else children.push(replacement); + // Replace in-place — slot was assigned on first push + children[slotIndex] = replacement; } }); - children.push(child); + // Only push on first launch; restarts replace the existing slot + if (slotIndex === -1) { + slotIndex = children.length; + children.push(child); + } + return child; }