agent-orchestrator/packages/web
Harshit Singh Bhandari 5c1d56aea4
fix(web): bound PTY re-attach loop with grace-period counter reset (#1640)
* fix(web): bound PTY re-attach loop with grace-period counter reset (#1639)

When `ao stop` (or any external action) kills a tmux session out from
under a still-subscribed dashboard, the mux server's PTY exit handler
attempts to re-attach. The MAX_REATTACH_ATTEMPTS=3 cap was supposed to
prevent unbounded respawning, but it was never engaging because the
counter was reset to 0 immediately after each "successful" `open()` —
where success only meant the new PTY was *spawned*, not that it
*survived*. When the underlying tmux session is gone, attach-session
exits ~40 ms after spawn, the exit handler fires again with counter=0,
and the loop runs at ~80 spawns/sec.

Diagnostic data captured on the issue: a single 1.5-second burst
produced 119 spawn↔exit cycles, raising the process's PTY fd count
from ~15 to ~153. Sustained for a few seconds, this exhausts the
macOS system PTY pool (kern.tty.ptmx_max=511), after which nothing
on the system can spawn a new PTY (tmux, VS Code terminal, ao spawn,
etc.) until the leaking process is killed.

Fix:
- Remove the `terminal.reattachAttempts = 0` reset inside the exit
  handler.
- Schedule a delayed reset via setTimeout in `open()`, gated on the
  closure-captured `pty` reference still being terminal.pty after
  REATTACH_RESET_GRACE_MS (5 s).

Effect: tight crash loops cannot reset the counter (PTY exits before
grace expires) and hit MAX_REATTACH_ATTEMPTS within ~150 ms, after
which the server emits "exited" and stops respawning. A long-lived
PTY that crashes hours later still gets a fresh retry budget.

Adds an integration test that reproduces the runaway scenario by
killing the tmux session externally and asserting "exited" arrives
within 2 s. Without this fix the test hangs and times out — the
exact symptom of the bug.

Note: this addresses the dominant runaway behaviour. A separate
~1 fd/cycle leak in node-pty 1.1.0 itself (each spawn opens 3
PTY-class fds in the parent, each exit releases only 2) remains and
will be tracked separately — likely a node-pty upgrade.

Fixes: #1639

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* fixup(web): track grace timer + add recovery test (#1639 PR review)

Address two PR review notes on #1640:

1. Greptile P2 — store the grace-period timer handle on ManagedTerminal
   and clearTimeout it in the unsubscribe cleanup path. The closure
   guard already prevented any incorrect counter reset, so this is
   tidiness rather than a bug fix: it eliminates the up-to-5 s window
   where the timer's closure kept the killed PTY and evicted terminal
   object reachable. Also clears any prior timer when scheduling a new
   one in open() so back-to-back re-attaches don't pile up dead closures.

2. Copilot — add an integration test for the recovery path. The
   existing runaway test exercises the case where the counter must NOT
   reset (PTY crashes inside grace); the new test exercises the case
   where the counter MUST reset (PTY survives grace, then crashes
   later). Without the grace timer firing correctly, a single transient
   blip during startup would permanently consume the retry budget.

Test takes ~5.5 s because it uses the production grace period; per-test
timeout raised to 15 s. Vitest runs tests in parallel so this doesn't
serialize the suite.

Refs: #1639, #1640

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 18:37:44 +05:30
..
e2e feat: seamless onboarding with enhanced documentation (#66) 2026-02-16 22:22:13 +05:30
public Fix web imports for service worker and project utils 2026-03-25 14:08:54 +05:30
screenshots fix: polish shimmer states, terminal PR inference, and gitignore .gstack 2026-04-06 22:49:09 -07:00
scripts fix: protect live dashboard artifacts (#1598) 2026-05-01 16:09:57 +05:30
server fix(web): bound PTY re-attach loop with grace-period counter reset (#1640) 2026-05-07 18:37:44 +05:30
src fix(core): adopt orphaned orchestrator worktrees (#1643) 2026-05-05 20:34:00 +05:30
.env.local.example feat: configurable terminal server ports for multi-dashboard support (#113) 2026-02-19 04:00:19 +05:30
.gitignore Normalize dashboard project filtering and align session layout 2026-03-25 14:34:55 +05:30
CHANGELOG.md chore: release 0.5.0 (#1676) 2026-05-06 16:44:45 +05:30
eslint.config.js fix: resolve three Next.js build warnings in web package (#1087) 2026-04-17 10:43:01 +05:30
next-env.d.ts feat: implement web dashboard with attention-zone UI and API routes (#1) 2026-02-14 14:26:59 +05:30
next.config.js fix(web): pin outputFileTracingRoot to repo root (#1497) 2026-04-25 20:36:19 +05:30
package.json chore: release 0.5.0 (#1676) 2026-05-06 16:44:45 +05:30
postcss.config.mjs feat: implement web dashboard with attention-zone UI and API routes (#1) 2026-02-14 14:26:59 +05:30
tsconfig.json fix: terminal servers compatible with hash-based architecture (#87) 2026-02-18 03:28:55 +05:30
tsconfig.server.json fix: scope node types to node packages 2026-04-13 18:25:21 +05:30
vitest.config.ts fix: register codex web activity plugin 2026-04-18 14:15:29 +05:30