* feat(core): add opt-in gh CLI tracer and migrate scm/tracker plugins
Introduces execGhObserved() in @aoagents/ao-core: a thin wrapper around
execFile("gh", ...) that writes a JSONL trace row to $AO_GH_TRACE_FILE
on both success and failure. Captures status line, HTTP status, ETag,
rate-limit headers, duration, stdout/stderr byte counts, exit code, and
signal. No-op when the env var is unset, so default behavior is
unchanged.
Migrates three call sites to the observer:
- scm-github/graphql-batch.ts — PR-list guard, commit-status guard,
GraphQL batch query
- scm-github/index.ts — gh() and ghInDir() helpers
- tracker-github/index.ts — internal gh() helper
This is Phase A1a of experiments/PLAN.md: tracer infrastructure +
migration. The full GhRunner contract (Promise<GhResult>,
GhRunnerError.ghResult on reject, body capture, redaction, 64 KB cap)
lands in A1b along with the scorecard baseline.
Also adds experiments/ reference docs: the v2.3 plan, the gh-CLI call
catalog, two ETag verification writeups, and a trace harness + summary
script.
* docs(experiments): add A1a validation status and A1b blockers
Record the five A1b pre-freeze blockers surfaced by Adil's 1,487-row
baseline and an independent drill run: graphql-batch missing -i,
extractOperation flag mis-bucketing, analyzer not segmenting burn by
reset window, CLI-subcommand opacity (GH_DEBUG=api stderr vs coarse
/rate_limit bracket — not equivalent), and sessionId/projectId not
threaded through plugin callsites. Note bare gh() helper cleanup as
known-open follow-up.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(tracer): close A1b blockers 1-4 — graphql-batch visibility, operation naming, analyzer segmentation
- Add -i flag to executeBatchQuery in graphql-batch.ts and split HTTP
headers from JSON body before parsing, making all gh.api.graphql-batch
rows visible to status and rate-limit analysis (was 186 invisible rows)
- Fix extractOperation() in gh-trace.ts to walk past -* flags before
picking the operation segment, eliminating the gh.api.--method bucket
- Add per-reset-window burn segmentation to both analyzers so runs
straddling a reset boundary produce per-window deltas instead of a
single invalid cross-reset delta
- Add experiment scripts: analyze-trace.mjs (deep trace analysis) and
drill-tracer.mjs (standalone tracer exerciser)
- Document Gap 1 decision in PLAN.md: accept CLI subcommands as opaque
for A1, bracket A2 runs with /rate_limit snapshots for coarse burn
- Add progress timeline to PLAN.md showing A→B→C track dependencies
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(experiments): add A2 baseline matrix runbook
Practical execution plan for the Phase A2 scenario x scale x topology
matrix: 7 priority cells, per-cell procedure, /rate_limit bracketing
for Gap 1 subcommand burn, output format for baseline.md, and the
scorecard that gates Track B.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(tracer): guard stderr/stdout against undefined, bound operation cardinality
Addresses code review findings:
1. Guard Buffer.byteLength and parseIncludedHttpResponse against
undefined stderr/stdout — fixes 48 SCM test regressions where
mocked execFile paths don't populate stderr
2. extractOperation() now takes only the first path segment of REST
URLs (e.g. "repos" from "repos/acme/repo/pulls/123/...") to keep
operation bucket cardinality bounded and stable across runs
3. Fix A2 runbook /rate_limit snapshots to produce valid JSON using
jq's now|todate instead of appending raw timestamp
4. Add blocker 5 dependency to runbook prereqs and per-session cells
All 140 SCM tests pass (0 failures).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(experiments): add rate-limiting research artifacts
Baseline measurements, discussion notes, benchmark harness spec,
and updated master plan from two independent trace runs at 5-6 sessions.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(experiments): add benchmark harness for GH rate-limit measurement
Three modes: setup (spawn sessions, wait for PRs), measure (trace API
calls over a fixed window, produce scorecard), report (recompute from
existing trace). Node.js stdlib only, shells out to ao CLI and gh CLI.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix(scm-github): handle 304 Not Modified in ETag guard catch blocks (B1)
`gh api -i` exits code 1 on HTTP 304 responses, causing the catch blocks
in checkPRListETag and checkCommitStatusETag to assume the resource changed
and trigger unnecessary GraphQL batch queries every poll cycle.
Fix: inspect stdout/stderr in the catch block for the 304 status line before
falling back to "assume changed". Also unifies the 304 detection regex to
handle HTTP/1.1, HTTP/2, and HTTP/2.0 status lines, and adds rateLimit
introspection to the batch GraphQL query.
Benchmark result (quiet-steady, 5 sessions, 15 min):
- GraphQL points/hr: 260/5,000 (5%) — down from 820–1,416 pre-fix
- ETag guard 304 rate: 100%
- GraphQL batch calls during measurement: 0
Also fixes the benchmark harness to create placeholder tmux sessions with a
claude symlink so the lifecycle actually polls sessions instead of
short-circuiting to "killed".
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(experiments): update plan and notes with B1 benchmark results
B1 fix validated at 5, 10, and 20 sessions in quiet-steady state:
- 5 sessions: 260 GraphQL pts/hr (5% budget)
- 10 sessions: 640 pts/hr (13%)
- 20 sessions: 680 pts/hr (14%) — sub-linear scaling confirmed
- 50-session projection: ~800-1000 pts/hr (16-20%)
- ETag guard 304 rate: 100% at all scale points
- graphql-batch calls: 0 during measurement at all scale points
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat(core): log gh wrapper invocations for D1
* fix(core): preserve wrapper logging for dash-prefixed gh args
* feat(core): add gh wrapper cache for PR discovery and issue context (D4)
Add read-through caching to the ~/.ao/bin/gh wrapper, targeting the two
largest agent-side waste buckets identified in D4 analysis:
1. PR discovery (gh pr list --head): infinite TTL for positive results.
598 calls → ~10 per 10-session run (98% reduction).
2. Issue context (gh issue view): 300s TTL.
75 calls → ~20 per 10-session run (73% reduction).
The wrapper now caches successful read-only responses in
$AO_DATA_DIR/.ghcache/$AO_SESSION/ and serves them on subsequent
identical calls. Negative results (empty []) are never cached.
gh pr create populates the PR discovery cache immediately.
Also lifts PATH wrapper installation from individual agent plugins into
session-manager, making it universal for all agents including Claude Code:
- session-manager injects PATH + GH_PATH into every runtime.create()
- session-manager calls setupPathWrapperWorkspace() for all agents
- Removes duplicate buildAgentPath/setupPathWrapperWorkspace boilerplate
from codex, aider, opencode, and cursor plugins
Includes D4 implementation plans in experiments/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): add full capacity discovery (5→50 sessions) and CI churn results
Complete scaling curve measured: 50 sessions uses only ~28% of GraphQL
budget with 100% ETag guard hit rate at every scale. Poll cycle lag
identified as first bottleneck (66s at 50 sessions vs 30s target).
CI churn benchmark shows ETag invalidation is a latency problem, not
a rate-limit problem (+9% GraphQL, +4.4x p50 latency).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs(experiments): record real-agent catastrophe and Track D handoff
5-real-agent run on todo-app exhausted GraphQL bucket in 31 min (~9572 pts/hr,
~37x quiet-steady at the same session count). AO polling consumed ~10 calls;
the rest came from agents themselves via the metadata-only ~/.ao/bin/gh
wrapper, which has no tracing. Captures findings, adds Track D (agent-side
gh consumption) plus B5 (migrate remaining bare gh callsites to
execGhObserved), and includes the runbook + benchmark scripts Adil will
build on for the cross-machine reproduction.
* feat(core): add cache-hit/miss tracing to gh wrapper (D4)
The wrapper trace now logs a cacheResult entry for every cacheable
command: hit, miss-stored, miss-negative, or miss-error. This makes
benchmark runs conclusive — you can count cache hits vs real gh calls
directly from the JSONL trace instead of inferring from rate-limit
deltas.
Bump wrapper version to 0.4.1.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(scm-github): replace repo-scoped Guard 1 with PR-scoped ETag checks (D4)
Guard 1 now checks GET /repos/{owner}/{repo}/pulls/{number} per PR
instead of GET /repos/{owner}/{repo}/pulls?... per repo. This means:
- Only changed PRs flow into the GraphQL batch
- Unchanged PRs are served directly from the enrichment cache
- shouldRefreshPREnrichment returns a refresh plan (prsToRefresh +
cachedResults) instead of a boolean
When 1 of 10 PRs changes, the old guard refreshed all 10 via GraphQL.
Now only the 1 changed PR is fetched; the other 9 are served from cache
at zero GraphQL cost.
Trade-off: more REST guard calls (1 per PR instead of 1 per repo), but
304 responses cost zero rate limit points.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Forward AO_AGENT_GH_TRACE to session runtimes
* revert: remove PR-scoped ETag guards (Change 3)
Reverts 25ae6013. The per-PR Guard 1 added more REST calls (1 per PR
instead of 1 per repo) without meaningful GraphQL savings at 10-session
scale. Core REST delta went from 16 to 142 while GraphQL rate stayed
flat. The repo-scoped guard is sufficient for current workloads.
Preserves the subsequent 6fc64f4f commit (AO_AGENT_GH_TRACE forwarding).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(core): use real gh binary in execGhObserved, bypass wrapper
execGhObserved() was calling bare "gh" which resolved to ~/.ao/bin/gh
(the wrapper) when that directory was in PATH. This caused:
- AO-side gh calls going through the agent wrapper
- All trace rows with aoSession=null polluting the agent trace
- Cache functions silently failing (no AO_SESSION in AO process)
Now strips ~/.ao/bin from PATH and resolves the real gh binary
(e.g. /opt/homebrew/bin/gh) at startup. Cached after first resolution.
AO process → execGhObserved → real gh → AO_GH_TRACE_FILE
Agent process → ~/.ao/bin/gh wrapper → AO_AGENT_GH_TRACE + cache
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(core): harden gh wrapper caching and agent-side tracing
Cache correctness:
- Include --json fields in cache key (prevents stale partial responses)
- Only cache stdout, not stderr (prevents warning contamination)
- Fix trailing newline inconsistency in PR discovery cache
- Support --key=value arg syntax for all cached flags
- Remove PR create cache pre-population (hardcoded fields, no JSON escaping)
- Log miss-write-failed when ao_cache_write fails (previously silent)
Agent trace improvements:
- Add operation field to invocation rows (gh.pr.list, gh.issue.view, etc.)
- Add durationMs, exitCode, ok to cache outcome rows
- Log passthrough for all non-cached code paths (pr/create, default case)
- Replace exec with child process in default case to enable post-call tracing
Bump wrapper version to 0.6.0.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(runtime-tmux): re-export PATH after shell init to survive macOS path_helper
macOS zsh runs path_helper during shell startup which resets PATH,
wiping entries set via tmux new-session -e. This caused ~/.ao/bin
to be lost, so the gh/git wrappers were never intercepting agent
calls — no caching, no tracing, no metadata auto-updates.
Fix: send `export PATH=...` via send-keys after the shell has
initialized but before the launch command, ensuring PATH sticks.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix(runtime-tmux): use launch script for PATH re-export instead of send-keys
The previous send-keys approach sent 1000+ literal keystrokes for the
PATH value, which broke terminal input buffers and caused stuck quote
prompts. Instead, include the PATH export in the launch script file
which is executed directly — no terminal buffer issues.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): add AO-side gh rate-limit trace report
5-session, 15-minute trace analysis with full call breakdown,
ETag guard effectiveness, anomaly investigation, and ranked
reduction opportunities.
Key findings:
- GraphQL at 41%/hr with 5 sessions (bottleneck at ~12 sessions)
- 47% of calls are individual REST fallbacks that batch should cover
- Review thread GraphQL calls (55/15min) can be folded into batch
- detectPR() and guard failures are working as designed
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): add AO rate-limit reduction plan with Step 1
Step 1: Remove individual REST fallback from determineStatus().
110 calls (65 pr view + 45 pr checks) eliminated per 15-min window.
Batch enrichment covers all PRs every 30s — fallback is unnecessary
insurance for an event that never occurred in real traces.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* experiments(m2): drop agent-trace gate for claude-code
Claude Code uses native PostToolUse hooks (.claude/settings.json), bypassing
the ~/.ao/bin/gh PATH wrapper, so AO_AGENT_GH_TRACE stays empty even when
Claude makes gh calls. The previous smoke gate required AGENT_ROWS>0 and
aborted every claude-code M2 batch at smoke.
- limit-finder.sh: add REQUIRE_AGENT_TRACE env + --no-require-agent-trace flag,
gate the AGENT_ROWS integrity check behind it.
- m2-ab-run.sh: bump SMOKE_DURATION to 420s; auto-pass --no-require-agent-trace
when AGENT=claude-code; simplify smoke_check to gate only on AO_ROWS>0
(B1 lives in AO-side scm-github, measured by AO trace).
Follow-up tracked as task #39: instrument Claude's hook/tool path or document
that AO_AGENT_GH_TRACE does not cover Claude Code.
* feat(tracker-github): cache issue reads in-process (5 min TTL)
The lifecycle worker polls getIssue/isCompleted repeatedly for the same
issue across a session. Trace data from a 5-session tier-5 bench run
showed the same (repo, issue) pair fetched 64+ times with >97% duplicate
rate — ~744 of 4,059 AO gh calls in 10 minutes were redundant issue views.
Adds an in-process Map<string, CachedIssue> per createGitHubTracker()
instance, keyed by `${repo}#${id}`, TTL 5 min, bounded to 500 entries
(LRU evict-oldest on overflow).
- getIssue: read-through cache, populate on miss
- isCompleted: routes through getIssue (was a separate narrow gh call)
- updateIssue: invalidate the entry before mutating
- createIssue: unchanged, naturally populates via the existing getIssue
- Failures are not cached
Cache lives inside createGitHubTracker so each create() returns an
isolated cache (test isolation comes for free).
Expected reduction: ~744 → ~15 gh issue view calls per tier-5 run.
Tests: 41 existing + 10 new cache tests, all passing.
* feat(scm-github): cache 5 gh pr view callsites with per-method TTLs
The lifecycle worker repeatedly polls each PR for state, summary, reviews,
and review decision. Trace data showed gh pr view was the single largest
AO-side endpoint at 1,280 calls per 5-session tier-5 run with >97% duplicate
rate (e.g. PR #184 polled 86× for --json state alone in 11.5 minutes).
Adds an in-process per-instance cache inside createGitHubSCM(), keyed by
${owner}/${repo}#${prKey}:${method} so different field-sets stay isolated.
Per-method TTLs balance reduction against staleness on decision-influencing
fields:
- resolvePR: 60s (identity metadata only)
- getPRState: 5s
- getPRSummary: 5s (includes state)
- getReviews: 5s
- getReviewDecision: 5s
assignPRToCurrentUser, mergePR, and closePR each invalidate the entire PR
cache for that PR after the mutation, so AO never sees stale state from its
own writes. Failures are not cached.
getCIChecksFromStatusRollup and getMergeability are intentionally NOT cached
here — those need ETag-based revalidation, not blind TTL, and will land
separately.
Expected reduction: ~1,165 of ~1,280 gh pr view calls per tier-5 run.
Tests: 73 existing + 12 new cache tests, all 153 passing.
* feat(scm-github): cache CI checks, mergeability, pending comments, detectPR
Completes the AO-side hot-read caching alongside the prior PR view cache.
All use 5s TTL per the approved policy for decision-influencing fields —
well under one lifecycle poll cycle so state transitions are still seen
next pass.
- getCIChecks (gh pr checks): 5s TTL
- getMergeability (composite pr view + CI + state): 5s TTL on the composite
- getPendingComments (gh api graphql review threads): 5s TTL —
ETag doesn't help on GraphQL per Experiment 2
- detectPR (gh pr list --head BRANCH): 5s TTL, POSITIVE-ONLY.
Empty results are never cached so a freshly created PR is discovered
on the very next poll. The branch-keyed cache entry is invalidated
by mergePR/closePR alongside the number-keyed entries.
Combined with the prior PR view cache, covers the top 6 AO-side gh
operation categories that accounted for ~85% of calls in tier-5 traces.
Tests: 85 existing + 9 new cache tests, all 162 passing.
* experiments(m2): parse REPO from yaml before using it in banner
m2-ab-run.sh referenced $REPO in the header banner before parsing it,
causing 'unbound variable' abort under 'set -u'. Parse it right after
CONFIG_FILE is set.
* test(core): mock full Issue shape in plugin-integration cleanup tests
After tracker-github routed isCompleted() through getIssue() to share
the issue cache, these mocks needed the full Issue shape (number, title,
body, url, state, stateReason, labels, assignees) instead of the narrow
{state} shape that worked when isCompleted made its own --json state call.
* perf(scm-github): tune cache TTLs based on trace replay
Replayed feat run1 + main run2 tier-5 traces (4059 + 1748 rows, 38 min, 5
sessions each) against the shipped cache logic. Three TTLs were materially
under-tuned for the actual lifecycle poll cadence:
- detectPR: 5s → 30s (was 0.5% hit rate; per-branch poll cadence
is ~90s, so 5s caught nothing. 30s catches
intra-cycle bursts when multiple sessions
share a branch. Positive-only stays.)
- getReviewDecision: 5s → 10s (within "10-30s TTL or ETag" policy)
- getPendingComments: 5s → 10s (same policy class)
All three are still well under one poll cycle; freshness contract unchanged
in practice. Other TTLs (5s on state/CI/mergeability, 60s on resolvePR,
5min on issue) hit the targets they were set for and stay as-is.
Replay results before/after:
- feat run1: 53.7% → 57.8% reduction (2179 → 2345 hits of 4059 calls)
- main run2: 47.4% → 52.6% reduction
- Net: ~55% AO-side gh calls eliminated across both traces
Adds experiments/cache-replay.mjs — a counterfactual replay tool that
walks an execGhObserved JSONL trace and simulates per-method cache hits
with the shipped TTLs. Useful as a regression check when tweaking cache
policy.
Tests: 162/162 passing.
* docs(experiments): add cache freshness check runbook
Seven-step manual runbook to validate the cache TTL contract doesn't
cause workflow lag. Covers each cached method with:
- exact gh CLI trigger command
- what to observe in the dashboard / lifecycle log
- pass/fail threshold (TTL + 30s poll cycle)
Companion to experiments/cache-replay.mjs — replay measures how much
we saved, runbook measures whether we lost anything in the process.
* docs(experiments): add Step 2 — consolidate review comment fetching
Single GraphQL call replaces GraphQL + REST for review comments.
Include comment data in agent reaction message to eliminate
agent-side gh read calls. Update future steps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): add duplicate API traffic analysis
Three independent sources hit GitHub API for the same PRs:
1. Dashboard serialize.ts — individual REST calls, no batch, no cache
2. CLI lifecycle manager — batch + guards
3. Web lifecycle manager — same batch + guards, 3s offset
~50% of all API traffic is pure duplication. Dashboard and dual
lifecycle managers are the root causes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): add full cache architecture to duplicate traffic analysis
Three independent cache layers across two processes with zero shared
state. Web process creates its own plugin registry, SCM plugin, lifecycle
manager, and dashboard cache — all hitting GitHub independently.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): add shared PR enrichment plan
Persist batch enrichment + review comments to session metadata files.
Dashboard reads from disk instead of making its own GitHub API calls.
Remove web's duplicate lifecycle manager.
Eliminates ~268 calls / 15 min (58% of all traffic). Dashboard data
gets fresher (30s vs 5min). Single writer (CLI lifecycle), web only reads.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): update Step 1 — remove all three fallback paths
Remove fallback in determineStatus(), maybeDispatchCIFailureDetails(),
and maybeDispatchMergeConflicts(). All three follow the same pattern:
batch cache hit → use it, cache miss → skip (wait 30s for next batch).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs(experiments): promote Step 3 (remove dead reviews field) + detail Step 5 (issue caching)
Step 3: Remove reviews(last: 5) from batch query — fetched but never
consumed, reduces GraphQL complexity on every batch call.
Step 5: Persist issue data to session metadata at spawn — eliminates
27 gh issue view calls per 15 min (both processes re-fetch independently).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(core): remove individual REST fallback from lifecycle polling
Remove fallback paths in determineStatus(), maybeDispatchCIFailureDetails(),
and maybeDispatchMergeConflicts() that made individual REST calls when the
batch enrichment cache missed. The batch runs every 30s — a cache miss
means the data arrives on the next cycle, not that it's lost.
Also add populatePREnrichmentCache() call to check() so single-session
checks also use the batch path.
Eliminates ~110 individual pr view/pr checks calls per 15-min window
(24% of all AO-side traffic).
* feat(core): consolidate review comment fetching into single GraphQL call
Add getReviewThreads() to SCM interface — returns all review threads
(human + bot) with isBot flag from a single GraphQL query. Lifecycle
manager splits locally for separate reaction pipelines.
- Eliminates the REST getAutomatedComments() call (40 calls / 15 min)
- Reaction messages now include inline comment data (file, line, author,
body, URL) so agents don't need to re-fetch via gh api
- Default config messages updated to not tell agents to call gh
- getAutomatedComments kept as optional for backward compatibility
* perf(scm-github): remove unused reviews(last: 5) from batch query
The batch query fetched reviews with author, state, submittedAt but
the data was never consumed — only used in a validation check.
The reviewDecision scalar field provides everything AO needs.
Reduces GraphQL complexity cost on every batch call.
* docs(experiments): add post-optimization trace report (Steps 1-3)
5-session, 17-minute trace after removing REST fallback, consolidating
review comments, and removing dead reviews field.
Results: GraphQL 35%/hr (was 41%), REST <1% (was 3%), automated
comment REST calls eliminated. 54% of remaining traffic is redundant
(duplicate lifecycle manager + dashboard individual calls).
* docs(experiments): add trace file gist link to post-optimization report
* feat(core,web): shared PR enrichment — dashboard reads from metadata
CLI lifecycle manager now persists batch enrichment data and review
comments to session metadata files (prEnrichment + prReviewComments
keys). The web dashboard reads from metadata instead of calling
GitHub API.
Changes:
- lifecycle-manager: add persistPREnrichmentToMetadata() after poll,
write prReviewComments in maybeDispatchReviewBacklog()
- serialize: replace enrichSessionPR (6 API calls) with metadata read
- services: stop web lifecycle polling (keep for webhook checks)
- cache: remove prCache (no longer needed)
- routes: remove timeout wrappers and cacheOnly pattern
Eliminates ~237 calls / 15 min (54% of all AO-side traffic).
Dashboard data freshness improves from 5min to 30s.
* fix(web): remove unused beforeEach import in serialize test
* docs(experiments): add final trace report — 56% GraphQL reduction achieved
5-session, 24-min trace after all optimizations including shared
enrichment. GraphQL 905/hr (was 2,072), REST 5/hr (was 168).
Single lifecycle manager confirmed. Dashboard API calls eliminated.
Max sessions before budget exhaustion: ~27 (was ~12).
* docs(experiments): add REST budget breakdown to final report
* fix(core): use storageKey for getSessionsDir in persistPREnrichmentToMetadata
* fix(test): use OpenCodeSessionManager type in plugin-integration tests
* fix(test): update bugbot-comments and auto-cleanup tests for new review API
* fix(web): fix syntax error and missing import from rebase
* docs(experiments): add complete rate-limiting change log and update final report numbers
* fix(web): fix tmux session resolution for legacy wrapped storageKeys
* fix(web): pass tmuxName directly to terminal server instead of reverse-resolving
* perf(core): gate detectPR behind Guard 1 ETag — skip when PR list unchanged
* perf(core): always run Guard 1 for all repos, dedup issue views, include threadId in review messages
* feat(core): add Guard 3 (review ETag), enrich review data with summaries, dedup issue views, gate detectPR for all repos
* fix(web): reuse cached tmuxSessionId on re-open, add 15-session trace report and comparison docs
* perf(scm-github): reduce contexts to first:10, add -i to review GraphQL for rate limit tracing
* feat(core): merge CI details into transition, enrich merge conflict message, reduce batch contexts, add graphqlCost tracing
* chore(experiments): remove working artifacts, keep final reports and reference docs
* chore: remove experiments directory
* refactor: remove getAutomatedComments from SCM interface and all implementations
* fix: address all PR review comments
- gh-trace: make binary resolution async via fs.access (no event loop
blocking, no shell injection), cache mkdir for trace writes, async
fire-and-forget appendFile, document 10MB maxBuffer rationale
- lifecycle-manager: log detectPR failures via observer instead of
silent catch, add getPRState fallback for terminal states
(merged/closed) when batch enrichment cache misses
- scm-gitlab: implement getReviewThreads with bot+human threads and
isBot flag, fixing silent feature regression after
getAutomatedComments removal
- scm-github: clear reviewThreadsCache in invalidatePRCache, document
first:11 CI checks cost budget
- runtime-tmux: use printf+JSON.stringify for PATH export to prevent
shell injection from single quotes
- agent-workspace-hooks: add cache timestamp sanity check, include
--repo in cache keys to prevent cross-repo collisions
- services: document dashboard dependency on CLI polling
- tests: update gh binary path assertions for resolved paths
* fix: address all 17 PR review comments
- gh-trace: use path.delimiter, cache-only-on-success, last HTTP status
line, await writes with warn-once, redact secrets, gate JSON.parse
- agent-workspace-hooks: validate cache keys, redact wrapper trace args,
sha256 cache keys to prevent collisions, 120s TTL ceiling
- session-manager: skip PATH wrappers for claude-code (native hooks)
- types: add deprecation JSDoc for getReviewThreads
- graphql-batch: clear Guard 3 in clearETagCache, re-read ETag on 304,
switch Guard 2 to check-runs endpoint, drop per_page=1 from Guard 3
- Add gh-trace unit tests for extractOperation, redactArgs, parseHttp
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: adil <adil.business4064@gmail.com>
Co-authored-by: iamasx <adilshaikh4064@gmail.com>
Renames all npm package scopes from @composio/* to @aoagents/* and
updates GitHub repo references from ComposioHQ/agent-orchestrator
to aoagents/ao throughout the codebase.
- All package.json names and dependencies
- README badges, links, and install instructions
- Documentation references
- Changeset config
- Source code imports and test files
* feat: implement seamless onboarding with enhanced documentation
- Add comprehensive README.md (18KB) with quick start, core concepts, and FAQ
- Add detailed SETUP.md (16.5KB) with prerequisites, integration guides, and troubleshooting
- Add examples/ directory with 5 ready-to-use config templates:
- simple-github.yaml: Minimal GitHub setup
- linear-team.yaml: Linear integration
- multi-project.yaml: Multiple repos
- auto-merge.yaml: Aggressive automation
- codex-integration.yaml: Using Codex agent
- Add environment detection (git repo, remote, branch, auth status)
- Auto-fill prompts with smart defaults from detected environment
- Add prerequisite validation (git, tmux, gh CLI)
- Show actionable next steps and warnings
- Parse owner/repo from git remote automatically
- Detect LINEAR_API_KEY and SLACK_WEBHOOK_URL in environment
- Prompt for Linear team ID when Linear tracker selected
- Format all files with Prettier for consistency
Reduces onboarding time from 30+ minutes to ~5 minutes:
1. Install CLI: `npm install -g @composio/ao-cli`
2. Run init: `ao init` (auto-detects everything)
3. Spawn agent: `ao spawn my-project ISSUE-123`
Users no longer need to:
- Manually parse git remote URLs
- Look up current branch names
- Remember YAML syntax
- Search for Linear team IDs
- Debug missing prerequisites
- ✅ pnpm build - All packages compile
- ✅ pnpm typecheck - No TypeScript errors
- ✅ pnpm lint - No new linting issues
- ✅ pnpm format - All files formatted
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* docs: update installation instructions to reflect npm not yet published
Package is not published to npm yet, so users must build from source.
Updated README.md and SETUP.md to:
- Make 'build from source' the primary installation method
- Add note that npm publishing is coming soon
- Include pnpm as a prerequisite
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat: add ao init --auto --smart for zero-config setup
Implements intelligent config generation with project type detection.
## What's New
### ao init --auto
- Zero prompts - auto-generates config with smart defaults
- Detects: git repo, remote, branch, languages, frameworks, tools
- Generates project-specific agentRules based on detected tech stack
### Project Detection
- Languages: TypeScript, JavaScript, Python, Go, Rust
- Frameworks: React, Next.js, Vue, Express, FastAPI, Django, Flask
- Tools: pnpm workspaces, test frameworks
- Package managers: pnpm, yarn, npm
### Rule Templates
Created templates for:
- base.md - Universal best practices
- typescript.md - TS strict mode, ESM, type imports
- javascript.md - Modern ES6+ patterns
- react.md - Hooks, composition, best practices
- nextjs.md - App Router, Server Components
- python.md - Type hints, PEP 8
- go.md - Error handling, defer patterns
- pnpm-workspaces.md - Monorepo commands
### Example Output
```bash
ao init --auto
# Detects:
# ✓ TypeScript + pnpm workspaces
# ✓ React + Next.js
# ✓ Vitest
# Generates:
agentRules: |
Always run tests before pushing.
Use TypeScript strict mode.
Use ESM modules with .js extensions.
Use React best practices (hooks, composition).
Before pushing: pnpm build && pnpm typecheck && pnpm lint && pnpm test
```
## Benefits
- **5 seconds** instead of 5 minutes
- **Zero config knowledge** required
- **Context-aware rules** tailored to your stack
- **Still customizable** - edit the generated config
## Future: --smart (AI-powered)
Flag added but not yet implemented. Will use Claude Code to:
- Analyze CLAUDE.md, CONTRIBUTING.md
- Read CI/CD config
- Generate custom rules based on project patterns
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: detect repo default branch instead of current branch
Fixes Bugbot issue: "Current branch wrongly suggested as default base branch"
## Problem
detectEnvironment was using `git branch --show-current` to suggest
defaultBranch in the config. If a user ran `ao init` while on a feature
branch like `feat/my-work`, the wizard would suggest that feature branch
as the default, causing agents to branch from the wrong base.
## Solution
Added detectDefaultBranch() function with 3 fallback methods:
1. git symbolic-ref refs/remotes/origin/HEAD (most reliable)
2. GitHub API via gh CLI (if ownerRepo known)
3. Check common branch names: main, master, next, develop
Now EnvironmentInfo tracks both:
- currentBranch: The checked-out branch (for display only)
- defaultBranch: The repo's base branch (for config)
## Testing
Tested on feat/seamless-onboarding branch:
- Current branch: feat/seamless-onboarding (displayed)
- Default branch: main (correctly detected for config)
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: prevent duplicate framework detection in Python projects
Fixes Bugbot issue: "Duplicate frameworks when multiple Python config files exist"
## Problem
When both requirements.txt and pyproject.toml exist and mention the same
framework (e.g., FastAPI), the detection loop added it to the frameworks
array twice, causing duplicate rules in the generated config.
## Solution
Added addFramework() helper that checks if framework already exists before
adding to the array. Also prevents pytest from being set multiple times as
testFramework.
## Testing
Verified with test repo containing both files with FastAPI:
- Before: Would add 'fastapi' twice
- After: Only adds 'fastapi' once ✓
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: address Bugbot review comments
- Remove redundant conditional in --smart flag (both branches were identical)
- Include templates directory in npm package files
* fix: add existence check for base.md template file
Add existsSync guard before reading base.md to handle missing templates gracefully, consistent with other template file reads.
* fix: use direct tool invocation instead of which command
Replace 'which' with direct tool invocation (tmux -V, gh --version)
for better portability on minimal Linux systems where 'which' may
not be installed.
* fix: address Bugbot review comments
- Simplify gh auth status check to rely on exit code instead of output string
- Remove async from synchronous functions (detectProjectType, generateRulesFromTemplates)
* feat: add setup script for one-command installation
Add scripts/setup.sh that:
- Installs pnpm if not present
- Installs dependencies
- Builds all packages
- Links CLI globally
Updated README with simplified setup instructions using the script.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: correct npm link command in setup script
Remove incorrect -g flag from npm link command. The correct syntax is to cd into the package directory and run npm link without flags.
* fix: address Bugbot review comments on init command
- Validate --smart flag requires --auto (prevents silent ignore)
- Fix path validation to check user-specified path (not CWD)
These fixes address medium and low severity issues found by Cursor Bugbot
in PR #66 review.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* docs: add DirectTerminal troubleshooting and fix setup script
- Add TROUBLESHOOTING.md documenting node-pty posix_spawnp error
- Update setup.sh to rebuild node-pty from source (fixes DirectTerminal)
- Ensures seamless onboarding with working terminal out-of-the-box
Resolves DirectTerminal WebSocket failures from incompatible prebuilt binaries.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: resolve variable scope issue in init command validation
- Move path variable outside if block to fix TypeScript scope error
- Only validate path existence if projectId is provided
- Use inline tilde expansion instead of missing expandHome import
Fixes build error that prevented setup.sh from completing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: automate node-pty rebuild to eliminate terminal issues
- Add postinstall hook to automatically rebuild node-pty after pnpm install
- Create scripts/rebuild-node-pty.js for automatic rebuild with error handling
- Remove manual node-pty rebuild from setup.sh (now automatic)
This ensures DirectTerminal works correctly on every installation without
manual intervention. Fixes posix_spawnp errors from incompatible prebuilt
binaries across different systems and installations.
Resolves issue where users would encounter blank terminals after setup.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* docs: update TROUBLESHOOTING with automatic node-pty rebuild
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* docs: add comprehensive README with quick start guide
- 3-line magical setup: clone → setup → init → start
- Architecture overview with plugin slots table
- Usage examples and auto-reaction configuration
- Links to detailed docs (SETUP.md, TROUBLESHOOTING.md, examples/)
- Philosophy: push not pull, amplify judgment
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: resolve ESLint errors in rebuild-node-pty script
- Add scripts directory configuration to eslint.config.js
- Configure Node.js globals (console, process) for scripts
- Remove unused error variable from catch block
Fixes lint CI failure.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: warn when auto mode uses placeholder repo value
- Detect when 'owner/repo' placeholder is used in --auto mode
- Show warning: 'Could not detect GitHub repository'
- Update next steps to emphasize editing config when placeholder used
- Prevents silent failures when spawning agents with invalid repo
Addresses Bugbot review comment about silent placeholder values.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat: add npm publishing support with @composio scope
Set up Changesets for version management, add publish metadata to all 20
packages under the @composio scope, create an unscoped wrapper package
(@composio/agent-orchestrator) for global install, and add a GitHub
Actions release workflow.
- Rename all packages from @agent-orchestrator/* to @composio/ao-*
- Add @composio/agent-orchestrator wrapper (bin shim → @composio/ao-cli)
- Add license, repository, homepage, bugs, files, engines to all packages
- Add .npmrc (access=public), MIT LICENSE file
- Add .changeset/ config with linked versioning for all packages
- Add .github/workflows/release.yml (changesets publish CI)
- Add changeset, version-packages, release scripts to root
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
* fix: exclude private web package from release build
The release script now filters out @composio/ao-web, matching the
workflow's existing exclusion and preventing a Next.js build failure
from blocking npm publishing.
Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
* feat: implement SCM and tracker plugins (github, linear)
Implement three plugin interfaces from packages/core/src/types.ts:
- scm-github: Full GitHub PR lifecycle via `gh` CLI — PR detection by
branch, state tracking, CI checks, reviews, review decision, pending
comments, automated bot comment detection (cursor[bot], codecov, etc.),
and merge readiness (CI + reviews + conflicts + draft).
- tracker-github: GitHub Issues tracker via `gh` CLI — issue CRUD,
completion check, branch naming (feat/issue-N), prompt generation,
list/filter/update/create with label and assignee support.
- tracker-linear: Linear issue tracker via GraphQL API (LINEAR_API_KEY) —
issue fetch, completion check, branch naming (feat/IDENTIFIER), prompt
generation, list with team/state/label filters, state transitions via
workflow state resolution, issue creation with teamId config.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review — GraphQL injection, N+1 queries, timeouts, tests
- tracker-linear: use GraphQL variables in listIssues to prevent injection
- tracker-linear: add 30s HTTP timeout to linearQuery
- tracker-linear: fix issueUrl to use workspace slug from project config
- tracker-linear: add labels/assignee support to createIssue
- scm-github: rewrite getPendingComments to use GraphQL reviewThreads
with real isResolved status instead of hardcoding false
- scm-github: simplify getAutomatedComments to single API call (N+1 fix)
- scm-github: add 30s timeout to gh CLI calls
- scm-github: fix parseDate to return epoch instead of fabricating dates
- scm-github: add repo format validation in detectPR
- scm-github: fix getCISummary to not count all-skipped as passing
- tracker-github: add 30s timeout to gh CLI calls
- tracker-github: fix createIssue to not use unsupported --json flag
- Add comprehensive vitest tests for scm-github and tracker-github
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address Codex review — assignee lookup, GraphQL vars, status checks
Iteration 1 fixes (8 issues from Codex review):
- tracker-linear: remove invalid assigneeDisplayName, resolve assignee
by display name to ID via users query after creation
- tracker-linear: add HTTP status code check in linearQuery
- tracker-linear: throw on missing workflow state in updateIssue
- scm-github: use GraphQL variables ($owner, $name, $number) in
getPendingComments instead of string interpolation
- scm-github: guard against empty comment nodes in review threads
- scm-github: use mergeStateStatus in getMergeability (BEHIND, BLOCKED)
- tracker-github: default description to "" in createIssue
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address PR review + lint — error context, GraphQL vars, mergeability
- scm-github/tracker-github: wrap gh() errors with command context + cause
- scm-github: use GraphQL variables ($owner, $name, $number) in
getPendingComments to prevent injection
- scm-github: guard against empty review thread nodes
- scm-github: incorporate mergeStateStatus (BEHIND/BLOCKED) in getMergeability
- tracker-linear: add identifier vs UUID comment in updateIssue
- Fix ESLint preserve-caught-error violations
- Remove unused mockGhRaw in scm-github tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: add pagination to getAutomatedComments and increase thread limit
- getAutomatedComments: add --paginate flag to REST API call to fetch
all review comments beyond the default 30-item first page
- getPendingComments: increase GraphQL reviewThreads limit from 100 to
250 (GitHub's max for connections)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use per_page=100 instead of --paginate for JSON-safe pagination
Replace --paginate with -F per_page=100 in getAutomatedComments to
avoid concatenated JSON arrays that break JSON.parse on multi-page
responses. 100 is GitHub's max per_page value.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: filter out resolved threads in getPendingComments
The method name implies it should only return unresolved/pending review
threads. The isResolved data was already fetched via GraphQL but was not
being filtered on, causing resolved threads to be included in results.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add comprehensive tracker-linear test suite (53 tests)
Covers all Tracker interface methods: getIssue, isCompleted, issueUrl,
branchName, generatePrompt, listIssues, updateIssue, createIssue.
Mocks node:https request to simulate Linear GraphQL API responses.
Tests state mapping, error handling, assignee/label resolution, and
GraphQL variable injection.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address 6 bugbot review comments on PR #4
- GraphQL reviewThreads(first: 250) → first: 100 (GitHub max)
- noConflicts false when mergeable is UNKNOWN (not just CONFLICTING)
- updateIssue now handles labels and assignee (not just state/comment)
- Add res.on("error") handler in linearQuery to prevent crashes
- createIssue returns only actually-applied labels, not all requested
- Tests added/updated for all fixes (144 total across 3 plugins)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: make Linear updateIssue labels additive to match GitHub behavior
Linear's issueUpdate replaces all labels, while GitHub's --add-label is
additive. Now fetches existing label IDs first and merges with new ones
before sending to issueUpdate, ensuring consistent behavior across both
tracker implementations.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: default listIssues to open state in tracker-linear
When no state filter is specified, tracker-github defaults to "open"
but tracker-linear was returning all issues. Now defaults to excluding
completed/canceled issues, matching tracker-github behavior.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add Composio SDK support to tracker-linear
Allow users with a COMPOSIO_API_KEY to use the Linear tracker without
a separate LINEAR_API_KEY. The plugin auto-detects which key is available
and routes through either the direct Linear GraphQL API or Composio's
LINEAR_RUN_QUERY_OR_MUTATION tool. All existing queries and response
parsing are reused via a GraphQLTransport abstraction.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: address 2 bugbot review comments on PR #4
1. getCIChecks now throws on error instead of silently returning [],
preventing a fail-open where CI appears healthy when we can't fetch
check status. getCISummary catches the error and returns "failing".
2. Handle GitHub's UNSTABLE mergeStateStatus (required checks failing)
as a merge blocker in getMergeability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: prevent unhandled promise rejection in Composio timeout race
When the timeout wins Promise.race in the Composio transport, the
resultPromise is left without a rejection handler. If the SDK call
later rejects, it becomes an unhandled promise rejection that crashes
Node.js 20+ with --unhandled-rejections=throw.
Attach no-op .catch() to both resultPromise and timeoutPromise before
the race so whichever promise loses has its rejection silently handled.
Also adds plugin integration tests (core -> real plugins -> mocked gh CLI).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: chain error cause in getCIChecks and fix core build
- getCIChecks catch now preserves the original error via { cause: err }
instead of discarding it, matching the pattern used by the gh() helper
- Add tsconfig.build.json to core that excludes __tests__/ from build
compilation, preventing TS5055 errors from circular workspace deps
(integration tests import plugin packages that depend back on core)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: remove circular devDeps, address bugbot comments
- Remove plugin devDependencies from core/package.json that created a
circular dependency (core -> plugins -> core), breaking CI build order
- Document in_progress state as intentional no-op in tracker-github
updateIssue (GitHub Issues only supports open/closed)
- Remove @composio/core optional peerDependency from tracker-linear;
the dynamic import() already handles the missing package gracefully,
and the peer dep was pulling composio + transitive deps into lockfile
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add real integration test for tracker-linear against Linear API
Creates a test that exercises the full tracker-linear plugin lifecycle
against the real Linear API — createIssue, getIssue, isCompleted,
listIssues, updateIssue (comment, close, reopen), generatePrompt,
branchName, issueUrl. Each run creates a throwaway test issue and
trashes it in cleanup. Skipped when LINEAR_API_KEY is not set.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* ci: pass LINEAR_API_KEY and LINEAR_TEAM_ID to integration tests
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: support both LINEAR_API_KEY and COMPOSIO_API_KEY in integration test
The tracker-linear integration test now runs with either credential:
- LINEAR_API_KEY: direct Linear API (full cleanup via trash)
- COMPOSIO_API_KEY: via Composio SDK (cleanup falls back to closing)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: don't pass COMPOSIO_API_KEY to CI integration tests
When both LINEAR_API_KEY and COMPOSIO_API_KEY are set, the plugin
prefers the Composio transport which requires @composio/core SDK.
The SDK isn't installed in CI, so use the direct LINEAR_API_KEY
transport which has no extra dependencies.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: use import.meta.url in vitest config, default createIssue description
- Replace __dirname with import.meta.url-derived dirname in ESM config
- Default createIssue description to "" for defensive safety
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: map CANCELLED and ACTION_REQUIRED CI conclusions to failed
Terminal check conclusions (CANCELLED, ACTION_REQUIRED) were falling
through to "pending", causing the lifecycle manager to wait forever.
Also changed the default else branch to "failed" (fail-closed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Dhruv Sharma
Prateek
Harsh Batheja