bkinnightskytw
/
agent-orchestrator
mirror of https://github.com/ComposioHQ/agent-orchestrator.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
agent-orchestrator/tests/integration
History
Prateek c9eac34e21 fix: use isolated ports (5000/5001/5003) for integration tests 
Prevents port conflicts with:
- Current orchestrator (ports 3000/3001/3003)
- Future orchestrator (ports 4000/4001/4003)

Changes:
- docker-compose.yml: Map ports 5000, 5001, 5003
- onboarding-test.sh: Configure port 5000, check 5003 for WebSocket
- All health checks updated to use new ports

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 		2026-02-18 03:27:21 +05:30
..
Dockerfile fix: configure npm user-local prefix for testuser (no sudo needed) 2026-02-18 03:27:16 +05:30
README.md feat: add Docker-based onboarding integration test 2026-02-18 03:27:07 +05:30
docker-compose.yml fix: use isolated ports (5000/5001/5003) for integration tests 2026-02-18 03:27:21 +05:30
onboarding-test.sh fix: use isolated ports (5000/5001/5003) for integration tests 2026-02-18 03:27:21 +05:30
run-test.sh feat: add Docker-based onboarding integration test 2026-02-18 03:27:07 +05:30

README.md

Integration Tests

Docker-based integration tests that simulate real-world onboarding and usage scenarios.

Onboarding Test

Tests the complete first-time developer experience:

  1. Fresh environment (no prior setup)
  2. Repository access
  3. ./scripts/setup.sh (install + build + link)
  4. ao init (configuration)
  5. ao start (dashboard + services)
  6. Verify dashboard responds
  7. Verify API endpoints work
  8. Measure total onboarding time

Run Locally

cd tests/integration
docker-compose up --build

Run Manually (for debugging)

# Build image
docker-compose build

# Run container interactively
docker-compose run --rm onboarding-test /bin/bash

# Inside container, run test manually
/workspace/agent-orchestrator/tests/integration/onboarding-test.sh

Expected Results

  • Total time: < 180s (3 minutes) for fresh install
  • All steps pass: ✓ green checkmarks
  • Dashboard accessible: http://localhost:4000

Metrics Tracked

  • Setup time (pnpm install + build)
  • Dashboard startup time
  • API response time
  • Total onboarding time

CI Integration

The test runs automatically on:

  • Pull requests (when packages/ or scripts/ change)
  • Pushes to main branch

View results: .github/workflows/onboarding-test.yml

Future Tests

  • Multi-project test: Verify multiple projects in one config
  • Port conflict test: Verify auto-port-selection
  • Config discovery test: Verify config found from subdirectories
  • Session spawn test: Verify ao spawn creates working sessions
  • Terminal test: Verify WebSocket terminal connection
  • Upgrade test: Verify upgrade from previous version

Adding New Tests

  1. Create test script in tests/integration/<test-name>.sh
  2. Add to docker-compose.yml as new service
  3. Add to .github/workflows/integration-tests.yml
  4. Document expected behavior and metrics

Troubleshooting

Test fails at setup step:

  • Check scripts/setup.sh for errors
  • Verify pnpm is installed in Dockerfile

Dashboard doesn't start:

  • Check port 4000 is not already in use
  • Verify pnpm dev runs all required services
  • Check dashboard logs: docker logs ao-onboarding-test

Test times out:

  • Increase timeout in docker-compose.yml
  • Check if any step hangs (tmux, npm link, etc.)