agent-orchestrator/changelog/hash-based-architecture-migration.md

Migration Guide: Hash-Based Architecture

Date: 2026-02-17
Breaking Change: Yes
Affects: All users with existing sessions

What Changed

Agent Orchestrator has migrated from flat, configurable directories (dataDir and worktreeDir) to a hash-based project isolation architecture. This change eliminates configuration overhead and prevents collisions when running multiple orchestrator instances from different directories.

Before (Flat Architecture)

# agent-orchestrator.yaml
dataDir: ~/.ao-sessions
worktreeDir: ~/.ao-worktrees

projects:
  my-app:
    path: ~/repos/my-app

Problems:

• Multiple configs sharing same dataDir caused session collisions
• Manual path configuration required
• No isolation between orchestrator instances
• Backwards compatibility code added complexity

After (Hash-Based Architecture)

# agent-orchestrator.yaml
# No dataDir or worktreeDir needed!

projects:
  my-app:
    path: ~/repos/my-app

Benefits:

• Zero configuration — paths auto-derived from config location
• Complete isolation — each config gets unique namespace
• Collision-free — SHA256 hash prevents conflicts
• Clean codebase — no backwards compatibility code

Architecture Details

Directory Structure

All orchestrator data now lives under ~/.agent-orchestrator/:

~/.agent-orchestrator/
├── {hash}-{projectId}/        # Unique per config+project
│   ├── .origin                # Stores config path (collision detection)
│   ├── sessions/              # Session metadata
│   │   ├── int-1              # Metadata for session int-1
│   │   ├── int-2
│   │   └── archive/           # Archived metadata
│   └── worktrees/             # Git worktrees
│       ├── int-1/             # Worktree for session int-1
│       └── int-2/

Hash Generation

The hash is the first 12 characters of SHA256(realpath(dirname(configPath))):

// Config at: ~/projects/acme/agent-orchestrator.yaml
// Hash of:   /Users/you/projects/acme
// Result:    a3b4c5d6e7f8

// Final path: ~/.agent-orchestrator/a3b4c5d6e7f8-my-app/

Why 12 chars? Balance between uniqueness (collision probability ~1 in 16 billion) and path length.

Session Naming Layers

Three levels of naming for compatibility:

1. User-facing: int-1, ao-42 (short, clean)
2. Tmux: a3b4c5d6e7f8-int-1 (globally unique across all configs)
3. Metadata file: int-1 (within project-specific sessions/ directory)

Breaking Changes

1. Config File Changes

REMOVED fields (will cause validation errors if present):

• dataDir
• worktreeDir

REQUIRED field:

• configPath (automatically set by loadConfig())

Migration:

# agent-orchestrator.yaml
- dataDir: ~/.ao-sessions
- worktreeDir: ~/.ao-worktrees

projects:
  my-app:
    path: ~/repos/my-app

2. Metadata File Locations

Before:

~/.ao-sessions/
├── int-1          # Flat directory, all projects mixed
├── int-2
├── ao-1
└── ao-2

After:

~/.agent-orchestrator/
├── a3b4c5d6e7f8-integrator/sessions/
│   ├── int-1      # Integrator project sessions
│   └── int-2
└── f9e8d7c6b5a4-my-app/sessions/
    ├── ao-1       # My-app project sessions
    └── ao-2

Impact: Existing metadata files will not be automatically migrated. See migration steps below.

3. Worktree Locations

Before:

~/.ao-worktrees/
├── integrator/
│   ├── int-1
│   └── int-2
└── my-app/
    ├── ao-1
    └── ao-2

After:

~/.agent-orchestrator/
├── a3b4c5d6e7f8-integrator/worktrees/
│   ├── int-1
│   └── int-2
└── f9e8d7c6b5a4-my-app/worktrees/
    ├── ao-1
    └── ao-2

Impact: Existing worktrees will not be automatically migrated. Git will report them as "missing" and they must be manually removed.

4. Environment Variables

Before:

AO_DATA_DIR=~/.ao-sessions          # Flat path

After:

AO_DATA_DIR=~/.agent-orchestrator/a3b4c5d6e7f8-integrator/sessions/

Impact: Scripts or hooks relying on AO_DATA_DIR pointing to a flat directory will break.

5. API Changes (For Plugin Developers)

Removed from OrchestratorConfig:

interface OrchestratorConfig {
  dataDir: string; // REMOVED
  worktreeDir: string; // REMOVED
  configPath: string; // NOW REQUIRED
}

New Path Utilities (use these instead):

import {
  getSessionsDir,
  getWorktreesDir,
  getProjectBaseDir,
  generateConfigHash,
  generateInstanceId,
  validateAndStoreOrigin,
} from "@composio/ao-core";

// Calculate paths dynamically
const sessionsDir = getSessionsDir(configPath, projectPath);
const worktreesDir = getWorktreesDir(configPath, projectPath);

Migration Steps

Step 1: Clean Up Config File

Remove dataDir and worktreeDir from your config:

# Edit your agent-orchestrator.yaml
vim ~/path/to/agent-orchestrator.yaml

Remove these lines:

dataDir: ~/.ao-sessions
worktreeDir: ~/.ao-worktrees

Step 2: Kill Existing Sessions

IMPORTANT: Existing sessions will not automatically migrate. You must kill them first.

# List all tmux sessions
tmux ls

# Kill all orchestrator sessions (adjust prefix as needed)
tmux ls | grep -E '^(int|ao|app)-[0-9]+:' | cut -d: -f1 | xargs -I{} tmux kill-session -t {}

# Or kill all tmux sessions (nuclear option)
tmux kill-server

Step 3: Clean Up Old Directories

Worktrees must be removed from git:

# For each project, remove old worktrees
cd ~/repos/integrator

# List worktrees
git worktree list

# Remove each old worktree
git worktree remove ~/.ao-worktrees/integrator/int-1 --force
git worktree remove ~/.ao-worktrees/integrator/int-2 --force
# ... repeat for all

# Prune stale references
git worktree prune

Metadata can be archived for reference:

# Archive old metadata (optional)
mv ~/.ao-sessions ~/.ao-sessions-backup-$(date +%Y%m%d)

# Or delete if you don't need it
rm -rf ~/.ao-sessions

Worktree directory can be removed after cleaning up git references:

# Remove old worktree directory (after git worktree remove)
rm -rf ~/.ao-worktrees

Step 4: Update to Latest Version

# Pull latest code
git pull origin main

# Reinstall dependencies
pnpm install

# Rebuild packages
pnpm build

Step 5: Start Fresh

# Start orchestrator (creates new directory structure)
ao start

# Spawn new sessions (uses hash-based paths)
ao spawn my-app INT-1234

Step 6: Verify New Structure

# Check that new directories were created
ls -la ~/.agent-orchestrator/

# You should see directories like:
# a3b4c5d6e7f8-my-app/
#   .origin
#   sessions/
#   worktrees/

# Check tmux sessions have hash prefix
tmux ls
# Should show: a3b4c5d6e7f8-int-1, a3b4c5d6e7f8-int-2, etc.

Rollback (If Needed)

If you need to rollback to the old architecture:

1. Checkout previous commit (before hash-based migration):

   git checkout <commit-before-migration>
   pnpm install
   pnpm build
   

2. Restore old config:

   # Add back to agent-orchestrator.yaml
   dataDir: ~/.ao-sessions
   worktreeDir: ~/.ao-worktrees
   

3. Restore old metadata (if archived):

   mv ~/.ao-sessions-backup-20260217 ~/.ao-sessions
   

4. Kill new hash-based sessions:

   tmux ls | grep -E '^[a-f0-9]{12}-' | cut -d: -f1 | xargs -I{} tmux kill-session -t {}
   

FAQ

Q: Why can't I use my old sessions?

A: The metadata file structure changed. Old sessions point to flat directories (~/.ao-sessions/int-1) but the new code expects hash-based paths (~/.agent-orchestrator/a3b4c5d6e7f8-integrator/sessions/int-1). You must kill old sessions and spawn new ones.

Q: Will my PRs be lost?

A: No! PRs are on GitHub, not in local sessions. You can:

1. Check gh pr list to see all open PRs
2. Spawn new sessions for PRs that need work: ao spawn integrator --branch feat/existing-branch

Q: What about in-progress work?

A: Git worktrees contain your code changes. Before killing sessions:

1. Commit or stash changes in each worktree
2. Note which issues/branches were being worked on
3. After migration, spawn new sessions and continue work

Q: Can I migrate metadata files manually?

A: Yes, but not recommended. The manual process:

# For each project
OLD_DIR=~/.ao-sessions
NEW_DIR=~/.agent-orchestrator/$(python3 -c "import hashlib; print(hashlib.sha256(b'/Users/you/path/to/config/dir').hexdigest()[:12])")-integrator/sessions

mkdir -p "$NEW_DIR"

# Copy metadata files
cp "$OLD_DIR"/int-* "$NEW_DIR/"

# Update worktree paths in each file (required!)
for file in "$NEW_DIR"/*; do
  sed -i '' 's|worktree=~/.ao-worktrees/integrator/|worktree=~/.agent-orchestrator/HASH-integrator/worktrees/|g' "$file"
done

This is error-prone. Recommended: Kill old sessions and spawn fresh ones.

Q: What happens if I have multiple config files?

A: Each config gets a unique hash! This is the main benefit of the new architecture:

# Config 1: ~/projects/acme/agent-orchestrator.yaml
# Hash: a3b4c5d6e7f8
# Sessions: ~/.agent-orchestrator/a3b4c5d6e7f8-my-app/

# Config 2: ~/experiments/test/agent-orchestrator.yaml
# Hash: 1f2e3d4c5b6a
# Sessions: ~/.agent-orchestrator/1f2e3d4c5b6a-my-app/

No conflicts, complete isolation!

Q: How do I find the hash for my config?

A:

# Calculate hash for your config directory
echo -n "/path/to/your/config/dir" | sha256sum | cut -c1-12

# Or let ao print it
ao status
# Output shows: Config: /path/to/config.yaml
# Hash will be in directory names: ~/.agent-orchestrator/{hash}-{project}/

Q: What if two configs have the same hash (collision)?

A: The .origin file detects this and throws an error with instructions:

Hash collision detected!
Directory: ~/.agent-orchestrator/a3b4c5d6e7f8-my-app
Expected config: /Users/you/config1/agent-orchestrator.yaml
Actual config: /Users/you/config2/agent-orchestrator.yaml
This is a rare hash collision. Please move one of the configs to a different directory.

Solution: Move one config to a different directory. Collision probability is ~1 in 16 billion with 12-character hashes.

Support

If you encounter issues during migration:

1. Check existing sessions: tmux ls
2. Check new directory structure: ls -la ~/.agent-orchestrator/
3. Check config validation: ao status
4. Review git worktrees: git worktree list (from project directory)
5. Check logs: journalctl -u ao-orchestrator or tmux session output

For bugs or questions, file an issue: https://github.com/composiohq/agent-orchestrator/issues

Summary

Action Required:

1. ✅ Remove dataDir and worktreeDir from config
2. ✅ Kill all existing sessions (tmux kill-server)
3. ✅ Clean up old git worktrees (git worktree remove --force)
4. ✅ Remove old directories (~/.ao-sessions, ~/.ao-worktrees)
5. ✅ Update to latest version (pnpm install && pnpm build)
6. ✅ Start fresh (ao start, ao spawn)

Expected Downtime: ~10 minutes (time to kill sessions, clean worktrees, respawn)

Risk Level: Low (PRs are safe, code is in git, only local sessions affected)

Benefit: Cleaner architecture, zero config, collision-free multi-instance support