AI agent storage incident

Parallel Agent Worktrees Cargo target Disk Full

When several AI agents build isolated Rust worktrees at the same time, each worktree can create a separate target/ tree. Four 12-16 GB build directories can saturate a 38 GB session volume before the code is wrong. Measure the per-worktree peak first, then choose shared target, cleanup-on-finish, or concurrency caps deliberately.

Evidence To Capture First

Before changing Cargo or deleting directories, prove the disk peak per agent. This keeps the fix focused on workflow policy rather than hiding a real test or linking failure.

df -h .
df -i .
du -xhd1 . 2>/dev/null | sort -h | tail -40
find . -maxdepth 3 -type d -name target -prune -print0 2>/dev/null \
  | xargs -0 du -xsh 2>/dev/null | sort -h

Policy Options

1. Shared CARGO_TARGET_DIR: best disk reduction, but validate concurrent Cargo lock behavior and cross-branch artifact invalidation.
2. Per-agent target cleanup: safe when the branch is pushed, tests are done, and no later step needs the compiled outputs.
3. Concurrency cap: choose floor((free_gb - reserve_gb) / peak_target_gb) for heavy Rust builds instead of a fixed agent count.
4. Larger session volume: useful for burst capacity, but still keep telemetry so target growth does not silently return.

Do Not Delete First

• Do not delete active target/ directories while a Cargo process is compiling or linking.
• Do not delete the worktree itself before the branch is pushed or the agent's diff is saved.
• Do not share CARGO_TARGET_DIR without a rollback path; subtle build-script or feature differences can make failures hard to attribute.
• Do not treat disk expansion as the only fix; the workflow still needs a reserve and cleanup contract.