Agent Container Disk Quota Resource Limits

AI agent runners can fill a shared Docker host with model downloads, package caches, build artifacts, logs, and task scratch. The safe fix is not one broad prune. Put disk promises, workspace volumes, cache paths, and cleanup boundaries under explicit policy.

Runbook: Limit The Path That Actually Fills

1. Measure Docker root, workspace, cache, and temp separately. A host can have free space while /var/lib/docker, a named volume, or a task workspace is the real boundary.
2. Do not rely on --storage-opt size=... alone. It depends on the storage driver and does not protect every mounted workspace, cache volume, or bind mount.
3. Give each agent task a writable budget: workspace bytes, temp bytes, package-cache bytes, log bytes, and artifact bytes. Treat model downloads as their own class.
4. Keep active task state out of broad cleanup. Delete only stale, known, owned paths after the task finishes or after an explicit operator review.
5. Add preflight before launch: block or queue tasks when Docker root free space, inode headroom, or the workspace reserve is below policy.
6. Emit before/after metrics: df, df -i, docker system df -v, workspace top buckets, and cache top buckets. Keep enough evidence to tell cache pressure from a product bug.
7. Make cleanup boring: named cache directories with TTLs, per-task workspaces, log rotation, artifact retention, and an emergency reserve that user tasks cannot consume.

I would split the disk part into three separate controls, because `--storage-opt size=...` is not a complete host-protection story by itself:

1. Docker-root preflight before starting a task:

```sh
df -h / /var/lib/docker 2>/dev/null || df -h
df -i / /var/lib/docker 2>/dev/null || df -i
docker system df -v
```

2. Per-task writable budgets:

- workspace volume or bind mount
- temp directory
- package caches such as npm/pip/cargo/go
- logs and captured command output
- large model/download paths

3. Cleanup boundary after task exit:

- remove only task-owned stale workspace/tmp/cache paths
- keep current task state and named volumes out of automatic prune
- log before/after bytes and inodes
- keep an emergency host reserve so cleanup and diagnostics can still run

For acceptance tests, I would add one task that fills its workspace/cache budget and prove it fails that task without filling Docker root or breaking a second concurrent task.

Do Not Delete First

• Named Docker volumes, databases, model stores, or persistent workspaces unless the operator has confirmed they are disposable.
• Current task output, live lock files, sockets, logs needed to debug the incident, or files held open by running agent processes.
• All package caches at once. Prefer TTL, per-project caches, and measured top-bucket cleanup.
• Docker.raw, overlay roots, or containerd directories directly. Use engine-level evidence and review first.