LSM Compaction Disk Full Deadlock

An LSM tree can deadlock near full disk when compaction is the operation that would free space, but a merge compaction first needs transient output space while the old SSTs still exist. The fix is a storage admission policy, not a blind cleanup script.

Runbook: Break The Compaction Reclaim Deadlock

1. Split compaction choices by disk effect. Drop and Move should stay runnable at the limit; Merge needs admission control.
2. Reserve a band for one minimal reclaiming compaction. User writes should be paused before they consume it.
3. Estimate a merge's transient need before starting. A conservative bound is the sum of input SST sizes while the output is being written.
4. Narrow instead of fail when possible: choose fewer inputs, smaller key ranges, or the compaction with the highest likely bytes reclaimed per byte written.
5. Keep the manifest atomic. If an output write fails, inputs remain valid, the manifest stays unchanged, and orphan outputs are separately cleaned.
6. Expose availability as a status bit: full compaction available, only reclaim compaction available, writes paused, or hard full.

I would make the compaction picker space-aware rather than treating all compactions like writes.

Acceptance checks I would add:
- Drop and Move remain eligible even when normal writes are paused.
- Merge estimates transient output space before starting.
- If the merge bound does not fit free-reserved, the picker skips or narrows the merge.
- The emergency reserve cannot be consumed by user writes.
- Injected mid-merge ENOSPC leaves inputs intact, output orphaned, manifest unchanged, and reopen succeeds.
- StorageStatus exposes whether full compaction is available or only reclaiming compaction is allowed.

Do Not Delete First

• Manifest, current files, table metadata, or compaction markers before proving reopen behavior.
• Input SSTs for a failed merge before confirming the output was committed to the manifest.
• Orphan outputs before recording whether cleanup is automatic and idempotent.
• WAL or memtable recovery files just to make room for compaction.