AI Agent Transcript JSONL Retention Policy

Long-running agent sessions, group chats, and topic-bound workers can append every message, tool call, and tool result into active transcript `.jsonl` files. When those files are uncapped, startup, maintenance, and gateway reads can burn CPU on an ever-growing tail.

Runbook: Cap The Transcript, Not Just The Session Index

1. Separate index retention from transcript retention. A `sessions.json` rotation or entry cap does not prove active `.jsonl` transcripts are bounded.
2. Choose a product policy: raw transcript rotation, compaction-only with successor transcript, or both.
3. Truncate or summarize large tool results before writing them to transcript files.
4. Protect active preserved sessions in the disk budget. Long-lived human topic sessions are often exactly the files that ordinary pruning skips.
5. Add a migration path for existing oversized transcripts: archive, summarize, split, or mark as cold before gateway startup scans them repeatedly.
6. Test with a deterministic oversized `.jsonl` fixture and assert maintenance/startup work is bounded.

I would split the fix into a product decision and a bounded-maintenance regression.

Acceptance checks I would want before closing this:

- Make the config surface explicit: sessions.json retention is separate from active transcript .jsonl retention.
- Pick one supported policy: raw transcript rotation, compaction-only successor transcripts, or both.
- Add an oversized .jsonl fixture and prove startup/maintenance does not repeatedly scan the full unbounded tail.
- Truncate or summarize large tool results before append, so one noisy tool call cannot create a 100 MB active transcript.
- Make the disk-budget path account for protected long-lived topic/group sessions, not only evictable inactive sessions.
- Document migration behavior for existing oversized transcripts: archive, summarize, split, or leave intentionally uncapped.

Do Not Delete First

• The only transcript that reproduces the replay or startup CPU spike.
• Session index files before confirming whether the oversized file is active, preserved, or cold.
• Compaction successor files before checking whether they are the intended mitigation path.
• Tool-result evidence that explains which command class is causing transcript growth.