teernisse 8fddd50193 Implement JSONL-first session discovery with tiered lookup ...

Rewrite session discovery to be filesystem-first, addressing the widespread
bug where Claude Code's sessions-index.json files are unreliable (87 MB of
unindexed sessions, 17% loss rate across all projects).

Architecture: Three-tier metadata lookup

Tier 1 - Index validation (instant):
  - Parse sessions-index.json into Map<sessionId, IndexEntry>
  - Validate entry.modified against actual file stat.mtimeMs
  - Use 1s tolerance to account for ISO string → filesystem mtime rounding
  - Trust content fields only (messageCount, summary, firstPrompt)
  - Timestamps always come from fs.stat, never from index

Tier 2 - Persistent cache hit (instant):
  - Check MetadataCache by (filePath, mtimeMs, size)
  - If match, use cached metadata
  - Survives server restarts

Tier 3 - Full JSONL parse (~5-50ms/file):
  - Call extractSessionMetadata() with shared parser helpers
  - Cache result for future lookups

Key correctness guarantees:
- All .jsonl files appear regardless of index state
- SessionEntry timestamps always from fs.stat (list ordering never stale)
- Message counts exact (shared helpers ensure parser parity)
- Duration computed from JSONL timestamps, not index

Performance:
- Bounded concurrency: 32 concurrent operations per project
- mapWithLimit() prevents file handle exhaustion
- Warm start <1s (stat all files, in-memory lookups)
- Cold start ~3-5s for 3,103 files (stat + parse phases)

TOCTOU handling:
- Files that disappear between readdir and stat: silently skipped
- Files that disappear between stat and read: silently skipped
- File actively being written: partial parse handled gracefully

Include PRD document that drove this implementation with detailed
requirements, edge cases, and verification plan.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

2026-02-28 00:53:20 -05:00

14 KiB

Raw Permalink Blame History

View Raw