Your plan is already strong, but I’d revise it in 10 places to reduce risk at scale and make it materially more useful. 1. Shared transport + retries for GraphQL (must-have) Reasoning: `REST` already has throttling/retry in `src/gitlab/client.rs`; your proposed GraphQL client would bypass that and can spike rate limits under concurrent project ingest (`src/cli/commands/ingest.rs`). Unifying transport prevents split behavior and cuts production incidents. ```diff @@ AC-1: GraphQL Client (Unit) - [ ] Network error → `LoreError::Other` + [ ] GraphQL requests use shared GitLab transport (same timeout, rate limiter, retry policy as REST) + [ ] Retries 429/502/503/504/network errors (max 3) with exponential backoff + jitter + [ ] 429 honors `Retry-After` before retrying + [ ] Exhausted network retries → `LoreError::GitLabNetworkError` @@ Decisions - 8. **No retry/backoff in v1** — DEFER. + 8. **Retry/backoff in v1** — YES (shared REST+GraphQL reliability policy). @@ Implementation Detail + File 15: `src/gitlab/transport.rs` (NEW) — shared HTTP execution and retry/backoff policy. ``` 2. Capability cache for unsupported projects (must-have) Reasoning: Free tier / older GitLab will repeatedly emit warning noise every sync and waste calls. Cache support status per project and re-probe on TTL. ```diff @@ AC-6: Enrichment in Orchestrator (Integration) - [ ] On any GraphQL error: logs warning, continues to next project (never fails the sync) + [ ] Unsupported capability responses (missing endpoint/type/widget) are cached per project + [ ] While cached unsupported, enrichment is skipped without repeated warning spam + [ ] Capability cache auto-expires (default 24h) and is re-probed @@ Migration Numbering - This feature uses **migration 021**. + This feature uses **migrations 021-022**. @@ Files Changed (Summary) + `migrations/022_project_capabilities.sql` | NEW — support cache table for project capabilities ``` 3. Delta-first enrichment with periodic full reconcile (must-have) Reasoning: Full GraphQL scan every sync is expensive for large projects. You already compute issue deltas in ingestion; use that as fast path and keep a periodic full sweep as safety net. ```diff @@ AC-6: Enrichment in Orchestrator (Integration) - [ ] Runs on every sync (not gated by `--full`) + [ ] Fast path: skip status enrichment when issue ingestion upserted 0 issues for that project + [ ] Safety net: run full reconciliation every `status_full_reconcile_hours` (default 24) + [ ] `--full` always forces reconciliation @@ AC-5: Config Toggle (Unit) + [ ] `SyncConfig` has `status_full_reconcile_hours: u32` (default 24) ``` 4. Strongly typed widget parsing via `__typename` (must-have) Reasoning: current “deserialize arbitrary widget JSON into `StatusWidget`” is fragile. Query/type by `__typename` for forward compatibility and fewer silent parse mistakes. ```diff @@ AC-3: Status Fetcher (Integration) - [ ] Extracts status from `widgets` array by matching `WorkItemWidgetStatus` fragment + [ ] Query includes `widgets { __typename ... }` and parser matches `__typename == "WorkItemWidgetStatus"` + [ ] Non-status widgets are ignored deterministically (no heuristic JSON-deserialize attempts) @@ GraphQL Query + widgets { + __typename + ... on WorkItemWidgetStatus { ... } + } ``` 5. Set-based transactional DB apply (must-have) Reasoning: row-by-row clear/update loops will be slow on large projects and hold write locks longer. Temp-table + set-based SQL inside one txn is faster and easier to reason about rollback. ```diff @@ AC-3: Status Fetcher (Integration) - `all_fetched_iids: Vec` + `all_fetched_iids: HashSet` @@ AC-6: Enrichment in Orchestrator (Integration) - [ ] Before applying updates, NULL out status fields ... (loop per IID) - [ ] UPDATE SQL: `SET status_name=?, ... WHERE project_id=? AND iid=?` + [ ] Use temp tables and set-based SQL in one transaction: + [ ] (1) clear stale statuses for fetched IIDs absent from status rows + [ ] (2) apply status values for fetched IIDs with status + [ ] One commit per project; rollback leaves prior state intact ``` 6. Fix index strategy for `COLLATE NOCASE` + default sorting (must-have) Reasoning: your proposed `(project_id, status_name)` index may not fully help `COLLATE NOCASE` + `ORDER BY updated_at`. Tune index to real query shape in `src/cli/commands/list.rs`. ```diff @@ AC-4: Migration 021 (Unit) - [ ] Adds compound index `idx_issues_project_status_name(project_id, status_name)` for `--status` filter performance + [ ] Adds covering NOCASE-aware index: + [ ] `idx_issues_project_status_name_nocase_updated(project_id, status_name COLLATE NOCASE, updated_at DESC)` + [ ] Adds category index: + [ ] `idx_issues_project_status_category_nocase(project_id, status_category COLLATE NOCASE)` ``` 7. Add stable/automation-friendly filters now (high-value feature) Reasoning: status names are user-customizable and renameable; category is more stable. Also add `--no-status` for quality checks and migration visibility. ```diff @@ AC-9: List Issues Filter (E2E) + [ ] `lore list issues --status-category in_progress` filters by category (case-insensitive) + [ ] `lore list issues --no-status` returns only issues where `status_name IS NULL` + [ ] `--status` + `--status-category` combine with AND logic @@ File 9: `src/cli/mod.rs` + Add flags: `--status-category`, `--no-status` @@ File 11: `src/cli/autocorrect.rs` + Register `--status-category` and `--no-status` for `issues` ``` 8. Better enrichment observability and failure accounting (must-have ops) Reasoning: only tracking `statuses_enriched` hides skipped/cleared/errors, and auth failures become silent partial data quality issues. Add counters and explicit progress events. ```diff @@ AC-6: Enrichment in Orchestrator (Integration) - [ ] `IngestProjectResult` gains `statuses_enriched: usize` counter - [ ] Progress event: `ProgressEvent::StatusEnrichmentComplete { enriched: usize }` + [ ] `IngestProjectResult` gains: + [ ] `statuses_enriched`, `statuses_cleared`, `status_enrichment_skipped`, `status_enrichment_failed` + [ ] Progress events: + [ ] `StatusEnrichmentStarted`, `StatusEnrichmentSkipped`, `StatusEnrichmentComplete`, `StatusEnrichmentFailed` + [ ] End-of-sync summary includes per-project enrichment outcome counts ``` 9. Add `status_changed_at` for immediately useful workflow analytics (high-value feature) Reasoning: without change timestamp, you can’t answer “how long has this been in progress?” which is one of the most useful agent/human queries. ```diff @@ AC-4: Migration 021 (Unit) + [ ] Adds nullable INTEGER column `status_changed_at` (ms epoch UTC) @@ AC-6: Enrichment in Orchestrator (Integration) + [ ] If status_name/category changes, update `status_changed_at = now_ms()` + [ ] If status is cleared, set `status_changed_at = NULL` @@ AC-9: List Issues Filter (E2E) + [ ] `lore list issues --stale-status-days N` filters by `status_changed_at <= now - N days` ``` 10. Expand test matrix for real-world failure/perf paths (must-have) Reasoning: current tests are good, but the highest-risk failures are retry behavior, capability caching, idempotency under repeated runs, and large-project performance. ```diff @@ TDD Plan — RED Phase + 26. `test_graphql_retries_429_with_retry_after_then_succeeds` + 27. `test_graphql_retries_503_then_fails_after_max_attempts` + 28. `test_capability_cache_skips_unsupported_project_until_ttl_expiry` + 29. `test_delta_skip_when_no_issue_upserts` + 30. `test_periodic_full_reconcile_runs_after_threshold` + 31. `test_set_based_enrichment_scales_10k_issues_without_timeout` + 32. `test_enrichment_idempotent_across_two_runs` + 33. `test_status_changed_at_updates_only_on_actual_status_change` ``` If you want, I can now produce a single consolidated revised plan document (full rewritten Markdown) with these changes merged in-place so it’s ready to execute.