gitlore/plans/work-item-status-graphql.feedback-4.md

Your plan is already strong, but I’d revise it in 10 places to reduce risk at scale and make it materially more useful.

1. Shared transport + retries for GraphQL (must-have) Reasoning: REST already has throttling/retry in src/gitlab/client.rs; your proposed GraphQL client would bypass that and can spike rate limits under concurrent project ingest (src/cli/commands/ingest.rs). Unifying transport prevents split behavior and cuts production incidents.

@@ AC-1: GraphQL Client (Unit)
- [ ] Network error → `LoreError::Other`
+ [ ] GraphQL requests use shared GitLab transport (same timeout, rate limiter, retry policy as REST)
+ [ ] Retries 429/502/503/504/network errors (max 3) with exponential backoff + jitter
+ [ ] 429 honors `Retry-After` before retrying
+ [ ] Exhausted network retries → `LoreError::GitLabNetworkError`

@@ Decisions
- 8. **No retry/backoff in v1** — DEFER.
+ 8. **Retry/backoff in v1** — YES (shared REST+GraphQL reliability policy).

@@ Implementation Detail
+ File 15: `src/gitlab/transport.rs` (NEW) — shared HTTP execution and retry/backoff policy.

2. Capability cache for unsupported projects (must-have) Reasoning: Free tier / older GitLab will repeatedly emit warning noise every sync and waste calls. Cache support status per project and re-probe on TTL.

@@ AC-6: Enrichment in Orchestrator (Integration)
- [ ] On any GraphQL error: logs warning, continues to next project (never fails the sync)
+ [ ] Unsupported capability responses (missing endpoint/type/widget) are cached per project
+ [ ] While cached unsupported, enrichment is skipped without repeated warning spam
+ [ ] Capability cache auto-expires (default 24h) and is re-probed

@@ Migration Numbering
- This feature uses **migration 021**.
+ This feature uses **migrations 021-022**.

@@ Files Changed (Summary)
+ `migrations/022_project_capabilities.sql` | NEW — support cache table for project capabilities

3. Delta-first enrichment with periodic full reconcile (must-have) Reasoning: Full GraphQL scan every sync is expensive for large projects. You already compute issue deltas in ingestion; use that as fast path and keep a periodic full sweep as safety net.

@@ AC-6: Enrichment in Orchestrator (Integration)
- [ ] Runs on every sync (not gated by `--full`)
+ [ ] Fast path: skip status enrichment when issue ingestion upserted 0 issues for that project
+ [ ] Safety net: run full reconciliation every `status_full_reconcile_hours` (default 24)
+ [ ] `--full` always forces reconciliation

@@ AC-5: Config Toggle (Unit)
+ [ ] `SyncConfig` has `status_full_reconcile_hours: u32` (default 24)

4. Strongly typed widget parsing via __typename (must-have) Reasoning: current “deserialize arbitrary widget JSON into StatusWidget” is fragile. Query/type by __typename for forward compatibility and fewer silent parse mistakes.

@@ AC-3: Status Fetcher (Integration)
- [ ] Extracts status from `widgets` array by matching `WorkItemWidgetStatus` fragment
+ [ ] Query includes `widgets { __typename ... }` and parser matches `__typename == "WorkItemWidgetStatus"`
+ [ ] Non-status widgets are ignored deterministically (no heuristic JSON-deserialize attempts)

@@ GraphQL Query
+       widgets {
+         __typename
+         ... on WorkItemWidgetStatus { ... }
+       }

5. Set-based transactional DB apply (must-have) Reasoning: row-by-row clear/update loops will be slow on large projects and hold write locks longer. Temp-table + set-based SQL inside one txn is faster and easier to reason about rollback.

@@ AC-3: Status Fetcher (Integration)
- `all_fetched_iids: Vec<i64>`
+ `all_fetched_iids: HashSet<i64>`

@@ AC-6: Enrichment in Orchestrator (Integration)
- [ ] Before applying updates, NULL out status fields ... (loop per IID)
- [ ] UPDATE SQL: `SET status_name=?, ... WHERE project_id=? AND iid=?`
+ [ ] Use temp tables and set-based SQL in one transaction:
+ [ ]   (1) clear stale statuses for fetched IIDs absent from status rows
+ [ ]   (2) apply status values for fetched IIDs with status
+ [ ] One commit per project; rollback leaves prior state intact

6. Fix index strategy for COLLATE NOCASE + default sorting (must-have) Reasoning: your proposed (project_id, status_name) index may not fully help COLLATE NOCASE + ORDER BY updated_at. Tune index to real query shape in src/cli/commands/list.rs.

@@ AC-4: Migration 021 (Unit)
- [ ] Adds compound index `idx_issues_project_status_name(project_id, status_name)` for `--status` filter performance
+ [ ] Adds covering NOCASE-aware index:
+ [ ]   `idx_issues_project_status_name_nocase_updated(project_id, status_name COLLATE NOCASE, updated_at DESC)`
+ [ ] Adds category index:
+ [ ]   `idx_issues_project_status_category_nocase(project_id, status_category COLLATE NOCASE)`

7. Add stable/automation-friendly filters now (high-value feature) Reasoning: status names are user-customizable and renameable; category is more stable. Also add --no-status for quality checks and migration visibility.

@@ AC-9: List Issues Filter (E2E)
+ [ ] `lore list issues --status-category in_progress` filters by category (case-insensitive)
+ [ ] `lore list issues --no-status` returns only issues where `status_name IS NULL`
+ [ ] `--status` + `--status-category` combine with AND logic

@@ File 9: `src/cli/mod.rs`
+ Add flags: `--status-category`, `--no-status`

@@ File 11: `src/cli/autocorrect.rs`
+ Register `--status-category` and `--no-status` for `issues`

8. Better enrichment observability and failure accounting (must-have ops) Reasoning: only tracking statuses_enriched hides skipped/cleared/errors, and auth failures become silent partial data quality issues. Add counters and explicit progress events.

@@ AC-6: Enrichment in Orchestrator (Integration)
- [ ] `IngestProjectResult` gains `statuses_enriched: usize` counter
- [ ] Progress event: `ProgressEvent::StatusEnrichmentComplete { enriched: usize }`
+ [ ] `IngestProjectResult` gains:
+ [ ]   `statuses_enriched`, `statuses_cleared`, `status_enrichment_skipped`, `status_enrichment_failed`
+ [ ] Progress events:
+ [ ]   `StatusEnrichmentStarted`, `StatusEnrichmentSkipped`, `StatusEnrichmentComplete`, `StatusEnrichmentFailed`
+ [ ] End-of-sync summary includes per-project enrichment outcome counts

9. Add status_changed_at for immediately useful workflow analytics (high-value feature) Reasoning: without change timestamp, you can’t answer “how long has this been in progress?” which is one of the most useful agent/human queries.

@@ AC-4: Migration 021 (Unit)
+ [ ] Adds nullable INTEGER column `status_changed_at` (ms epoch UTC)

@@ AC-6: Enrichment in Orchestrator (Integration)
+ [ ] If status_name/category changes, update `status_changed_at = now_ms()`
+ [ ] If status is cleared, set `status_changed_at = NULL`

@@ AC-9: List Issues Filter (E2E)
+ [ ] `lore list issues --stale-status-days N` filters by `status_changed_at <= now - N days`

10. Expand test matrix for real-world failure/perf paths (must-have) Reasoning: current tests are good, but the highest-risk failures are retry behavior, capability caching, idempotency under repeated runs, and large-project performance.

@@ TDD Plan — RED Phase
+ 26. `test_graphql_retries_429_with_retry_after_then_succeeds`
+ 27. `test_graphql_retries_503_then_fails_after_max_attempts`
+ 28. `test_capability_cache_skips_unsupported_project_until_ttl_expiry`
+ 29. `test_delta_skip_when_no_issue_upserts`
+ 30. `test_periodic_full_reconcile_runs_after_threshold`
+ 31. `test_set_based_enrichment_scales_10k_issues_without_timeout`
+ 32. `test_enrichment_idempotent_across_two_runs`
+ 33. `test_status_changed_at_updates_only_on_actual_status_change`

If you want, I can now produce a single consolidated revised plan document (full rewritten Markdown) with these changes merged in-place so it’s ready to execute.