Your plan is already strong and implementation-aware. The best upgrades are mostly about reliability under real-world API instability, large-scale performance, and making the feature more useful for automation. 1. Promote retry/backoff from deferred to in-scope now. Reason: Right now, transient failures cause silent status gaps until a later sync. Bounded retries with jitter and a time budget dramatically improve successful enrichment without making syncs hang. ```diff @@ AC-1: GraphQL Client (Unit) @@ - [ ] Network error → `LoreError::Other` + [ ] Transient failures (`429`, `502`, `503`, `504`, timeout, connect reset) retry with exponential backoff + jitter (max 3 attempts) + [ ] `Retry-After` supports both delta-seconds and HTTP-date formats + [ ] Per-request retry budget capped (e.g. 120s total) to preserve cancellation responsiveness @@ AC-6: Enrichment in Orchestrator (Integration) @@ - [ ] On any GraphQL error: logs warning, continues to next project (never fails the sync) + [ ] On transient GraphQL errors: retry policy applied before warning/skip behavior @@ Decisions @@ - 8. **No retry/backoff in v1** — DEFER. + 8. **Retry/backoff in v1** — YES. Required for reliable enrichment under normal GitLab/API turbulence. ``` 2. Add a capability cache so unsupported projects stop paying repeated GraphQL cost. Reason: Free tier / older instances will never return status widgets. Re-querying every sync is wasted time and noisy logs. ```diff @@ Acceptance Criteria @@ + ### AC-11: Capability Probe & Cache (Integration) + - [ ] Add `project_capabilities` cache with `supports_work_item_status`, `checked_at`, `cooldown_until` + - [ ] 404/403/known-unsupported responses update capability cache and suppress repeated warnings until TTL expires + - [ ] Supported projects still enrich every run (subject to normal schedule) @@ Future Enhancements (Not in Scope) @@ - **Capability probe/cache**: Detect status-widget support per project ... (deferred) + (moved into scope as AC-11) ``` 3. Make enrichment delta-aware with periodic forced reconciliation. Reason: Full pagination every sync is expensive on large projects. You can skip unnecessary status fetches when no issue changes occurred, while still doing periodic safety sweeps. ```diff @@ AC-6: Enrichment in Orchestrator (Integration) @@ - [ ] Runs on every sync (not gated by `--full`) + [ ] Runs when issue ingestion reports project issue deltas OR reconcile window elapsed + [ ] New config: `status_reconcile_hours` (default: 24) for periodic full sweep + [ ] `--refresh-status` forces enrichment regardless of delta/reconcile window ``` 4. Replace row-by-row update loops with set-based SQL via temp table. Reason: Current per-IID loops are simple but slow at scale and hold locks longer. Set-based updates are much faster and reduce lock contention. ```diff @@ File 6: `src/ingestion/orchestrator.rs` (MODIFY) @@ - for iid in all_fetched_iids { ... UPDATE issues ... } - for (iid, status) in statuses { ... UPDATE issues ... } + CREATE TEMP TABLE temp_issue_status_updates(...) + bulk INSERT temp rows (iid, name, category, color, icon_name) + single set-based UPDATE for enriched rows + single set-based NULL-clear for fetched-without-status rows + commit transaction ``` 5. Add strict mode and explicit partial-failure reporting. Reason: “Warn and continue” is good default UX, but automation needs a fail-fast option and machine-readable failure output. ```diff @@ AC-5: Config Toggle (Unit) @@ + - [ ] `SyncConfig` adds `status_enrichment_strict: bool` (default false) @@ AC-6: Enrichment in Orchestrator (Integration) @@ - [ ] On any GraphQL error: logs warning, continues to next project (never fails the sync) + [ ] Default mode: warn + continue + [ ] Strict mode: status enrichment error fails sync for that run @@ AC-6: IngestProjectResult @@ + - [ ] Adds `status_enrichment_error: Option` @@ AC-8 / Robot sync envelope @@ + - [ ] Robot output includes `partial_failures` array with per-project enrichment failures ``` 6. Fix case-insensitive matching robustness and track freshness. Reason: SQLite `COLLATE NOCASE` is ASCII-centric; custom statuses may be non-ASCII. Also you need visibility into staleness. ```diff @@ AC-4: Migration 021 (Unit) @@ - [ ] Migration adds 4 nullable TEXT columns to `issues` + [ ] Migration adds 6 columns: + `status_name`, `status_category`, `status_color`, `status_icon_name`, + `status_name_fold`, `status_synced_at` - [ ] Adds compound index `idx_issues_project_status_name(project_id, status_name)` + [ ] Adds compound index `idx_issues_project_status_name_fold(project_id, status_name_fold)` @@ AC-9: List Issues Filter (E2E) @@ - [ ] Filter uses case-insensitive matching (`COLLATE NOCASE`) + [ ] Filter uses `status_name_fold` (Unicode-safe fold normalization done at write time) ``` 7. Expand filtering to category and missing-status workflows. Reason: Name filters are useful, but automation is better on semantic categories and “missing data” detection. ```diff @@ AC-9: List Issues Filter (E2E) @@ + - [ ] `--status-category in_progress` filters by `status_category` (case-insensitive) + - [ ] `--no-status` returns only issues where `status_name IS NULL` + - [ ] `--status` and `--status-category` can be combined with AND logic ``` 8. Change robot payload from flat status fields to a nested `status` object. Reason: Better schema evolution and less top-level field sprawl as you add metadata (`synced_at`, future lifecycle fields). ```diff @@ AC-7: Show Issue Display (E2E) @@ - [ ] JSON includes `status_name`, `status_category`, `status_color`, `status_icon_name` fields - [ ] Fields are `null` (not absent) when status not available + [ ] JSON includes `status` object: + `{ "name", "category", "color", "icon_name", "synced_at" }` + [ ] `status: null` when not available @@ AC-8: List Issues Display (E2E) @@ - [ ] `--fields` supports: `status_name`, `status_category`, `status_color`, `status_icon_name` + [ ] `--fields` supports: `status.name,status.category,status.color,status.icon_name,status.synced_at` ``` If you want, I can produce a fully rewritten “Iteration 5” plan document with these changes integrated end-to-end (ACs, files, migrations, TDD batches, and updated decisions/future-scope).