feat(ingestion): add progress reporting for status enrichment pipeline

Previously the status enrichment phase (GraphQL work item status fetch)
ran silently — users saw no feedback between "syncing issues" and the
final enrichment summary. For projects with hundreds of issues and
adaptive page-size retries, this felt like a hang.

Changes across three layers:

GraphQL (graphql.rs):
  - Extract fetch_issue_statuses_with_progress() accepting an optional
    on_page callback invoked after each paginated fetch with the
    running count of fetched IIDs
  - Original fetch_issue_statuses() preserved as a zero-cost
    delegation wrapper (no callback overhead)

Orchestrator (orchestrator.rs):
  - Three new ProgressEvent variants: StatusEnrichmentStarted,
    StatusEnrichmentPageFetched, StatusEnrichmentWriting
  - Wire the page callback through to the new _with_progress fn

CLI (ingest.rs):
  - Handle all three new events in the progress callback, updating
    both the per-project spinner and the stage bar with live counts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Taylor Eernisse
2026-02-11 10:22:20 -05:00
parent 70271c14d6
commit e9af529f6e
3 changed files with 57 additions and 1 deletions

View File

@@ -45,6 +45,9 @@ pub enum ProgressEvent {
MrDiffsFetchStarted { total: usize },
MrDiffFetched { current: usize, total: usize },
MrDiffsFetchComplete { fetched: usize, failed: usize },
StatusEnrichmentStarted,
StatusEnrichmentPageFetched { items_so_far: usize },
StatusEnrichmentWriting { total: usize },
StatusEnrichmentComplete { enriched: usize, cleared: usize },
StatusEnrichmentSkipped,
}
@@ -150,6 +153,8 @@ pub async fn ingest_project_issues_with_progress(
if config.sync.fetch_work_item_status && !signal.is_cancelled() {
use rusqlite::OptionalExtension;
emit(ProgressEvent::StatusEnrichmentStarted);
let project_path: Option<String> = conn
.query_row(
"SELECT path_with_namespace FROM projects WHERE id = ?1",
@@ -170,7 +175,16 @@ pub async fn ingest_project_issues_with_progress(
}
Some(path) => {
let graphql_client = client.graphql_client();
match crate::gitlab::graphql::fetch_issue_statuses(&graphql_client, &path).await {
let page_cb = |items_so_far: usize| {
emit(ProgressEvent::StatusEnrichmentPageFetched { items_so_far });
};
match crate::gitlab::graphql::fetch_issue_statuses_with_progress(
&graphql_client,
&path,
Some(&page_cb),
)
.await
{
Ok(fetch_result) => {
if let Some(ref reason) = fetch_result.unsupported_reason {
result.status_enrichment_mode = "unsupported".into();
@@ -199,6 +213,9 @@ pub async fn ingest_project_issues_with_progress(
cleared: 0,
});
} else {
emit(ProgressEvent::StatusEnrichmentWriting {
total: fetch_result.all_fetched_iids.len(),
});
match enrich_issue_statuses_txn(
conn,
project_id,