feat(surgical-sync): add per-IID surgical sync pipeline with preflight validation
Add the ability to sync specific issues or merge requests by IID without
running a full incremental sync. This enables fast, targeted data refresh
for individual entities — useful for agent workflows, debugging, and
real-time investigation of specific issues or MRs.
Architecture:
- New CLI flags: --issue <IID> and --mr <IID> (repeatable, up to 100 total)
scoped to a single project via -p/--project
- Preflight phase validates all IIDs exist on GitLab before any DB writes,
with TOCTOU-aware soft verification at ingest time
- 6-stage pipeline: preflight -> fetch -> ingest -> dependents -> docs -> embed
- Each stage is cancellation-aware via ShutdownSignal
- Dedicated SyncRunRecorder extensions track surgical-specific counters
(issues_fetched, mrs_ingested, docs_regenerated, etc.)
New modules:
- src/ingestion/surgical.rs: Core surgical fetch/ingest/dependent logic
with preflight_fetch(), ingest_issue_by_iid(), ingest_mr_by_iid(),
and fetch_dependents_for_{issue,mr}()
- src/cli/commands/sync_surgical.rs: Full CLI orchestrator with progress
spinners, human/robot output, and cancellation handling
- src/embedding/pipeline.rs: embed_documents_by_ids() for scoped embedding
- src/documents/regenerator.rs: regenerate_dirty_documents_for_sources()
for scoped document regeneration
Database changes:
- Migration 027: Extends sync_runs with mode, phase, surgical_iids_json,
per-entity counters, and cancelled_at column
- New indexes: idx_sync_runs_mode_started, idx_sync_runs_status_phase_started
GitLab client:
- get_issue_by_iid() and get_mr_by_iid() single-entity fetch methods
Error handling:
- New SurgicalPreflightFailed error variant with entity_type, iid, project,
and reason fields. Shares exit code 6 with GitLabNotFound.
Includes comprehensive test coverage:
- 645 lines of surgical ingestion tests (wiremock-based)
- 184 lines of scoped embedding tests
- 85 lines of scoped regeneration tests
- 113 lines of GitLab client single-entity tests
- 236 lines of sync_run surgical column/counter tests
- Unit tests for SyncOptions, error codes, and CLI validation
This commit is contained in:
@@ -15,6 +15,7 @@ pub mod show;
|
||||
pub mod stats;
|
||||
pub mod sync;
|
||||
pub mod sync_status;
|
||||
pub mod sync_surgical;
|
||||
pub mod timeline;
|
||||
pub mod trace;
|
||||
pub mod who;
|
||||
@@ -39,7 +40,7 @@ pub use ingest::{
|
||||
DryRunPreview, IngestDisplay, print_dry_run_preview, print_dry_run_preview_json,
|
||||
print_ingest_summary, print_ingest_summary_json, run_ingest, run_ingest_dry_run,
|
||||
};
|
||||
pub use init::{InitInputs, InitOptions, InitResult, run_init};
|
||||
pub use init::{InitInputs, InitOptions, InitResult, run_init, run_token_set, run_token_show};
|
||||
pub use list::{
|
||||
ListFilters, MrListFilters, NoteListFilters, open_issue_in_browser, open_mr_in_browser,
|
||||
print_list_issues, print_list_issues_json, print_list_mrs, print_list_mrs_json,
|
||||
@@ -55,6 +56,7 @@ pub use show::{
|
||||
pub use stats::{print_stats, print_stats_json, run_stats};
|
||||
pub use sync::{SyncOptions, SyncResult, print_sync, print_sync_json, run_sync};
|
||||
pub use sync_status::{print_sync_status, print_sync_status_json, run_sync_status};
|
||||
pub use sync_surgical::run_sync_surgical;
|
||||
pub use timeline::{TimelineParams, print_timeline, print_timeline_json_with_meta, run_timeline};
|
||||
pub use trace::{parse_trace_path, print_trace, print_trace_json};
|
||||
pub use who::{WhoRun, print_who_human, print_who_json, run_who};
|
||||
|
||||
Reference in New Issue
Block a user