9ec1344945
Add the ability to sync specific issues or merge requests by IID without
running a full incremental sync. This enables fast, targeted data refresh
for individual entities — useful for agent workflows, debugging, and
real-time investigation of specific issues or MRs.
Architecture:
- New CLI flags: --issue <IID> and --mr <IID> (repeatable, up to 100 total)
scoped to a single project via -p/--project
- Preflight phase validates all IIDs exist on GitLab before any DB writes,
with TOCTOU-aware soft verification at ingest time
- 6-stage pipeline: preflight -> fetch -> ingest -> dependents -> docs -> embed
- Each stage is cancellation-aware via ShutdownSignal
- Dedicated SyncRunRecorder extensions track surgical-specific counters
(issues_fetched, mrs_ingested, docs_regenerated, etc.)
New modules:
- src/ingestion/surgical.rs: Core surgical fetch/ingest/dependent logic
with preflight_fetch(), ingest_issue_by_iid(), ingest_mr_by_iid(),
and fetch_dependents_for_{issue,mr}()
- src/cli/commands/sync_surgical.rs: Full CLI orchestrator with progress
spinners, human/robot output, and cancellation handling
- src/embedding/pipeline.rs: embed_documents_by_ids() for scoped embedding
- src/documents/regenerator.rs: regenerate_dirty_documents_for_sources()
for scoped document regeneration
Database changes:
- Migration 027: Extends sync_runs with mode, phase, surgical_iids_json,
per-entity counters, and cancelled_at column
- New indexes: idx_sync_runs_mode_started, idx_sync_runs_status_phase_started
GitLab client:
- get_issue_by_iid() and get_mr_by_iid() single-entity fetch methods
Error handling:
- New SurgicalPreflightFailed error variant with entity_type, iid, project,
and reason fields. Shares exit code 6 with GitLabNotFound.
Includes comprehensive test coverage:
- 645 lines of surgical ingestion tests (wiremock-based)
- 184 lines of scoped embedding tests
- 85 lines of scoped regeneration tests
- 113 lines of GitLab client single-entity tests
- 236 lines of sync_run surgical column/counter tests
- Unit tests for SyncOptions, error codes, and CLI validation
39 lines
1.3 KiB
Rust
39 lines
1.3 KiB
Rust
pub mod dirty_tracker;
|
|
pub mod discussion_queue;
|
|
pub mod discussions;
|
|
pub mod issues;
|
|
pub mod merge_requests;
|
|
pub mod mr_diffs;
|
|
pub mod mr_discussions;
|
|
pub mod orchestrator;
|
|
pub(crate) mod surgical;
|
|
|
|
pub use discussions::{IngestDiscussionsResult, ingest_issue_discussions};
|
|
pub use issues::{IngestIssuesResult, IssueForDiscussionSync, ingest_issues};
|
|
pub use merge_requests::{
|
|
IngestMergeRequestsResult, MrForDiscussionSync, get_mrs_needing_discussion_sync,
|
|
ingest_merge_requests,
|
|
};
|
|
pub use mr_discussions::{IngestMrDiscussionsResult, ingest_mr_discussions};
|
|
/// Format a set of named counters as a compact human-readable summary,
|
|
/// filtering out zero values and joining with middle-dot separators.
|
|
/// Returns `"nothing to update"` when all values are zero.
|
|
pub(crate) fn nonzero_summary(pairs: &[(&str, usize)]) -> String {
|
|
let parts: Vec<String> = pairs
|
|
.iter()
|
|
.filter(|(_, v)| *v > 0)
|
|
.map(|(k, v)| format!("{v} {k}"))
|
|
.collect();
|
|
if parts.is_empty() {
|
|
"nothing to update".to_string()
|
|
} else {
|
|
parts.join(" \u{b7} ")
|
|
}
|
|
}
|
|
|
|
pub use orchestrator::{
|
|
DrainResult, IngestMrProjectResult, IngestProjectResult, ProgressCallback, ProgressEvent,
|
|
ingest_project_issues, ingest_project_issues_with_progress, ingest_project_merge_requests,
|
|
ingest_project_merge_requests_with_progress,
|
|
};
|