feat(observability): Add metrics, logging, and sync-run core modules
Introduce the foundational observability layer for the sync pipeline: - MetricsLayer: Custom tracing subscriber layer that captures span timing and structured fields, materializing them into a hierarchical Vec<StageTiming> tree for robot-mode performance data output - logging: Dual-layer subscriber infrastructure with configurable stderr verbosity (-v/-vv/-vvv) and always-on JSON file logging with daily rotation and configurable retention (default 30 days) - SyncRunRecorder: Compile-time enforced lifecycle recorder for sync_runs table (start -> succeed|fail), with correlation IDs and aggregate counts - LoggingConfig: New config section for log_dir, retention_days, and file_logging toggle - get_log_dir(): Path helper for log directory resolution - is_permanent_api_error(): Distinguish retryable vs permanent API failures (only 404 is truly permanent; 403/auth errors may be environmental) Database changes: - Migration 013: Add resource_events_synced_for_updated_at watermark columns to issues and merge_requests tables for incremental resource event sync - Migration 014: Enrich sync_runs with run_id correlation ID, aggregate counts (total_items_processed, total_errors), and run_id index - Wrap file-based migrations in savepoints for rollback safety Dependencies: Add uuid (run_id generation), tracing-appender (file logging) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
@@ -47,6 +47,10 @@ const MIGRATIONS: &[(&str, &str)] = &[
|
||||
"013",
|
||||
include_str!("../../migrations/013_resource_event_watermarks.sql"),
|
||||
),
|
||||
(
|
||||
"014",
|
||||
include_str!("../../migrations/014_sync_runs_enrichment.sql"),
|
||||
),
|
||||
];
|
||||
|
||||
/// Create a database connection with production-grade pragmas.
|
||||
@@ -190,13 +194,35 @@ pub fn run_migrations_from_dir(conn: &Connection, migrations_dir: &Path) -> Resu
|
||||
|
||||
let sql = fs::read_to_string(entry.path())?;
|
||||
|
||||
conn.execute_batch(&sql)
|
||||
// Wrap each migration in a savepoint to prevent partial application,
|
||||
// matching the safety guarantees of run_migrations().
|
||||
let savepoint_name = format!("migration_{}", version);
|
||||
conn.execute_batch(&format!("SAVEPOINT {}", savepoint_name))
|
||||
.map_err(|e| LoreError::MigrationFailed {
|
||||
version,
|
||||
message: e.to_string(),
|
||||
message: format!("Failed to create savepoint: {}", e),
|
||||
source: Some(e),
|
||||
})?;
|
||||
|
||||
match conn.execute_batch(&sql) {
|
||||
Ok(()) => {
|
||||
conn.execute_batch(&format!("RELEASE {}", savepoint_name))
|
||||
.map_err(|e| LoreError::MigrationFailed {
|
||||
version,
|
||||
message: format!("Failed to release savepoint: {}", e),
|
||||
source: Some(e),
|
||||
})?;
|
||||
}
|
||||
Err(e) => {
|
||||
let _ = conn.execute_batch(&format!("ROLLBACK TO {}", savepoint_name));
|
||||
return Err(LoreError::MigrationFailed {
|
||||
version,
|
||||
message: e.to_string(),
|
||||
source: Some(e),
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
info!(version, file = %filename_str, "Migration applied");
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user