fix: Project-scoped job claiming, structured rate-limit logging, RRF total_cmp

Targeted fixes across multiple subsystems:

dependent_queue:
- Add project_id parameter to claim_jobs() for project-scoped job claiming,
  preventing cross-project job theft during concurrent multi-project ingestion
- Add project_id parameter to count_pending_jobs() with optional scoping
  (None returns global counts, Some(pid) returns per-project counts)

gitlab/client:
- Downgrade rate-limit log from warn to info (429s are expected operational
  behavior, not warnings) and add structured fields (path, status_code)
  for better log filtering and aggregation

gitlab/transformers/discussion:
- Add tracing::warn on invalid timestamp parse instead of silent fallback
  to epoch 0, making data quality issues visible in logs

ingestion/merge_requests:
- Remove duplicate doc comment on upsert_label_tx

search/rrf:
- Replace partial_cmp().unwrap_or() with total_cmp() for f64 sorting,
  eliminating the NaN edge case entirely (total_cmp treats NaN consistently)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
This commit is contained in:
teernisse
2026-02-04 10:01:28 -05:00
parent f6d19a9467
commit 86a51cddef
6 changed files with 102 additions and 38 deletions

View File

@@ -144,11 +144,12 @@ impl GitLabClient {
if response.status() == StatusCode::TOO_MANY_REQUESTS && attempt < Self::MAX_RETRIES {
let retry_after = Self::parse_retry_after(&response);
tracing::warn!(
retry_after_secs = retry_after,
tracing::info!(
path = %path,
attempt,
path,
"Rate limited by GitLab, retrying"
retry_after_secs = retry_after,
status_code = 429u16,
"Rate limited, retrying"
);
sleep(Duration::from_secs(retry_after)).await;
continue;
@@ -565,11 +566,12 @@ impl GitLabClient {
if response.status() == StatusCode::TOO_MANY_REQUESTS && attempt < Self::MAX_RETRIES {
let retry_after = Self::parse_retry_after(&response);
tracing::warn!(
retry_after_secs = retry_after,
tracing::info!(
path = %path,
attempt,
path,
"Rate limited by GitLab, retrying"
retry_after_secs = retry_after,
status_code = 429u16,
"Rate limited, retrying"
);
sleep(Duration::from_secs(retry_after)).await;
continue;

View File

@@ -1,5 +1,7 @@
//! Discussion and note transformers: convert GitLab discussions to local schema.
use tracing::warn;
use crate::core::time::{iso_to_ms, iso_to_ms_strict, now_ms};
use crate::gitlab::types::{GitLabDiscussion, GitLabNote};
@@ -60,7 +62,13 @@ pub struct NormalizedNote {
/// Parse ISO 8601 timestamp to milliseconds, defaulting to 0 on failure.
fn parse_timestamp(ts: &str) -> i64 {
iso_to_ms(ts).unwrap_or(0)
match iso_to_ms(ts) {
Some(ms) => ms,
None => {
warn!(timestamp = ts, "Invalid timestamp, defaulting to epoch 0");
0
}
}
}
/// Transform a GitLab discussion into normalized schema.