Add the ability to sync specific issues or merge requests by IID without
running a full incremental sync. This enables fast, targeted data refresh
for individual entities — useful for agent workflows, debugging, and
real-time investigation of specific issues or MRs.
Architecture:
- New CLI flags: --issue <IID> and --mr <IID> (repeatable, up to 100 total)
scoped to a single project via -p/--project
- Preflight phase validates all IIDs exist on GitLab before any DB writes,
with TOCTOU-aware soft verification at ingest time
- 6-stage pipeline: preflight -> fetch -> ingest -> dependents -> docs -> embed
- Each stage is cancellation-aware via ShutdownSignal
- Dedicated SyncRunRecorder extensions track surgical-specific counters
(issues_fetched, mrs_ingested, docs_regenerated, etc.)
New modules:
- src/ingestion/surgical.rs: Core surgical fetch/ingest/dependent logic
with preflight_fetch(), ingest_issue_by_iid(), ingest_mr_by_iid(),
and fetch_dependents_for_{issue,mr}()
- src/cli/commands/sync_surgical.rs: Full CLI orchestrator with progress
spinners, human/robot output, and cancellation handling
- src/embedding/pipeline.rs: embed_documents_by_ids() for scoped embedding
- src/documents/regenerator.rs: regenerate_dirty_documents_for_sources()
for scoped document regeneration
Database changes:
- Migration 027: Extends sync_runs with mode, phase, surgical_iids_json,
per-entity counters, and cancelled_at column
- New indexes: idx_sync_runs_mode_started, idx_sync_runs_status_phase_started
GitLab client:
- get_issue_by_iid() and get_mr_by_iid() single-entity fetch methods
Error handling:
- New SurgicalPreflightFailed error variant with entity_type, iid, project,
and reason fields. Shares exit code 6 with GitLabNotFound.
Includes comprehensive test coverage:
- 645 lines of surgical ingestion tests (wiremock-based)
- 184 lines of scoped embedding tests
- 85 lines of scoped regeneration tests
- 113 lines of GitLab client single-entity tests
- 236 lines of sync_run surgical column/counter tests
- Unit tests for SyncOptions, error codes, and CLI validation
Change the `who` command's --limit flag from default=20 to optional,
so omitting it returns all results. This matches the behavior users
expect when they want a complete expert/workload/active/overlap listing
without an arbitrary cap.
Also applies clippy-recommended sort improvements:
- who/reviews: sort_by(|a,b| b.count.cmp(&a.count)) -> sort_by_key with Reverse
- drift: same pattern for frequency sorting
Adds Theme::color_icon() helper to DRY the stage-icon coloring pattern
used in sync output (was inline closure, now shared method).
Introduce a centralized token resolution system that supports both
environment variables and config-file-stored tokens with clear priority
(env var wins). This enables cron-based sync which runs in minimal
shell environments without env vars.
Core changes:
- GitLabConfig gains optional `token` field and `resolve_token()` method
that checks env var first, then config file, returning trimmed values
- `token_source()` returns human-readable provenance ("environment variable"
or "config file") for diagnostics
- `ensure_config_permissions()` enforces 0600 on config files containing
tokens (Unix only, no-op on other platforms)
New CLI commands:
- `lore token set [--token VALUE]` — validates against GitLab API, stores
in config, enforces file permissions. Supports flag, stdin pipe, or
interactive entry.
- `lore token show [--unmask]` — displays masked token with source label
Consumers updated to use resolve_token():
- auth_test: removes manual env var lookup
- doctor: shows token source in health check output
- ingest: uses centralized resolution
Includes 10 unit tests for resolve/source logic and 2 for mask_token.
Add lore cron {install,uninstall,status} to manage a crontab entry that
runs lore sync on a configurable interval. Supports both human and robot
output modes.
Core implementation (src/core/cron.rs):
- install_cron: appends a tagged crontab entry, detects existing entries
- uninstall_cron: removes the tagged entry
- cron_status: reads crontab + checks last-sync time from the database
- Unix-only (#[cfg(unix)]) — compiles out on Windows
CLI wiring:
- CronAction enum and CronArgs in cli/mod.rs with after_help examples
- Robot JSON envelope with RobotMeta timing for all 3 sub-actions
- Dispatch in main.rs
Also in this commit:
- Add after_help example blocks to Status, Auth, Doctor, Init, Migrate,
Health commands for better discoverability
- Add LORE_ICONS env var documentation to CLI help text
- Simplify notes format dispatch in main.rs (removed csv/jsonl paths)
- Update commands/mod.rs re-exports for cron + notes cleanup
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove print_list_notes_csv, print_list_notes_jsonl, and csv_escape from
the notes list command. The --format flag's csv and jsonl variants added
complexity without meaningful adoption — robot mode already provides
structured JSON output. Notes now have two output paths: human (default)
and JSON (--robot).
Also removes the corresponding test coverage (csv_escape, csv_output).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace serde_json::to_string(&output).unwrap() with match-based error
handling across all robot-mode JSON printers. On serialization failure,
the error is now written to stderr instead of panicking. This hardens
the CLI against unexpected Serialize failures in production.
Affected commands: count (2), embed, generate-docs, ingest (2), search,
stats, sync (2), sync-status, timeline.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add structured tracing spans to trace and file-history pipelines so debug
logging (-vv) shows path resolution counts, MR match counts, and discussion
counts at each stage. This makes empty-result debugging straightforward.
Add a hints field to TraceResult and FileHistoryResult that carries
machine-readable diagnostic strings explaining *why* results may be empty
(e.g., "Run 'lore sync' to fetch MR file changes"). The CLI renders these
as info lines; robot mode includes them in JSON when non-empty.
Also: fix filter_map(Result::ok) → collect::<Result> in trace.rs (same
pattern fixed in prior commit for file_history/path_resolver), and switch
conn.prepare → conn.prepare_cached for the MR query.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Split the monolithic who.rs into a who/ directory module with 7 focused
files. The 5 query modes (expert, workload, reviews, active, overlap) share
no query-level code — only types and a few small helpers — making this a
clean mechanical extraction.
New structure:
who/types.rs — all pub result structs/enums (~185 lines)
who/mod.rs — dispatch, shared helpers, JSON envelope (~428 lines)
who/expert.rs — query + render + json for expert mode (~839 lines)
who/workload.rs — query + render + json for workload mode (~370 lines)
who/reviews.rs — query + render + json for reviews mode (~214 lines)
who/active.rs — query + render + json for active mode (~299 lines)
who/overlap.rs — query + render + json for overlap mode (~323 lines)
Token savings: an agent working on any single mode now loads ~400-960 lines
instead of 2,598 (63-85% reduction). Public API unchanged — parent mod.rs
re-exports are identical.
Test re-exports use #[cfg(test)] use (not pub use) to avoid visibility
conflicts with pub(super) items in submodules. All 79 who tests pass.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Workload and active modes now exclude discussions on closed issues and
merged/closed MRs by default. Adds --include-closed flag to restore
the previous behavior when needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Overhaul the sync command's human output to use semantic colors and a
cleaner rendering architecture. The changes fall into four areas:
Stage lines: Replace direct finish_stage() calls with an
emit_stage_line/emit_stage_block pattern that clears the spinner first,
then prints static lines via MultiProgress::suspend. Stage icons are
now color-coded green (success) or yellow (warning) via color_icon().
A separate "Status" stage line now appears after Issues, summarizing
work-item status enrichment across all projects.
Sub-rows: Replace the imperative print_issue_sub_rows/print_mr_sub_rows
functions with functional issue_sub_rows(), mr_sub_rows(), and new
status_sub_rows() that return Vec<String>. Project paths use
Theme::muted(), error/failure counts use Theme::warning(), and
separators use the dim middle-dot style. Sub-rows are printed atomically
with their parent stage line to avoid interleaving with spinners.
Summary: In print_sync(), counts now use Theme::info().bold() for visual
pop, detail-line separators are individually styled (dim middle-dot),
and a new "Sync completed with issues" headline appears when any stage
had failures. Document errors and embedding failures are surfaced in
both the doc-parts line and the errors line.
Tests: Full coverage for append_failures, summarize_status_enrichment,
should_print_timings, issue_sub_rows, mr_sub_rows, and status_sub_rows.
Add -t/--timings flag to the sync subcommand, allowing users to opt
into a per-stage timing breakdown after the sync summary. Wire the flag
through main.rs into print_sync() which passes it to the new
should_print_timings() gate.
Enrich the data structures that flow through the sync pipeline so
downstream renderers have full error visibility:
- ProjectSummary gains status_errors (issue-side status enrichment
failures per project)
- ProjectStatusEnrichment gains path (project path for sub-row display)
- SyncResult gains documents_errored and embedding_failed so the
summary can surface doc-gen and embed failures separately
- Autocorrect table updated with --timings for fuzzy flag matching
Miscellaneous fixes across CLI and core modules:
- Timeline: widen TAG_WIDTH from 10 to 11 to accommodate longer event
type labels without truncation
- render.rs: save and restore LORE_ICONS env var in glyph_mode test to
prevent interference from the test environment leaking into or from
other tests that set LORE_ICONS
- logging.rs: adjust verbose=1 to info level (was debug), verbose=2 to
debug — this reduces noise at -v while keeping -vv as the full debug
experience
- issues.rs, merge_requests.rs: use infodebug! macro consistently for
ingestion summary logging
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add per-project detail rows beneath stage completion lines during multi-project
syncs, showing itemized counts (issues/MRs, discussions, events, statuses, diffs)
for each project. Previously, only aggregate totals were visible, making it hard
to diagnose which project contributed what during a sync.
Status enrichment gets proper progress bars replacing the old spinner-only
display: StatusEnrichmentStarted now carries a total count so the CLI can
render a determinate bar with rate and ETA. The enrichment SQL is tightened
to use IS NOT comparisons for diff-only UPDATEs (skip rows where values
haven't changed), and a follow-up touch_stmt ensures status_synced_at is
updated even for unchanged rows so staleness detection works correctly.
Other improvements:
- New ProjectSummary struct aggregates per-project metrics during ingestion
- SyncResult gains statuses_enriched + per-project summary vectors
- "Already up to date" message when sync finds zero changes
- Remove Arc<AtomicBool> tick_started pattern from docs/embed stages
(enable_steady_tick is idempotent, the guard was unnecessary)
- Progress bar styling: dim spinner, dark_gray track, per_sec + eta display
- Tick intervals tightened from 100ms to 60ms for smoother animation
- statuses_without_widget calculation uses fetch_result.statuses.len()
instead of subtracting enriched (more accurate when some statuses lack
work item widgets)
- Status enrichment completion log downgraded from info to debug
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 7 cleanup: migrate timeline.rs and main.rs search spinner
from stage_spinner() to stage_spinner_v2() with proper icon labels,
then remove the now-unused stage_spinner() function and its tests.
No external callers remain for the old numbered-stage API.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 6 of the UX overhaul. Applies consistent visual treatment across
the remaining command outputs: stats, doctor, timeline, who, count,
and drift.
Stats (stats.rs):
- Apply render::format_number() to all numeric values (documents,
FTS indexed, embedding counts, chunks) for thousand-separator
formatting in large databases
Doctor (doctor.rs):
- Replace Unicode check/warning/cross symbols with Icons::success(),
Icons::warning(), Icons::error() for glyph-mode awareness
- Add summary line after checks showing "Ready/Not ready" with counts
of passed, warnings, and failed checks separated by middle dots
- Remove "lore doctor" title header for cleaner output
Count (count.rs):
- Right-align numeric values with {:>10} format for columnar output
in count and state breakdown displays
Timeline (timeline.rs):
- Add entity icons (issue/MR) before entity references in event rows
- Refactor format_event_tag to pad plain text before applying style,
preventing ANSI codes from breaking column alignment
- Extract style_padded() helper for width-then-style pattern
Who (who.rs):
- Add Icons::user() before usernames in expert, workload, reviews,
and overlap displays
- Replace manual bold section headers with render::section_divider()
in workload view (Assigned Issues, Authored MRs, Reviewing MRs,
Unresolved Discussions)
Drift (drift.rs):
- Add Icons::error()/success() before drift detection status line
- Replace '#' bar character with Unicode full block for similarity
curve visualization
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 4 of the UX overhaul. Restructures the show issue and show MR
detail displays with consistent section layout, state icons, and
improved typography.
Issue detail changes:
- Replace bold header + box-drawing underline with indented title using
Theme::bold() for the title text only
- Organize fields into named sections using render::section_divider():
Details, Development, Description, Discussions
- Add state icons (Icons::issue_opened/closed) alongside text labels
- Add relative time in parentheses next to Created/Updated dates
- Switch labels from "Labels: (none)" to only showing when present,
using format_labels_bare for clean comma-separated output
- Move URL and confidential indicator into Details section
- Closing MRs show state-colored icons (merged/opened/closed)
- Discussions use section_divider instead of bold text, remove colons
from author lines, adjust wrap widths for consistent indentation
MR detail changes:
- Same section-divider layout: Details, Description, Discussions
- State icons for opened/merged/closed using Icons::mr_* helpers
- Draft indicator uses Icons::mr_draft() instead of [Draft] text prefix
- Relative times added to Created, Updated, Merged, Closed dates
- Reviewers and Assignees fields aligned with fixed-width labels
- Labels shown only when present, using format_labels_bare
- Discussion formatting matches issue detail style
Both views use 5-space left indent for field alignment and consistent
wrap widths (72 for descriptions, 68/66 for discussion notes/replies).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 3 of the UX overhaul. Enhances the issues, merge requests, and
notes list displays with visual indicators and improved formatting.
List display changes (src/cli/commands/list.rs):
- Add state icons to issues (opened/closed) and merge requests
(opened/merged/closed) using Icons:: helpers alongside text labels
- Replace [DRAFT] prefix with Icons::mr_draft() glyph for draft MRs
- Switch from format_relative_time to format_relative_time_compact for
tighter column widths in tabular output
- Switch from format_labels to format_labels_bare for unlabeled style
- Change format_discussions() return type from String to StyledCell so
unresolved counts render with Theme::warning() color inline
- Bold the section headers ("Issues", "Merge Requests", "Notes")
with count separated from the label for cleaner scanning
- Import Icons from render module
Test updates (src/cli/commands/list_tests.rs):
- Update format_discussions tests to assert on StyledCell.text field
instead of raw String, since the function now returns styled output
- The unresolved-count test checks starts_with/contains to handle
embedded ANSI escape codes from Theme::warning()
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 2 of the UX overhaul. Replaces the old numbered-stage progress
system (1/4, 2/4...) and manual indicatif ProgressBar/ProgressStyle
setup with the new centralized progress helpers.
Sync command changes (src/cli/commands/sync.rs):
- Replace stage_spinner(n, total, msg) with stage_spinner_v2(icon, label, status)
removing the rigid numbered-stage counter in favor of named stages
- Replace manual ProgressBar::new + ProgressStyle::default_bar for docs
and embed sub-progress with nested_progress(label, len, robot_mode)
- Add finish_stage() calls that display a completion summary with
elapsed time, e.g. "Issues 42 issues from 3 projects 1.2s"
- Each stage (Issues, MRs, Docs, Embed) now reports what it did on
completion rather than just clearing the spinner silently
- Embed failure path uses Icons::warning() instead of inline Theme
formatting, keeping error display consistent with success path
- Remove indicatif direct dependency from sync.rs (now handled by
progress module)
Main entry point changes (src/main.rs):
- Add GlyphMode detection: auto-detect Unicode/Nerd Font support or
fall back to ASCII based on --icons flag, --color=never, NO_COLOR,
or robot mode
- Update all LoreRenderer::init() calls to pass GlyphMode alongside
ColorMode for icon-aware rendering throughout the CLI
- Overhaul handle_error() formatting: use Icons::error() glyph,
bold error text, arrow prefixed action suggestions, and breathing
room with blank lines for scannability
- Migrate handle_embed() progress bar from manual ProgressBar +
ProgressStyle to nested_progress() helper, matching sync command
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Phase 5 of the UX overhaul. Migrates search result display from raw
console styling to the centralized Theme system with semantic methods,
improving visual consistency and readability.
Search result changes:
- Type badges now use semantic styles (issue_ref, mr_ref) with
fixed-width alignment for clean columnar layout
- Snippet rendering uses Theme::highlight() for matched terms and
Theme::muted() for surrounding context, replacing bold+underline
- Metadata line uses Theme::username() for authors and per-part
styling with middle-dot separators instead of a single dim line
- Result numbering uses muted style with right-aligned width
- Consistent 8-space indent for metadata, snippets, and explain lines
- Header line uses muted style for search mode instead of dim+parens
- Trailing blank line moved after the result loop instead of per-result
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace all console::style() calls in command modules with the centralized
Theme API and render:: utility functions. This ensures consistent color
behavior across the entire CLI, proper NO_COLOR/--color never support via
the LoreRenderer singleton, and eliminates duplicated formatting code.
Changes per module:
- count.rs: Theme for table headers, render::format_number replacing local
duplicate. Removed local format_number implementation.
- doctor.rs: Theme::success/warning/error for check status symbols and
messages. Unicode escapes for check/warning/cross symbols.
- drift.rs: Theme::bold/error/success for drift detection headers and
status messages.
- embed.rs: Compact output format — headline with count, zero-suppressed
detail lines, 'nothing to embed' short-circuit for no-op runs.
- generate_docs.rs: Same compact pattern — headline + detail + hint for
next step. No-op short-circuit when regenerated==0.
- ingest.rs: Theme for project summaries, sync status, dry-run preview.
All console::style -> Theme replacements.
- list.rs: Replace comfy-table with render::LoreTable for issue/MR listing.
Remove local colored_cell, colored_cell_hex, format_relative_time,
truncate_with_ellipsis, and format_labels (all moved to render.rs).
- list_tests.rs: Update test assertions to use render:: functions.
- search.rs: Add render_snippet() for FTS5 <mark> tag highlighting via
Theme::bold().underline(). Compact result layout with type badges.
- show.rs: Theme for entity detail views, delegate format_date and
wrap_text to render module.
- stats.rs: Section-based layout using render::section_divider. Compact
middle-dot format for document counts. Color-coded embedding coverage
percentage (green >=95%, yellow >=50%, red <50%).
- sync.rs: Compact sync summary — headline with counts and elapsed time,
zero-suppressed detail lines, visually prominent error-only section.
- sync_status.rs: Theme for run history headers, removed local
format_number duplicate.
- timeline.rs: Theme for headers/footers, render:: for date/truncate,
standard format! padding replacing console::pad_str.
- who.rs: Theme for all expert/workload/active/overlap/review output
modes, render:: for relative time and truncation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds issue:N / i:N / mr:N / m:N query syntax to bypass hybrid search
and seed the timeline directly from a known entity. All discussions for
the entity are gathered without needing Ollama.
- parse_timeline_query() detects entity-direct patterns
- resolve_entity_by_iid() resolves IID to EntityRef with ambiguity handling
- seed_timeline_direct() gathers all discussions for the entity
- 20 new tests (5 resolve, 6 direct seed, 9 parse)
- Updated CLI help text and robot-docs manifest
Adds numbered stage spinners ([1/3], [2/3], [3/3]) to the timeline
pipeline stages (seed, expand, collect) so users see activity during
longer queries. TimelineParams gains a robot_mode field to suppress
spinners in JSON output mode.
Adds a [1/1] spinner to the search command for consistency, using the
shared stage_spinner from cli/progress.
Also refactors wrap_snippet() to delegate to wrap_text() with a 4-line
cap, eliminating the duplicated word-wrapping logic.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Moves stage_spinner() from a private function in sync.rs to a pub function
in cli/progress.rs so it can be reused by the timeline and search commands.
The function creates a numbered spinner (e.g. [1/3]) for pipeline stages,
returning a hidden no-op bar in robot mode to keep caller code path-uniform.
sync.rs now imports from crate::cli::progress::stage_spinner instead of
defining its own copy. Adds unit tests for robot mode (hidden bar), human
mode (prefix/message properties), and prefix formatting.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Implements the downstream consumption of matched discussions from the seed
phase, completing the discussion thread feature across collect, CLI, and
integration tests.
Collect phase (timeline_collect.rs):
- New collect_discussion_threads() function assembles full threads by
querying notes for each matched discussion_id, filtering out system notes
(is_system = 0), ordering chronologically, and capping at THREAD_MAX_NOTES
with a synthetic "[N more notes not shown]" summary note
- build_entity_lookup() creates a (type, id) -> (iid, path) map from seed
and expanded entities to provide display metadata for thread events
- Thread timestamp is set to the first note's created_at for correct
chronological interleaving with other timeline events
- collect_events() gains a matched_discussions parameter; threads are
collected after entity events and before evidence note merging
CLI rendering (cli/commands/timeline.rs):
- Human mode: threads render with box-drawing borders, bold @author tags,
date-stamped notes, and word-wrapped bodies (60 char width)
- Robot mode: DiscussionThread serializes as discussion_thread kind with
note_count, full notes array (note_id, author, body, ISO created_at)
- THREAD tag in yellow for human event tag styling
- TimelineMeta gains discussion_threads_included count
Tests:
- 8 new collect tests: basic thread assembly, system note filtering, empty
thread skipping, body truncation to THREAD_NOTE_MAX_CHARS, note cap with
synthetic summary, timestamp from first note, chronological sort position,
and deduplication of duplicate discussion_ids
- Integration tests updated for new collect_events signature
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace FTS-only seed entity discovery with hybrid search (FTS + vector
via RRF), using the same search_hybrid infrastructure as the search
command. Falls back gracefully to FTS-only when Ollama is unavailable.
Changes:
- seed_timeline() now accepts OllamaClient, delegates to search_hybrid
- New resolve_documents_to_entities() replaces find_seed_entities()
- SeedResult gains search_mode field tracking actual mode used
- TimelineResult carries search_mode through to JSON renderer
- run_timeline wires up OllamaClient from config
- handle_timeline made async for the hybrid search await
- Tests updated for new function signatures
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extract shared path resolution logic from who.rs into a new
core::path_resolver module for cross-module reuse. Functions moved:
escape_like, normalize_repo_path, PathQuery, SuffixResult,
build_path_query, suffix_probe. Duplicate escape_like copies removed
from list.rs, project.rs, and filters.rs — all now import from
path_resolver.
Additionally fixes two bugs in query_expert_details() and
query_overlap() where only position_new_path was checked (missing
old_path matches for renamed files) and state filter excluded 'closed'
MRs despite the main scoring query including them with a decay
multiplier.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move inline #[cfg(test)] mod tests { ... } blocks from 22 source files
into dedicated _tests.rs companion files, wired via:
#[cfg(test)]
#[path = "module_tests.rs"]
mod tests;
This keeps implementation-focused source files leaner and more scannable
while preserving full access to private items through `use super::*;`.
Modules extracted:
core: db, note_parser, payloads, project, references, sync_run,
timeline_collect, timeline_expand, timeline_seed
cli: list (55 tests), who (75 tests)
documents: extractor (43 tests), regenerator
embedding: change_detector, chunking
gitlab: graphql (wiremock async tests), transformers/issue
ingestion: dirty_tracker, discussions, issues, mr_diffs
Also adds conflicts_with("explain_score") to the --detail flag in the
who command to prevent mutually exclusive flags from being combined.
All 629 unit tests pass. No behavior changes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make run_search async, replace hardcoded lexical mode with SearchMode::parse(),
wire search_hybrid() with OllamaClient for semantic/hybrid modes, graceful
degradation when Ollama unavailable.
Closes: bd-1ksf
Implement drift detection using cosine similarity between issue description
embedding and chronological note embeddings. Sliding window (size 3) identifies
topic drift points. Includes human and robot output formatters.
New files: drift.rs, similarity.rs
Closes: bd-1cjx
Add references_full, user_notes_count, merge_requests_count computed
fields to show issue. Add closed_at and confidential columns via
migration 023.
Closes: bd-2g50
Query optimizer fixes for the `who` and `stats` commands based on
a systematic performance audit of the SQLite query plans.
who command (expert/reviews/detail modes):
- Add INDEXED BY idx_notes_diffnote_path_created hints to all DiffNote
queries. SQLite's planner was selecting idx_notes_system (38% of rows)
over the far more selective partial index (9.3% of rows). Measured
50-133x speedup on expert queries, 26x on reviews queries.
- Reorder JOIN clauses in detail mode's MR-author sub-select to match
the index scan direction (notes -> discussions -> merge_requests).
stats command:
- Replace 12+ sequential COUNT(*) queries with conditional aggregates
(COALESCE + SUM + CASE). Documents, dirty_sources, pending_discussion_
fetches, and pending_dependent_fetches tables each scanned once instead
of 2-3 times. Measured 1.7x speedup (109ms -> 65ms warm cache).
- Switch FTS document count from COUNT(*) on the virtual table to
COUNT(*) on documents_fts_docsize shadow table (B-tree scan vs FTS5
virtual table overhead). Measured 19x speedup for that single query.
Database: 61652 docs, 282K notes, 211K discussions, 1.5GB.
Integrates the defaultProject config field across the entire CLI
surface so that omitting `-p` now falls back to the configured default.
Init command:
- New `--default-project` flag on `lore init` (and robot-mode variant)
- InitInputs.default_project: Option<String> passed through to run_init
- Validation in run_init ensures the default matches a configured path
- Interactive mode: when multiple projects are configured, prompts
whether to set a default and which project to use
- Robot mode: InitOutputJson now includes default_project (omitted when
null) for downstream automation
- Autocorrect dictionary updated with `--default-project`
Command handlers applying effective_project():
- handle_issues: list filters use config default when -p omitted
- handle_mrs: same cascading resolution for MR listing
- handle_ingest: dry-run and full sync respect the default
- handle_timeline: TimelineParams.project resolved via effective_project
- handle_search: SearchCliFilters.project resolved via effective_project
- handle_generate_docs: project filter cascades
- handle_who: falls back to config.default_project when -p omitted
- handle_count: both count subcommands respect the default
- handle_discussions: discussion count filters respect the default
Robot-docs:
- init command schema updated with --default-project flag and
response_schema showing default_project as string?
- New config_notes section documents the defaultProject field with
type, description, and example
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
(Supersedes empty commit f3788eb — jj auto-snapshot race.)
Three related refinements to how work item status is presented:
1. available_statuses in meta (list.rs, main.rs):
Robot-mode issue list responses now include meta.available_statuses —
a sorted array of all distinct status_name values in the database.
Agents can use this to validate --status filter values or display
valid options without a separate query.
2. Hide status_category from JSON (list.rs, show.rs):
status_category is a GitLab internal classification that duplicates
the state field. Switched to skip_serializing so it never appears
in JSON output while remaining available internally.
3. Simplify human-readable status display (show.rs):
Removed the "(category)" parenthetical from the Status line.
4. robot-docs schema updates (main.rs):
Documented --status filter semantics and meta.available_statuses.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Previously the status enrichment phase (GraphQL work item status fetch)
ran silently — users saw no feedback between "syncing issues" and the
final enrichment summary. For projects with hundreds of issues and
adaptive page-size retries, this felt like a hang.
Changes across three layers:
GraphQL (graphql.rs):
- Extract fetch_issue_statuses_with_progress() accepting an optional
on_page callback invoked after each paginated fetch with the
running count of fetched IIDs
- Original fetch_issue_statuses() preserved as a zero-cost
delegation wrapper (no callback overhead)
Orchestrator (orchestrator.rs):
- Three new ProgressEvent variants: StatusEnrichmentStarted,
StatusEnrichmentPageFetched, StatusEnrichmentWriting
- Wire the page callback through to the new _with_progress fn
CLI (ingest.rs):
- Handle all three new events in the progress callback, updating
both the per-project spinner and the stage bar with live counts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Work item status integration across all CLI output:
Issue listing (lore list issues):
- New Status column appears when any issue has status data, with
hex-color rendering using ANSI 256-color approximation
- New --status flag for case-insensitive filtering (OR logic for
multiple values): lore issues --status "In progress" --status "To do"
- Status fields (name, category, color, icon_name, synced_at) in issue
list query and JSON output with conditional serialization
Issue detail (lore show issue):
- Displays "Status: In progress (in_progress)" with color-coded output
using ANSI 256-color approximation from hex color values
- Status fields included in robot mode JSON with ISO timestamps
- IssueRow, IssueDetail, IssueDetailJson all carry status columns
Robot mode field selection expanded to new commands:
- search: --fields with "minimal" preset (document_id, title, source_type, score)
- timeline: --fields with "minimal" preset (timestamp, type, entity_iid, detail)
- who: --fields with per-mode presets (expert_minimal, workload_minimal, etc.)
- robot-docs: new --brief flag strips response_schema from output (~60% smaller)
- strip_schemas() utility in robot.rs for --brief mode
- expand_fields_preset() extended for search, timeline, and all who modes
Robot-docs manifest updated with --status flag documentation, --fields
flags for search/timeline/who, fields_presets sections, and corrected
search response schema field names.
Note: replaces empty commit dcfd449 which lost staging during hook execution.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add a "Phase 1.5" status enrichment step to the issue ingestion pipeline
that fetches work item statuses via the GitLab GraphQL API after the
standard REST API ingestion completes.
Schema changes (migration 021):
- Add status_name, status_category, status_color, status_icon_name, and
status_synced_at columns to the issues table (all nullable)
Ingestion pipeline changes:
- New `enrich_issue_statuses_txn()` function that applies fetched
statuses in a single transaction with two phases: clear stale statuses
for issues that no longer have a status widget, then apply new/updated
statuses from the GraphQL response
- ProgressEvent variants for status enrichment (complete/skipped)
- IngestProjectResult tracks enrichment metrics (seen, enriched, cleared,
without_widget, partial_error_count, enrichment_mode, errors)
- Robot mode JSON output includes per-project status enrichment details
Configuration:
- New `sync.fetchWorkItemStatus` config option (defaults true) to disable
GraphQL status enrichment on instances without Premium/Ultimate
- `LoreError::GitLabAuthFailed` now treated as permanent API error so
status enrichment auth failures don't trigger retries
Also removes the unnecessary nested SAVEPOINT in store_closes_issues_refs
(already runs within the orchestrator's transaction context).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Label upsert (issues + merge_requests): Replace INSERT ... ON CONFLICT DO
UPDATE RETURNING with INSERT OR IGNORE + SELECT. The prior RETURNING-based
approach relied on last_insert_rowid() matching the returned id, which is
not guaranteed when ON CONFLICT triggers an update (SQLite may return 0).
The new two-step approach is unambiguous and correctly tracks created_count.
Init: Add ON CONFLICT(gitlab_project_id) DO UPDATE to the project insert
so re-running `lore init` updates path/branch/url instead of failing with
a unique constraint violation.
MR discussions sync: Reset discussions_sync_attempts to 0 when clearing a
sync health error, so previously-failed MRs get a fresh retry budget after
successful sync.
Count: format_number now handles negative numbers correctly by extracting
the sign before inserting thousand-separators.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Expert mode now surfaces the specific MR references (project/path!iid) that
contributed to each expert's score, capped at 50 per user. A new --detail flag
adds per-MR breakdowns showing role (Author/Reviewer/both), note count, and
last activity timestamp.
Scoring weights (author_weight, reviewer_weight, note_bonus) are now
configurable via the config file's `scoring` section with validation that
rejects negative values. Defaults shift to author_weight=25, reviewer_weight=10,
note_bonus=1 — better reflecting that code authorship is a stronger expertise
signal than review assignment alone.
Path resolution gains suffix matching: typing "login.rs" auto-resolves to
"src/auth/login.rs" when unambiguous, with clear disambiguation errors when
multiple paths match. Project-scoping (-p) narrows the candidate set.
The MAX_MR_REFS_PER_USER constant is promoted to module scope for reuse
across expert and overlap modes. Human output shows MR refs inline and detail
sub-rows when requested. Robot JSON includes mr_refs, mr_refs_total,
mr_refs_truncated, and optional details array.
Includes comprehensive tests for suffix resolution, scoring weight
configurability, MR ref aggregation across projects, and detail mode.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds mr_diffs_fetched and mr_diffs_failed fields to IngestResult and
SyncResult, threads them through the orchestrator aggregation, includes
them in the structured tracing span and human-readable sync summary.
Previously MR diff failures were silently swallowed — now they appear
alongside resource event counts for full pipeline observability.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The --since naming is more intuitive (matches git log --since) and
consistent with the list commands which already use --since. Renames
the CLI flags, SearchCliFilters fields, SearchFilters fields,
autocorrect registry, and robot-docs manifest. No behavioral change.
Affected paths:
- cli/mod.rs: SearchArgs field + clap attribute rename
- cli/commands/search.rs: SearchCliFilters + run_search plumbing
- search/filters.rs: SearchFilters struct + apply_filters logic
- main.rs: handle_search + robot-docs JSON
- cli/autocorrect.rs: COMMAND_FLAGS entry for search
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signal 4 (mr_reviewers + mr_file_changes) was missing the self-review
exclusion that signal 1 (DiffNote reviewer) already had. An MR author
listed as their own reviewer would be double-counted as both author
and reviewer, inflating their score.
Also removes redundant SELECT DISTINCT from signal 2 (GROUP BY
already ensures uniqueness).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Chain: bd-jec (config flag) -> bd-2yo (fetch MR diffs) -> bd-3qn6 (rewrite who queries)
- Add fetch_mr_file_changes config option and --no-file-changes CLI flag
- Add GitLab MR diffs API fetch pipeline with watermark-based sync
- Create migration 020 for diffs_synced_for_updated_at watermark column
- Rewrite query_expert() and query_overlap() to use 4-signal UNION ALL:
DiffNote reviewers, DiffNote MR authors, file-change authors, file-change reviewers
- Deduplicate across signal types via COUNT(DISTINCT CASE WHEN ... THEN mr_id END)
- Add insert_file_change test helper, 8 new who tests, all 397 tests pass
- Also includes: list performance migration 019, autocorrect module, README updates
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Comprehensive peer code review identified and fixed the following:
1. who.rs: @-prefixed path routing used `target` (with @) instead of
`clean` (stripped) when checking for '/' and passing to Expert mode,
causing `lore who @src/auth/` to silently return zero results because
the SQL LIKE matched against `@src/auth/%` which never exists.
2. db.rs: After ROLLBACK TO savepoint on migration failure, the savepoint
was never RELEASEd, leaving it active on the connection. Fixed in both
run_migrations() and run_migrations_from_dir().
3. lock.rs: Multiple acquire() calls (e.g. re-acquiring a stale lock)
replaced the heartbeat_handle without stopping the old thread, causing
two concurrent heartbeat writers competing on the same lock row. Now
signals the old thread to stop and joins it before spawning a new one.
4. chunk_ids.rs: encode_rowid() had no guard for chunk_index >= 1000
(CHUNK_ROWID_MULTIPLIER), which would cause rowid collisions between
adjacent documents. Added range assertion [0, 1000).
5. main.rs: Fallback JSON error formatting in handle_auth_test
interpolated LoreError Display output without escaping quotes or
backslashes, potentially producing malformed JSON for robot-mode
consumers. Now escapes both characters before interpolation.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
