SPEC_discussion_analysis.md defines a pre-computed enrichment pipeline that
replaces the current key_decisions heuristic in explain with actual
LLM-extracted discourse analysis (decisions, questions, consensus).
Key design choices:
- Dual LLM backend: Claude Haiku via AWS Bedrock (primary) or Anthropic API
- Pre-computed batch enrichment (lore enrich), never runtime LLM calls
- Staleness detection via notes_hash to skip unchanged threads
- New discussion_analysis SQLite table with structured JSON results
- Configurable via config.json enrichment section
Status: DRAFT — open questions on Bedrock model ID, auth mechanism, rate
limits, cost ceiling, and confidence thresholds.
Validates that the projects table schema uses gitlab_project_id (not
gitlab_id) and that queries filtering by this column return the correct
project. Uses the test helper convention where insert_project sets
gitlab_project_id = id * 100.
Guards against regression in the wiring chain run_me -> print_me_json ->
MeJsonEnvelope where the gitlab_base_url meta field could silently
disappear.
- me_envelope_includes_gitlab_base_url_in_meta: verifies full envelope
serialization preserves the base URL in meta
- activity_event_carries_url_construction_fields: verifies activity events
contain entity_type + entity_iid + project fields, then demonstrates
URL construction by combining with meta.gitlab_base_url
Previously, query_mentioned_in returned mentions from any time in the
entity's history as long as the entity was still open (or recently closed).
This caused noise: a mention from 6 months ago on a still-open issue would
appear in the dashboard indefinitely.
Now the SQL filters notes by created_at > mention_cutoff_ms, defaulting to
30 days. The recency_cutoff (7 days) still governs closed/merged entity
visibility — this new cutoff governs mention note age on open entities.
Signature change: query_mentioned_in gains a mention_cutoff_ms parameter.
All existing test call sites updated. Two new tests verify the boundary:
- mentioned_in_excludes_old_mention_on_open_issue (45-day mention filtered)
- mentioned_in_includes_recent_mention_on_open_issue (5-day mention kept)
Multiple improvements to the explain command's data richness:
- Add project_path to EntitySummary so consumers can construct URLs from
project + entity_type + iid without extra lookups
- Include first_note_excerpt (first 200 chars) in open threads so agents
and humans get thread context without a separate query
- Add state and direction fields to RelatedIssue — consumers now see
whether referenced entities are open/closed/merged and whether the
reference is incoming or outgoing
- Filter out self-references in both outgoing and incoming related entity
queries (entity referencing itself via cross-reference extraction)
- Wrap timeline excerpt in TimelineExcerpt struct with total_events and
truncated fields — consumers know when events were omitted
- Keep most recent events (tail) instead of oldest (head) when truncating
timeline — recent activity is more actionable
- Floor activity summary first_event at entity created_at — label events
from bulk operations can predate entity creation
- Human output: show project path in header, thread excerpt preview,
state badges on related entities, directional arrows, truncation counts
CEO memory notes for 2026-03-11 and 2026-03-12 capture the full timeline of
GIT-2 (founding engineer evaluation), GIT-3 (calibration task), and GIT-6
(plan reviewer hire).
Founding Engineer: AGENTS.md rewritten from 25-line boilerplate to 3-layer
progressive disclosure model (AGENTS.md core -> DOMAIN.md reference ->
SOUL.md persona). Adds HEARTBEAT.md checklist, TOOLS.md placeholder. Key
changes: memory system reference, async runtime warning, schema gotchas,
UTF-8 boundary safety, search import privacy.
Plan Reviewer: new agent created with AGENTS.md (review workflow, severity
levels, codebase context), HEARTBEAT.md, SOUL.md. Reviews implementation
plans in Paperclip issues before code is written.
The old truncation counted <mark></mark> HTML tags (~13 chars per keyword)
as visible characters, causing over-aggressive truncation. When a cut
landed inside a tag pair, render_snippet would render highlighted text
as muted gray instead of bold yellow.
New truncate_snippet() walks through markup counting only visible
characters, respects tag boundaries, and always closes an open <mark>
before appending ellipsis. Includes 6 unit tests.
Phase 1: Add source_entity_iid to search results via CASE subquery on
hydrate_results() for all 4 source types (issue, MR, discussion, note).
Phase 2: Fix visual alignment - compute indent from prefix visible width.
Phase 3: Show compact relative time on title line.
Phase 4: Add drill-down hint footer (lore issues <iid>).
Phase 5: Move labels to --explain mode, limit snippets to 2 terminal lines.
Phase 6: Use section_divider() for results header.
Also: promote strip_ansi/visible_width to public render utils, update
robot mode --fields minimal search preset with source_entity_iid.
The `me` dashboard robot output now includes `meta.gitlab_base_url` so
consuming agents can construct clickable issue/MR links without needing
access to the lore config file. The pattern is:
{gitlab_base_url}/{project}/-/issues/{iid}
{gitlab_base_url}/{project}/-/merge_requests/{iid}
This uses the new RobotMeta::with_base_url() constructor. The base URL
is sourced from config.gitlab.base_url (already available in the me
command's execution context) and normalized to strip trailing slashes.
robot-docs updated to document the new meta field and URL construction
pattern for the me command's response schema.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
RobotMeta previously required direct struct literal construction with only
elapsed_ms. This made it impossible to add optional fields without updating
every call site to include them.
Introduce two constructors:
- RobotMeta::new(elapsed_ms) — standard meta with timing only
- RobotMeta::with_base_url(elapsed_ms, base_url) — meta enriched with the
GitLab instance URL, enabling consumers to construct entity links without
needing config access
The gitlab_base_url field uses #[serde(skip_serializing_if = "Option::is_none")]
so existing JSON envelopes are byte-identical — no breaking change for any
robot mode consumer.
All 22 call sites across handlers, count, cron, drift, embed, generate_docs,
ingest, list (mrs/notes), related, show, stats, sync_status, and who are
updated from struct literals to RobotMeta::new(). Three tests verify the
new constructors and trailing-slash normalization.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Three robot-mode print functions used `serde_json::to_string().unwrap_or_default()`
which silently outputs an empty string on failure (exit 0, no error). This
diverged from the codebase standard in handlers.rs which uses `?` propagation.
Changed to return Result<()> with proper LoreError::Other mapping:
- explain.rs: print_explain_json()
- file_history.rs: print_file_history_json()
- trace.rs: print_trace_json()
Updated callers in handlers.rs and explain.rs to propagate with `?`.
While serde_json::to_string on a json!() Value is unlikely to fail in practice
(only non-finite floats trigger it), the unwrap_or_default pattern violates the
robot mode contract: callers expect either valid JSON on stdout or a structured
error on stderr with a non-zero exit code, never empty output with exit 0.
SQLite does not guarantee row order without ORDER BY, even with LIMIT.
This was a systemic issue found during a multi-pass bug hunt:
Production queries (explain.rs):
- Outgoing reference query: ORDER BY target_entity_type, target_entity_iid
- Incoming reference query: ORDER BY source_entity_type, COALESCE(iid)
Without these, robot mode output was non-deterministic across calls,
breaking clients expecting stable ordering.
Test helper queries (5 locations across 3 files):
- discussions_tests.rs: get_discussion_id()
- mr_discussions.rs: get_mr_discussion_id()
- queue.rs: setup_db_with_job(), release_all_locked_jobs_clears_locks()
Currently safe (single-row inserts) but would break silently if tests
expanded to multi-row fixtures.
1. fetch_open_threads: replace N+1 loop (2 queries per thread) with a
single query using correlated subqueries for note_count and started_by.
2. extract_key_decisions: track consumed notes so the same note is not
matched to multiple events, preventing duplicate decision entries.
3. build_timeline_excerpt_from_pipeline: log tracing::warn on seed/collect
failures instead of silently returning empty timeline.
entity_references.target_entity_iid is nullable (unresolved cross-project
refs), and COALESCE(i.iid, mr.iid) returns NULL for orphaned refs.
Both paths caused rusqlite InvalidColumnType errors when fetching i64.
Added IS NOT NULL filters to both outgoing and incoming reference queries.
Update planning docs and audit tables to reflect the removal of
`lore show`:
- CLI_AUDIT.md: remove show row, renumber remaining entries
- plan-expose-discussion-ids.md: replace `show` with
`issues <IID>`/`mrs <IID>`
- plan-expose-discussion-ids.feedback-3.md: replace `show` with
"detail views"
- work-item-status-graphql.md: update example commands from
`lore show issue 123` to `lore issues 123`
The `show` command (`lore show issue 42` / `lore show mr 99`) was
deprecated in favor of the unified entity commands (`lore issues 42` /
`lore mrs 99`). This commit fully removes the command entry point:
- Remove `Commands::Show` variant from clap CLI definition
- Remove `Commands::Show` match arm and deprecation warning in main.rs
- Remove `handle_show_compat()` forwarding function from robot_docs.rs
- Remove "show" from autocorrect known-commands and flags tables
- Rename response schema keys from "show" to "detail" in robot-docs
- Update command descriptions from "List or show" to "List ... or
view detail with <IID>"
The underlying detail-view module (`src/cli/commands/show/`) is
preserved — its types (IssueDetail, MrDetail) and query/render
functions are still used by `handle_issues` and `handle_mrs` when
an IID argument is provided.
The show command's NoteDetail and MrNoteDetail structs were missing
gitlab_id, making individual notes unaddressable in robot mode output.
This was inconsistent with the notes list command which already exposed
gitlab_id. Without an identifier, agents consuming show output could
not construct GitLab web URLs or reference specific notes for follow-up
operations via glab.
Added gitlab_id to:
- NoteDetail / NoteDetailJson (issue discussions)
- MrNoteDetail / MrNoteDetailJson (MR discussions)
- Both SQL queries (shifted column indices accordingly)
- Both From<&T> conversion impls
Deliberately scoped to show command only — me/timeline/trace structs
were evaluated and intentionally left unchanged because they serve
different consumption patterns where note-level identity is not needed.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Document that the activity feed and since-last-check inbox cover items
in any state (open, closed, merged), while the issues and MRs sections
show only open items. Add the previously undocumented since-last-check
inbox section to the dashboard description.
The activity feed and since-last-check inbox previously filtered to
only open items via state = 'opened' checks in the SQL subqueries.
This meant comments on merged MRs (post-merge follow-ups, questions)
and closed issues were silently dropped from the feed.
Remove the state filter from the association checks in both
query_activity() and query_since_last_check(). The user-association
checks (assigned, authored, reviewing) remain — activity still only
appears for items the user is connected to, regardless of state.
The simplified subqueries also eliminate unnecessary JOINs to the
issues/merge_requests tables that were only needed for the state
check, resulting in slightly more efficient index-only scans on
issue_assignees and mr_reviewers.
Add 4 tests covering: merged MR (authored), closed MR (reviewer),
closed issue (assignee), and merged MR in the since-last-check inbox.
CLI audit scoring the current command surface across human ergonomics,
robot/agent ergonomics, documentation quality, and flag design. Paired
with a detailed implementation plan for restructuring commands into a
more consistent, discoverable hierarchy.
Add pre-flight FTS count check before expensive bm25-ranked search.
Queries matching >10,000 documents are rejected instantly with a
suggestion to use a more specific query or --since filter.
Prevents multi-minute CPU spin on queries like 'merge request' that
match most of the corpus (106K/178K documents).
is_multiple_of(N) returns true for 0, which caused debug/info
progress messages to fire at doc_num=0 (the start of every page)
rather than only at the intended 50/100 milestones. Add != 0
check to both the debug (every 50) and info (every 100) log sites.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Major additions to the migration plan based on review feedback:
Alternative analysis:
- Add "Why not tokio CancellationToken + JoinSet?" section explaining
why obligation tracking and single-migration cost favor asupersync
over incremental tokio fixes.
Error handling depth:
- Add NetworkErrorKind enum design for preserving error categories
(timeout, DNS, TLS, connection refused) without coupling LoreError
to any HTTP client.
- Add response body size guard (64 MiB) to prevent unbounded memory
growth from misconfigured endpoints.
Adapter layer refinements:
- Expand append_query_params with URL fragment handling, edge case
docs, and doc comments.
- Add contention constraint note for std::sync::Mutex rate limiter.
Cancellation invariants (INV-1 through INV-4):
- Atomic batch writes, no .await between tx open/commit,
ShutdownSignal + region cancellation complementarity.
- Concrete test plan for each invariant.
Semantic ordering concerns:
- Document 4 behavioral differences when replacing join_all with
region-spawned tasks (ordering, error aggregation, backpressure,
late result loss on cancellation).
HTTP behavior parity:
- Replace informational table with concrete acceptance criteria and
pass/fail tests for redirects, proxy, keep-alive, DNS, TLS, and
Content-Length.
Phasing refinements:
- Add Cx threading sub-steps (orchestration path first, then
command/embedding layer) for blast radius reduction.
- Add decision gate between Phase 0d and Phase 1 requiring compile +
behavioral smoke tests before committing to runtime swap.
Rollback strategy:
- Per-phase rollback guidance with concrete escape hatch triggers
(nightly breakage > 7d, TLS incompatibility, API instability,
wiremock issues).
Testing depth:
- Adapter-layer test gap analysis with 5 specific asupersync-native
integration tests.
- Cancellation integration test specifications.
- Coverage gap documentation for wiremock-on-tokio tests.
Risk register additions:
- Unbounded response body buffering, manual URL/header handling
correctness.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The `query_mentioned_in` SQL previously joined notes directly against
the full issues/merge_requests tables, with per-row subqueries for
author/assignee/reviewer exclusion. On large databases this produced
pathological query plans where SQLite scanned the entire notes table
before filtering to relevant entities.
Refactor into a dedicated `build_mentioned_in_sql()` builder that:
1. Pre-filters candidate issues and MRs into MATERIALIZED CTEs
(state open OR recently closed, not authored by user, not
assigned/reviewing). This narrows the working set before any
notes join.
2. Computes note timestamps (my_ts, others_ts, any_ts) as separate
MATERIALIZED CTEs scoped to candidate entities only, rather than
scanning all notes.
3. Joins mention-bearing notes against the pre-filtered candidates,
avoiding the full-table scans.
Also adds a test verifying that authored issues are excluded from the
mentions results, and a unit test asserting all four CTEs are
materialized.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The issue and MR ingestion paths previously inserted labels, assignees,
and reviewers one row at a time inside a transaction. For entities with
many labels or assignees, this issued N separate SQLite statements where
a single multi-row INSERT suffices.
Replace the per-row loops with batch INSERT functions that build a
single `INSERT OR IGNORE ... VALUES (?1,?2),(?1,?3),...` statement per
chunk. Chunks are capped at 400 rows (BATCH_LINK_ROWS_MAX) to stay
comfortably below SQLite's default 999 bind-parameter limit.
Affected paths:
- issues.rs: link_issue_labels_batch_tx, insert_issue_assignees_batch_tx
- merge_requests.rs: insert_mr_labels_batch_tx,
insert_mr_assignees_batch_tx, insert_mr_reviewers_batch_tx
New tests verify deduplication (OR IGNORE), multi-chunk correctness,
and equivalence with the old per-row approach. A perf benchmark
(bench_issue_assignee_insert_individual_vs_batch) demonstrates the
speedup across representative assignee set sizes.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove configuration files that are no longer used:
- .opencode/rules: OpenCode rules file, superseded by project CLAUDE.md
and ~/.claude/ rules directory structure
- .roam/fitness.yaml: Roam fitness tracking config, unrelated to this
project
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove iterative feedback files that were used during plan development.
These files captured review rounds but are no longer needed now that the
plans have been finalized:
- plans/lore-service.feedback-{1,2,3,4}.md
- plans/time-decay-expert-scoring.feedback-{1,2,3,4}.md
- plans/tui-prd-v2-frankentui.feedback-{1,2,3,4,5,6,7,8,9}.md
The canonical plan documents remain; only the review iteration artifacts
are removed to reduce clutter.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Draft plan for replacing Tokio + Reqwest with Asupersync, a cancel-correct
async runtime with structured concurrency guarantees.
Motivation:
- Current Ctrl+C during join_all silently drops in-flight HTTP requests
- ShutdownSignal is a hand-rolled AtomicBool with no structured cancellation
- No deterministic testing for concurrent ingestion patterns
- Tokio provides no structured concurrency guarantees
Plan structure:
- Complete inventory of tokio/reqwest usage in production and test code
- Phase 0: Preparation (reduce tokio surface before swap)
- Extract signal handler to single function
- Replace tokio::sync::Mutex with std::sync::Mutex where appropriate
- Create HTTP adapter trait for pluggable backends
- Phase 1-5: Progressive migration with detailed implementation steps
Trade-offs accepted:
- Nightly Rust required (asupersync dependency)
- Pre-1.0 runtime dependency (mitigated by adapter layer + version pinning)
- Deeper function signature changes for Cx threading
This is a reference document for future implementation, not an immediate
change to the runtime.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update CLAUDE.md and README.md with documentation for recently added
features:
CLAUDE.md:
- Add robot mode examples for `lore --robot related`
- Add example for `lore --robot init --refresh`
README.md:
- Add full documentation section for `lore me` command including all
flags (--issues, --mrs, --mentions, --activity, --since, --project,
--all, --user, --reset-cursor) and section descriptions
- Add documentation section for `lore related` command with entity mode
and query mode examples
- Expand `lore init` section with --refresh flag documentation explaining
project registration workflow
- Add quick examples in the features section
- Update version number in example output (0.9.2)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Extend the CLI autocorrection pipeline with two new correction rules that
help agents recover from common typos and misunderstandings:
1. SubcommandFuzzy (threshold 0.85): Fuzzy-matches typo'd subcommands
against the canonical list. Examples:
- "issuess" → "issues"
- "timline" → "timeline"
- "serach" → "search"
Guards prevent false positives:
- Words that look like misplaced global flags are skipped
- Valid command prefixes are left to clap's infer_subcommands
2. FlagAsSubcommand: Detects when agents type subcommands as flags.
Some agents (especially Codex) assume `--robot-docs` is a flag rather
than a subcommand. This rule converts:
- "--robot-docs" → "robot-docs"
- "--generate-docs" → "generate-docs"
Also improves error messages in main.rs:
- MissingRequiredArgument: Contextual example based on detected subcommand
- MissingSubcommand: Lists common commands
- TooFewValues/TooManyValues: Command-specific help hints
Added CANONICAL_SUBCOMMANDS constant enumerating all valid subcommands
(including hidden ones) for fuzzy matching. This ensures agents that know
about hidden commands still get typo correction.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a new --mentions flag to the `lore me` command that surfaces items
where the user is @-mentioned but NOT already assigned, authoring, or
reviewing. This fills an important gap in the personal work dashboard:
cross-team requests and callouts that don't show up in the standard
issue/MR sections.
Implementation details:
- query_mentioned_in() scans notes for @username patterns, then filters
out entities where the user is already an assignee, author, or reviewer
- MentionedInItem type captures entity_type (issue/mr), iid, title, state,
project path, attention state, and updated timestamp
- Attention state computation marks items as needs_attention when there's
recent activity from others
- Recency cutoff (7 days) prevents surfacing stale mentions
- Both human and robot renderers include the new section
The robot mode schema adds mentioned_in array with me_mentions field
preset for token-efficient output.
Test coverage:
- mentioned_in_finds_mention_on_unassigned_issue: basic case
- mentioned_in_excludes_assigned_issue: no duplicate surfacing
- mentioned_in_excludes_author_on_mr: author already sees in authored MRs
- mentioned_in_excludes_reviewer_on_mr: reviewer already sees in reviewing
- mentioned_in_uses_recency_cutoff: old mentions filtered
- mentioned_in_respects_project_filter: scoping works
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When new projects are added to the config file, `lore sync` doesn't pick
them up because project discovery only happens during `lore init`.
Previously, users had to use `--force` to overwrite their entire config.
The new `--refresh` flag reads the existing config and updates the
database to match, without modifying the config file itself.
Features:
- Validates GitLab authentication before processing
- Registers new projects from config into the database
- Detects orphan projects (in DB but removed from config)
- Interactive mode: prompts to delete orphans (default: No)
- Robot mode: returns JSON with orphan info, no prompts
Usage:
lore init --refresh # Interactive
lore --robot init --refresh # JSON output
Improved UX: When running `lore init` with an existing config and no
flags, the error message now suggests using `--refresh` to register
new projects or `--force` to overwrite the config file.
Implementation:
- Added RefreshOptions and RefreshResult types to init module
- Added run_init_refresh() for core refresh logic
- Added delete_orphan_projects() helper for orphan cleanup
- Added handle_init_refresh() in main.rs for CLI handling
- Added JSON output types for robot mode
- Registered --refresh in autocorrect.rs command flags registry
- --refresh conflicts with --force (mutually exclusive)
These HTML files were generated for one-time analysis/review purposes
and should not be tracked in the repository.
Files removed:
- api-review.html
- gitlore-sync-explorer.html
- phase-a-review.html
Issue discussion sync was ~10x slower than MR discussion sync because it
used a fully sequential pattern: fetch one issue's discussions, write to
DB, repeat. MR sync already used a prefetch pattern with concurrent HTTP
requests followed by sequential DB writes.
This commit brings issue discussion sync to parity with MRs:
Architecture (prefetch pattern):
1. HTTP phase: Concurrent fetches via `join_all()` with batch size
controlled by `dependent_concurrency` config (default 8)
2. Transform phase: Normalize discussions and notes during prefetch
3. DB phase: Sequential writes with proper transaction boundaries
Changes:
- gitlab/client.rs: Add `fetch_all_issue_discussions()` to mirror
the existing MR pattern for API consistency
- discussions.rs: Replace `ingest_issue_discussions()` with:
* `prefetch_issue_discussions()` - async HTTP fetch + transform
* `write_prefetched_issue_discussions()` - sync DB writes
* New structs: `PrefetchedIssueDiscussions`, `PrefetchedDiscussion`
- orchestrator.rs: Update `sync_discussions_sequential()` to use
concurrent prefetch for each batch instead of sequential calls
- surgical.rs: Update single-issue surgical sync to use new functions
- mod.rs: Update public exports
Expected improvement: 5-10x speedup on issue discussion sync (from ~50s
to ~5-10s for large projects) due to concurrent HTTP round-trips.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When `--limit` is omitted, the default value is `usize::MAX` to mean
"unlimited". The previous code used `(limit + 1) as i64` to fetch one
extra row for "has more" detection. This caused integer overflow:
usize::MAX + 1 = 0 (wraps around)
The resulting `LIMIT 0` clause returned zero rows, making the `who`
subcommands appear to find nothing even when data existed.
Fix: Use `saturating_add(1)` to cap at `usize::MAX` instead of wrapping,
then `.min(i64::MAX as usize)` to ensure the value fits in SQLite's
signed 64-bit LIMIT parameter.
Includes regression tests that verify `usize::MAX` limit returns results.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update beads issue tracking state and expand the GitLab TODOs
notifications integration design document with additional
implementation details.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
