Commit Graph

61 Commits

Author SHA1 Message Date
Taylor Eernisse
9d4755521f feat(config): Add fetchResourceEvents config flag with --no-events CLI override
Adds a new boolean field to SyncConfig that controls whether resource
event fetching is performed during sync:

- SyncConfig.fetch_resource_events: defaults to true via serde
  default_true helper, serialized as "fetchResourceEvents" in JSON
- SyncArgs.no_events: --no-events CLI flag that overrides the config
  value to false when present
- SyncOptions.no_events: propagates the flag through the sync pipeline
- handle_sync_cmd: mutates loaded config when --no-events is set,
  ensuring the flag takes effect regardless of config file contents

This follows the existing pattern established by --no-embed and
--no-docs flags, where CLI flags override config file defaults.
The config is loaded as mutable specifically to support this override.

Also adds "events" to the count command's entity type value_parser,
enabling `lore count events` (implementation in a separate commit).

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:07:06 -05:00
Taylor Eernisse
ce5cd9c95d feat(schema): Add migration 011 for resource events, entity references, and dependent fetch queue
Introduces five new tables that power temporal queries (timeline,
file-history, trace) via GitLab Resource Events APIs:

- resource_state_events: State transitions (opened/closed/reopened/merged/locked)
  with actor tracking, source commit, and source MR references
- resource_label_events: Label add/remove history per entity
- resource_milestone_events: Milestone assignment changes per entity
- entity_references: Cross-reference table (Gate 2 prep) linking
  source/target entity pairs with reference type and discovery method
- pending_dependent_fetches: Generic job queue for resource_events,
  mr_closes_issues, and mr_diffs with exponential backoff retry

All event tables enforce entity exclusivity via CHECK constraints
(exactly one of issue_id or merge_request_id must be non-NULL).
Deduplication handled via UNIQUE indexes on (gitlab_id, project_id).
FK cascades ensure cleanup when parent entities are removed.

The dependent fetch queue uses a UNIQUE constraint on
(project_id, entity_type, entity_iid, job_type) for idempotent
enqueue, with partial indexes optimizing claim and retry queries.

Registered as migration 011 in the embedded MIGRATIONS array in db.rs.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 12:06:43 -05:00
Taylor Eernisse
2a52594a60 feat(db): Add migration 010 for chunk config tracking columns
Add chunk_max_bytes and chunk_count columns to embedding_metadata to
support config drift detection and adaptive dedup sizing. Includes a
partial index on sentinel rows (chunk_index=0) to accelerate the drift
detection and max-chunk queries.

Also exports LATEST_SCHEMA_VERSION as a public constant derived from
the MIGRATIONS array length, replacing the previously hardcoded magic
number in the health check.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-03 09:34:48 -05:00
Taylor Eernisse
51c370fac2 feat(project): Add substring matching and use Ambiguous error for resolution
Extend resolve_project() with a 4th cascade step: case-insensitive
substring match when exact, case-insensitive, and suffix matches all
fail. This allows shorthand like "typescript" to match
"vs/typescript-code" when unambiguous. Multi-match still returns an
error with all candidates listed.

Also change ambiguity errors from LoreError::Other to LoreError::Ambiguous
so they get the proper AMBIGUOUS error code (exit 18) instead of
INTERNAL_ERROR.

Includes tests for unambiguous substring, case-insensitive substring,
ambiguous substring, and suffix-preferred-over-substring ordering.

Co-Authored-By: Claude (us.anthropic.claude-opus-4-5-20251101-v1:0) <noreply@anthropic.com>
2026-01-30 16:55:23 -05:00
Taylor Eernisse
730ddef339 fix(error): Remap ConfigNotFound to exit 20 and add NotFound/Ambiguous codes
ConfigNotFound previously used exit code 2 which collides with clap's
usage error code. Remap it to exit 20 to avoid ambiguity. Also add
dedicated NotFound (exit 17) and Ambiguous (exit 18) error codes with
proper ErrorCode variants and Display implementations, replacing the
previous incorrect mapping of these errors to GitLabNotFound.

Co-Authored-By: Claude (us.anthropic.claude-opus-4-5-20251101-v1:0) <noreply@anthropic.com>
2026-01-30 16:54:02 -05:00
Taylor Eernisse
6e22f120d0 refactor(core): Rename GiError to LoreError and add search infrastructure
Mechanical rename of GiError -> LoreError across the core module to
match the project's rebranding from gitlab-inbox to gitlore/lore.
Updates the error enum name, all From impls, and the Result type alias.

Additionally introduces:

- New error variants for embedding pipeline: OllamaUnavailable,
  OllamaModelNotFound, EmbeddingFailed, EmbeddingsNotBuilt. Each
  includes actionable suggestions (e.g., "ollama serve", "ollama pull
  nomic-embed-text") to guide users through recovery.

- New error codes 14-16 for programmatic handling of Ollama failures.

- Savepoint-based migration execution in db.rs: each migration now
  runs inside a SQLite SAVEPOINT so a failed migration rolls back
  cleanly without corrupting the schema_version tracking. Previously
  a partial migration could leave the database in an inconsistent
  state.

- core::backoff module: exponential backoff with jitter utility for
  retry loops in the embedding pipeline and discussion queues.

- core::project module: helper for resolving project IDs and paths
  from the local database, used by the document regenerator and
  search filters.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-30 15:45:54 -05:00
Taylor Eernisse
d3a05cfb87 fix(error): Improve error suggestions with inline examples
Error suggestions now include concrete CLI examples so users
(and robot-mode consumers) can act immediately without consulting
docs. For instance, ConfigNotFound now shows the expected path
and the exact command to run, TokenNotSet shows the export syntax,
and Ambiguous shows the -p flag with example project paths.

Also fixes the error code for Ambiguous errors: it now maps to
GitLabNotFound instead of InternalError, since the entity exists
but the user needs to disambiguate -- not an internal failure.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 08:41:45 -05:00
Taylor Eernisse
390f8a9288 refactor(core): Centralize timestamp parsing in core::time
Duplicate ISO 8601 timestamp parsing functions existed in both
discussion.rs and merge_request.rs transformers. This extracts
iso_to_ms_strict() and iso_to_ms_opt_strict() into core::time
as the single source of truth, and updates both transformer
modules to use the shared implementations.

Also removes the private now_ms() from merge_request.rs in
favor of the existing core::time::now_ms(), and replaces the
local parse_timestamp_opt() in discussion.rs with the public
iso_to_ms() from core::time.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-29 08:41:34 -05:00
teernisse
55b895a2eb Update name to gitlore instead of gitlab-inbox 2026-01-28 15:49:14 -05:00
Taylor Eernisse
5fe76e46a3 fix(core): Add structured error handling and responsive lock release
Improves core infrastructure with robot-friendly error output and
faster lock release for better sync behavior.

Error handling improvements (error.rs):
- ErrorCode::exit_code(): Unique exit codes per error type (1-13)
  for programmatic error handling in scripts/agents
- GiError::suggestion(): Helpful hints for common error recovery
- GiError::to_robot_error(): Structured JSON error conversion
- RobotError/RobotErrorOutput: Serializable error types with code,
  message, and optional suggestion fields

Lock improvements (lock.rs):
- Heartbeat thread now polls every 100ms for release flag, only
  updating database heartbeat at full interval (5s default)
- Eliminates 5-10s delay after sync completion when waiting for
  heartbeat thread to notice release
- Reduces lock hold time after operation completes

Database (db.rs):
- Bump expected schema version to 6 for MR migration

The exit code mapping enables shell scripts and CI/CD pipelines to
distinguish between configuration errors (2-4), GitLab API errors
(5-8), and database errors (9-11) for appropriate retry/alert logic.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 22:46:08 -05:00
Taylor Eernisse
7aaa51f645 feat(core): Implement infrastructure layer for CLI operations
Establishes foundational modules that all other components depend on.

src/core/config.rs - Configuration management:
- JSON-based config file with Zod-like validation via serde
- GitLab settings: base URL, token environment variable
- Project list with paths to track
- Sync settings: backfill days, stale lock timeout, cursor rewind
- Storage settings: database path, payload compression toggle
- XDG-compliant config path resolution via dirs crate
- Loads GITLAB_TOKEN from configured environment variable

src/core/db.rs - Database connection and migrations:
- Opens or creates SQLite database with WAL mode for concurrency
- Embeds migration SQL as const strings (001-005)
- Runs migrations idempotently with checksum verification
- Provides thread-safe connection management

src/core/error.rs - Unified error handling:
- GiError enum with variants for all failure modes
- Config, Database, GitLab, Ingestion, Lock, IO, Parse errors
- thiserror derive for automatic Display/Error impls
- Result type alias for ergonomic error propagation

src/core/lock.rs - Distributed sync locking:
- File-based locks to prevent concurrent syncs
- Stale lock detection with configurable timeout
- Force override for recovery scenarios
- Lock file contains PID and timestamp for debugging

src/core/paths.rs - Path resolution:
- XDG Base Directory Specification compliance
- Config: ~/.config/gi/config.json
- Data: ~/.local/share/gi/gi.db
- Creates parent directories on first access

src/core/payloads.rs - Raw payload storage:
- Optional gzip compression for storage efficiency
- SHA-256 content addressing for deduplication
- Type-prefixed keys (issue:, discussion:, note:)
- Batch insert with UPSERT for idempotent ingestion

src/core/time.rs - Timestamp utilities:
- Relative time parsing (7d, 2w, 1m) for --since flag
- ISO 8601 date parsing for absolute dates
- Human-friendly relative time formatting

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 11:28:07 -05:00