Taylor Eernisse cd60350c6d feat(ingestion): Implement cursor-based incremental sync from GitLab
Provides efficient data synchronization with minimal API calls.

src/ingestion/issues.rs - Issue sync logic:
- Cursor-based incremental sync using updated_at timestamp
- Fetches only issues modified since last sync
- Configurable cursor rewind for overlap safety (default 2s)
- Batched database writes with transaction wrapping
- Upserts issues, labels, milestones, and assignees
- Maintains issue_labels and issue_assignees junction tables
- Returns IngestIssuesResult with counts and issues needing discussion sync
- Identifies issues where discussion count changed

src/ingestion/discussions.rs - Discussion sync logic:
- Fetches discussions for issues that need sync
- Compares discussion count vs stored to detect changes
- Batched note insertion with raw payload preservation
- Updates discussion metadata (resolved state, note counts)
- Tracks sync state per discussion to enable incremental updates
- Returns IngestDiscussionsResult with fetched/skipped counts

src/ingestion/orchestrator.rs - Sync coordination:
- Two-phase sync: issues first, then discussions
- Progress callback support for CLI progress bars
- ProgressEvent enum for fine-grained status updates:
  - IssueFetch, IssueProcess, DiscussionFetch, DiscussionSkip
- Acquires sync lock before starting
- Updates sync watermark on successful completion
- Handles partial failures gracefully (watermark not updated)
- Returns IngestProjectResult with detailed statistics

The architecture supports future additions:
- Merge request ingestion (parallel to issues)
- Full-text search indexing hooks
- Vector embedding pipeline integration

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-26 11:28:34 -05:00
2026-01-20 13:11:40 -05:00
2026-01-21 15:56:11 -05:00
2026-01-21 15:56:11 -05:00
2026-01-21 15:56:11 -05:00

gi - GitLab Inbox

A command-line tool for managing GitLab issues locally. Syncs issues, discussions, and notes from GitLab to a local SQLite database for fast, offline-capable querying and filtering.

Features

  • Local-first: All data stored in SQLite for instant queries
  • Incremental sync: Cursor-based sync only fetches changes since last sync
  • Multi-project: Track issues across multiple GitLab projects
  • Rich filtering: Filter by state, author, assignee, labels, milestone, due date
  • Raw payload storage: Preserves original GitLab API responses for debugging

Installation

cargo install --path .

Or build from source:

cargo build --release
./target/release/gi --help

Quick Start

# Initialize configuration (interactive)
gi init

# Verify authentication
gi auth-test

# Sync issues from GitLab
gi ingest --type issues

# List recent issues
gi list issues --limit 10

# Show issue details
gi show issue 123 --project group/repo

Configuration

Configuration is stored in ~/.config/gi/config.json (or $XDG_CONFIG_HOME/gi/config.json).

Example Configuration

{
  "gitlab": {
    "baseUrl": "https://gitlab.com",
    "tokenEnvVar": "GITLAB_TOKEN"
  },
  "projects": [
    { "path": "group/project" },
    { "path": "other-group/other-project" }
  ],
  "sync": {
    "backfillDays": 14,
    "staleLockMinutes": 10
  },
  "storage": {
    "compressRawPayloads": true
  }
}

Configuration Options

Section Field Default Description
gitlab baseUrl GitLab instance URL (required)
gitlab tokenEnvVar GITLAB_TOKEN Environment variable containing API token
projects path Project path (e.g., group/project)
sync backfillDays 14 Days to backfill on initial sync
sync staleLockMinutes 10 Minutes before sync lock considered stale
sync cursorRewindSeconds 2 Seconds to rewind cursor for overlap safety
storage dbPath ~/.local/share/gi/gi.db Database file path
storage compressRawPayloads true Compress stored API responses

GitLab Token

Create a personal access token with read_api scope:

  1. Go to GitLab → Settings → Access Tokens
  2. Create token with read_api scope
  3. Export it: export GITLAB_TOKEN=glpat-xxxxxxxxxxxx

Commands

gi init

Initialize configuration and database interactively.

gi init                    # Interactive setup
gi init --force            # Overwrite existing config
gi init --non-interactive  # Fail if prompts needed

gi auth-test

Verify GitLab authentication is working.

gi auth-test
# Authenticated as @username (Full Name)
# GitLab: https://gitlab.com

gi doctor

Check environment health and configuration.

gi doctor          # Human-readable output
gi doctor --json   # JSON output for scripting

gi ingest

Sync data from GitLab to local database.

gi ingest --type issues                    # Sync all projects
gi ingest --type issues --project group/repo  # Single project
gi ingest --type issues --force            # Override stale lock

gi list issues

Query issues from local database.

gi list issues                              # Recent issues (default 50)
gi list issues --limit 100                  # More results
gi list issues --state opened               # Only open issues
gi list issues --state closed               # Only closed issues
gi list issues --author username            # By author
gi list issues --assignee username          # By assignee
gi list issues --label bug                  # By label (AND logic)
gi list issues --label bug --label urgent   # Multiple labels
gi list issues --milestone "v1.0"           # By milestone title
gi list issues --since 7d                   # Updated in last 7 days
gi list issues --since 2w                   # Updated in last 2 weeks
gi list issues --since 2024-01-01           # Updated since date
gi list issues --due-before 2024-12-31      # Due before date
gi list issues --has-due-date               # Only issues with due dates
gi list issues --project group/repo         # Filter by project
gi list issues --sort created --order asc   # Sort options
gi list issues --open                       # Open first result in browser
gi list issues --json                       # JSON output

gi show issue

Display detailed issue information.

gi show issue 123                      # Show issue #123
gi show issue 123 --project group/repo # Disambiguate if needed

gi count

Count entities in local database.

gi count issues                    # Total issues
gi count discussions               # Total discussions
gi count discussions --type issue  # Issue discussions only
gi count notes                     # Total notes

gi sync-status

Show current sync state and watermarks.

gi sync-status

gi migrate

Run pending database migrations.

gi migrate

gi version

Show version information.

gi version

Database Schema

Data is stored in SQLite with the following main tables:

  • projects: Tracked GitLab projects
  • issues: Issue metadata (title, state, author, assignee info, due date, milestone)
  • milestones: Project milestones with state and due dates
  • issue_assignees: Many-to-many issue-assignee relationships
  • labels: Project labels with colors
  • issue_labels: Many-to-many issue-label relationships
  • discussions: Issue/MR discussions
  • notes: Individual notes within discussions
  • raw_payloads: Compressed original API responses

The database is stored at ~/.local/share/gi/gi.db by default.

Global Options

gi --config /path/to/config.json <command>  # Use alternate config

Development

# Run tests
cargo test

# Run with debug logging
RUST_LOG=gi=debug gi list issues

# Check formatting
cargo fmt --check

# Lint
cargo clippy

Tech Stack

  • Rust (2024 edition)
  • SQLite via rusqlite (bundled)
  • clap for CLI parsing
  • reqwest for HTTP
  • tokio for async runtime
  • serde for serialization
  • tracing for logging

License

MIT

Description
No description provided
Readme 42 MiB
Languages
Rust 100%