Skip to content

Development

Jump to bottom
sysid edited this page Sep 28, 2026 · 6 revisions

Development

This guide covers development setup, testing, contributing, and the technical architecture of bkmr for developers who want to contribute or understand the codebase.

Overview

bkmr is built with:

  • Language: Rust 2021 edition
  • Database: SQLite with Diesel ORM + rusqlite (for sqlite-vec)
  • Search: FTS5 full-text search + sqlite-vec vector similarity search
  • Embeddings: fastembed (ONNX Runtime, local, offline)
  • CLI Framework: Clap v4
  • LSP: tower-lsp with async tokio runtime
  • Testing: Cargo test with shared database strategy

Getting Started

Prerequisites

Required:

  • Rust 1.70+ (latest stable recommended)
  • SQLite 3.35+
  • Git

Optional (for full development):

  • fzf (for fuzzy finder testing)
  • Python 3.8+ (for LSP test scripts)
  • jq (for JSON processing in examples)

Clone and Build

# Clone repository
git clone https://github.com/sysid/bkmr.git
cd bkmr

# Build debug version (includes all functionality)
cargo build

# Build release version (optimized)
cargo build --release

# Binary location
ls -la target/debug/bkmr        # Debug
ls -la target/release/bkmr      # Release

Development Setup

1. Create test database:

# Create at standard test location
./target/debug/bkmr create-db ../db/bkmr.db

# Set environment variable
export BKMR_DB_URL=../db/bkmr.db

2. Verify build:

# Check version
./target/debug/bkmr --version

# Test basic functionality
./target/debug/bkmr add "https://example.com" test
./target/debug/bkmr search "test"

3. Configure editor:

# For code navigation and development
# Rust Analyzer recommended for VS Code/Neovim

Project Structure

bkmr/
├── bkmr/                    # Main Rust project
│   ├── src/
│   │   ├── main.rs          # Entry point, CLI routing
│   │   ├── cli/             # CLI commands and handlers
│   │   ├── application/     # Use cases and business logic
│   │   ├── domain/          # Core entities and traits
│   │   ├── infrastructure/  # External systems (DB, HTTP, embeddings)
│   │   └── lsp/             # LSP server implementation
│   ├── migrations/          # Diesel database migrations
│   ├── tests/               # Integration tests
│   └── Cargo.toml           # Dependencies and configuration
├── docs/                    # Documentation (to be deprecated)
├── bkmr.wiki/               # GitHub wiki (main documentation)
├── scripts/                 # Utility scripts
│   └── lsp/                 # LSP testing scripts
├── Makefile                 # Development commands
└── README.md                # Project overview

Architecture Overview

bkmr follows Clean Architecture principles:

Layer Structure:

  1. Domain (src/domain/) - Core business entities and traits

    • No external dependencies
    • Pure business logic
    • Entity definitions (Bookmark, Tag, etc.)

  2. Application (src/application/) - Use cases and orchestration

    • Service traits and implementations
    • Depends only on Domain interfaces
    • Business workflow coordination

  3. Infrastructure (src/infrastructure/) - External systems

    • SQLite repository implementations (Diesel + rusqlite for sqlite-vec)
    • Local embeddings via fastembed (ONNX Runtime)
    • sqlite-vec virtual table for vector storage and similarity search
    • HTTP client for metadata fetching
    • Clipboard operations

  4. CLI (src/cli/) - User interface

    • Command parsing (Clap)
    • Command handlers
    • Output formatting

  5. LSP (src/lsp/) - Editor integration

    • tower-lsp protocol implementation
    • Async/sync bridging
    • Language-aware filtering

Dependency Injection:

  • ServiceContainer pattern for dependency wiring
  • All services accept dependencies via constructors
  • No global state (eliminated in v4.31+)
  • Arc for shared service ownership

Running Tests

make test        # all unit + integration tests, single-threaded (also runs in CI)

Everything else — why tests must run single-threaded, test layout and conventions, ignored tests, scenario suites (hsearch), fixtures and manual editor testing — is in TESTING.md, the single source for testing.

Development Commands

Makefile Targets

# Build
make build              # Build release version
make                    # List all targets (help)

# Testing (see TESTING.md)
make test               # L1: unit + integration tests (single-threaded)
make test-scenarios     # L2: hsearch scenario checks
make test-env           # L3: seed a throwaway DB for manual editor testing

# Code Quality
make format             # Format code with cargo fmt
make lint               # Run clippy linter with auto-fix

# Maintenance
make clean              # Clean build artifacts
make install            # Install to ~/bin/ with version suffix

Manual Commands

Format code:

cargo fmt
cargo fmt --check  # Check without modifying

Linting:

cargo clippy --all-targets --all-features
cargo clippy --fix  # Auto-fix warnings

Build variations:

# Debug build (fast compile, unoptimized)
cargo build

# Release build (optimized)
cargo build --release

# With specific features (no longer needed, all features always enabled)
cargo build --release

Run from source:

# Debug binary
./target/debug/bkmr search "test"

# Release binary
./target/release/bkmr search "test"

# Or use cargo run
cargo run -- search "test"
cargo run --release -- search "test"

Database Management

Schema Changes

Create new migration:

# Install diesel CLI
cargo install diesel_cli --no-default-features --features sqlite

# Create migration
diesel migration generate add_new_column

# Edit generated files in migrations/
# - up.sql: Schema changes
# - down.sql: Rollback changes

# Apply migration
diesel migration run

# Rollback if needed
diesel migration revert

Migration Best Practices:

  1. Always create backup before schema changes
  2. Test migrations on copy of production database
  3. Write both up.sql and down.sql
  4. Document complex migrations in comments

Test Database

make test deletes and recreates db/bkmr.db; see TESTING.md.

Debugging

Debug Logging

Log level comes only from -d / -dd / -ddd; the binary ignores RUST_LOG. Details, including LSP server logs, in TESTING.md › Debug logging.

Common Debug Scenarios

Search not working:

# Debug search execution (global -dd, narrowed with grep)
cargo run -- -dd search "test" 2>&1 | grep -i search

Template not interpolating:

# Debug template rendering (global -dd, narrowed with grep)
cargo run -- -dd open <id> 2>&1 | grep -i template

Database issues:

# Debug database operations (global -dd, narrowed with grep)
cargo run -- -dd search "test" 2>&1 | grep -i database

Contributing

Before Starting

  1. Check existing issues: Look for similar feature requests or bug reports
  2. Open discussion: For significant changes, open an issue first
  3. Read architecture: Understand Clean Architecture principles
  4. Run tests: Ensure existing tests pass

Development Workflow

1. Create feature branch:

git checkout -b feature/your-feature-name
# or
git checkout -b fix/your-bug-fix

2. Make changes:

  • Write code following Rust conventions
  • Add tests for new functionality
  • Update documentation
  • Run tests frequently

3. Test thoroughly:

# Format code
make format

# Run linter
make lint

# Run all tests (CRITICAL)
make test

# All tests must pass before committing

4. Commit changes:

# Write clear commit messages
git commit -m "Add: New feature description"
git commit -m "Fix: Bug description"
git commit -m "Docs: Documentation update"

5. Push and create PR:

git push origin feature/your-feature-name

# Create pull request on GitHub
# Fill in PR template with:
# - Description of changes
# - Related issues
# - Testing performed

Code Style Guidelines

Rust Conventions:

  • Follow rustfmt formatting (run make format)
  • Use snake_case for functions and variables
  • Use PascalCase for types and traits
  • Add documentation comments for public APIs
  • Use thiserror for error types
  • Prefer Result<T> over panics

Error Handling:

// Good: Propagate errors with context
let bookmark = repository.get_bookmark(id)
    .context("Failed to retrieve bookmark")?;

// Good: Descriptive error messages
return Err(DomainError::NotFound(format!("Bookmark {} not found", id)));

// Avoid: Generic errors
return Err(DomainError::Unknown);

Dependency Injection:

// Good: Explicit dependencies via constructor
pub struct BookmarkServiceImpl {
    repository: Arc<dyn BookmarkRepository>,
    embedder: Arc<dyn Embedder>,
}

impl BookmarkServiceImpl {
    pub fn new(
        repository: Arc<dyn BookmarkRepository>,
        embedder: Arc<dyn Embedder>,
    ) -> Arc<dyn BookmarkService> {
        Arc::new(Self { repository, embedder })
    }
}

// Avoid: Global state or factory methods
// These patterns have been completely eliminated

Testing:

// Good: Clear test names and structure
#[test]
fn given_valid_url_when_adding_bookmark_then_succeeds() {
    // Arrange
    let test_container = TestServiceContainer::new();
    let service = test_container.bookmark_service.clone();

    // Act
    let result = service.add_bookmark(
        "https://example.com",
        &["test"],
        Some("Test Bookmark"),
        None,
        false,
    );

    // Assert
    assert!(result.is_ok());
}

Documentation

Code Documentation:

/// Retrieves a bookmark by its ID.
///
/// # Arguments
/// * `id` - The unique identifier of the bookmark
///
/// # Returns
/// Returns the bookmark if found, or an error if not found or inaccessible.
///
/// # Errors
/// * `DomainError::NotFound` - Bookmark with given ID doesn't exist
/// * `DomainError::DatabaseError` - Database access failed
pub fn get_bookmark(&self, id: i32) -> DomainResult<Bookmark> {
    // Implementation
}

Wiki Documentation:

  • Update wiki pages for user-facing changes
  • Add examples for new features
  • Update troubleshooting for new issues
  • Keep documentation in sync with code

Pull Request Guidelines

PR Description Should Include:

  • Clear description of changes
  • Motivation and context
  • Related issues (closes #123)
  • Breaking changes (if any)
  • Testing performed

Before Submitting:

  • ✅ All tests pass (make test)
  • ✅ Code formatted (make format)
  • ✅ No clippy warnings (make lint)
  • ✅ Documentation updated
  • ✅ Commit messages are clear

Review Process:

  1. Automated CI checks must pass
  2. Code review by maintainers
  3. Address feedback
  4. Final approval and merge

Release Process

Version Numbers

Follow Semantic Versioning (SemVer):

  • Major: Breaking changes (e.g., 4.0.0 → 5.0.0)
  • Minor: New features, backward compatible (e.g., 4.31.0 → 4.32.0)
  • Patch: Bug fixes (e.g., 4.31.0 → 4.31.1)

Release Checklist

1. Prepare release:

  • Update CHANGELOG.md
  • Update version in Cargo.toml
  • Run full test suite
  • Test on multiple platforms

2. Create release:

# Tag version
git tag -a v4.32.0 -m "Release 4.32.0"
git push origin v4.32.0

# GitHub Actions handles:
# - Building binaries
# - Publishing to crates.io
# - Creating GitHub release
# - Building Python wheels

3. Verify release:

  • Check crates.io publication
  • Verify GitHub release artifacts
  • Test installation from crates.io

Architecture Details

Clean Architecture Layers

Domain Layer (innermost):

// Pure business logic, no external dependencies
pub trait BookmarkRepository: Send + Sync {
    fn get_bookmark(&self, id: i32) -> DomainResult<Bookmark>;
    fn add_bookmark(&self, bookmark: &Bookmark) -> DomainResult<i32>;
}

Application Layer:

// Use cases and orchestration
pub trait BookmarkService: Send + Sync {
    fn get_all_bookmarks(&self, limit: Option<i32>, offset: Option<i32>)
        -> ApplicationResult<Vec<Bookmark>>;
}

Infrastructure Layer:

// External system implementations
pub struct SqliteBookmarkRepository {
    pool: Pool<ConnectionManager<SqliteConnection>>,
}

impl BookmarkRepository for SqliteBookmarkRepository {
    fn get_bookmark(&self, id: i32) -> DomainResult<Bookmark> {
        // SQLite-specific implementation
    }
}

Dependency Flow

main.rs
  ↓
ServiceContainer (composition root)
  ↓ Creates all services with dependencies
CLI Commands
  ↓ Receives ServiceContainer + Settings
Application Services
  ↓ Business logic orchestration
Domain Entities
  ↓ Pure business logic
Infrastructure
  ↓ External systems

Error Handling Strategy

Layer-Specific Errors:

  • DomainError - Business rule violations
  • ApplicationError - Use case failures
  • InfrastructureError - External system errors
  • CliError - User interface errors

Error Conversion:

// Infrastructure → Domain
impl From<diesel::result::Error> for DomainError {
    fn from(err: diesel::result::Error) -> Self {
        DomainError::DatabaseError(err.to_string())
    }
}

// Domain → Application
impl From<DomainError> for ApplicationError {
    fn from(err: DomainError) -> Self {
        ApplicationError::DomainError(err)
    }
}

Advanced Topics

LSP Implementation

Architecture:

  • BkmrLspBackend - Main LSP server
  • tower-lsp for protocol handling
  • tokio runtime for async operations
  • Sync bkmr services wrapped in spawn_blocking

Key Files:

  • src/lsp/backend.rs - LSP backend implementation
  • src/lsp/services/ - LSP-specific services
  • src/lsp/domain/ - Language registry and mappings

Testing Philosophy

See TESTING.md: tests are specifications (given_X_when_Y_then_Z, Arrange/Act/Assert), dependencies are injected via TestServiceContainer, the LSP server is tested black-box over stdio.

Performance Considerations

Database:

  • Connection pooling (r2d2, max_size=15)
  • FTS5 for efficient search
  • Indexed columns for common queries

Memory:

  • Arc for shared ownership
  • Lazy loading for large content
  • Efficient string handling

Concurrency:

  • Single-threaded database access (SQLite limitation)
  • Arc for thread-safe sharing
  • No global mutable state

Community

Resources:

Getting Help:

  • Check wiki for documentation
  • Search existing issues
  • Open new issue with details
  • Join discussions

Related Pages

bkmr

Getting Started
Using bkmr
Agent Integration
Advanced
Reference

Clone this wiki locally