From f4b2a21eaad27732437ae26aa65fafea196ff583 Mon Sep 17 00:00:00 2001 From: Lasse Larsen Date: Tue, 30 Jun 2026 02:40:08 +0200 Subject: [PATCH] docs: backfill missing project documentation Add CONTRIBUTING.md (dev loop, Makefile targets, pre-commit hook, branch/PR workflow, commit trailer convention) and ARCHITECTURE.md (module overview, cmd layer, internal packages grouped by responsibility, with inferred relationships flagged as assumptions). Add the missing package doc comment to cmd/provider-calibration, and backfill website/docs/cli-reference.md with the missing config/init/report/install/uninstall/busfactor commands plus daemon and config subcommand details. Nightshift-Task: docs-backfill Nightshift-Ref: https://github.com/marcus/nightshift --- ARCHITECTURE.md | 156 +++++++++++++++++++++++++++++++ CONTRIBUTING.md | 133 ++++++++++++++++++++++++++ cmd/provider-calibration/main.go | 10 ++ website/docs/cli-reference.md | 52 ++++++++++- 4 files changed, 350 insertions(+), 1 deletion(-) create mode 100644 ARCHITECTURE.md create mode 100644 CONTRIBUTING.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..8af5c76 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,156 @@ +# Architecture + +This document describes the high-level structure of Nightshift: the module +layout, the command layer, and how the `internal/` packages group by +responsibility. It is intended as a map for contributors; it deliberately does +not duplicate the runtime walkthrough, which lives in +[docs/guides/run-lifecycle.md](docs/guides/run-lifecycle.md). + +> **Accuracy note.** Package descriptions below are taken from each package's +> own `// Package ...` doc comment, and the dependency edges are taken from each +> package's imports. Statements about *how* components compose at runtime that +> are not directly supported by those two sources are marked **(assumption)**. + +## Overview + +Nightshift is a single Go module, `github.com/marcus/nightshift` (see `go.mod`, +Go 1.24). It is a CLI that orchestrates AI coding agents (Claude Code, Codex, +GitHub Copilot) to work on development tasks during off-hours, within a +configurable token budget. Runs are triggered on a schedule (cron / launchd / +systemd / the built-in daemon), execute a plan → implement → review loop through +an agent, and record their results as structured logs, SQLite state, and Markdown +reports. + +Everything lands as a branch or pull request; the primary branch is never written +to directly. + +## The command layer (`cmd/`) + +There are two binaries: + +| Path | Package | Role | +|------|---------|------| +| `cmd/nightshift/` | `main` | CLI entry point; calls `commands.Execute()`. | +| `cmd/nightshift/commands/` | `commands` | All cobra subcommands (`setup`, `run`, `preview`, `budget`, `task`, `doctor`, `status`, `logs`, `stats`, `daemon`, `config`, `report`, `snapshot`, `install`, `init`, `busfactor`, …). | +| `cmd/provider-calibration/` | `main` | Standalone diagnostic tool that aggregates token/session metrics from provider logs to suggest a budget-calibration multiplier. Not built into the main binary; run via `make calibrate-providers`. | + +The `commands` package is the **composition root** for a run: `run.go` wires +together the `config`, `budget`, `calibrator`, `db`, `logging`, `orchestrator`, +`providers`, `reporting`, `state`, `tasks`, `trends`, and `agents` packages to +carry out a scheduled or manual run. + +## Internal packages (`internal/`) + +The packages under `internal/` are grouped below by responsibility. Each bullet's +description is the package's own doc comment; the import relationships are +verified from source. + +### Configuration, state, and storage + +- **`config`** — handles loading and validating nightshift configuration. +- **`state`** — manages persistent state for nightshift runs. +- **`db`** — provides SQLite-backed storage for nightshift state and snapshots. + Used as the shared persistence layer: `tasks`, `projects`, `snapshots`, + `trends`, `calibrator`, `stats`, and the run command all depend on it. + +`config` is a foundational dependency — `budget`, `scheduler`, `tasks`, +`reporting`, `integrations`, `projects`, and `calibrator` all import it. + +### Agents, providers, and integrations + +- **`agents`** — provides interfaces and implementations for spawning AI agents. +- **`providers`** — defines interfaces and implementations for AI coding agents + (Claude, Codex, Copilot). A near-leaf package (no `internal/` dependencies). +- **`integrations`** — provides readers for external configuration and task + sources (e.g. `AGENTS.md`, `CLAUDE.md`, GitHub, td). Imports only `config`. + +`agents` and `providers` are distinct: `providers` defines the agent backends and +interfaces, while `agents` provides the spawning machinery. **(assumption)** The +`orchestrator` drives `agents` to execute work through the chosen provider. + +### Task selection and execution + +- **`tasks`** — provides task selection and priority scoring. Imports `config`, + `db`, `state`. +- **`scheduler`** — handles time-based job scheduling. Imports `config`. +- **`orchestrator`** — coordinates AI agents working on tasks. Imports `agents`, + `budget`, `logging`, `tasks`. + +### Budget and calibration + +- **`budget`** — implements token budget calculation and allocation. Imports + `config` and `providers`. +- **`calibrator`** — tunes task budgets and scheduling based on historical usage + data. Imports `budget`, `config`, `db`. +- **`snapshots`** — collects and stores periodic usage data from AI providers. + Imports `db` and `tmux`. +- **`trends`** — analyzes historical snapshot data to surface usage patterns and + anomalies. Imports `db`. +- **`stats`** — computes aggregate statistics from nightshift run data. Imports + `budget`, `db`, `reporting`. +- **`tmux`** — scrapes tmux sessions to detect running AI agent processes and + their usage. A leaf package (no `internal/` dependencies). + +These form a **usage feedback loop**: `tmux` scrapes live usage → `snapshots` +persists it to `db` → `trends` analyzes the history in `db` → `calibrator` reads +that history to retune `budget`. **(assumption)** The exact trigger and cadence +of calibration is governed by configuration (`budget.calibrate_enabled`, +`budget.snapshot_interval`). + +### Security + +- **`security`** — provides credential management for nightshift. A leaf package + (no `internal/` dependencies). + +### Observability and reporting + +- **`logging`** — provides structured logging with file rotation. +- **`reporting`** — generates morning summary reports for nightshift runs. + Imports `config` and `logging`. + +### Analysis + +- **`analysis`** — provides code ownership and bus-factor analysis tools (backing + the `busfactor` command). + +### Projects and setup + +- **`projects`** — handles multi-project discovery, resolution, and budget + allocation. Imports `config`, `db`, `state`. +- **`setup`** — provides interactive configuration and task preset selection for + new projects (backing the guided `setup` command). + +## Runtime lifecycle + +The sequence from a scheduled trigger to a finished run — config/logging init, +SQLite state, budget allowance, provider selection, the orchestrator's +plan → implement → review loop, and report generation — is documented with a +sequence diagram in +[docs/guides/run-lifecycle.md](docs/guides/run-lifecycle.md). Refer there rather +than reproducing the flow here. + +## Where output goes + +(From [run-lifecycle.md](docs/guides/run-lifecycle.md).) + +- **Structured logs**: `~/.local/share/nightshift/logs/nightshift-YYYY-MM-DD.log` +- **Run report**: `~/.local/share/nightshift/reports/run-YYYY-MM-DD-HHMMSS.md` +- **Daily summary** (when `reporting.morning_summary: true`): + `~/.local/share/nightshift/summaries/summary-YYYY-MM-DD.md` +- **State**: SQLite under `~/.local/share/nightshift/` (managed by `db`) + +## Open assumptions + +To avoid fabrication, the following are explicitly assumptions rather than +verified facts, and should be confirmed against the code before being relied on: + +- The `orchestrator` drives `agents` through the selected `providers` backend. +- The calibration feedback loop (`tmux` → `snapshots` → `trends`/`calibrator` → + `budget`) is triggered and cadenced by `budget.calibrate_enabled` and + `budget.snapshot_interval`. +- The `daemon` cobra command wraps `scheduler` to run tasks in the background + (per the run-lifecycle guide, which mentions "the daemon scheduler calls + `runScheduledTasks`"). + +If you verify any of these while working in the code, please update this file to +convert the assumption into a stated fact. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..c65174a --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,133 @@ +# Contributing to Nightshift + +Thanks for helping improve Nightshift! This guide covers the local development +loop, the checks your changes must pass, and the commit/PR conventions the +project uses. + +## Prerequisites + +- **Go** 1.24 or newer (see `go.mod`) + +Optional but recommended: + +- **golangci-lint** — for `make lint` (install with + `go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest`) +- **gum** — used by the CLI's preview pager (the CLI degrades gracefully without it) + +## Getting the code + +```bash +git clone https://github.com/marcus/nightshift.git +cd nightshift +make deps # go mod download && go mod tidy +``` + +## Development loop + +All common tasks are wrapped as `make` targets: + +| Command | What it does | +|---------|--------------| +| `make build` | Build the `nightshift` binary | +| `make test` | Run the full test suite (`go test ./...`) | +| `make test-verbose` | Run tests with `-v` | +| `make test-race` | Run tests with the race detector (`go test -race ./...`) | +| `make coverage` | Run tests with a coverage report (`go tool cover -func`) | +| `make coverage-html` | Generate an HTML coverage report | +| `make lint` | Run `golangci-lint` (if installed) | +| `make check` | Run `test` + `lint` | +| `make install` | Build and install the binary to your Go bin directory | +| `make calibrate-providers` | Run the provider-calibration diagnostic tool | +| `make help` | List all available targets | + +Typical flow before opening a PR: + +```bash +make build +make test-race +make lint +``` + +## Pre-commit hook + +Install the git pre-commit hook to catch formatting and build issues before +pushing: + +```bash +make install-hooks +``` + +This symlinks `scripts/pre-commit.sh` into `.git/hooks/pre-commit`. On every +commit it runs: + +- **gofmt** — flags any staged `.go` files that need formatting +- **go vet** — catches common correctness issues +- **go build** — ensures the whole module compiles + +To bypass the hook in a pinch: + +```bash +git commit --no-verify +``` + +## Code style + +Follow standard Go conventions (the project enforces `gofmt` and `go vet`): + +- Run `gofmt -w .` before committing (the hook checks this for you). +- Wrap errors with context rather than swallowing them. +- Write table-driven tests in `_test.go` files alongside the code they cover. +- Keep log messages hyper-concise but informative. + +## Branching and pull requests + +1. **Never commit directly to `main`.** Branch off the latest `main`: + + ```bash + git checkout main + git pull + git checkout -b + ``` + + Use a descriptive branch name, e.g. `docs/backfill-missing-docs`, + `fix/budget-overflow`, or `feat/new-task`. + +2. Make your changes, keeping commits focused. +3. Push your branch and **open a pull request** against `marcus/nightshift` + `main`. Describe what changed and why; link any relevant issue. +4. Ensure CI and the pre-commit checks pass. Address review feedback with + additional commits (or a tidy rebase if the maintainer prefers). + +## Commit trailers + +Every commit should include the Nightshift task trailers so it can be traced +back to the work item that produced it. Add them to your commit message: + +``` +Fix budget rollover at midnight boundaries + +Nightshift-Task: budget-rollover +Nightshift-Ref: https://github.com/marcus/nightshift +``` + +- `Nightshift-Task` — the short id/slug of the task this change belongs to. +- `Nightshift-Ref` — the canonical project URL (`https://github.com/marcus/nightshift`). + +You can add trailers interactively in your editor, or pass them on the command line: + +```bash +git commit -m "Fix budget rollover at midnight boundaries" \ + -m "Nightshift-Task: budget-rollover" \ + -m "Nightshift-Ref: https://github.com/marcus/nightshift" +``` + +## Reporting issues + +Found a bug or have a feature idea? Please open an issue on +[github.com/marcus/nightshift/issues](https://github.com/marcus/nightshift/issues) +with a clear description and, for bugs, steps to reproduce. + +## License + +By contributing you agree that your contributions will be licensed under the +project's [MIT license](LICENSE). diff --git a/cmd/provider-calibration/main.go b/cmd/provider-calibration/main.go index ee573b2..9761947 100644 --- a/cmd/provider-calibration/main.go +++ b/cmd/provider-calibration/main.go @@ -1,3 +1,13 @@ +// Command provider-calibration is a standalone diagnostic tool that aggregates +// token and session metrics from local provider logs (Codex sessions under +// ~/.codex/sessions and Claude projects under ~/.claude/projects) and compares +// per-session and per-user-turn token usage between the two providers. It +// derives the ratio of Codex to Claude token consumption and suggests a budget +// calibration multiplier, which feeds the budget manager's calibration. +// +// It is not part of the main nightshift binary; run it via +// "make calibrate-providers" or "go run ./cmd/provider-calibration". +// See docs/guides/provider-calibration.md for details. package main import ( diff --git a/website/docs/cli-reference.md b/website/docs/cli-reference.md index d5a2cd4..9699244 100644 --- a/website/docs/cli-reference.md +++ b/website/docs/cli-reference.md @@ -10,15 +10,21 @@ title: CLI Reference | Command | Description | |---------|-------------| | `nightshift setup` | Guided global configuration | +| `nightshift init` | Create a configuration file | | `nightshift run` | Execute scheduled tasks | | `nightshift preview` | Show upcoming runs | | `nightshift budget` | Check token budget status | | `nightshift task` | Browse and run tasks | +| `nightshift config` | View and modify configuration | | `nightshift doctor` | Check environment health | | `nightshift status` | View run history | +| `nightshift report` | View reports from recent runs | | `nightshift logs` | Stream or export logs | | `nightshift stats` | Token usage statistics | -| `nightshift daemon` | Background scheduler | +| `nightshift daemon` | Manage background daemon | +| `nightshift install` | Install system service (launchd/systemd/cron) | +| `nightshift uninstall` | Remove system service | +| `nightshift busfactor` | Analyze code ownership concentration | ## Run Options @@ -83,6 +89,50 @@ nightshift budget history -n 10 nightshift budget calibrate ``` +## Configuration Commands + +```bash +nightshift config get budget.max_percent # Read a value by key path +nightshift config set budget.max_percent 80 # Write a value by key path +nightshift config validate # Validate the configuration file +nightshift init # Create a new configuration file +``` + +| Subcommand | Description | +|------------|-------------| +| `config get KEY` | Get a configuration value by dotted key path | +| `config set KEY VALUE` | Set a configuration value by dotted key path | +| `config validate` | Validate the current configuration file | + +`config get`/`config set` read and write project-local config by default. Use +`--global` (`-g`) to target the global config instead: + +```bash +nightshift config set budget.max_percent 80 --global +``` + +## Daemon Commands + +```bash +nightshift daemon start # Start as a background process +nightshift daemon start --foreground # Run in the foreground (for debugging) +nightshift daemon stop # Stop the running daemon +nightshift daemon status # Show whether the daemon is running +``` + +| Subcommand | Description | +|------------|-------------| +| `daemon start` | Start the nightshift daemon | +| `daemon stop` | Stop the running daemon (sends SIGTERM) | +| `daemon status` | Check daemon status | + +`daemon start` flags: + +| Flag | Default | Description | +|------|---------|-------------| +| `--foreground`, `-f` | `false` | Run in the foreground (don't daemonize) | +| `--timeout` | `30m` | Per-agent execution timeout | + ## Global Flags | Flag | Description |