Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
156 changes: 156 additions & 0 deletions ARCHITECTURE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Architecture

This document describes the high-level structure of Nightshift: the module
layout, the command layer, and how the `internal/` packages group by
responsibility. It is intended as a map for contributors; it deliberately does
not duplicate the runtime walkthrough, which lives in
[docs/guides/run-lifecycle.md](docs/guides/run-lifecycle.md).

> **Accuracy note.** Package descriptions below are taken from each package's
> own `// Package ...` doc comment, and the dependency edges are taken from each
> package's imports. Statements about *how* components compose at runtime that
> are not directly supported by those two sources are marked **(assumption)**.

## Overview

Nightshift is a single Go module, `github.com/marcus/nightshift` (see `go.mod`,
Go 1.24). It is a CLI that orchestrates AI coding agents (Claude Code, Codex,
GitHub Copilot) to work on development tasks during off-hours, within a
configurable token budget. Runs are triggered on a schedule (cron / launchd /
systemd / the built-in daemon), execute a plan → implement → review loop through
an agent, and record their results as structured logs, SQLite state, and Markdown
reports.

Everything lands as a branch or pull request; the primary branch is never written
to directly.

## The command layer (`cmd/`)

There are two binaries:

| Path | Package | Role |
|------|---------|------|
| `cmd/nightshift/` | `main` | CLI entry point; calls `commands.Execute()`. |
| `cmd/nightshift/commands/` | `commands` | All cobra subcommands (`setup`, `run`, `preview`, `budget`, `task`, `doctor`, `status`, `logs`, `stats`, `daemon`, `config`, `report`, `snapshot`, `install`, `init`, `busfactor`, …). |
| `cmd/provider-calibration/` | `main` | Standalone diagnostic tool that aggregates token/session metrics from provider logs to suggest a budget-calibration multiplier. Not built into the main binary; run via `make calibrate-providers`. |

The `commands` package is the **composition root** for a run: `run.go` wires
together the `config`, `budget`, `calibrator`, `db`, `logging`, `orchestrator`,
`providers`, `reporting`, `state`, `tasks`, `trends`, and `agents` packages to
carry out a scheduled or manual run.

## Internal packages (`internal/`)

The packages under `internal/` are grouped below by responsibility. Each bullet's
description is the package's own doc comment; the import relationships are
verified from source.

### Configuration, state, and storage

- **`config`** — handles loading and validating nightshift configuration.
- **`state`** — manages persistent state for nightshift runs.
- **`db`** — provides SQLite-backed storage for nightshift state and snapshots.
Used as the shared persistence layer: `tasks`, `projects`, `snapshots`,
`trends`, `calibrator`, `stats`, and the run command all depend on it.

`config` is a foundational dependency — `budget`, `scheduler`, `tasks`,
`reporting`, `integrations`, `projects`, and `calibrator` all import it.

### Agents, providers, and integrations

- **`agents`** — provides interfaces and implementations for spawning AI agents.
- **`providers`** — defines interfaces and implementations for AI coding agents
(Claude, Codex, Copilot). A near-leaf package (no `internal/` dependencies).
- **`integrations`** — provides readers for external configuration and task
sources (e.g. `AGENTS.md`, `CLAUDE.md`, GitHub, td). Imports only `config`.

`agents` and `providers` are distinct: `providers` defines the agent backends and
interfaces, while `agents` provides the spawning machinery. **(assumption)** The
`orchestrator` drives `agents` to execute work through the chosen provider.

### Task selection and execution

- **`tasks`** — provides task selection and priority scoring. Imports `config`,
`db`, `state`.
- **`scheduler`** — handles time-based job scheduling. Imports `config`.
- **`orchestrator`** — coordinates AI agents working on tasks. Imports `agents`,
`budget`, `logging`, `tasks`.

### Budget and calibration

- **`budget`** — implements token budget calculation and allocation. Imports
`config` and `providers`.
- **`calibrator`** — tunes task budgets and scheduling based on historical usage
data. Imports `budget`, `config`, `db`.
- **`snapshots`** — collects and stores periodic usage data from AI providers.
Imports `db` and `tmux`.
- **`trends`** — analyzes historical snapshot data to surface usage patterns and
anomalies. Imports `db`.
- **`stats`** — computes aggregate statistics from nightshift run data. Imports
`budget`, `db`, `reporting`.
- **`tmux`** — scrapes tmux sessions to detect running AI agent processes and
their usage. A leaf package (no `internal/` dependencies).

These form a **usage feedback loop**: `tmux` scrapes live usage → `snapshots`
persists it to `db` → `trends` analyzes the history in `db` → `calibrator` reads
that history to retune `budget`. **(assumption)** The exact trigger and cadence
of calibration is governed by configuration (`budget.calibrate_enabled`,
`budget.snapshot_interval`).

### Security

- **`security`** — provides credential management for nightshift. A leaf package
(no `internal/` dependencies).

### Observability and reporting

- **`logging`** — provides structured logging with file rotation.
- **`reporting`** — generates morning summary reports for nightshift runs.
Imports `config` and `logging`.

### Analysis

- **`analysis`** — provides code ownership and bus-factor analysis tools (backing
the `busfactor` command).

### Projects and setup

- **`projects`** — handles multi-project discovery, resolution, and budget
allocation. Imports `config`, `db`, `state`.
- **`setup`** — provides interactive configuration and task preset selection for
new projects (backing the guided `setup` command).

## Runtime lifecycle

The sequence from a scheduled trigger to a finished run — config/logging init,
SQLite state, budget allowance, provider selection, the orchestrator's
plan → implement → review loop, and report generation — is documented with a
sequence diagram in
[docs/guides/run-lifecycle.md](docs/guides/run-lifecycle.md). Refer there rather
than reproducing the flow here.

## Where output goes

(From [run-lifecycle.md](docs/guides/run-lifecycle.md).)

- **Structured logs**: `~/.local/share/nightshift/logs/nightshift-YYYY-MM-DD.log`
- **Run report**: `~/.local/share/nightshift/reports/run-YYYY-MM-DD-HHMMSS.md`
- **Daily summary** (when `reporting.morning_summary: true`):
`~/.local/share/nightshift/summaries/summary-YYYY-MM-DD.md`
- **State**: SQLite under `~/.local/share/nightshift/` (managed by `db`)

## Open assumptions

To avoid fabrication, the following are explicitly assumptions rather than
verified facts, and should be confirmed against the code before being relied on:

- The `orchestrator` drives `agents` through the selected `providers` backend.
- The calibration feedback loop (`tmux` → `snapshots` → `trends`/`calibrator` →
`budget`) is triggered and cadenced by `budget.calibrate_enabled` and
`budget.snapshot_interval`.
- The `daemon` cobra command wraps `scheduler` to run tasks in the background
(per the run-lifecycle guide, which mentions "the daemon scheduler calls
`runScheduledTasks`").

If you verify any of these while working in the code, please update this file to
convert the assumption into a stated fact.
133 changes: 133 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
# Contributing to Nightshift

Thanks for helping improve Nightshift! This guide covers the local development
loop, the checks your changes must pass, and the commit/PR conventions the
project uses.

## Prerequisites

- **Go** 1.24 or newer (see `go.mod`)

Optional but recommended:

- **golangci-lint** — for `make lint` (install with
`go install github.com/golangci/golangci-lint/cmd/golangci-lint@latest`)
- **gum** — used by the CLI's preview pager (the CLI degrades gracefully without it)

## Getting the code

```bash
git clone https://github.com/marcus/nightshift.git
cd nightshift
make deps # go mod download && go mod tidy
```

## Development loop

All common tasks are wrapped as `make` targets:

| Command | What it does |
|---------|--------------|
| `make build` | Build the `nightshift` binary |
| `make test` | Run the full test suite (`go test ./...`) |
| `make test-verbose` | Run tests with `-v` |
| `make test-race` | Run tests with the race detector (`go test -race ./...`) |
| `make coverage` | Run tests with a coverage report (`go tool cover -func`) |
| `make coverage-html` | Generate an HTML coverage report |
| `make lint` | Run `golangci-lint` (if installed) |
| `make check` | Run `test` + `lint` |
| `make install` | Build and install the binary to your Go bin directory |
| `make calibrate-providers` | Run the provider-calibration diagnostic tool |
| `make help` | List all available targets |

Typical flow before opening a PR:

```bash
make build
make test-race
make lint
```

## Pre-commit hook

Install the git pre-commit hook to catch formatting and build issues before
pushing:

```bash
make install-hooks
```

This symlinks `scripts/pre-commit.sh` into `.git/hooks/pre-commit`. On every
commit it runs:

- **gofmt** — flags any staged `.go` files that need formatting
- **go vet** — catches common correctness issues
- **go build** — ensures the whole module compiles

To bypass the hook in a pinch:

```bash
git commit --no-verify
```

## Code style

Follow standard Go conventions (the project enforces `gofmt` and `go vet`):

- Run `gofmt -w .` before committing (the hook checks this for you).
- Wrap errors with context rather than swallowing them.
- Write table-driven tests in `_test.go` files alongside the code they cover.
- Keep log messages hyper-concise but informative.

## Branching and pull requests

1. **Never commit directly to `main`.** Branch off the latest `main`:

```bash
git checkout main
git pull
git checkout -b <your-branch>
```

Use a descriptive branch name, e.g. `docs/backfill-missing-docs`,
`fix/budget-overflow`, or `feat/new-task`.

2. Make your changes, keeping commits focused.
3. Push your branch and **open a pull request** against `marcus/nightshift`
`main`. Describe what changed and why; link any relevant issue.
4. Ensure CI and the pre-commit checks pass. Address review feedback with
additional commits (or a tidy rebase if the maintainer prefers).

## Commit trailers

Every commit should include the Nightshift task trailers so it can be traced
back to the work item that produced it. Add them to your commit message:

```
Fix budget rollover at midnight boundaries

Nightshift-Task: budget-rollover
Nightshift-Ref: https://github.com/marcus/nightshift
```

- `Nightshift-Task` — the short id/slug of the task this change belongs to.
- `Nightshift-Ref` — the canonical project URL (`https://github.com/marcus/nightshift`).

You can add trailers interactively in your editor, or pass them on the command line:

```bash
git commit -m "Fix budget rollover at midnight boundaries" \
-m "Nightshift-Task: budget-rollover" \
-m "Nightshift-Ref: https://github.com/marcus/nightshift"
```

## Reporting issues

Found a bug or have a feature idea? Please open an issue on
[github.com/marcus/nightshift/issues](https://github.com/marcus/nightshift/issues)
with a clear description and, for bugs, steps to reproduce.

## License

By contributing you agree that your contributions will be licensed under the
project's [MIT license](LICENSE).
10 changes: 10 additions & 0 deletions cmd/provider-calibration/main.go
Original file line number Diff line number Diff line change
@@ -1,3 +1,13 @@
// Command provider-calibration is a standalone diagnostic tool that aggregates
// token and session metrics from local provider logs (Codex sessions under
// ~/.codex/sessions and Claude projects under ~/.claude/projects) and compares
// per-session and per-user-turn token usage between the two providers. It
// derives the ratio of Codex to Claude token consumption and suggests a budget
// calibration multiplier, which feeds the budget manager's calibration.
//
// It is not part of the main nightshift binary; run it via
// "make calibrate-providers" or "go run ./cmd/provider-calibration".
// See docs/guides/provider-calibration.md for details.
package main

import (
Expand Down
52 changes: 51 additions & 1 deletion website/docs/cli-reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,21 @@ title: CLI Reference
| Command | Description |
|---------|-------------|
| `nightshift setup` | Guided global configuration |
| `nightshift init` | Create a configuration file |
| `nightshift run` | Execute scheduled tasks |
| `nightshift preview` | Show upcoming runs |
| `nightshift budget` | Check token budget status |
| `nightshift task` | Browse and run tasks |
| `nightshift config` | View and modify configuration |
| `nightshift doctor` | Check environment health |
| `nightshift status` | View run history |
| `nightshift report` | View reports from recent runs |
| `nightshift logs` | Stream or export logs |
| `nightshift stats` | Token usage statistics |
| `nightshift daemon` | Background scheduler |
| `nightshift daemon` | Manage background daemon |
| `nightshift install` | Install system service (launchd/systemd/cron) |
| `nightshift uninstall` | Remove system service |
| `nightshift busfactor` | Analyze code ownership concentration |

## Run Options

Expand Down Expand Up @@ -83,6 +89,50 @@ nightshift budget history -n 10
nightshift budget calibrate
```

## Configuration Commands

```bash
nightshift config get budget.max_percent # Read a value by key path
nightshift config set budget.max_percent 80 # Write a value by key path
nightshift config validate # Validate the configuration file
nightshift init # Create a new configuration file
```

| Subcommand | Description |
|------------|-------------|
| `config get KEY` | Get a configuration value by dotted key path |
| `config set KEY VALUE` | Set a configuration value by dotted key path |
| `config validate` | Validate the current configuration file |

`config get`/`config set` read and write project-local config by default. Use
`--global` (`-g`) to target the global config instead:

```bash
nightshift config set budget.max_percent 80 --global
```

## Daemon Commands

```bash
nightshift daemon start # Start as a background process
nightshift daemon start --foreground # Run in the foreground (for debugging)
nightshift daemon stop # Stop the running daemon
nightshift daemon status # Show whether the daemon is running
```

| Subcommand | Description |
|------------|-------------|
| `daemon start` | Start the nightshift daemon |
| `daemon stop` | Stop the running daemon (sends SIGTERM) |
| `daemon status` | Check daemon status |

`daemon start` flags:

| Flag | Default | Description |
|------|---------|-------------|
| `--foreground`, `-f` | `false` | Run in the foreground (don't daemonize) |
| `--timeout` | `30m` | Per-agent execution timeout |

## Global Flags

| Flag | Description |
Expand Down