Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ brew install go node uv
3. Add or update tests for changed behavior.
4. Update the documentation when you change commands, flags, output, or
requirements.
5. Run `make test`.
5. Run `make test`. It runs the engine tests in parallel; set
`PYTEST_XDIST_AUTO_NUM_WORKERS` to choose the worker count.

## Open a pull request

Expand Down
6 changes: 3 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
# The OpenCode client is built for Apple Silicon only.
ifeq ($(shell uname -sm),Darwin arm64)
test: test-cli test-engine
CLIENTS = opencode pi
CLIENTS = opencode pi omp
else
test: test-cli
CLIENTS = pi
CLIENTS = pi omp
endif

test-cli:
Expand All @@ -24,7 +24,7 @@ PYTHON_ROOT = $(shell uv python dir)/cpython-3.12.12-macos-aarch64-none

test-engine:
cd engine && uv venv --allow-existing --python 3.12 && uv pip install '.[validation]'
cd engine && .venv/bin/pytest -v -rs
cd engine && nice -n 10 .venv/bin/pytest -v -rs -n auto --dist loadgroup
uv python install 3.12.12
rm -rf .build/bundle-mlx .build/smoke
$(PYTHON_ROOT)/bin/python3.12 packaging/build-mlx-runtime.py --python-root $(PYTHON_ROOT) --output .build/bundle-mlx
Expand Down
21 changes: 17 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,11 +48,12 @@ terminals. All other data lives under `~/.local/share/rec`.

We find the best performance from our models when paired with the
[pi](https://pi.dev/) harness. For convenience, rec provides pre-configured
recipes for running pi and [OpenCode](https://opencode.ai) harnesses. Run
recipes for running pi, omp, and [OpenCode](https://opencode.ai) harnesses. Run
them using:

```bash
rec launch pi # preferred
rec launch omp
rec launch opencode
```

Expand All @@ -65,7 +66,7 @@ run downloads the model and harness. The model server continues to run after
you close the terminal. Run `rec stop` to stop it.

These harnesses are kept separate from anything on your system. Use
`rec exec pi|opencode ARGS` to run their own commands, for example:
`rec exec pi|omp|opencode ARGS` to run their own commands, for example:

```bash
rec exec pi install npm:PACKAGE
Expand Down Expand Up @@ -109,6 +110,18 @@ The response contains generated text in `choices`. For other clients, use
base URL `http://127.0.0.1:11500/v1`, model `flint`, and the API key from
`rec auth token`.

To bound the reasoning of a chat request, add `"thinking_budget": N`, a
nonnegative integer. It needs `grammar=flint` (the default). N counts the
tokens after the model's own `<think>`; when N is reached, Flint closes the
`</think>` block itself. A budget of 0 skips reasoning. A prompt that already
ends with `<think>` is rejected with HTTP 400. A prompt with an
unclosed `<think>` anywhere is also rejected. `/v1/messages` rejects only
`thinking_budget` with 400; `thinking.budget_tokens` is ignored as before. When
the request sets `thinking_budget` (0 included), the response `usage` reports
`reasoning_tokens` and `thinking_forced_close`. Streaming responses report them
only in the final usage chunk, so set `stream_options.include_usage`. Without a
budget these keys do not appear.

The Anthropic-compatible API is available at
`http://127.0.0.1:11500/v1/messages`. Point `ANTHROPIC_BASE_URL` at
`http://127.0.0.1:11500` and send the token from `rec auth token` as a bearer
Expand All @@ -119,8 +132,8 @@ token or as `x-api-key`. The model is `flint`.

| Command | What it does |
|---|---|
| `rec launch pi\|opencode` | Start the server if needed and open a harness |
| `rec exec pi\|opencode ARGS` | Run a harness command, such as installing a plugin |
| `rec launch pi\|omp\|opencode` | Start the server if needed and open a harness |
| `rec exec pi\|omp\|opencode ARGS` | Run a harness command, such as installing a plugin |
| `rec serve` | Start the model server in this terminal |
| `rec status` | Show whether the server is running |
| `rec stop` | Stop the server |
Expand Down
6 changes: 6 additions & 0 deletions cli/clients.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ var openCodeLogo embed.FS
//go:embed clients/pi/package.json
var piPackage []byte

//go:embed clients/omp/package.json
var ompPackage []byte

//go:embed clients/omp/package-lock.json
var ompLock []byte

//go:embed clients/pi/package-lock.json
var piLock []byte

Expand Down
Loading