Promote development to main

.github/workflows/ci.yml

@@ -16,16 +16,22 @@ jobs:
           go-version: "1.23"
 
       - name: Build
-        run: go build -o /dev/null ./cmd/coginfer
+        run: CGO_ENABLED=0 go build -o /dev/null ./cmd/coginfer
 
       - name: Lint
-        uses: golangci/golangci-lint-action@v6
+        uses: golangci/golangci-lint-action@v9
+        env:
+          CGO_ENABLED: 0
         with:
           version: latest
           args: --timeout=3m
 
       - name: Test
+        env:
+          CGO_ENABLED: 0
         run: go test ./... -v -count=1
 
       - name: Vet
+        env:
+          CGO_ENABLED: 0
         run: go vet ./...

.gitignore

@@ -5,13 +5,15 @@
 *.test
 *.out
 /vendor/
+!/vendor/llama.cpp/
 /go/pkg/
 *.sum
 !go.sum
 go.work
 
 # C/C++ build artifacts
 /build/
+/vendor/llama.cpp/build/
 *.o
 *.a
 *.so

AGENTS.md

@@ -1,42 +1,70 @@
-# CogInfer — CognitiveOS Inference Engine
+# CognitiveOS Inference Engine
 
-LLM inference server wrapping llama.cpp as a child process, exposing an Ollama-compatible HTTP API with CognitiveOS extensions.
+This repo contains two binaries: **coginfer** (Wide Model) and **cograw** (Raw Model firmware guardrail).
 
 ## Architecture
 
 ```
-cmd/coginfer         — Entry point, flag parsing
-internal/server/     — HTTP server with all API handlers
-  ├── /api/*         — Ollama-compatible: generate, chat, tags, pull, ps, delete
-  ├── /cognitiveos/* — Extensions: status, capabilities
-  ├── /api/negotiate — Resource negotiation
-  └── /health        — Healthcheck for cognitiveosd
-internal/llm/        — Backend interface + implementations
-  ├── mock           — Simulated generation (for testing/dev)
-  └── cli            — llama-cli subprocess (production)
-internal/model/      — Model scanning and metadata (.gguf file discovery)
+cmd/coginfer             — Wide Model HTTP inference server (Ollama-compatible)
+cmd/cograw               — Raw Model RPC server (firmware guardrail)
+internal/server/         — HTTP server with all API handlers
+  ├── /api/*             — Ollama-compatible: generate, chat, tags, pull, ps, delete
+  ├── /cognitiveos/*     — Extensions: status, capabilities
+  ├── /api/negotiate     — Resource negotiation
+  ├── /health            — Healthcheck for cognitiveosd
+  ├── backend_cgo.go     — CgoBackend constructor (build tag: cgo)
+  └── backend_stub.go    — Stub fallback (build tag: !cgo)
+internal/llm/            — Backend interface + implementations
+  ├── llm.go             — Backend interface, MockBackend
+  ├── bridge.go          — Single import "C" file wrapping llama.h API (build tag: cgo)
+  ├── cgobackend.go      — CgoBackend: Backend impl calling bridge functions (build tag: cgo)
+  └── loadopts.go        — LoadOptions{NumCtx, GPULayers, Threads}
+internal/model/          — Model scanning and metadata (.gguf file discovery)
+vendor/llama.cpp/        — Manually cloned (NOT a git submodule — no `.gitmodules` file)
 ```
 
-## Build
+### coginfer (Wide Model)
 
-```go
-go build -o bin/coginfer ./cmd/coginfer
+LLM inference server linking llama.cpp via a vendored CGo bridge, exposing an Ollama-compatible HTTP API with CognitiveOS extensions.
+
+#### Build
+
+```bash
+# With CGo (production — requires cmake + gcc + llama.cpp cloned into vendor/)
+cd vendor/llama.cpp && cmake -B build -DLLAMA_NO_ACCELERATE=1 \
+  -DLLAMA_STATIC=1 -DLLAMA_NATIVE=0 \
+  -DBUILD_SHARED_LIBS=0 -DLLAMA_BUILD_TESTS=0 \
+  -DLLAMA_BUILD_EXAMPLES=0 -DLLAMA_BUILD_SERVER=0 \
+  -DCMAKE_ARCHIVE_OUTPUT_DIRECTORY="$PWD/build"
+cmake --build build --target llama --config Release -j$(nproc)
+cd ../.. && CGO_ENABLED=1 go build -tags=cgo -o bin/coginfer ./cmd/coginfer
+
+# Without CGo (CI, development — mock backend only)
+CGO_ENABLED=0 go build -o bin/coginfer ./cmd/coginfer
 ```
 
-## Usage
+**CGo flags** (in `internal/llm/bridge.go`):
+- `-lllama` — NOT `-llama` (cmake target `llama` → `libllama.a`)
+- `-L.../build/src` — modern llama.cpp puts static archives in `build/src/`
+  (workaround: `CMAKE_ARCHIVE_OUTPUT_DIRECTORY` puts it in `build/`)
+- `-I.../ggml/include` — `llama.h` includes `ggml.h`, `ggml-cpu.h`, etc.
+- `CGO_LDFLAGS` is appended **before** `#cgo LDFLAGS` by Go toolchain;
+  `-L` in env var establishes search path before `-l` in source
+
+#### Usage
 
 ```bash
 # Start with mock backend (default, no llama.cpp needed)
 ./bin/coginfer --backend mock --models /cognitiveos/models
 
-# Start with real llama-cli backend
-./bin/coginfer --backend cli --models /cognitiveos/models
+# Start with CGo llama.cpp bridge (production)
+./bin/coginfer --backend cgo --models /cognitiveos/models
 
 # Custom port and log file
 ./bin/coginfer --addr 127.0.0.1:11434 --log /cognitiveos/logs/inference.log
 ```
 
-## API
+#### API
 
 | Method | Endpoint | Description |
 |--------|----------|-------------|
@@ -51,7 +79,79 @@ go build -o bin/coginfer ./cmd/coginfer
 | GET  | `/cognitiveos/capabilities` | Hardware capabilities |
 | GET  | `/health` | Healthcheck |
 
-## Backends
+#### Backends
+
+- **mock**: Simulated token generation with delays, no external dependencies. Default for development. Always available.
+- **cgo**: In-process llama.cpp via CGo bridge. Requires `CGO_ENABLED=1` and `vendor/llama.cpp/build/libllama.a`. Production default.
+
+#### Build Tags
+
+- `bridge.go`, `cgobackend.go`, `backend_cgo.go` — `//go:build cgo`
+- `backend_stub.go` — `//go:build !cgo`
+- CI runs `CGO_ENABLED=0` (Ubuntu defaults to `CGO_ENABLED=1`, which includes `cgo` tag — must be explicit)
+- `MockBackend` has no build tag — always available
+
+### cograw (Raw Model — Firmware Guardrail)
+
+Always-on, root-level RPC server that provides the firmware guardrail between the human and the Wide Model. Communicates over a Unix socket using JSON-RPC 2.0.
+
+**Key constraint:** cograw has **no knowledge of MCP tools or registries** — it is a pure guardrail GGUF. Tool routing is the daemon's responsibility.
+
+#### Location
+
+`cmd/cograw/main.go`
+
+#### Transport
+
+JSON-RPC 2.0 over Unix socket at `/cognitiveos/run/raw.sock` (mode 0600, root-owned).
+
+#### RPC Methods
+
+| Method | Purpose |
+|--------|---------|
+| `validate_system_code` | Validate system codes (wake/idle/security/reset/unlock) |
+| `check_unlock_code` | Validate unlock codes for paid patches |
+| `audit_resources` | Check hardware resources before Wide Model load |
+| `healthcheck` | Return running status |
+| `version` | Return version, model path, quantization |
+| `validate_prompt` | Guardrail check — allow/deny/modify prompts before they reach the Wide Model |
+
+**Full parameter/response schemas**: [product-specs/specs/raw-model.md](https://github.com/CognitiveOS-Project/product-specs/blob/development/specs/raw-model.md#rpc-methods)
+
+#### Build
+
+```bash
+# With CGo (production)
+cd vendor/llama.cpp && cmake -B build ... && cmake --build build --config Release -j$(nproc)
+CGO_ENABLED=1 go build -tags=cgo -o bin/cograw ./cmd/cograw
+
+# Without CGo (mock mode, no real guardrail)
+CGO_ENABLED=0 go build -o bin/cograw ./cmd/cograw
+```
+
+#### Flags
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--socket` | `/cognitiveos/run/raw.sock` | Unix socket path |
+| `--model` | `/cognitiveos/models/raw/raw-model.gguf` | Raw model GGUF path |
+| `--log` | (stderr) | Log file path |
+| `--audit-log` | `/cognitiveos/logs/raw/audit.log` | Audit log file path |
+
+#### Startup Order
+
+cograw starts **before** cognitiveosd. The daemon hard-fails if the raw socket is unavailable at boot.
+
+#### Model File
+
+- Path: `/cognitiveos/models/raw/raw-model.gguf`
+- Read-only squashfs partition
+- `root:root 0400`
+- Wide Model has **no read access**
+- Updated only via firmware flash (not `.cgp` packages)
+
+#### Spec Reference
 
-- **mock**: Simulated token generation with delays, no external dependencies. Default for development.
-- **cli**: Shells out to `llama-cli` for real inference. Pass `--backend cli` in production.
+- [raw-model.md](https://github.com/CognitiveOS-Project/product-specs/blob/development/specs/raw-model.md) — authoritative Raw Model specification
+- [architecture.md](https://github.com/CognitiveOS-Project/product-specs/blob/development/specs/architecture.md) — system architecture and layer diagram
+- [cognitiveosd-api.md](https://github.com/CognitiveOS-Project/product-specs/blob/development/specs/cognitiveosd-api.md) — daemon protocol and message types

README.md

@@ -1,36 +1,50 @@
-# inference — CogInfer
+# inference — CogInfer + CogRaw
 
-CognitiveOS inference engine — local LLM runtime for Raw Model and Wide Model execution. Exposes an Ollama-compatible HTTP API with CognitiveOS extensions.
+CognitiveOS inference engine — local LLM runtime for both **Raw Model** (firmware guardrail) and **Wide Model** (operational inference). Contains two binaries:
+
+- `coginfer` — Ollama-compatible HTTP inference server for the Wide Model
+- `cograw` — Root-level RPC server for the Raw Model (firmware GGUF guardrail)
 
 ## Architecture
 
 ```
-cmd/coginfer             — Entry point, flag parsing
+cmd/coginfer             — Wide Model inference server (HTTP, Ollama-compatible)
+cmd/cograw               — Raw Model RPC server (Unix socket, JSON-RPC 2.0)
 internal/server/         — HTTP server with all API handlers
-internal/llm/            — Backend interface + implementations (mock, cli)
+internal/llm/            — Backend interface + implementations (mock, cgo)
 internal/model/          — Model scanning and .gguf metadata discovery
 ```
 
-## Build
+## Binaries
+
+### coginfer (Wide Model)
+
+Exposes an Ollama-compatible HTTP API for the general-purpose Wide Model.
+
+#### Build
 
 ```bash
-go build -o bin/coginfer ./cmd/coginfer
+# With CGo (production — requires vendored llama.cpp build)
+CGO_ENABLED=1 go build -tags=cgo -o bin/coginfer ./cmd/coginfer
+
+# Without CGo (mock backend only, no llama.cpp needed)
+CGO_ENABLED=0 go build -o bin/coginfer ./cmd/coginfer
 ```
 
-## Usage
+#### Usage
 
 ```bash
 # Start with mock backend (no llama.cpp needed)
 ./bin/coginfer --backend mock --models /cognitiveos/models
 
-# Start with llama-cli backend (production)
-./bin/coginfer --backend cli --models /cognitiveos/models
+# Start with CGo llama.cpp backend (production)
+./bin/coginfer --backend cgo --models /cognitiveos/models
 
 # Custom port and log file
 ./bin/coginfer --addr 127.0.0.1:11434 --log /cognitiveos/logs/inference.log
 ```
 
-## API
+#### API
 
 | Method | Endpoint | Description |
 |--------|----------|-------------|
@@ -45,17 +59,78 @@ go build -o bin/coginfer ./cmd/coginfer
 | GET  | `/cognitiveos/capabilities` | Hardware capabilities |
 | GET  | `/health` | Healthcheck |
 
-## Backends
+#### Backends
 
 - **mock** — Simulated token generation with delays. Default for development.
-- **cli** — Shells out to `llama-cli` for real inference on device.
+- **cgo** — In-process llama.cpp via CGo bridge. Requires `CGO_ENABLED=1` and vendored llama.cpp build.
+
+### cograw (Raw Model — Firmware Guardrail)
+
+A small, always-on RPC server that acts as the firmware-level guardrail between the human and the Wide Model. Runs as root, communicates over a Unix socket via JSON-RPC 2.0.
+
+**Key constraint:** cograw has **no knowledge of MCP tools or registries** — it is a pure guardrail GGUF. Tool routing is the daemon's responsibility.
+
+For the authoritative specification, see [raw-model.md](https://github.com/CognitiveOS-Project/product-specs/blob/development/specs/raw-model.md) in the product-specs repo.
+
+#### Build
+
+Requires CGo and vendored llama.cpp. See [cograw build docs](cmd/cograw/) for details.
+
+#### Usage
+
+```bash
+./bin/cograw \
+  --socket /cognitiveos/run/raw.sock \
+  --model /cognitiveos/models/raw/raw-model.gguf
+```
+
+#### Flags
+
+| Flag | Default | Description |
+|------|---------|-------------|
+| `--socket` | `/cognitiveos/run/raw.sock` | Unix socket path for JSON-RPC |
+| `--model` | `/cognitiveos/models/raw/raw-model.gguf` | Path to raw model GGUF file |
+| `--log` | (stderr) | Log file path |
+
+#### RPC Methods
+
+| Method | Description |
+|--------|-------------|
+| `validate_system_code` | Validate a system code (wake/idle/security/reset/unlock) |
+| `check_unlock_code` | Validate an unlock code for a paid patch |
+| `audit_resources` | Check hardware resources before Wide Model load |
+| `healthcheck` | Return whether the Raw Model is running and ready |
+| `version` | Return Raw Model version and model info |
+| `validate_prompt` | Guardrail check: allow/deny/modify a prompt before it reaches the Wide Model |
+
+Detailed parameter and response schemas are in the [product-specs RPC methods table](https://github.com/CognitiveOS-Project/product-specs/blob/development/specs/raw-model.md#rpc-methods).
+
+#### Role in System
+
+```
+Human → cli → cognitiveosd → cograw (validate) → Wide Model (generate) → cognitiveosd (parse, route MCP) → cli
+```
+
+1. Human input arrives at the daemon via CLI
+2. Daemon sends the prompt to cograw for guardrail validation
+3. If allowed, daemon forwards to Wide Model for inference
+4. Daemon parses the Wide Model response and routes tool calls to MCP servers
+5. Final output returned to CLI
+
+cograw **must start before** cognitiveosd — the daemon hard-fails if the raw socket is unavailable at boot.
+
+#### Model Protection
+
+The raw model file is stored at `/cognitiveos/models/raw/raw-model.gguf` on a read-only squashfs partition:
+- Owner: `root:root`
+- Permissions: `0400`
+- The Wide Model has **no read access**
+- Only a full firmware update (physical reflash or signed update) can change it
 
 ## Related
 
-- [CognitiveOS](https://github.com/CognitiveOS-Project/cognitiveos) — main project repository
-- [cognitive-os.org](https://cognitive-os.org) — project website
-- [cognitiveosd](https://github.com/CognitiveOS-Project/cognitiveosd) — daemon that manages model lifecycle
-- [Product Specs](https://github.com/CognitiveOS-Project/product-specs) — inference API specification
+- [Product Specs](https://github.com/CognitiveOS-Project/product-specs) — authoritative specs for raw-model, architecture, API, and security model
+- [cognitiveosd](https://github.com/CognitiveOS-Project/cognitiveosd) — daemon that manages the Wide Model and routes tool calls
 - [CognitiveOS Project](https://github.com/CognitiveOS-Project) — GitHub organization
 
 ## Contributing

cmd/coginfer/main.go

@@ -11,7 +11,7 @@ import (
 func main() {
 	addr := flag.String("addr", "127.0.0.1:11434", "HTTP listen address")
 	modelDir := flag.String("models", "/cognitiveos/models", "model directory")
-	backend := flag.String("backend", "mock", "inference backend (mock, cli)")
+	backend := flag.String("backend", "mock", "inference backend (mock, cgo)")
 	logFile := flag.String("log", "", "log file path")
 	flag.Parse()
 
@@ -20,7 +20,7 @@ func main() {
 		if err != nil {
 			log.Fatal(err)
 		}
-		defer f.Close()
+		defer func() { _ = f.Close() }()
 		log.SetOutput(f)
 	}
 

cmd/cograw/cograw_llm.go

@@ -0,0 +1,9 @@
+//go:build cgo
+
+package main
+
+import "github.com/CognitiveOS-Project/inference/internal/llm"
+
+func newBackend() llm.Backend {
+	return llm.NewCgoBackend()
+}