diff --git a/AGENTS.md b/AGENTS.md index b9aa97aa4b..99f22bc408 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,6 +20,7 @@ Prefer `make` targets at repo root: - `make build`: regenerates protos/codegen and builds `opentdf` + `sdk` + `examples`. - `make lint`: runs `buf lint`, `golangci-lint`, and `govulncheck` across modules. - `make test`: runs `go test ./... -race` across core modules (does **not** include `tests-bdd/`). +- `make fuzz`: fuzzes every `FuzzXxx` target across modules. Not part of `make test` or CI; run it deliberately. - `docker compose up`: brings up local infra (Postgres + Keycloak). See `docs/Contributing.md`. ## Coding Style & Naming Conventions @@ -55,6 +56,7 @@ Prefer `make` targets at repo root: - **BDD tests**: run `cd tests-bdd && go test ./...` (requires Docker; feature files are `tests-bdd/features/*.feature`). - **Integration tests** may require the compose stack; follow module README(s) under `service/`. - **README tests**: verify code examples in documentation compile and work correctly. +- **Fuzz tests**: `FuzzXxx` functions in `*_test.go`; run `make fuzz` (override the per-target budget with `make fuzz FUZZTIME=10m`). A plain `make test` only replays their seed corpus — it does not fuzz. When a run finds a crasher, Go writes it to `testdata/fuzz//`, and every later `go test` replays it as a seed; commit that file together with the fix, never ahead of it. - **Cross-SDK e2e (xtest)**: the `opentdf/tests` repo runs the platform against the go/java/js SDKs. Use it to validate cross-language behavior (e.g. DPoP, TDF interop) that unit tests can't cover. ### Running xtest on a branch diff --git a/Makefile b/Makefile index 779a88d308..1af1039d0d 100644 --- a/Makefile +++ b/Makefile @@ -1,7 +1,7 @@ # make # To run all lint checks: `LINT_OPTIONS= make lint` -.PHONY: all buf-check build clean connect-wrapper-generate docker-build fix fmt go-lint license lint otdfctl/otdfctl policy-sql-gen proto-generate proto-helper-generate proto-lint sdk/sdk sqlc-check test tidy toolcheck +.PHONY: all buf-check build clean connect-wrapper-generate docker-build fix fmt fuzz go-lint license lint otdfctl/otdfctl policy-sql-gen proto-generate proto-helper-generate proto-lint sdk/sdk sqlc-check test tidy toolcheck MODS=protocol/go lib/ocrypto lib/fixtures lib/flattening lib/identifier sdk service examples otdfctl tests-bdd HAND_MODS=lib/ocrypto lib/fixtures lib/flattening lib/identifier sdk service examples otdfctl tests-bdd @@ -115,8 +115,42 @@ policy-erd-gen: test: for m in $(HAND_MODS); do (cd $$m && go test ./... -race) || exit 1; done +# Budget per fuzz target, not for the run as a whole: `make fuzz` costs roughly +# FUZZTIME multiplied by the number of targets. Override for a longer soak, e.g. +# `make fuzz FUZZTIME=10m`. +FUZZTIME?=30s + +# `go test -fuzz` mutates input only when -fuzz is passed; without it a FuzzXxx +# function is an ordinary test that replays its seed corpus and -fuzztime is +# ignored. It also accepts exactly one package and one target per invocation +# (`go test ./... -fuzz=...` is rejected), so the nested loop is required rather +# than stylistic. +# +# Targets are discovered instead of listed so a new FuzzXxx is picked up without +# editing this file. The grep narrows to candidate packages first because +# `go test -list` has to build a test binary per package and most packages here +# have no fuzz targets. testdata and vendor are skipped since they hold corpus +# files and third-party code, never targets. A failing `go test -list` (e.g. a +# package that does not compile) aborts the run with its errors visible rather +# than being treated as "no targets", which would let `make fuzz` pass having +# fuzzed nothing. +# +# A crasher gets written to testdata/fuzz// and is replayed as a +# seed by every later `go test`, so it turns the suite red until the underlying +# bug is fixed. Keep a crasher in the same commit as its fix. fuzz: - cd sdk && go test ./... -fuzztime=2m + @for m in $(HAND_MODS); do \ + (cd $$m && \ + dirs=$$(grep -rl --include='*_test.go' --exclude-dir=testdata --exclude-dir=vendor '^func Fuzz' . \ + | while read -r f; do dirname "$$f"; done | sort -u); \ + for d in $$dirs; do \ + listing=$$(go test -list '^Fuzz' "$$d") || { echo "go test -list failed in $$m $$d" >&2; exit 1; }; \ + for fn in $$(echo "$$listing" | grep '^Fuzz'); do \ + echo "==> $$m $$d $$fn ($(FUZZTIME))"; \ + go test -run '^$$' -fuzz "^$$fn$$" -fuzztime=$(FUZZTIME) "$$d" || exit 1; \ + done; \ + done) || exit 1; \ + done bench: for m in $(HAND_MODS); do (cd $$m && go test -bench ./... -benchmem) || exit 1; done