Skip to content

polyworld: shared neural policy tier - #78

Open
daveey wants to merge 1 commit into
polyworld-nnfrom
polyworld-neural-tier
Open

daveey wants to merge 1 commit into
polyworld-nnfrom
polyworld-neural-tier

Conversation

@daveey

@daveey daveey commented Sep 25, 2026 •

Copy link
Copy Markdown

Stacked on #72. The base is polyworld-nn (#72's head), so this diff is only the tier: one commit. #76 (GotA neural policies) is stacked on this PR and is its first client.

This PR adds a game-agnostic tier for neural policies. A player can submit a small network next to a BASIC program, and it runs natively inside a Polyworld game under a fixed operation budget. There is no game code in the tier: a game plugs in with one NeuralContract. The how-to is docs/neural-policies.md.

What's in it

  • src/polyworld/neural_package.nim: the package loader. A package is a ZIP with exactly manifest.json, policy.bas and model.bin.
    • The manifest pins the SHA-256 of the other two files.
    • Every entry's declared uncompressed size is checked against a cap before anything is inflated (manifest 64 KiB, policy 256 KiB, model at most the largest valid model). Inflating stops at the declared size (boundedinflate.nim, zippy's inflate with an output limit), so a zip bomb never expands in memory. A package over 16 MiB is rejected as "package exceeds 16 MiB".
    • Keys are strict: an unknown key at any level is rejected. A game adds its own manifest and decoder keys and an options parser.
    • JSON kinds are checked before any value is read or iterated: 3.0 and true are not integers, temperature: true is not a number, and a non-finite number (NaN, Infinity, 1e999) is rejected. A malformed manifest can't raise a defect and take the server down; every failure is a ValueError with the reason.
    • parsePackage(bytes, contract) returns a read-only NeuralPackage (getters). The raw model bytes are not kept, only the loaded actor.
    • The decoder is a variant object: argmax, or sample with its temperature.
  • src/polyworld/neural_actor.nim: GOTANET1, the Polyworld neural model format (GotA introduced it; the byte layout and magic are unchanged). The actor is an FP32 MinGRU with its dimensions in the model header. Actor is read-only.
  • src/polyworld/neural_host.nim: the host side.
    • NeuralContract has one constructor, initNeuralContract. It computes the observation and action hashes from the contract texts and asserts that the game's logit count equals the sum of the head sizes.
    • A contract holds the schema, observation size, head sizes, both hashes, the mask size, and the callbacks buildObservation, decodeAction, head, tick, plus the optional log, actionMask and telemetryExtra.
    • loadActor(bytes, contract) checks the op budget, both contract hashes, the input size and the head sizes.
    • NeuralBrain is the per-seat state: the decision frame and the recurrent-state lifecycle (zero on the first frame and on the first frame after a death; resetBrain gives a fresh seat).
    • Also here: the per-seat op budget, the telemetry line, the argmax / sampling / masked decoders, and the BASIC functions run_neural_net, neuralObservation, neuralLogits, neuralState, neuralModel.
  • coworld.nim:
    • A staged player file that starts with PK\x03\x04 is read up to 16 MiB. A bigger file is read one byte past the cap, so it is reported as too large rather than as a broken ZIP.
    • failPlayer(slot, detail, message) ends an episode before play with a real reason. compilePlayer now uses it; its seat log line, status and failure.json are the same as before.
  • coworld/runtime/neural_package.py: the Python staging validator, parameterized by a Contract. Raw zipfile / zlib / JSON errors come out as PackageError.
  • Tests (both in build.yml):
    • tests/test_neural_tier.nim (in tests.nim) runs the whole tier on a toy contract (3 inputs, heads [2, 3]): load, strict keys and JSON kinds, in-memory ZIP and model corruption, budget rejection, the recurrent-state lifecycle and reset, the telemetry line format, decoders and the BASIC functions.
    • tests/neural_cases.py + tests/test_neural_cases.nim: one corruption suite that the Python and Nim validators must agree on.

Evidence

At this PR's head.

check result
tests/test_neural_tier.nim (toy contract) pass
Shared corruption suite, Python vs Nim 50/50 same verdict: 3 accepted, 47 rejected. Covers encrypted entry, method 12, duplicate entry, zip bomb (8 MiB declared as 1 KiB), declared size over the cap, lying sizes, corrupt deflate, truncated ZIP, not a ZIP, over 16 MiB, bad model magic, NaN weight, truncated model, file-hash, contract-hash, input and head mismatches, over budget, unknown keys at every level, 3.0 / 2.0 / true as integers, temperature: true, NaN and 1e999 in the manifest, and a 30,000-deep nested array
The build.yml test list all pass (Linux, Nim 2.2.10)
GitHub Actions build.yml on this branch (ubuntu, macOS, Windows; dispatched, since the workflow only runs on PRs into main) pass: run 36196222941
GotA rebuilt on this tier (#76) vs its previous implementation, 20 seeds x 6 lineups, full 28,800 ticks byte-identical wherever the behaviour was not changed on purpose; see #76

Notes

  • Nothing calls the tier yet apart from the tests. GotA neural policies (stacked on #78) #76 is the first user.
  • failPlayer is new public API in coworld.nim. The only visible change in compilePlayer is the text of the CoworldError raised after collection, which now carries the failure reason instead of "Player compilation failed".

🤖 Generated with Claude Code

Game-agnostic neural seats for any Polyworld game: a strict package
loader, the GOTANET1 model format with its FP32 MinGRU actor, and a
host layer that a game drives through one NeuralContract.

- neural_package.nim: ZIP of exactly manifest.json + policy.bas +
  model.bin; sha256 file pins; per-entry size caps checked before
  decompressing and a bounded inflate (boundedinflate.nim, adapted
  from zippy); strict keys and JSON kinds; game-supplied extra
  manifest/decoder keys and an options parser. Read-only results.
- neural_actor.nim: GOTANET1 loader and inference, dims from the
  header, read-only Actor; loadActor(bytes, contract) checks the
  budget, contract hashes, inputs and heads.
- neural_host.nim: NeuralContract (one constructor that hashes the
  contract texts and checks the logit count), NeuralBrain with the
  frame and recurrent-state lifecycle, the per-seat op budget, the
  telemetry line, argmax/sampling/masked decoders, and the BASIC
  functions run_neural_net / neuralObservation / neuralLogits /
  neuralState / neuralModel.
- coworld.nim: PK-prefixed staged players are read up to 16 MiB;
  failPlayer ends an episode with a real reason.
- coworld/runtime/neural_package.py: the Python staging validator,
  parameterized by contract.
- tests: a toy contract proves the tier is game-agnostic; one
  corruption suite runs through both validators in CI.
- docs/neural-policies.md: how to add neural seats to a game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant