conformance: probe environment (SPEC 8.1) and driven scenarios - the checker sees what a client does with each action

[tactino]

SPEC.md section 8 lists what a client must do. The checker watched one well-behaved connection, so it could not tell several of those clauses from their violations:

• chunk-summed reward;
• the terminal observation on a done step;
• time-major action order;
• reading metadata before sending anything;
• the connection options;
• the two close reasons;
• dropping held feedback across a reconnect;
• text frames.

A client breaking all of these still printed "no violations", and SPEC.md said so.

The probe environment (new SPEC 8.1). A client under test runs n probe envs instead of its real ones. Env i counts the steps of its episode in states["t"], echoes the action it applied last in states["a"], pays 1 per step, and terminates at t = 3 + 2(i mod 3). The checker sends actions whose values encode their position (10000 k + 10 j + d). From each feedback it then works out:

• how many steps the client ran: they must be between 1 and H, and the reward must equal them;
• which action it applied last: this checks time-major, in-order application, and the dtype;
• whether a done env reported its own terminal observation, and restarted at t = 0.

Driven scenarios (--scenario basic|resync|text|all). The checker drives the connection instead of watching it:

• basic:
  • speaks 0.3 s late, to catch a client that sends first;
  • sends a 2 MiB metadata frame with an unknown key, which catches a kept 1 MiB frame cap or a client that rejects unknown keys;
  • flags a handshake that offers permessage-deflate;
  • stops the run with plugrl-server-stop, after which the client must exit 0 and not reconnect.
• resync: halfway, closes with plugrl-server-resync just after sending an action. The client must reconnect, and start the new connection with an infer, not the feedback it was holding.
• text: answers an infer with a text frame. The client must stop.
• all runs each in turn, starting the client once per scenario.

Passive mode is unchanged and remains the default, so the C++ client's CI steps are untouched.

Proving each check works. examples/raw_client.py --probe implements the probe env, executes chunks as SPEC 5.3 says, and handles the close reasons. It passes all three scenarios with the existing text note. Its --bug option breaks one clause at a time:

• early-infer, compress, frame-cap, float32, env-major;
• last-step-reward, reset-in-step, resend-feedback, stale-chunk, ignore-stop, ignore-text.

tests/test_conformance_probe.py runs the checker against each bug and expects it to name that clause. All eleven are caught, and the conforming client passes.

What probe mode still cannot check is listed in SPEC 8:

• reading env_ids, which must equal env_indices anyway;
• a feedback env set that differs from the infer set, which the client chooses;
• requiring no metadata key;
• dropping feedback after closes other than a resync.

CI.

• The test job installs the conformance extra, so the new tests run rather than skip.
• The cross-language job adds a probe run of the Python reference client over all scenarios.

Local run: 42 passed (30 existing, 12 new).

It has already found two real bugs.

• In this PR's own reference client. An ended episode was reset only after its feedback was sent, so a resync close that arrived during the send carried the episode onto the next connection. It failed about one run in two on Linux, and is fixed here.
• In plugrl-env-client (a reconnect voids every chunk in flight (ConnectionReplaced) - found by the probe checker; probe-v1 env plugrl-env-client#15). After reconnecting, it fed back chunks answered on the old connection, which SPEC 7.6 forbids. The checker now names that 7.6 rather than 4.3, and the stale-chunk bug reproduces it.

Next: the C++ reference client gets the probe env, and plugrl-env-client gets a probe-v1 env so the main client can be graded the same way.

[tactino]

Added one commit. On Linux the resync scenario failed the conforming client about one run in two, and the checker was right to fail it. When the resync close arrived while the client was sending its feedback, the send raised, and the reset after it never ran. Env 0 then carried a finished episode onto the next connection, reaching t = 7, 11 and 15 in an episode of 3. The client now resets ended episodes before sending; the packed feedback already holds the terminal observation. After the fix, the probe tests passed five runs in five on Linux.