conformance: grade a training server (SPEC 8.2), with a reference server written against SPEC - #10
Merged
Merged
Conversation
…the checker sees what a client does with each action and how it handles closes The passive checker watched one connection and could not tell chunk-summed reward, terminal observations, time-major order, metadata-first, the close reasons or text frames from their violations. With --probe the client runs a probe env whose observation counts steps and echoes the applied action, and the checker drives the connection: it speaks late, sends a 2 MiB metadata frame, closes for a resync, stops the run, and answers with a text frame. raw_client.py implements the probe env, and its --bug option breaks each clause in turn; tests/test_conformance_probe.py checks that each is named.
…fter Found by the probe checker itself, on Linux, in about one run in two of the resync scenario: when the server's resync close arrived while the feedback was being sent, the send raised, the reset after it never ran, and env 0 carried its finished episode onto the next connection (t = 7, 11, 15 for an episode of 3). The packed feedback already holds the terminal observation, so resetting first changes nothing the server sees.
…ed in CI in every scenario `plugrl_client --probe host port batch` runs the probe environment, executes each chunk time-major and flushes it at the terminal step, and handles the section 7 close reasons: it reconnects after plugrl-server-resync with nothing resent, exits 0 on plugrl-server-stop, and exits 2 on a text frame. A close now raises ServerClosed with its code and reason, and a text frame TextFrame, instead of one generic error. Passive mode is unchanged.
…s a 7.6 violation; raw_client --bug stale-chunk plugrl-env-client failed the resync scenario this way: after reconnecting it fed back chunks answered on the old connection, which the checker reported under 4.3. On a reconnected connection, feedback for an env that was never given an action there is now named for the clause it breaks. The new stale-chunk bug does exactly that, and the test expects the 7.6 clause.
… 8.2), and examples/reference_server.py is one written against SPEC alone A client that drives a server the way SPEC lets a client behave - ragged batches, a feedback set unlike the infer's, large frames, three kinds of malformed message, two connections sharing env 0, and the end of the run - and checks the action layout and env_ids, the resync closes, that the server survives a bad client, and the stop reason. The reference server trains nothing and passes; its --bug option breaks each server clause in turn. plugrl-server main passes; its version before #108 fails two checks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The checker graded clients only. This grades a training server, so both ends of the protocol can be checked: plugrl-server, the RLinf bridge to come, or anyone else's. Stacked on #9 (and #8): merge those first.
plugrl-conformance-server(new SPEC 8.2) is a client that drives a server the way SPEC lets a client behave, and checks what comes back. It runs in phases:[H, n, *da], withenv_idsequal toenv_indicesin order, and agrees with the declaredaction_horizonandaction_dim;infothat cannot be split per env. Each must close that connection with 1001 andplugrl-server-resync, and a new connection must still get metadata afterwards.--until-stop): the run must end with 1001 andplugrl-server-stop.It sends a states-only observation,
--state-dimvalues under--state-key(defaultobs), and can add an image with--image-key. The phases never leave a connection idle while another waits, because plugrl-server waits for every connected client before it infers.examples/reference_server.pyis a server written against SPEC that trains nothing. It answers with random chunks, validates every message, closes for a resync on anything malformed, survives a bad client, and stops after--stepsframes. It passes every phase. Its--bugoption breaks one server clause at a time:no-metadata,env-major,no-env-ids,wrong-horizon,lenient,crash-on-infoandplain-stop.tests/test_server_conformance.pychecks that the grader names each one.Against plugrl-server (dummy policy, CPU, on this laptop):
infoit could not split closed the connection with 1011;Local run: 50 passed.
Next:
WebSocketAgentServer, once it pins a plugrl-protocol that has it.