a reconnect voids every chunk in flight (ConnectionReplaced) - found by the probe checker; probe-v1 env

[tactino]

The stricter checker from plugrl-protocol#8 graded this client for the first time and failed it in the resync scenario. The bug is real.

What went wrong. The agent reconnected inside its next call, without telling the rollout. Chunks from the old connection were finished and fed back on the new one, in two ways:

• Envs mid-chunk. An env still executing a chunk from the old connection kept executing it, then sent its feedback on the new connection. In the probe run, envs 0 and 2 did this right after the reconnect.
• A failed feedback send. It was dropped, but the next feedback call reconnected, so the first message on the new connection could be a stale feedback.

SPEC.md section 7.6 forbids both. The server has no step state for those envs on the new connection. Since plugrl-server#104 it warns and discards them; before #104 it stored transitions built from empty observations.

The fix.

• ConnectionReplaced. WebSocketEnvClientAgent numbers its connections and remembers which one answered the caller's last infer. When that connection is gone, it raises ConnectionReplaced:
  • from the first infer after a reconnect, before sending anything;
  • from a feedback whose send fails;
  • from a feedback called after the connection closed.
• feedback never reconnects. A feedback is only sent on the connection that answered its infer.
• The rollout drops every chunk in flight on ConnectionReplaced, so its next infer re-plans every env on the new connection. Episodes that ended are still reset when their feedback is dropped, which is the bug the checker found in the Python reference client (protocol#8, last commit).
• The SPEC 7.6 Gap is fixed. With --reconnect-on-server-stop, a stop during a feedback send used to resend that feedback. It is now dropped like any other.

probe-v1. A new env, envs/probe_env.py, implements the SPEC 8.1 probe environment so this client can be graded the same way as the reference clients. It needs no extra.

Checked.

• Before the fix, plugrl-conformance --probe --scenario all against probe-v1, with 3 envs and horizon 4, failed the resync scenario: feedback for envs 0 and 2 had no action on the new connection.
• After it, there are no violations in all three scenarios, and none in three further resync runs.
• Tests:
  • test_reconnect_drops_feedback.py now expects ConnectionReplaced, and adds the stop-with-reconnect case, feedback never reconnecting, and the first infer after a reconnect sending nothing.
  • test_rollout_connection_replaced.py runs the rollout on probe-v1 with an agent that replaces the connection mid-run. No stale chunk is fed back, the first infer on the new connection names every env, and an ended episode is reset when its feedback is dropped.
  • The full suite passes: 136 passed.

Follow-ups.

• Once protocol#8 is merged and this repository pins a plugrl-protocol that has it, CI can run plugrl-conformance --probe --scenario all against probe-v1 on every change.
• The SPEC 7.6 Gap text in plugrl-protocol can then be removed.