[castler]

Problem

Flaky nightly failures in eclipse-score/communication (#1248, #1084, #986):

• execute_async() fails with ValueError: invalid literal for int() ...: '2026/10/26 ... lola log info ...' – the first stdout chunk is application output instead of the PID line.
• target.execute("echo Hello World!") returns exit code 0 with empty output.

Root cause

docker-py 7.1.0 reads exec output from the raw socket (response.raw._fp.fp.raw), bypassing the BufferedReader that http.client used to parse the HTTP response headers. That reader reads up to 8 KiB at once, so any output the daemon sent in the same burst as the headers (i.e. the very first output of the command) is already in its buffer and is silently dropped. This happens whenever the client reads the reply late, e.g. on a loaded CI host; the output that is lost is exactly the first bytes (echo $$ PID line, or the complete output of a short command).

Reproduction

Delaying http.client.HTTPConnection.getresponse by 0.1 s makes docker-py lose the PID line 50/50 times and the first stdout chunk becomes the next output line (the #1248/#1084 symptom); with 0.5 s the whole output of a short command is lost (the #986 symptom).

Fix

A small APIClient subclass that parses the exec frames from the buffered reader (read/read1), so nothing is dropped. As a side effect frames are never split mid-PID. The ITF Docker clients use it.

Tests

test_exec_output_not_lost.py emulates the slow reader. It fails on the old code ('' == 'Hello World!\n', Failed to extract PID) and passes with the fix; test_docker, test_ssh, test_dlt, test_dlt_on_target pass.

[limdor]

I'm not familiar with pydocker APIClient and I do not know how _read_from_socket should look like. However, I took a look to the PR and this is my understanding and proposal.

I first reviewed the newly added test. To me the tests make total sense and I also checked that without the additional changes, the tests are broken (10 out of 10). With the additional changes, the tests are passing (10 out of 10).

I did a bit of research and it seems to be a known issue with pydocker (see docker/docker-py#3332 and docker/docker-py#2042) and there are event 2 open PRs to address the issue (see docker/docker-py#3333 and docker/docker-py#3438).

How I see it, the newly added code in this PR, is basically reimplementing the _read_from_socket on the docker api client but disabling the timeout and not missing content. I cannot say if this is fully correct but considering that the tests are passing I wouls propose to merge it.

The only change that I would suggest before merging, is to add a comment in self._client = _ExecDockerClient.from_env(timeout=DOCKER_CLIENT_TIMEOUT) mentioning the existing issues in docker-py and the opened PRs, and mention that this is a workaround until the issues are fixed and it should be removed once the upstream issues/PRs are merged.

[castler]

Thanks for the review. Added a comment on _ExecDockerClient (used by both from_env call sites) linking docker-py#3332, #2042 and the open PRs #3333/#3438, and noting the workaround should be removed once they are resolved.