Skip to content

[Bug]: Python data channel masks saved reader exception with Channel closed prematurely #40774

Open

Description

@bgeng777

What happened?

In Beam 2.75.0 (Python 3.12.7), a data-plane read failure is saved in self._exception, but input_elements checks self._closed first. The final bundle traceback therefore shows Channel closed prematurely and queue.Empty without the saved cause.

Reproducer

Save as repro.py and run python repro.py with apache-beam==2.75.0:

import unittest

from apache_beam.runners.worker import data_plane


class ChannelErrorTest(unittest.TestCase):
    def test_saved_reader_failure_is_preserved(self):
        channel = data_plane._GrpcDataChannel()
        error = ValueError("original read failure")
        self.addCleanup(self.assertRaises, ValueError, channel.close)

        def failed_inputs():
            raise error
            yield

        with self.assertLogs(data_plane._LOGGER, level="ERROR"):
            with self.assertRaises(ValueError):
                channel._read_inputs(failed_inputs())
        with self.assertRaisesRegex(
                RuntimeError, "Channel closed prematurely") as caught:
            list(channel.input_elements("instruction", ["input"]))
        self.assertIs(caught.exception.__cause__, error)


if __name__ == "__main__":
    unittest.main()

The test fails with None is not ValueError('original read failure'). The saved exception should be preserved as the cause.

Suggested fix

if self._closed:
    raise RuntimeError(
        'Channel closed prematurely.') from self._exception

This retains the original exception and suppresses the unrelated queue timeout when no cause was recorded. See #40775 for the fix and regression tests.

Issue Priority

Priority: 3 (minor)

Issue Components

Component: Python SDK

AI assistance: OpenAI Codex.

Activity

  1. schizophrenicmaniac commented on Oct 25, 2026

    @schizophrenicmaniac
    Contributor

    .take-issue

    Hi, I'd like to work on this. In _GrpcDataChannel.input_elements the self._closed check runs before self._exception is looked at, so I'd raise the "Channel closed prematurely" error from the saved exception and add a regression test in data_plane_test.py.

  2. bgeng777 commented on Oct 25, 2026

    @bgeng777
    Author

    2. .take-issue
    Hi, I'd like to work on this. In _GrpcDataChannel.input_elements the self._closed check runs before self._exception is looked at, so I'd raise the "Channel closed prematurely" error from the saved exception and add a regression test in data_plane_test.py.

    Hi @schizophrenicmaniac , thanks for the quick reply but sorry for the overlap. I had already prepared the fix before seeing your comment and have now opened PR #40775. I’d really appreciate your review and feedback. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions