Skip to content

gh-148508: Add another common pattern for iOS SSL failures - #150442

Merged
vstinner merged 4 commits into
python:mainfrom
freakboy3742:ios-ssl-tests
Jun 1, 2026
Merged

vstinner merged 4 commits into
python:mainfrom
freakboy3742:ios-ssl-tests

Conversation

@freakboy3742

@freakboy3742 freakboy3742 commented May 26, 2026 •

Copy link
Copy Markdown
Contributor

The test_ssl.TestPreHandshakeClose.test_preauth_data_to_tls_server test fails intermittently on the iOS buildbot because an SSL error doesn't raise the expected TLS handshake error.

This test already has an escape clause that allows a skip under certain error conditions (raising a ConnectionError, an OSError, or an SSL error with "wrong version number"). #148536 added a pattern for one type of iOS failure; this PR adds a second failure mode that is observed (simplifying the test into a single regex).

It also tweaks the iOS logging regex to strip an additional space; this should aid Bevedere in reporting stack traces when failures occur.

@freakboy3742

Copy link
Copy Markdown
Contributor Author

!buildbot iOS

@bedevere-bot

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @freakboy3742 for commit 5e9b009 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F150442%2Fmerge

The command will test the builders whose names match following regular expression: iOS

The builders matched are:

  • iOS ARM64 Simulator PR

@freakboy3742
freakboy3742 requested a review from vstinner May 26, 2026 02:49
Comment thread Lib/test/test_ssl.py
Comment thread Lib/test/test_ssl.py Outdated
re.search(
r'wrong.version.number|record.layer.failure|http.request',
str(getattr(err, "reason", "")),
re.I

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick:

Suggested change
re.I
re.IGNORECASE

@freakboy3742
freakboy3742 requested a review from vstinner May 31, 2026 03:58

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@vstinner
vstinner merged commit 540b3d0 into python:main Jun 1, 2026
53 checks passed
@miss-islington-app

Copy link
Copy Markdown

Thanks @freakboy3742 for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@bedevere-app

bedevere-app Bot commented Jun 1, 2026

Copy link
Copy Markdown

GH-150697 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Jun 1, 2026
@bedevere-app

bedevere-app Bot commented Jun 1, 2026

Copy link
Copy Markdown

GH-150698 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jun 1, 2026
@bedevere-app

bedevere-app Bot commented Jun 1, 2026

Copy link
Copy Markdown

GH-150699 is a backport of this pull request to the 3.13 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.13 only security fixes label Jun 1, 2026
@vstinner

vstinner commented Jun 1, 2026

Copy link
Copy Markdown
Member

Merged, thanks for the fix @freakboy3742.

vstinner pushed a commit that referenced this pull request Jun 1, 2026
…test_ssl (GH-150442) (#150697)

gh-148508: Add another common pattern for iOS SSL failures to test_ssl (GH-150442)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
vstinner pushed a commit that referenced this pull request Jun 1, 2026
…test_ssl (GH-150442) (#150698)

gh-148508: Add another common pattern for iOS SSL failures to test_ssl (GH-150442)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
@bedevere-bot

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.15 (tier-1) has failed when building commit a4f3336.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/2019/builds/120) and take a look at the build logs.
  4. Check if the failure is related to this commit (a4f3336) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/2019/builds/120

Failed tests:

  • test.test_multiprocessing_fork.test_processes

Failed subtests:

  • test_interrupt - test.test_multiprocessing_fork.test_processes.WithProcessesTestProcess.test_interrupt

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/contextlib.py", line 116, in inner
    return func(*args, **kwds)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 649, in test_interrupt
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/contextlib.py", line 116, in inner
    return func(*args, **kwds)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 630, in _kill_process
    self.assertEqual(join(), None)
                     ~~~~^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 303, in __call__
    return self.func(*args, **kwds)
           ~~~~~~~~~^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/multiprocessing/process.py", line 156, in join
    res = self._popen.wait(timeout)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/multiprocessing/popen_fork.py", line 44, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
               ~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 626, in handler
    raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-158' pid=1657692 parent=1657134 started daemon>


Traceback (most recent call last):
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/contextlib.py", line 116, in inner
    return func(*args, **kwds)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 649, in test_interrupt
    exitcode = self._kill_process(multiprocessing.Process.interrupt)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/contextlib.py", line 116, in inner
    return func(*args, **kwds)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 630, in _kill_process
    self.assertEqual(join(), None)
                     ~~~~^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 303, in __call__
    return self.func(*args, **kwds)
           ~~~~~~~~~^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/multiprocessing/process.py", line 156, in join
    res = self._popen.wait(timeout)
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/multiprocessing/popen_fork.py", line 44, in wait
    return self.poll(os.WNOHANG if timeout == 0.0 else 0)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/multiprocessing/popen_fork.py", line 28, in poll
    pid, sts = os.waitpid(self.pid, flag)
               ~~~~~~~~~~^^^^^^^^^^^^^^^^
  File "/root/buildarea/3.15.angelico-debian-amd64/build/Lib/test/_test_multiprocessing.py", line 626, in handler
    raise RuntimeError('join took too long: %s' % p)
RuntimeError: join took too long: <Process name='Process-1' pid=1680348 parent=1680346 started daemon>

freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Jun 12, 2026
…es to test_ssl (pythonGH-150442) (python#150699)

pythongh-148508: Add another common pattern for iOS SSL failures to test_ssl (pythonGH-150442)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Jun 12, 2026
…es to test_ssl (pythonGH-150442) (python#150699)

pythongh-148508: Add another common pattern for iOS SSL failures to test_ssl (pythonGH-150442)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
@hugovk

hugovk commented Sep 26, 2026

Copy link
Copy Markdown
Member

The ARM64 macOS 3.11 builder has failed three times in a row:

======================================================================
FAIL: test_preauth_data_to_tls_server (test.test_ssl.TestPreHandshakeClose.test_preauth_data_to_tls_server)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/buildbot/buildarea/3.11.pablogsal-macos-m1.macos-with-brew/build/Lib/test/test_ssl.py", line 5099, in test_preauth_data_to_tls_server
    self.assertIn("before TLS handshake with data", wrap_error.args[1])
AssertionError: 'before TLS handshake with data' not found in '[SSL: HTTP_REQUEST] http request (_ssl.c:1016)'

----------------------------------------------------------------------

https://buildbot.python.org/#/builders/1030

This follows upgrades for macOS 26.2->26.6.2 and OpenSSL 3.6.1->3.6.3 on the worker, and the 3.12 builder also failed for its first build after the upgrade:
https://buildbot.python.org/#/builders/1206/builds/1627

3.10 is likely affected, but hasn't run since.

Not every run fails, but these security branches don't run very often.

We've already made the test more robust, and backported it to 3.13. Let's now backport the Lib/test/test_ssl.py change for 3.12-3.10.

@hugovk hugovk added needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes needs backport to 3.12 only security fixes labels Sep 26, 2026
@miss-islington-app

Copy link
Copy Markdown

Thanks @freakboy3742 for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Thanks @freakboy3742 for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Thanks @freakboy3742 for the PR, and @vstinner for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

@miss-islington-app

Copy link
Copy Markdown

Sorry, @freakboy3742 and @vstinner, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 540b3d0a7fa7cd842f064f79b1410cbd6868bffa 3.10

@miss-islington-app

Copy link
Copy Markdown

Sorry, @freakboy3742 and @vstinner, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 540b3d0a7fa7cd842f064f79b1410cbd6868bffa 3.11

@miss-islington-app

Copy link
Copy Markdown

Sorry, @freakboy3742 and @vstinner, I could not cleanly backport this to 3.12 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 540b3d0a7fa7cd842f064f79b1410cbd6868bffa 3.12

@bedevere-app

bedevere-app Bot commented Sep 26, 2026

Copy link
Copy Markdown

GH-158246 is a backport of this pull request to the 3.12 branch.

@bedevere-app

bedevere-app Bot commented Sep 26, 2026

Copy link
Copy Markdown

GH-158247 is a backport of this pull request to the 3.11 branch.

@bedevere-app

bedevere-app Bot commented Sep 26, 2026

Copy link
Copy Markdown

GH-158248 is a backport of this pull request to the 3.10 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.10 only security fixes label Sep 26, 2026
Yhg1s pushed a commit that referenced this pull request Sep 30, 2026
…test_ssl (GH-150442) (#158246)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
pablogsal pushed a commit that referenced this pull request Oct 1, 2026
…test_ssl (GH-150442) (#158248)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
pablogsal pushed a commit that referenced this pull request Oct 1, 2026
…test_ssl (GH-150442) (#158247)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Oct 2, 2026
…es to test_ssl (pythonGH-150442) (python#150699)

pythongh-148508: Add another common pattern for iOS SSL failures to test_ssl (pythonGH-150442)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
freakboy3742 added a commit to freakboy3742/cpython that referenced this pull request Oct 2, 2026
…es to test_ssl (pythonGH-150442) (python#150699)

pythongh-148508: Add another common pattern for iOS SSL failures to test_ssl (pythonGH-150442)

Match also '[SSL: HTTP_REQUEST] http request (_ssl.c:1143)'.
(cherry picked from commit 540b3d0)

Co-authored-by: Russell Keith-Magee <russell@keith-magee.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants