upgrade requests-ntlm to 1.3.0 to fix MD4 failure under OpenSSL 3.x - #46313
upgrade requests-ntlm to 1.3.0 to fix MD4 failure under OpenSSL 3.x#46313shmuel44 wants to merge 12 commits into
Conversation
requests-ntlm <1.3.0 pulls in ntlm-auth, which calls hashlib.new('md4')
and fails under OpenSSL 3.x. requests-ntlm 1.3.0 uses pyspnego instead,
and pyspnego >=0.5.2 ships its own pure-Python MD4.
Drops the direct ntlm-auth dependency and relocks. pyspnego retains its
["kerberos"] extra (required by requests-kerberos).
XSUP-75734
requests-ntlm <1.3.0 pulls in ntlm-auth, which calls hashlib.new('md4')
and fails under OpenSSL 3.x. requests-ntlm 1.3.0 uses pyspnego instead,
and pyspnego >=0.5.2 ships its own pure-Python MD4.
Relocking drops ntlm-auth and adds pyspnego. cryptography, cffi, and
pycparser were already present via requests-ntlm 1.1.0 and are unchanged
apart from a cryptography patch bump (50.0.0 -> 50.0.1).
XSUP-75734
requests-ntlm <1.3.0 pulls in ntlm-auth, which calls hashlib.new('md4')
and fails under OpenSSL 3.x. requests-ntlm 1.3.0 uses pyspnego instead,
and pyspnego >=0.5.2 ships its own pure-Python MD4.
Drops the direct ntlm-auth dependency. verify.py no longer asserts that
OpenSSL exposes md4; it now checks NTLM hashing end to end via
spnego's ntowfv1, whose "pass" test vector was verified against RFC 1320
MD4 and MS-NLMP.
XSUP-75734
The 1.1.0 pin is lifted across ntlm, py-ews, and py3ews, so dependabot can manage requests-ntlm again. XSUP-75734
The openssl.cnf legacy_sect activation existed only so hashlib could expose md4 for ntlm-auth, which requests-ntlm <1.3.0 depended on. requests-ntlm 1.3.0 uses pyspnego, which ships its own pure-Python MD4, so the image no longer needs OpenSSL's legacy provider. XSUP-75734
The ntlm image had no verify.py, so CI produced no signal that the requests-ntlm 1.3.0 / pyspnego upgrade actually works there. Assert NTOWFv1 directly, mirroring the py3ews check. XSUP-75734
This comment has been minimized.
This comment has been minimized.
Docker Image Ready - DevDocker automatic build has deployed your docker image: devdemisto/py-ews:5.6.0.12505882 Docker Metadata
|
Docker Image Ready - DevDocker automatic build has deployed your docker image: devdemisto/py3ews:5.6.0.12505882 Docker Metadata
|
This comment has been minimized.
This comment has been minimized.
| [packages] | ||
| requests-ntlm = "==1.1.0" # lock because of https://github.com/requests/requests-ntlm/issues/136 | ||
| pyspnego = ">=0.5.2" # <0.5.2 calls hashlib.new('md4'), fails under OpenSSL 3.x (XSUP-75734) | ||
| requests-ntlm = "==1.3.0" |
There was a problem hiding this comment.
| requests-ntlm = "==1.3.0" | |
| requests-ntlm = ">=1.3.0" |
| @@ -0,0 +1,7 @@ | |||
| import requests_ntlm | |||
There was a problem hiding this comment.
Should we do something with requests_ntlm?
| from spnego._ntlm_raw.crypto import ntowfv1 | ||
|
|
||
| # Make sure NTLM hashing works without OpenSSL's md4 (XSUP-75734): | ||
| assert ntowfv1("pass").hex() == "36aa83bdcab3c9fdaf321ca42a31c3fc" |
There was a problem hiding this comment.
Is there any other usage of the integration code that can be added here to enhance the verify script?
Same comment for the py3ews verify
| requests = ">=2.32.3" | ||
| requests-kerberos = "*" | ||
| requests-ntlm = "==1.1.0" # lock because of https://github.com/requests/requests-ntlm/issues/136 | ||
| requests-ntlm = "==1.3.0" |
There was a problem hiding this comment.
| requests-ntlm = "==1.3.0" | |
| requests-ntlm = ">=1.3.0" |
| && apk --update add --no-cache --virtual .build-dependencies python3-dev gcc build-base wget git libffi-dev openssl-dev python3-dev libxml2-dev \ | ||
| && pip install --no-cache-dir -r requirements.txt \ | ||
| && apk del .build-dependencies | ||
| # Handling the issue described here: https://github.com/ecederstrand/exchangelib/issues/608 |
There was a problem hiding this comment.
Does this show elsewhere in the repo?
| requests-ntlm = "==1.1.0" # lock because of https://github.com/requests/requests-ntlm/issues/136 | ||
| ntlm-auth = ">=1.5.0" | ||
| pyspnego = ">=0.5.2" # <0.5.2 calls hashlib.new('md4'), fails under OpenSSL 3.x (XSUP-75734) | ||
| requests-ntlm = "==1.3.0" |
There was a problem hiding this comment.
| requests-ntlm = "==1.3.0" | |
| requests-ntlm = ">=1.3.0" |
This reverts commit 97bd884. Narrowing XSUP-75734 to the ntlm image only. py3ews keeps requests-ntlm 1.1.0 and therefore still needs OpenSSL's legacy provider for md4. XSUP-75734
This reverts commit dc2e5ab. Narrowing XSUP-75734 to the ntlm image only. py3ews stays on requests-ntlm 1.1.0 with ntlm-auth, and verify.py keeps asserting md4 availability. XSUP-75734
This reverts commit c5bef38. Narrowing XSUP-75734 to the ntlm image only. py-ews stays on requests-ntlm 1.1.0 with ntlm-auth. XSUP-75734
Narrowing XSUP-75734 to the ntlm image only. py-ews and py3ews stay pinned at requests-ntlm 1.1.0, so their ignore rules are restored. The ntlm image is upgraded to 1.3.0, so its rule stays removed. XSUP-75734
The exact ==1.3.0 pin was only needed to force the jump off requests-ntlm
1.1.0, which pulled in ntlm-auth and its hashlib.new('md4') call that fails
under OpenSSL 3.x. Now that 1.3.0+ routes NTLM hashing through pyspnego's
pure-Python MD4, a floor is sufficient and lets dependabot manage future
updates.
Relock is a no-op for the resolved graph: 1.3.0 is still the newest release,
so the only Pipfile.lock change is the _meta hash. ntlm-auth remains absent.
Relates to XSUP-75734
Beyond the NTOWFv1 known-answer check, assert the API the consuming integrations actually use: 2-arg HttpNtlmAuth with a DOMAIN\user name, the no-op session argument, and that applying the auth handler forces Connection: Keep-Alive and registers the 401 response hook. All checks are offline, so they run under the CI image with no network. XSUP-75734
This comment has been minimized.
This comment has been minimized.
Docker Image Ready - DevDocker automatic build has deployed your docker image: devdemisto/ntlm:1.0.0.12535775 Docker Metadata
|
🔍 AI Triage Report AvailableAn automated triage report has been generated for this pipeline. Status:
|
Related Issues
fixes: XSUP-75734
Problem
docker/ntlm,docker/py-ewsanddocker/py3ewspinnedrequests-ntlm==1.1.0,which pulls the deprecated
ntlm-auth. It callshashlib.new('md4')whilebuilding the NTLM Type-3 message, and the base image ships OpenSSL 3.x where MD4
sits in the unloaded legacy provider:
Varonis DSP is fully non-functional (NTLM runs on every command, including
test-module); CyberArk CCP pins the same tag. Reproduce:The pin cited requests-ntlm #136, a 1.2.0-specific regression fixed in 1.3.0.
Stale for ~2 years.
Fix
Upgrade all three to
requests-ntlm==1.3.0, which uses pyspnego's pure-Python MD4.Drops the direct
ntlm-authdep from both EWS images and removes the three voiddependabot ignore rules that let this rot.
The
pyspnego>=0.5.2floor is load-bearing - 1.3.0 declares only>=0.4.0, but0.5.1 still calls
hashlib.new("md4"). Without it a resolver could reintroduce the bug.py3ews/verify.pyasserted on hashlib MD4 and importedntlm_auth, which would nowfail the build. Replaced with
assert ntowfv1("pass").hex() == "36aa83bd...".