Skip to content

upgrade requests-ntlm to 1.3.0 to fix MD4 failure under OpenSSL 3.x - #46313

Open
shmuel44 wants to merge 12 commits into
masterfrom
fix/xsup-75734-requests-ntlm-md4
Open

upgrade requests-ntlm to 1.3.0 to fix MD4 failure under OpenSSL 3.x#46313
shmuel44 wants to merge 12 commits into
masterfrom
fix/xsup-75734-requests-ntlm-md4

Conversation

@shmuel44

@shmuel44 shmuel44 commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Related Issues

fixes: XSUP-75734

Problem

docker/ntlm, docker/py-ews and docker/py3ews pinned requests-ntlm==1.1.0,
which pulls the deprecated ntlm-auth. It calls hashlib.new('md4') while
building the NTLM Type-3 message, and the base image ships OpenSSL 3.x where MD4
sits in the unloaded legacy provider:

ValueError: unsupported hash type md4

Varonis DSP is fully non-functional (NTLM runs on every command, including
test-module); CyberArk CCP pins the same tag. Reproduce:

docker run --rm demisto/ntlm:1.0.0.9067966 \
  python -c 'from ntlm_auth.compute_hash import _ntowfv1; _ntowfv1("pass")'

The pin cited requests-ntlm #136, a 1.2.0-specific regression fixed in 1.3.0.
Stale for ~2 years.

Fix

Upgrade all three to requests-ntlm==1.3.0, which uses pyspnego's pure-Python MD4.
Drops the direct ntlm-auth dep from both EWS images and removes the three void
dependabot ignore rules that let this rot.

The pyspnego>=0.5.2 floor is load-bearing - 1.3.0 declares only >=0.4.0, but
0.5.1 still calls hashlib.new("md4"). Without it a resolver could reintroduce the bug.

py3ews/verify.py asserted on hashlib MD4 and imported ntlm_auth, which would now
fail the build. Replaced with assert ntowfv1("pass").hex() == "36aa83bd...".

requests-ntlm <1.3.0 pulls in ntlm-auth, which calls hashlib.new('md4')
and fails under OpenSSL 3.x. requests-ntlm 1.3.0 uses pyspnego instead,
and pyspnego >=0.5.2 ships its own pure-Python MD4.

Drops the direct ntlm-auth dependency and relocks. pyspnego retains its
["kerberos"] extra (required by requests-kerberos).

XSUP-75734
requests-ntlm <1.3.0 pulls in ntlm-auth, which calls hashlib.new('md4')
and fails under OpenSSL 3.x. requests-ntlm 1.3.0 uses pyspnego instead,
and pyspnego >=0.5.2 ships its own pure-Python MD4.

Relocking drops ntlm-auth and adds pyspnego. cryptography, cffi, and
pycparser were already present via requests-ntlm 1.1.0 and are unchanged
apart from a cryptography patch bump (50.0.0 -> 50.0.1).

XSUP-75734
requests-ntlm <1.3.0 pulls in ntlm-auth, which calls hashlib.new('md4')
and fails under OpenSSL 3.x. requests-ntlm 1.3.0 uses pyspnego instead,
and pyspnego >=0.5.2 ships its own pure-Python MD4.

Drops the direct ntlm-auth dependency. verify.py no longer asserts that
OpenSSL exposes md4; it now checks NTLM hashing end to end via
spnego's ntowfv1, whose "pass" test vector was verified against RFC 1320
MD4 and MS-NLMP.

XSUP-75734
The 1.1.0 pin is lifted across ntlm, py-ews, and py3ews, so dependabot
can manage requests-ntlm again.

XSUP-75734
The openssl.cnf legacy_sect activation existed only so hashlib could
expose md4 for ntlm-auth, which requests-ntlm <1.3.0 depended on.
requests-ntlm 1.3.0 uses pyspnego, which ships its own pure-Python MD4,
so the image no longer needs OpenSSL's legacy provider.

XSUP-75734
The ntlm image had no verify.py, so CI produced no signal that the
requests-ntlm 1.3.0 / pyspnego upgrade actually works there. Assert
NTOWFv1 directly, mirroring the py3ews check.

XSUP-75734
Comment thread docker/ntlm/verify.py Fixed
@shmuel44 shmuel44 changed the title Fix/xsup 75734 requests ntlm md4 upgrade requests-ntlm to 1.3.0 to fix MD4 failure under OpenSSL 3.x Aug 30, 2026
@content-bot

This comment has been minimized.

@content-bot

Copy link
Copy Markdown
Contributor

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/py-ews:5.6.0.12505882
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py-ews/tags
Get started by pulling the image:

docker pull devdemisto/py-ews:5.6.0.12505882

Docker Metadata

  • Image Size: 144.12M
  • Image ID: sha256:2036a1e2975353979c46ec9de78c96053cf3ad6eddcd383027032f36793d1c34
  • Created: 2026-08-30T14:27:12.970641851Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • LANG=C.UTF-8
    • GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305
    • PYTHON_VERSION=3.12.13
    • PYTHON_SHA256=c08bc65a81971c1dd5783182826503369466c7e67374d1646519adf05207b684
    • DOCKER_IMAGE=devdemisto/py-ews:5.6.0.12505882
  • Labels:
    • io.buildah.version:1.37.5
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:90563bd8300092f60ef36ab13f3ffd4bcbd1576f
    • org.opencontainers.image.version:5.6.0.12505882
    • panw.builtby.pipeline:11694983
    • panw.builtby.project:xdr/cortex-content/dockerfiles
    • panw.builtby.template:build-scan-publish

@content-bot

Copy link
Copy Markdown
Contributor

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/py3ews:5.6.0.12505882
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/py3ews/tags
Get started by pulling the image:

docker pull devdemisto/py3ews:5.6.0.12505882

Docker Metadata

  • Image Size: 63.72M
  • Image ID: sha256:962494510212e509ef7eeaf42f06cd52a7e2a0c23321dd5da1e89cf1904126c1
  • Created: 2026-08-30T14:28:19.69967604Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • LANG=C.UTF-8
    • GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305
    • PYTHON_VERSION=3.12.13
    • PYTHON_SHA256=c08bc65a81971c1dd5783182826503369466c7e67374d1646519adf05207b684
    • DOCKER_IMAGE=devdemisto/py3ews:5.6.0.12505882
  • Labels:
    • io.buildah.version:1.37.5
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:90563bd8300092f60ef36ab13f3ffd4bcbd1576f
    • org.opencontainers.image.version:5.6.0.12505882
    • panw.builtby.pipeline:11658749
    • panw.builtby.project:xdr/cortex-content/dockerfiles
    • panw.builtby.template:build-scan-publish

@content-bot

This comment has been minimized.

Comment thread docker/ntlm/Pipfile Outdated
[packages]
requests-ntlm = "==1.1.0" # lock because of https://github.com/requests/requests-ntlm/issues/136
pyspnego = ">=0.5.2" # <0.5.2 calls hashlib.new('md4'), fails under OpenSSL 3.x (XSUP-75734)
requests-ntlm = "==1.3.0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
requests-ntlm = "==1.3.0"
requests-ntlm = ">=1.3.0"

Comment thread docker/ntlm/verify.py
@@ -0,0 +1,7 @@
import requests_ntlm

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we do something with requests_ntlm?

Comment thread docker/ntlm/verify.py
from spnego._ntlm_raw.crypto import ntowfv1

# Make sure NTLM hashing works without OpenSSL's md4 (XSUP-75734):
assert ntowfv1("pass").hex() == "36aa83bdcab3c9fdaf321ca42a31c3fc"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any other usage of the integration code that can be added here to enhance the verify script?
Same comment for the py3ews verify

Comment thread docker/py-ews/Pipfile Outdated
requests = ">=2.32.3"
requests-kerberos = "*"
requests-ntlm = "==1.1.0" # lock because of https://github.com/requests/requests-ntlm/issues/136
requests-ntlm = "==1.3.0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
requests-ntlm = "==1.3.0"
requests-ntlm = ">=1.3.0"

Comment thread docker/py3ews/Dockerfile
&& apk --update add --no-cache --virtual .build-dependencies python3-dev gcc build-base wget git libffi-dev openssl-dev python3-dev libxml2-dev \
&& pip install --no-cache-dir -r requirements.txt \
&& apk del .build-dependencies
# Handling the issue described here: https://github.com/ecederstrand/exchangelib/issues/608

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this show elsewhere in the repo?

Comment thread docker/py3ews/Pipfile Outdated
requests-ntlm = "==1.1.0" # lock because of https://github.com/requests/requests-ntlm/issues/136
ntlm-auth = ">=1.5.0"
pyspnego = ">=0.5.2" # <0.5.2 calls hashlib.new('md4'), fails under OpenSSL 3.x (XSUP-75734)
requests-ntlm = "==1.3.0"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
requests-ntlm = "==1.3.0"
requests-ntlm = ">=1.3.0"

This reverts commit 97bd884.

Narrowing XSUP-75734 to the ntlm image only. py3ews keeps
requests-ntlm 1.1.0 and therefore still needs OpenSSL's legacy
provider for md4.

XSUP-75734
This reverts commit dc2e5ab.

Narrowing XSUP-75734 to the ntlm image only. py3ews stays on
requests-ntlm 1.1.0 with ntlm-auth, and verify.py keeps asserting
md4 availability.

XSUP-75734
This reverts commit c5bef38.

Narrowing XSUP-75734 to the ntlm image only. py-ews stays on
requests-ntlm 1.1.0 with ntlm-auth.

XSUP-75734
Narrowing XSUP-75734 to the ntlm image only. py-ews and py3ews stay
pinned at requests-ntlm 1.1.0, so their ignore rules are restored.
The ntlm image is upgraded to 1.3.0, so its rule stays removed.

XSUP-75734
The exact ==1.3.0 pin was only needed to force the jump off requests-ntlm
1.1.0, which pulled in ntlm-auth and its hashlib.new('md4') call that fails
under OpenSSL 3.x. Now that 1.3.0+ routes NTLM hashing through pyspnego's
pure-Python MD4, a floor is sufficient and lets dependabot manage future
updates.

Relock is a no-op for the resolved graph: 1.3.0 is still the newest release,
so the only Pipfile.lock change is the _meta hash. ntlm-auth remains absent.

Relates to XSUP-75734
Beyond the NTOWFv1 known-answer check, assert the API the consuming
integrations actually use: 2-arg HttpNtlmAuth with a DOMAIN\user name,
the no-op session argument, and that applying the auth handler forces
Connection: Keep-Alive and registers the 401 response hook.

All checks are offline, so they run under the CI image with no network.

XSUP-75734
@content-bot

This comment has been minimized.

@content-bot

Copy link
Copy Markdown
Contributor

Docker Image Ready - Dev

Docker automatic build has deployed your docker image: devdemisto/ntlm:1.0.0.12535775
It is available now on docker hub at: https://hub.docker.com/r/devdemisto/ntlm/tags
Get started by pulling the image:

docker pull devdemisto/ntlm:1.0.0.12535775

Docker Metadata

  • Image Size: 62.40M
  • Image ID: sha256:1ab59d07e583d706d7a32761d70dc99bed8b0b5b5c867dfa7c1975f701347874
  • Created: 2026-08-31T08:06:02.089978715Z
  • Arch: linux/amd64
  • Command: ["python3"]
  • Environment:
    • PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
    • LANG=C.UTF-8
    • GPG_KEY=7169605F62C751356D054A26A821E680E5FA6305
    • PYTHON_VERSION=3.12.13
    • PYTHON_SHA256=c08bc65a81971c1dd5783182826503369466c7e67374d1646519adf05207b684
    • DOCKER_IMAGE=devdemisto/ntlm:1.0.0.12535775
  • Labels:
    • io.buildah.version:1.37.5
    • org.opencontainers.image.authors:Demisto <containers@demisto.com>
    • org.opencontainers.image.revision:ff580291053976d1ae2a7de820b983285a45ba5d
    • org.opencontainers.image.version:1.0.0.12535775
    • panw.builtby.pipeline:11694983
    • panw.builtby.project:xdr/cortex-content/dockerfiles
    • panw.builtby.template:build-scan-publish

@content-bot

Copy link
Copy Markdown
Contributor

🔍 AI Triage Report Available

An automated triage report has been generated for this pipeline.

Status: success
Report ID: e787b82e9f01ff0b

📋 Triage Report

⚠️ AI-generated triage. Validate before acting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants