Skip to content

chore(deps): update nltk to 3.9.4 - #968

Merged
Patrick Nilan (pnilan) merged 8 commits into
mainfrom
devin/1774667708-update-nltk-cryptography
Aug 28, 2026
Merged

Patrick Nilan (pnilan) merged 8 commits into
mainfrom
devin/1774667708-update-nltk-cryptography

Conversation

@aaronsteers

@aaronsteers Aaron ("AJ") Steers (aaronsteers) commented Mar 28, 2026 •

Copy link
Copy Markdown
Member

Resolves https://github.com/airbytehq/oncall/issues/11795:


Summary

Bumps the optional nltk extra from 3.9.1 to 3.9.4, unblocking CVE fixes for downstream users. nltk==3.9.1 propagates as a hard pin through the CDK into PyAirbyte and connectors, so those consumers currently cannot resolve CVE-2026-0846, CVE-2025-14009 (zip slip, GHSA-7p94-766c-hgjp, CVSS 10.0 — the advisory tracked by the oncall issue above), CVE-2026-33231, or CVE-2026-33230. See airbytehq/PyAirbyte#1022.

-nltk = { version = "3.9.1", optional = true }
+nltk = { version = "3.9.4", optional = true }

The branch has been merged forward with current main and the lockfile regenerated with Poetry 2.1.4 (the version that produced main's lockfile), so the lock diff is 5 lines each way: the nltk version, its wheel/sdist hashes, its python-versions metadata (>=3.8 → >=3.10, satisfied by the CDK's >=3.10,<3.14), and the content hash. No transitive dependency versions change. Earlier revisions of this branch also touched the cryptography range and regenerated the lock with Poetry 2.3.3; both are gone — the diff vs main is nltk only.

3.9.3 is the minimum version that patches the zip-slip advisory; 3.9.4 additionally clears GHSA-jm6w-m3j8-898g and GHSA-gfwx-w7gr-fvh7 at no extra cost. Several remaining high-severity nltk advisories are only patched in 3.10.0, which is deliberately out of scope here (see below).

Breaking changes

None expected. 3.9.2–3.9.4 are patch releases (security hardening, tokenizer/metric bugfixes, Python 3.14 support). The strict nltk.pathsec policy that is breaking landed in 3.10.0, which this stays below.

The one hardening worth checking against CDK usage is the path-traversal / absolute-path restrictions added to nltk.data and the downloader in 3.9.3, because unstructured_parser.py appends an absolute dir to nltk.data.path and downloads into it at import time. Verified under 3.9.4 with an empty data dir: the download branch runs and punkt / punkt_tab download and are then found successfully.

Separately noted, not fixed here: unstructured_parser.py looks for tokenizers/averaged_perceptron_tagger_eng.zip, but nltk stores that resource under taggers/. That lookup has always missed; behavior is identical on 3.9.1, so it is pre-existing and out of scope for this bump.

Test plan

  • poetry install --all-extras, poetry check, poetry run ruff check ., poetry run ruff format --check . — all clean (re-run after the merge-forward to current main).
  • poetry run pytest unit_tests/sources/file_based/file_types/test_unstructured_parser.py unit_tests/sources/file_based/test_file_based_scenarios.py — 280 passed.
  • poetry run pytest -m 'not slow' — 4179 passed, 2 skipped.
  • Manual import of unstructured_parser against an empty nltk data dir to exercise the download-and-find path under 3.9.4.
  • Confirmed the installed nltk/downloader.py _unzip_iter now validates each member (_validate_member(...) + per-member zf.extract(...)) instead of calling zipfile.extractall(), i.e. the zip-slip fix is present.

Test Coverage

No new unit test is added: the change is a dependency version pin with no CDK code change, so there is no new CDK behavior to assert. Coverage comes from the existing test_unstructured_parser.py / test_file_based_scenarios.py suites (280 tests) exercising the nltk-dependent parser against 3.9.4, plus the direct check that the patched _unzip_iter implementation is what gets installed.

Requested by: Patrick Nilan (@pnilan) (via /ai-fix on the oncall issue)

Link to Devin session: https://app.devin.ai/sessions/7c79317d1dc04c4d8147f3a3354b673d
Open in Devin Desktop: https://app.devin.ai/desktop/session/7c79317d1dc04c4d8147f3a3354b673d?variant=devin

Summary by CodeRabbit

  • Chores
    • Updated the optional language-processing dependency to a newer compatible version.

Important

Auto-merge enabled.

This PR is set to merge automatically when all requirements are met.

@devin-ai-integration

Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

@github-actions

Copy link
Copy Markdown

👋 Greetings, Airbyte Team Member!

Here are some helpful tips and reminders for your convenience.

💡 Show Tips and Tricks

Testing This CDK Version

You can test this version of the CDK using the following:

# Run the CLI from this branch:
uvx 'git+https://github.com/airbytehq/airbyte-python-cdk.git@devin/1774667708-update-nltk-cryptography#egg=airbyte-python-cdk[dev]' --help

# Update a connector to use the CDK from this branch ref:
cd airbyte-integrations/connectors/source-example
poe use-cdk-branch devin/1774667708-update-nltk-cryptography

PR Slash Commands

Airbyte Maintainers can execute the following slash commands on your PR:

  • /autofix - Fixes most formatting and linting issues
  • /poetry-lock - Updates poetry.lock file
  • /test - Runs connector tests with the updated CDK
  • /prerelease - Triggers a prerelease publish with default arguments
  • /poe build - Regenerate git-committed build artifacts, such as the pydantic models which are generated from the manifest JSON schema in YAML.
  • /poe <command> - Runs any poe command in the CDK environment
📚 Show Repo Guidance

Helpful Resources

📝 Edit this welcome message.

@github-actions

github-actions Bot commented Mar 28, 2026 •

Copy link
Copy Markdown

PyTest Results (Fast)

4 363 tests  ±0   4 351 ✅  - 1   7m 34s ⏱️ - 1m 13s
    1 suites ±0      12 💤 +1 
    1 files   ±0       0 ❌ ±0 

Results for commit 375efb9. ± Comparison against base commit 4855c2d.

This pull request skips 1 test.
unit_tests.sources.declarative.test_concurrent_declarative_source ‑ test_read_with_concurrent_and_synchronous_streams

♻️ This comment has been updated with latest results.

@github-actions

github-actions Bot commented Mar 28, 2026 •

Copy link
Copy Markdown

PyTest Results (Full)

4 366 tests  ±0   4 354 ✅ ±0   14m 22s ⏱️ +22s
    1 suites ±0      12 💤 ±0 
    1 files   ±0       0 ❌ ±0 

Results for commit 375efb9. ± Comparison against base commit 4855c2d.

♻️ This comment has been updated with latest results.

@devin-ai-integration devin-ai-integration Bot changed the title chore(deps): update nltk to 3.9.4 and widen cryptography range to <47.0.0 chore(deps): update nltk to 3.9.4 Mar 30, 2026
@ShaneMac-Anaplan

Copy link
Copy Markdown

Aaron ("AJ") Steers (@aaronsteers) When do we expect to get this in? I'm starting to get security alerts on 3.9.1?

devin-ai-integration Bot and others added 2 commits August 6, 2026 00:09
Co-Authored-By: AJ Steers <aj@airbyte.io>
Copilot AI review requested due to automatic review settings August 6, 2026 00:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the optional nltk dependency from 3.9.1 to 3.9.4, with a regenerated poetry.lock reflecting Poetry 2.3.3 marker-expression rewrites, to keep the CDK’s file-based “unstructured” parsing dependency set current.

Changes:

  • Bump nltk from 3.9.1 → 3.9.4 in pyproject.toml.
  • Refresh poetry.lock to lock nltk==3.9.4 (and update the lock metadata/content hash as part of regeneration).

Reviewed changes

Copilot reviewed 1 out of 2 changed files in this pull request and generated 1 comment.

File Description
pyproject.toml Updates the pinned optional nltk version to 3.9.4.
poetry.lock Locks nltk to 3.9.4 and updates lock metadata from regeneration.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pyproject.toml

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 2 changed files in this pull request and generated no new comments.

@pnilan
Patrick Nilan (pnilan) marked this pull request as ready for review August 28, 2026 18:19
@pnilan
Patrick Nilan (pnilan) requested a review from a team as a code owner August 28, 2026 18:19

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

@pnilan
Patrick Nilan (pnilan) enabled auto-merge (squash) August 28, 2026 18:21
@coderabbitai

coderabbitai Bot commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f025a2cf-7f7f-4b00-85ec-5e4690a9ff09

📥 Commits

Reviewing files that changed from the base of the PR and between 4855c2d and 375efb9.

⛔ Files ignored due to path filters (1)
  • poetry.lock is excluded by !**/*.lock
📒 Files selected for processing (1)
  • pyproject.toml

Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review.


📝 Walkthrough

Walkthrough

The optional nltk dependency constraint in pyproject.toml changes from 3.9.1 to 3.9.4.

Changes

NLTK dependency update

Layer / File(s) Summary
Update optional nltk constraint
pyproject.toml
The optional nltk dependency constraint changes from 3.9.1 to 3.9.4.

Estimated code review effort: 1 (Trivial) | ~2 minutes

Merge Risk: ⚪ Minimal · up to 375ef

This narrowly scoped dependency update changes NLTK from 3.9.1 to 3.9.4 to reduce vulnerability exposure without changing CDK code or public behavior. No actionable merge-blocking risk remains after normal checks and review.

Suggested reviewers: bazarnov, danylogl, darynaishchenko

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: updating the optional nltk dependency to version 3.9.4.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)

✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch devin/1774667708-update-nltk-cryptography

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@pnilan
Patrick Nilan (pnilan) merged commit 7c96f9d into main Aug 28, 2026
33 of 35 checks passed
@pnilan
Patrick Nilan (pnilan) deleted the devin/1774667708-update-nltk-cryptography branch August 28, 2026 18:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants