Security hardening backlog: committed 7 MB tarball, XXE-prone XML parsing, no zip-bomb guard, unsanitised stored file names #92

Part of #90 (PR #61 catch-up). The §3.5 security pass from catchup/ndi-python-2026-08 (commit 66db4b3) never landed; each item was re-verified against main and resolved.

Items

1. pythonArtifacts.tar.gz is committed at the repo root — ✅ Done in #184 — 7,308,051-byte binary deleted from the tree and .gitignored. Nothing in MANIFEST.in, pyproject.toml, or any code path referenced the file. (Removing it from git history is a separate, more invasive call — flagged, not assumed.)

2. XML is parsed with xml.etree.ElementTree on attacker-reachable paths. — ✅ Nothing to do on main; relocated to Waltham-Data-Science/ndi-ontology-python#6.

• src/ndi/ontology/providers.py no longer exists — commit 3784390 Depend on ndi-ontology-python instead of vendoring ndi.ontology moved the whole subpackage to the external ndi-ontology-python package. The defusedxml swap belongs there and is filed as issue improvement: Pydantic #6 against NCBITaxonProvider._lookup_taxid (providers.py:324) and _search_name (providers.py:357).
• src/ndi/cloud/admin/crossref.py:15 only imports Element, SubElement, tostring — pure XML construction, which the original scope already carved out as safe.
• Repo-wide grep -rn "xml.etree\|ElementTree\|defusedxml\|lxml" src/ returns only that one construction import.

3. No zip-bomb guard on cloud bulk download. — ✅ Done in #186 — added _check_zip_ratio in src/ndi/cloud/download.py, called before extraction inside _download_chunk_zip. Reads file_size / compress_size from infolist() (no bytes extracted), refuses when the summed ratio exceeds 100×. Bulk-download payloads are JSON — real ratios sit under 15×, bombs need 1000×+, so the ceiling is orders of magnitude clear of legitimate data. The three-check pattern from the original catch-up commit (member count, per-member size, total size) was rejected because each shape trips on real NDI datasets (thousands of per-epoch docs; multi-GB recording binaries). Regression tests in tests/test_cloud_download_zip_ratio.py.

4. Stored file names are not sanitised. — ✅ Relocated to VH-Lab/DID-python#58. Tracing the traversal risk from an untrusted document down to the filesystem calls: filename itself is no longer joined into any path in DID (used only as a SQL/is_in_file_list lookup key), so the classic filename="../../.ssh/authorized_keys" bug the original ndi-python fix guarded against genuinely doesn't reach the filesystem via DID's API. Two real holes remain in did.implementations.sqlitedb: _ingest_location writing to os.path.join(file_dir, uid) (sqlitedb.py:564) and _resolve_local following any location string without containment (sqlitedb.py:1013), both driven by untrusted document-JSON fields. DID-python#58 documents both with line numbers and the parity concern in DID-matlab (sqlitedb.m:430/694/726).

5. Smaller items from the same commit re-audit. — ✅ Scoped as #187. Walkthrough of src/ndi/cloud/profile.py, auth.py, client.py, ndi_install.py, and tests/_matlab_license_guard.py turned up five concrete findings: (i) login() writes NDI_CLOUD_TOKEN into os.environ so every subprocess inherits it; (ii) same problem for NDI_CLOUD_PASSWORD env var; (iii) secrets file written with 0644 permissions on multi-user systems; (iv) login error echoes raw resp.text verbatim into the exception message; (v) CloudAPIError.response_body carries raw server payload into every traceback. ndi_install.py and _matlab_license_guard.py were scanned clean.

Closing as complete — every original concern is either landed or filed as a scoped follow-up.