Skip to content

fix: support torch 2.13 without torch_sparse and make mlflow optional - #3

Merged
maxg0811 merged 6 commits into
mainfrom
task/update-for-torch-213-compatibility
Nov 16, 2026
Merged

maxg0811 merged 6 commits into
mainfrom
task/update-for-torch-213-compatibility

Conversation

@maxg0811

@maxg0811 maxg0811 commented Nov 10, 2026 •

Copy link
Copy Markdown

Why

PyG publishes no torch-sparse or torch-scatter wheels for torch 2.13. Without torch_sparse, importing nichecompass fails with "No module named 'torch_sparse'", and the neighbor loaders cannot sample the induced subgraphs nichecompass trains on.

Changes

  • Replace SparseTensor with torch.sparse. The adjacency in SpatialAnnTorchDataset is a coalesced COO tensor. The node label aggregators compute the GCN normalization with plain torch, with the same semantics as gcn_norm (existing self loops are replaced by weight-1 self loops). The aggregation uses a sorted CSR matrix.
  • Sample induced subgraphs with pyg-lib. PyG samples induced subgraphs (directed=False) only with torch_sparse. The new InducedNeighborSampler samples the nodes with the directional sampler, which pyg-lib supports, and adds all edges between the sampled nodes in the order torch_sparse returns them. It is used by the node loaders, and by the link loaders when edges_directed is False.
  • Make mlflow optional. mlflow is only used for tracking when an mlflow_experiment_id is given. It is now imported lazily and lives in the new tracking extra (also in all). This drops the mlflow server stack (flask, gunicorn, sqlalchemy, alembic and more) from default installs.

Users need pyg-lib for the neighbor loaders, e.g. pip install pyg-lib -f https://data.pyg.org/whl/torch-2.13.0+cpu.html.

Checks

  • Aggregation matches torch_sparse 0.6.18 on 50 random graphs with duplicate edges and self loops, largest difference 3.6e-7. On CPU it is faster: 8, 62 and 122 ms for graphs with 4k, 20k and 100k nodes, against 15, 77 and 219 ms with torch_sparse.
  • Node loader batches are identical to torch_sparse on 12 random graphs with 1 and 2 hops (200 batches).
  • All 620 link loader batches hold the full neighborhood of their label edges and the induced subgraph. Negative edges differ from the second batch on, because pyg-lib draws from torch's random generator while sampling.
  • New tests: tests/test_aggregators.py, tests/test_dataloaders.py, tests/test_tracking.py.
  • A two-epoch training on a small synthetic slide runs through on CPU with mlflow uninstalled. A GPU smoke test passed as well.

🤖 Generated with Claude Code

maxg0811 and others added 3 commits November 9, 2026 13:54
PyG publishes no torch-sparse wheels for torch 2.13, so importing nichecompass failed with
"No module named 'torch_sparse'". The adjacency matrix in SpatialAnnTorchDataset is now a
coalesced torch sparse COO tensor. The node label aggregators compute the GCN normalization
with plain torch, with the same semantics as gcn_norm on a SparseTensor: existing self loops
are replaced by weight-1 self loops and the aggregation weights keep their row-major order.
The transposed product uses a CSR matrix with sorted indices, so the sparse product does not
sort the edges again.

Checked against torch_sparse 0.6.18 with torch 2.5.1 on 50 random graphs, with and without
duplicate edges and self loops: same weights in the same order, largest difference 3.6e-7.
On CPU the aggregation takes 8, 62 and 122 ms for graphs of 4k, 20k and 100k nodes, against
15, 77 and 219 ms with torch_sparse.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The loaders sample induced subgraphs (directed=False), which PyG supports
only with torch_sparse. torch_sparse has no wheels for torch 2.13.
InducedNeighborSampler samples the nodes with the directional sampler, which
pyg-lib supports, and adds all edges between the sampled nodes in the order
torch_sparse returns them.

Checked against torch_sparse 0.6.18 on torch 2.13 for 12 random graphs with
1 and 2 hops: the node loader batches are identical (200 batches). All 620
link loader batches hold the full neighborhood of their label edges and the
induced subgraph. Their negative edges differ from the second batch on,
because pyg-lib draws from torch's random generator while sampling and
torch_sparse does not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nichecompass only calls mlflow to log parameters, metrics and figures, and
only when an mlflow_experiment_id is given. As a required dependency, mlflow
pulled in its server stack (flask, werkzeug, gunicorn, waitress,
sqlalchemy, alembic, graphene, docker and more), which has frequent security
advisories. mlflow is now imported only when tracking is used, and it is in
the new 'tracking' extra. Without it, tracking fails with an ImportError that
names the extra.

Checked with mlflow uninstalled: nichecompass imports without loading
mlflow, the tests pass and a two-epoch training on a small synthetic slide
runs through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ox-security

ox-security Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

OX Security Logo

OX Security reviewed this pull request — nothing to fix.

✔ No issues found

Branch task/update-for-torch-213-compatibility → main

View scan in OX Security →

@maxg0811 maxg0811 changed the title Support torch 2.13 without torch_sparse and make mlflow optional fix: support torch 2.13 without torch_sparse and make mlflow optional Nov 10, 2026
@maxg0811
maxg0811 force-pushed the task/update-for-torch-213-compatibility branch from 3603af5 to 0b853dc Compare November 10, 2026 09:36
maxg0811 and others added 2 commits November 10, 2026 10:38
The repository only allows actions pinned to a full-length commit SHA, so the
build job failed before it started. Each action is pinned to the commit of
the latest release in its current major version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_sparse

The symmetry check compared the nnz of the matrix and its transpose, which are
always equal, so it never failed. It now compares the indices and raises a
ValueError.

CI, the conda environment and the docs still installed torch_sparse. They now
install pyg-lib, which the induced neighbor sampler needs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@philippj2

philippj2 commented Nov 10, 2026 •

Copy link
Copy Markdown

can we do a quick smoketest on GPU?
If you want me to do that let me know

@philippj2

Copy link
Copy Markdown

The following things need a follow-up MR on the target-identification repo:

  • Our scripts/niche_compass/patches/lowmem.py re-implements SpatialAnnTorchDataset.init and still calls self.adj.to_torch_sparse_coo_tensor(); this becomes (becomes self.adj.indices() here)
  • I guess the other one is the image rebuild which doesn't require changes 😄

@philippj2

Copy link
Copy Markdown

https://github.com/aignostics/nichecompass/actions tests are disabled here
do we disable_fork false here or irrelevant?
(claude question)

Comment thread src/nichecompass/data/dataloaders.py Resolved
Comment thread docs/installation.md Resolved
@maxg0811

Copy link
Copy Markdown
Author

can we do a quick smoketest on GPU? If you want me to do that let me know

Can you please have a look as you trained it before?

InducedNeighborSampler overrides the private NeighborSampler._sample, so
pin torch-geometric to the versions it was tested with (2.5 to 2.8).
nichecompass never imports torch_scatter, so remove it from the docs,
the conda environment and the CI setup.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@maxg0811

Copy link
Copy Markdown
Author

On the CI question: the Test workflow has state disabled_fork because this repo is a fork of Lotfollahi-lab/nichecompass. GitHub disables workflows in forks until a repo admin enables them (Actions tab, or gh workflow enable test.yaml). Worth enabling, but note the setup action still defaults to torch 2.0.0, so it would not cover torch 2.13 without bumping torch-version.

@maxg0811

maxg0811 commented Nov 11, 2026 •

Copy link
Copy Markdown
Author

Thanks! Both follow-ups are handled on our side.

@philippj2

Copy link
Copy Markdown

can we do a quick smoketest on GPU? If you want me to do that let me know

Can you please have a look as you trained it before?

ran smoke test, everything passed 🙌

@maxg0811
maxg0811 merged commit 9489ba3 into main Nov 16, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants