Repository navigation
fix: support torch 2.13 without torch_sparse and make mlflow optional - #3
Conversation
PyG publishes no torch-sparse wheels for torch 2.13, so importing nichecompass failed with "No module named 'torch_sparse'". The adjacency matrix in SpatialAnnTorchDataset is now a coalesced torch sparse COO tensor. The node label aggregators compute the GCN normalization with plain torch, with the same semantics as gcn_norm on a SparseTensor: existing self loops are replaced by weight-1 self loops and the aggregation weights keep their row-major order. The transposed product uses a CSR matrix with sorted indices, so the sparse product does not sort the edges again. Checked against torch_sparse 0.6.18 with torch 2.5.1 on 50 random graphs, with and without duplicate edges and self loops: same weights in the same order, largest difference 3.6e-7. On CPU the aggregation takes 8, 62 and 122 ms for graphs of 4k, 20k and 100k nodes, against 15, 77 and 219 ms with torch_sparse. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The loaders sample induced subgraphs (directed=False), which PyG supports only with torch_sparse. torch_sparse has no wheels for torch 2.13. InducedNeighborSampler samples the nodes with the directional sampler, which pyg-lib supports, and adds all edges between the sampled nodes in the order torch_sparse returns them. Checked against torch_sparse 0.6.18 on torch 2.13 for 12 random graphs with 1 and 2 hops: the node loader batches are identical (200 batches). All 620 link loader batches hold the full neighborhood of their label edges and the induced subgraph. Their negative edges differ from the second batch on, because pyg-lib draws from torch's random generator while sampling and torch_sparse does not. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nichecompass only calls mlflow to log parameters, metrics and figures, and only when an mlflow_experiment_id is given. As a required dependency, mlflow pulled in its server stack (flask, werkzeug, gunicorn, waitress, sqlalchemy, alembic, graphene, docker and more), which has frequent security advisories. mlflow is now imported only when tracking is used, and it is in the new 'tracking' extra. Without it, tracking fails with an ImportError that names the extra. Checked with mlflow uninstalled: nichecompass imports without loading mlflow, the tests pass and a two-epoch training on a small synthetic slide runs through. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
OX Security reviewed this pull request — nothing to fix.
Branch |
3603af5 to
0b853dc
Compare
The repository only allows actions pinned to a full-length commit SHA, so the build job failed before it started. Each action is pinned to the commit of the latest release in its current major version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…_sparse The symmetry check compared the nnz of the matrix and its transpose, which are always equal, so it never failed. It now compares the indices and raises a ValueError. CI, the conda environment and the docs still installed torch_sparse. They now install pyg-lib, which the induced neighbor sampler needs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
a72ba94 to
9a6c4cc
Compare
|
can we do a quick smoketest on GPU? |
|
The following things need a follow-up MR on the target-identification repo:
|
|
https://github.com/aignostics/nichecompass/actions tests are disabled here |
Can you please have a look as you trained it before? |
InducedNeighborSampler overrides the private NeighborSampler._sample, so pin torch-geometric to the versions it was tested with (2.5 to 2.8). nichecompass never imports torch_scatter, so remove it from the docs, the conda environment and the CI setup. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
On the CI question: the Test workflow has state |
|
Thanks! Both follow-ups are handled on our side. |
ran smoke test, everything passed 🙌 |
Why
PyG publishes no torch-sparse or torch-scatter wheels for torch 2.13. Without torch_sparse, importing nichecompass fails with "No module named 'torch_sparse'", and the neighbor loaders cannot sample the induced subgraphs nichecompass trains on.
Changes
SpatialAnnTorchDatasetis a coalesced COO tensor. The node label aggregators compute the GCN normalization with plain torch, with the same semantics as gcn_norm (existing self loops are replaced by weight-1 self loops). The aggregation uses a sorted CSR matrix.InducedNeighborSamplersamples the nodes with the directional sampler, which pyg-lib supports, and adds all edges between the sampled nodes in the order torch_sparse returns them. It is used by the node loaders, and by the link loaders whenedges_directedis False.mlflow_experiment_idis given. It is now imported lazily and lives in the newtrackingextra (also inall). This drops the mlflow server stack (flask, gunicorn, sqlalchemy, alembic and more) from default installs.Users need pyg-lib for the neighbor loaders, e.g.
pip install pyg-lib -f https://data.pyg.org/whl/torch-2.13.0+cpu.html.Checks
tests/test_aggregators.py,tests/test_dataloaders.py,tests/test_tracking.py.🤖 Generated with Claude Code