Repository navigation
tools.igrog compiled gridders; packed grid-normal kernel; empty_cuda_memcache - #171
Conversation
… double only on the host) Consumer cards run complex128 at 1/64 rate: on an RTX 4060 the 256^3 Deli-CS case spent 86 s gridding. Single and double agree to 2e-6 relative on the test object. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
…e kernels Saves solving every kernel a second time to propagate noise through the gridding. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…rch_grog_grid) Ported from fast-encoding's grog.cpp to C, in single precision: per target, Q at the source-pair and source-target displacements by cubic Lagrange in the table, R_ss + mu I by Cholesky, the kernel applied to every set (kspace and also=) and the noise gain. Targets are spread over OpenMP threads. igrog now grids on the host with it by default (device=None); device= a card keeps the batched torch solve, and double=True the torch double solve. The torch kernels interpolate only the pairs i < j (Q(-d) = Q(d)^H) and gather table rows with index_select. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
Each thread interpolates Q into the packed lower triangle of R_ss + mu I, factors it, and solves W a row at a time with that row of R_ts, then applies it to every set: no workspace, the card holds the samples and one batch of targets. igrog grids on the samples' device by default; torch remains for double=True and builds without the card gridder. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
|
CodeFactor reports 2 minor issues on 75d5917. Its page renders them client-side only and the check run carries no text, so I can't see which lines they are. They are most likely complexity notes on the new gridders ( Generated by Claude Code |
…ounding (2e-4) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
|
codecov/patch reports 60 % of the diff hit (target 93 %) on 037c83b. The lines it misses are the card path: Generated by Claude Code |
The per-thread kernel kept 1.8 KB of system per thread in local memory, which streamed through L2 (0.7 s per million targets on an A40). Now a warp shares a target: a lane per coil pair interpolates Q (coalesced table rows), a lane per row factors and solves, and the sets are applied by a warp reduction; the packed triangle, one row of W and the offsets sit in shared memory, 1.6 KB per warp at 18. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
…pty_cuda_memcache The positions normal kept a full R x R complex kernel per kept place. It is Hermitian, so only r <= c is stored (packed upper triangle), and when every entry is real (real subspace basis) it is stored as floats: R(R+1)/2 instead of R^2 complex entries, a further 2x for a real basis. The host normal and kern_contract_grid read the packed layout and conjugate for r > c. bartorch.empty_cuda_memcache() frees BART's per-stream cache of device blocks, so a caller can hand VRAM back between stages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm
Summary
tools.igrognow grids with compiled loops instead of batched torch:src/csrc/ops/grog.c,bartorch_grog_grid): ported from fast-encoding'sgrog.cpp, in single precision, with OpenMP across targets. For each target it interpolates Q (cubic Lagrange in the table), factors R_ss + mu I by Cholesky, applies W to every set and returns the noise gain.src/csrc/ops/grog.cu,bartorch_grog_grid_cuda): one target per warp. The packed lower triangle of the system, one row of W and the offsets sit in shared memory (1.6 KB per warp at 18). Lanes split the coil-pair interpolation (coalesced table rows), the Cholesky rows, the solves and the per-set reductions. The card holds only the samples and one batch of targets.igrog(..., device=None)grids on the samples' device.double=Truekeeps the torch double-precision path for checks, and the torch path is also the fallback on a build without the card gridder.also=moves extra sample sets (e.g. a noise draw) with the same kernels.Two more changes for the gridded-positions normal (
CartesianSense(positions=)):kern_contract_gridread the packed layout.bartorch.empty_cuda_memcache()returns BART's per-stream cache of device blocks, so a caller can free VRAM between stages without turning the cache off.Measured (Deli-CS case, 256³, cycled R8, 6 coils, 3 sources, A40)
Card and host agree to 2e-6 on the values. The noise draw differs by 2.5e-4, which is single-precision rounding: host and card are each 2.3e-4 from double.
Full recon (mri2qmap, at e7cf777): the grid kernel goes from 0.35 to 0.11 GiB (2.92 M places, real basis). Emptying the memcache after each CG solve frees about 2.5 GB during the other stages, and CG is not slower (10.1 s vs 11.8 s). The process peak drops from 6.5 to 6.0 GB, or to 5.55 GB with coil kernels on the caller's side. The maps match the previous run within run-to-run noise.
Tests
test_single_precision_matches_double,test_also_moves_extra_samples_by_the_same_kernels,test_the_card_gridder_matches_the_host_loop(skips without a card).test_points_normal_with_a_real_basis_keeps_a_real_kernel_and_is_the_two_applications,test_emptying_barts_device_cache_is_safe_with_or_without_a_device.CUDAHOSTCXX=clang++, otherwise cuFINUFFT's templates mangle differently.🤖 Generated with Claude Code
https://claude.ai/code/session_01MFB6QLjhTxFMFDEzathbEm