Cuda pr 8 cuda headers - #653
Merged
Merged
Conversation
The SPH linear smoothing kernels returned a non-zero gradient at zero separation, so every particle pushed on itself. This PR makes all SPH kernel gradients vanish at r=0, which matches the symmetric value the trigonometric and gaussian kernels already give. Closes #437 ### What was wrong - `grad_linear_uni(0, h)` returned `+1/h**2`, the left derivative. That value carries into `grad_linear_1d`, `grad_linear_2d_{1,2}` and `grad_linear_3d_{1,2,3}`, and it applies whenever the component's own coordinate is 0, not only at the origin. - `grad_linear_isotropic_3d_{1,2,3}` returned `-1/h / (C h^3)` at `r == 0`. - As a result the SPH gradient of a lattice-loaded constant density was not zero: it came out at about 3.5 for 1d linear, 3.8 for 3d tensor linear and -0.58 for isotropic in the check below. ### What changed - `src/struphy/pic/sph_smoothing_kernels.py`: at the cusp the gradient now returns 0 (`x == 0` in `grad_linear_uni`, `r == 0` in the isotropic variants). Values away from 0 are unchanged. The trigonometric and gaussian gradients were checked too and already give 0 there. - `src/struphy/pic/tests/test_kernel_setup.py`: - New test `test_sph_kernel_gradients_vanish_at_zero` covers every gradient type in `smoothing_kernel`, at the origin and on the coordinate planes. - `test_sph_tensor_destinations` relied on the old self-gradient `1/h**2`. It now uses a second neighbouring particle so the viscosity-tensor destinations still get non-trivial values. ### Verification - A standalone check calls all 21 gradient kernel types at the origin and on the coordinate planes. Before the fix, 9 types were non-zero at the origin and all 4 plane cases were non-zero. After the fix all of them are 0. At random points away from 0 the values are bit-identical before and after. - The gradient of a lattice-loaded constant density is now ~1e-16 in 1d linear, 3d tensor linear and 3d isotropic. - `pytest src/struphy/pic/tests/test_kernel_setup.py`: 74 passed. The new test fails without the fix. - A subset of `test_sph.py::test_sph_evaluation_1d` (linear_1d, periodic) passes. ### Risks / follow-ups - SPH results that use the linear kernels change slightly, because the self-contribution is gone. That removal is the intended correction. - #436 (viscosity strain without DF^-T) is in the same area and is not addressed here. - This branch includes the commit bumping `feectools` to 0.1.11 so that CI can run. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ld just be a proof of concept
CudaKernel no longer flattens argument objects or casts scalars at call time: the arguments must already be in the format of a cupy.RawKernel (flat tuple of CuPy arrays and NumPy scalars, built once at setup), and the number of threads is passed as n_threads. Add docstrings to the kernel and argument classes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Converting the scalars and joining the .values of the CUDA argument classes costs about 1 us per call (the kernel launch alone about 70 us), prevents silently misaligned arguments when a Python int is passed as a 64-bit value, and keeps the calls the same on both backends. Arrays are still never converted or copied; n_threads stays an explicit argument. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…sses CUDA reads each kernel argument with the size declared in the signature, so Python int/float already arrive correctly in int/double parameters; the casts did not change the result, and did not prevent the actual failure cases (e.g. an integer passed to a double parameter). Checking scalars against the kernel signature is noted as a follow-up in CUDA_STRATEGY.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Add CudaKernel.from_file, which reads the CUDA source from a <name>_cuda.cu file and takes the kernel name from the file name, and ship .cu/.cuh files as package data. Step 2 of CUDA_STRATEGY.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
KernelCatalog.from_package pairs <name>/<name>_kernels.py (pyccel) with <name>/<name>_cuda.cu (CUDA, optional). The CUDA kernel of a Kernel is now optional: on the CuPy backend, a kernel without a CUDA version raises NotImplementedError naming the expected .cu file. Step 3 of CUDA_STRATEGY.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Pusher now takes a Kernel or, as before, a PyccelKernel (wrapped into a Kernel without CUDA version) and calls get_kernel() in its constructor. On the CuPy backend, a pusher whose kernel has no CUDA version thus fails when it is created instead of in the time loop. No changes to the propagators, no behaviour change on the CPU. Step 4 of CUDA_STRATEGY.md. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Test only analytic mappings on the CuPy backend: spline mappings such as IGAPolarCylinder cannot be created there yet (interp_mapping passes CuPy arrays to scipy.sparse.csc_matrix); correct CUDA_STRATEGY.md. - Build the CUDA domain arguments with cupy instead of xp, so they can be built whichever backend is active (the arrays are on the device); test this. - Move the new tests above the __main__ block of test_domain.py. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Merged
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
# Conflicts: # CUDA_STRATEGY.md
…o cuda-pr-7-derham
max-models
added a commit
that referenced
this pull request
Oct 2, 2026
**Solves the following issue(s):** PR 7 of the CUDA strategy (`CUDA_STRATEGY.md`). No issue closed. Stack: #649 → #651 → this PR → #653. The commits `Derham on the GPU: create Derham on CuPy, add Derham.cuda_args_derham` and `CUDA_STRATEGY: PR 7 depends on feectools#85` are new relative to #651. Depends on struphy-hub/feectools#85 (feectools on the CuPy backend). Without it, feectools itself fails while the spline spaces are built on CuPy. **Core changes:** `Derham` (`src/struphy/feec/psydac_derham.py`) can be created on the CuPy backend. The rule: data that describes the spline spaces stays on the host on every backend, because it goes to pyccel kernels, SciPy or MPI. Only coefficients and stencil matrices live on the device. - The projection and quadrature grids (`get_pts_and_wts`, `get_pts_and_wts_quasi`, `get_span_and_basis`, the local-projector weights) and `spline_types_pyccel` are NumPy arrays. This removes the ad-hoc `cupy` conversions in them. - `domain_array`, `index_array`, `index_array_N`, `index_array_D` and `neighbours` are gathered with NumPy MPI buffers, so no CUDA-aware MPI is needed. They are then converted with `xp.asarray`, so they are device arrays on CuPy, like `Particles.domain_array`. The neighbour search runs on the host because it builds an object array with `None` entries, which CuPy cannot hold. - `args_derham` is built from the host knots, degrees and starts directly; `_to_numpy_for_kernel` is gone. - New `Derham.cuda_args_derham`: `CudaDerhamArguments` (new in `src/struphy/utils/cuda_arguments.py`) with one device copy of the degrees, knots and start indices, built on first access. On the NumPy backend it raises `TypeError`, since host arrays are never copied to the device. The pyccel scratch arrays (`bn1`, ..., `bd3`) are not part of it; they become per-thread local arrays in CUDA (PR 10). - Local projectors (`DerhamOptions.local_projectors=True`) raise `NotImplementedError` on CuPy when the `Derham` is created, instead of failing later inside `CommutingProjectorLocal`. Not in this PR: - Local projectors on CuPy. `CommutingProjectorLocal` builds its data with `xp` and calls pyccel kernels on it, as `Derham` did. - Polar splines on CuPy. They need a spline mapping, which cannot be created on CuPy yet (see PR 5). - Field evaluation (`SplineFunction.__call__`, ...) on the device. It still calls pyccel kernels with the coefficients, so it needs CUDA evaluation kernels (PR 10+). Tests: - New `src/struphy/feec/tests/test_derham_gpu.py`: - `test_cuda_args_derham_needs_device_arrays` (runs everywhere): raises on the NumPy backend. - `test_derham_on_cupy` (GPU only), with periodic and with Dirichlet/free boundaries: the decomposition tables and kernel arguments agree between the backends, and `cuda_args_derham` is built once and holds device copies of the right values. - `test_local_projectors_not_supported_on_cupy` (GPU only). - The GPU-only tests have not run on a GPU yet. On the CPU they were run with a strict host stand-in for CuPy, which rejects host/device mixing and cannot run kernels, on top of feectools#85. With it, a `Derham` created on the "CuPy" backend matches the NumPy one on 1, 2 and 4 MPI processes. - `test_kernel_backends.py::test_pusher_accepts_kernel` fails with and without this PR (`ValueError: array is too big` in `draw_markers`). It is unrelated. **Model-specific changes:** None 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
max-models
enabled auto-merge (squash)
October 2, 2026 14:13
spossann
approved these changes
Oct 2, 2026
9 of 20 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Solves the following issue(s):
PR 8 of the CUDA strategy (
CUDA_STRATEGY.md). No issue closed.Stack: #649 → #651 → #652 → this PR. Only the commit
Shared CUDA header for the kernel argument classesis new relative to #652 (plus a merge of #652).Core changes:
CUDA kernels used to repeat the full flat signature of the argument classes (26 parameters for markers and domain alone). CuPy does not check kernel signatures, so adding a field to
MarkerArgumentswould have shifted all following arguments of every CUDA kernel without an error. Now each argument class is one C struct, defined once.src/struphy/kernel_arguments/pusher_args.cuhwithstruct MarkerArgs,DerhamArgsandDomainArgs. Kernels take them by value in place of the pyccel argument classes, e.g.void push_eta_linear(double dt, int stage, MarkerArgs args_markers, DomainArgs args_domain), and readargs_markers.markers,args_markers.n_markers, and so on.MarkerArgs.n_cols, since pyccel kernels take it frommarkers.shape[1]. Accordingly,CudaMarkerArguments.first_pusher_idxis nowfirst_init_idx, as inMarkerArguments.src/struphy/utils/cuda_arguments.py:Argumentsubclass lists its members once, e.g.fields = (("double*", "markers"), ...).struct_dtype()builds a NumPy structured dtype withalign=True, which gives C alignment and padding. Pointer members areuint64device addresses (array.data.ptr).numpy.void, which CuPy passes by value. It is packed once in the constructor, so kernel calls do no extra work.floatfor anintmember raisesTypeError, and a value that does not fit raisesOverflowError. Before, they arrived in the kernel truncated or wrapped around; plain NumPy assignment also truncates1.7to1. This solves the "scalar types" open question for everything inside the argument classes. Scalars passed directly to a kernel (dt,stage) are still unchecked.__getstate__/__setstate__).CudaKernel(src/struphy/utils/kernel_backends.py) compiles with the folder that contains thestruphypackage on the NVRTC include path, so.cufiles can#include "struphy/kernel_arguments/pusher_args.cuh"..cuhfiles are already package data.Why structs, and why
_pack()and the copy handling are needed:The underlying problem: CuPy passes kernel arguments positionally and never checks them against the signature. With flat parameter lists, a new field in an argument class shifts every later argument of every CUDA kernel, silently and on the GPU only. Some single, checked definition of each argument layout is therefore necessary. A C struct is the simplest one that also works for device helper functions.
What each piece is for:
struct_dtype()withalign=True: CuPy passes a struct by value only as a NumPy structured scalar, andalign=Truegives the C padding (theintmembers are followed by pointers).uint64(data.ptr): a struct cannot hold a CuPy array, only its device address.__setstate__: a direct consequence of storing addresses. The arrays of a deepcopied or unpickled object have new addresses, and without the repack a copy would silently read the original's arrays.For:
fields.args_domainand readargs_domain.kind_map, and CUDA reads exactly the same. This decides it for PR 10. The mapping evaluation, B-spline and boundary helpers take these argument objects, so a__device__helper takes oneconst DomainArgs&instead of 12+ parameters forwarded at every call site.fieldsand the header, not every kernel.Against:
_cupy_array()checks at construction that the array is a C-contiguous CuPy array of the right dtype, but not which device it is on. That will matter with one GPU per MPI rank; adding a device check is a small follow-up.sizeof/offsetofcheck and the NVRTC layout test cover it.Alternatives considered:
#define MARKER_ARGS double* markers, ...), withget_cuda_args()returning the arrays infieldsorder. This also prevents silent shifts and keeps CuPy's checks, without pointers or repacking. But every device helper then needs a long parameter list, or a second macro that rebuilds a struct inside the kernel. That is more macro machinery, and the CUDA code no longer mirrors the pyccel helpers.Tests (
src/struphy/pic/tests/test_kernel_backends.py):test_cuda_argument_structs_match_header: parsespusher_args.cuhand compares member names, C types and order withfields.test_cuda_struct_members_are_pyccel_attributes: the member names exist as attributes inpusher_args_kernels.py, exceptn_cols.test_cuda_struct_layout: an NVRTC-compiled kernel reportssizeofand the member offsets and sizes, which are compared with the NumPy dtype.test_cuda_struct_scalars_are_checkedandtest_cuda_struct_follows_copies.push_eta_linearandwrite_scalarsnow use the structs, andwrite_scalarsalso reads a pointer member of each struct.geometry/tests/test_domain.pystill used thevaluesattribute, whichget_cuda_args()replaced. They would have failed on a GPU.The GPU-only tests have not run on a GPU yet. On the host:
sizeof/offsetofas the NumPy dtypes. x86-64 and arm64 lay these structs out like CUDA.test_pusher_accepts_kernelfails with and without this PR (see Cuda pr 7 derham #652). It is unrelated.Model-specific changes:
None
🤖 Generated with Claude Code