Detect kernel dependencies imported via 'from x_kernels import func' - #661
Merged
Merged
Conversation
get_dependencies only counted "from a.b import c" as a dependency when c is a module. Kernels importing a function from another kernel, e.g. "from struphy.bsplines.evaluation_kernels_3d import get_spans", therefore had no make prerequisite on it, and "struphy compile -j N" could fail with "Imported files must be pyccelised before they can be used", depending on the build order. Affected: particle_to_mat_kernels, accum_kernels, pusher_utilities_kernels. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
struphy compile -j Nfailed at random in CI, e.g. in this job, while another shard of the same run with the same packages passed:dependencies.pyonly countedfrom a.b import cas a dependency whencis a module.particle_to_mat_kernels.pydoesfrom struphy.bsplines.evaluation_kernels_3d import get_spans, a function, so make had no prerequisite onevaluation_kernels_3d.so. Whether it was built first depended on how make scheduled the parallel jobs.Kernels affected:
particle_to_mat_kernels,accum_kernels,pusher_utilities_kernels.Fix
dependencies.py: if the imported namecinfrom a.b import cis not a module, thena.bitself is a dependency.test_compile.py:test_dependencies_static_rulesnow expectsfrom struphy.pkg.h_kernels import some_functionto give a dependency.test_dependencies_match_import_based_resultnow also collects the__module__of imported functions and classes, compiled kernels included.Testing
src/struphy/console/tests/test_compile.pypass, including the cycle check.struphy compile -d,struphy compile -j 4built all 56 kernels locally.particle_to_mat_kernelsnow listsevaluation_kernels_3damong its dependencies.evaluation_kernels_3dwas built before all three affected kernels.🤖 Generated with Claude Code