Skip to content

feature: detect MPI desynchronizations #4091

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
kdrienCG wants to merge 13 commits into
base: develop
Choose a base branch
from
+723 −591
Draft

feature: detect MPI desynchronizations #4091

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
24f46c6
add build option to enable MPI desync detection
kdrienCG Jun 12, 2026
61701a3
add desync dectection for barrier()
kdrienCG Jun 12, 2026
71ac289
move detectMpiDesync in the .hpp
kdrienCG Jun 12, 2026
e3a406e
add desync detection to all collective operations
kdrienCG Jun 12, 2026
72e6581
complete collective calls desync detection
kdrienCG Jun 12, 2026
a19e09e
rename variables for readability
kdrienCG Jun 12, 2026
c881621
fix max Allreduce call
kdrienCG Jun 12, 2026
27ec2ee
add MpiDesyncGuard RAII helper for readability
kdrienCG Jun 12, 2026
f7cf49f
fix global counter name
kdrienCG Jun 17, 2026
e4753b7
replace standard MPI_* collective operations use with MpiWrapper ones
kdrienCG Jun 17, 2026
24f60b7
store stacktraces on desync detections
kdrienCG Jun 18, 2026
b2574ef
wip: use a timeout as desync detection method
kdrienCG Jun 19, 2026
067d70f
wip: use a timeout as desync detection method
kdrienCG Jun 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/cmake/GeosxConfig.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ set( PREPROCESSOR_DEFINES BOUNDS_CHECK
METIS
MKL
MPI
MPI_DESYNC_DETECTION
PARMETIS
PETSC
PYGEOSX
Expand Down
2 changes: 2 additions & 0 deletions src/cmake/GeosxOptions.cmake
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,8 @@ endif()

option( ENABLE_MPI "" ON )

option( ENABLE_MPI_DESYNC_DETECTION "" OFF )

option( ENABLE_CUDA "" OFF )

option( ENABLE_HIP "" OFF )
Expand Down
3 changes: 3 additions & 0 deletions src/coreComponents/common/GeosxConfig.hpp.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@
/// Enables use of MPI (CMake option ENABLE_MPI)
#cmakedefine GEOS_USE_MPI

/// Enables use of MPI (CMake option ENABLE_MPI_DESYNC_DETECTION)
#cmakedefine GEOS_USE_MPI_DESYNC_DETECTION

/// Enables use of OpenMP (CMake option ENABLE_OPENMP)
#cmakedefine GEOS_USE_OPENMP

Expand Down
Loading
Loading