Repository navigation
MPI Parallelization MVP for Matlab Grid - #202
Merged
Merged
Conversation
BenjaminRodenberg
force-pushed
the
mpi-parallelization-mvp-matlab
branch
2 times, most recently
from
October 21, 2026 15:19
756e32e to
704d732
Compare
setup-mpi makes Open MPI use TCP between ranks, and on GitHub's runners Open MPI's TCP transport also picks the docker0 interface, over which messages never arrive. Small messages still get through, but large ones hang: in #197 a gather on 4 ranks hung until GitHub's limit of 6 hours, while the same run passed locally, where the ranks use shared memory. The next commits run the MATLAB example on up to 4 ranks, with gathers large enough to hang. Open MPI now only uses the loopback interface in both workflows. It is set in mca-params.conf, since tox does not pass OMPI_MCA_* variables to the tests. In addition, the build-and-test job now stops after 20 minutes and its MPI steps after 5, so a hang fails quickly instead of blocking a runner for 6 hours. Taken from 5e28f4d and 47b4c8f of #197. Assisted-by: Claude Code:claude-opus-5-5
MATLAB and NetCDF grids now keep the flat indices of their columns in the 2-D fields as `ind` and take every per-column field from the 2-D fields with them, instead of applying the mask to each field again. A subset of the columns then only needs a subset of `ind`, which the next commits use to split a MATLAB grid over MPI ranks. The columns, their order and the model output are unchanged. Assisted-by: Claude Code:claude-opus-5-5
The output writer now gathers the columns of all ranks with ColumnGather and rank 0 writes them to the file a single rank would write, one variable at a time to limit its memory. Only structured grids are gathered; a partitioned mesh still skips the output file. Taken from 8b5a9eb of #197. Assisted-by: Claude Code:claude-opus-5-5
The shading look-up table is computed by walking from every glacier cell of the 2-D grid, not only from the columns of one rank, so a rank holding part of the columns cannot build its rows of the table. Shading on several ranks is now a usage error asking for --no-shading, and _precompute_shading_matlab asserts that it is not called in such a run. No grid that supports shading runs on several ranks yet; the next commits split MATLAB grids over the ranks. Taken from 43a676e of #197. Assisted-by: Claude Code:claude-opus-5-5
On several ranks, each rank now computes a contiguous block of the columns of a MATLAB grid, the first block on rank 0. The 2-D fields stay whole on every rank: the slope needs the neighbours of each cell, and the output writer places the columns of all ranks into them. The domain-wide latitude mean, lapse rate and random seed of #203 make the results equal those of a single rank. Taken from 46f47a2 of #197. Assisted-by: Claude Code:claude-opus-5-5
The MATLAB example runs for one day on a single rank and on 2, 3 and 4 ranks under mpirun; each parallel run must give a byte-identical output file and an equal --dump-reference snapshot. CI also runs the MATLAB example on 4 ranks. Taken from db2b582 and e26d3e8 of #197. Assisted-by: Claude Code:claude-opus-5-5
BenjaminRodenberg
force-pushed
the
mpi-parallelization-mvp-matlab
branch
from
October 21, 2026 16:04
704d732 to
25d7e27
Compare
BenjaminRodenberg
changed the title
5 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Split off from #197 (#197), which has grown too large: MPI parallelization for the MATLAB grid only. Builds on the MPI usage errors of #201 and on the domain-wide values of #203.
mpirun -np N ebfm --matlab-mesh examples/dem_and_mask.mat --no-shadingnow splits the columns of the grid into contiguous blocks, one per rank. Rank 0 gathers the columns of all ranks and writes the output file; it is byte-identical to that of a run on a single rank, and so is--dump-reference(tested on 2, 3 and 4 ranks). The latitude mean, lapse rate and random seed are computed for the whole domain since #203.Also in this PR:
--no-shadingis required).Output and restart files are still skipped for partitioned Elmer meshes. Everything else from #197 (splitting non-partitioned Elmer meshes, XIOS DEMs on several ranks, output of partitioned meshes, restart files in parallel runs) stays there.
🤖 Generated with Claude Code