Repository navigation
Make the results on a partitioned Elmer mesh independent of the partitioning - #203
Merged
Merged
Conversation
Every column gets an id that is unique over the whole domain, grid["column_id"]: the Elmer element id, or the flat index into the 2-D fields of a MATLAB or NetCDF grid. ColumnGather collects per-column arrays of all ranks on rank 0 in id order. --dump-reference and --print-diagnostics now use it, so on a partitioned mesh rank 0 writes and reports the whole domain instead of every rank its own partition. timelastsnow_netCDF is now always float64: date2num returns integers when all times are whole days, ranks can disagree on that, and ColumnGather requires equal dtypes. Taken from 261e83f and 88e576c of mpi-parallelization-mvp-matlab, from #197. Assisted-by: Claude Code:claude-opus-5-5
On a partitioned Elmer mesh, each rank used the mean latitude of its own partition to select the water vapour coefficient of SWin, fitted the lapse rate of potential temperature over its own columns, and, without --random-seed, drew its own random seed. The results thus depended on the partitioning. The mean latitude is now reduced over all ranks in init_grid (grid["lat_mean_whole_domain"]), the lapse rate is fitted on rank 0 in column id order and broadcast, and rank 0 draws the seed and shares it. On a single rank each of these is a no-op. A new test splits a small generated mesh into 3 partitions and compares --dump-reference against a serial run on the whole mesh. Taken from 3cc2b1e of mpi-parallelization-mvp-matlab, from #197. Assisted-by: Claude Code:claude-opus-5-5
BenjaminRodenberg
force-pushed
the
partitioned-mesh-domain-wide-values
branch
from
October 21, 2026 14:48
2de0667 to
fe85e8d
Compare
BenjaminRodenberg
added a commit
that referenced
this pull request
Oct 21, 2026
On several ranks, each rank now computes a contiguous block of the columns of a MATLAB grid, the first block on rank 0. The 2-D fields stay whole on every rank: the slope needs the neighbours of each cell, and the output writer places the columns of all ranks into them. The domain-wide latitude mean, lapse rate and random seed of #203 make the results equal those of a single rank. Taken from 46f47a2 of #197. Assisted-by: Claude Code:claude-opus-5-5
BenjaminRodenberg
added a commit
that referenced
this pull request
Oct 21, 2026
On several ranks, each rank now computes a contiguous block of the columns of a MATLAB grid, the first block on rank 0. The 2-D fields stay whole on every rank: the slope needs the neighbours of each cell, and the output writer places the columns of all ranks into them. The domain-wide latitude mean, lapse rate and random seed of #203 make the results equal those of a single rank. Taken from 46f47a2 of #197. Assisted-by: Claude Code:claude-opus-5-5
BenjaminRodenberg
added a commit
that referenced
this pull request
Oct 21, 2026
On several ranks, each rank now computes a contiguous block of the columns of a MATLAB grid, the first block on rank 0. The 2-D fields stay whole on every rank: the slope needs the neighbours of each cell, and the output writer places the columns of all ranks into them. The domain-wide latitude mean, lapse rate and random seed of #203 make the results equal those of a single rank. Taken from 46f47a2 of #197. Assisted-by: Claude Code:claude-opus-5-5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On
main,mpirun -np N ebfm --elmer-mesh .../partitioning.N --netcdf-dem-mesh dem.nc --is-partitioned-elmer-meshcomputes values that belong to the whole domain from each rank's own partition, so the results depend on the partitioning:LOOP_EBM_SWinselects the water vapour coefficient from the mean latitude of the rank's columns.LOOP_climate_forcingfits the lapse rate of potential temperature (Theta_lapse, used by SHF and LHF) over the rank's columns.--random-seed, every rank seeds NumPy itself, so the random forcing differs between partitions.--dump-referencehas every rank write its own partition to the same file, and--print-diagnosticsreports each rank's own partition.This PR:
grid["column_id"], unique over the whole domain (Elmer element id, or flat index for MATLAB/NetCDF grids), andColumnGather, which collects per-column arrays on rank 0 in id orderinit_grid(domain_mean,grid["lat_mean_whole_domain"]), which now takes EBFM's communicator--random-seedis not given--dump-referenceand--print-diagnosticson rank 0timelastsnow_netCDFas float64, sincedate2numreturns integers for whole days and ranks could disagree on the dtypeAll collectives run over
ebfm_comm, and with one rank each is a no-op: serial MATLAB output (Output/model_output.nc,--dump-reference,--print-diagnostics) is identical tomain. The only change a serial run can see is the dtype oftimelastsnow_netCDFin the restart file, which was an integer when all its times were whole days. Output and restart files are still skipped for partitioned meshes.A new test generates a small Elmer mesh crossing 70 N and a DEM, splits the mesh into 3 partitions, and compares
--dump-referenceofmpirun -np 3against a serial run on the whole mesh. It fails with the per-rank latitude mean or lapse rate ofmain.The pieces are taken from #197 (#197). The MATLAB MPI MVP (#202) will build on this PR.
Generated with Claude Code