Repository navigation
Fix total energy at t=0 in Vlasov-Ampère/Maxwell models (ghost regions, control variate) - #738
Open
max-models wants to merge 4 commits into
Open
max-models wants to merge 4 commits into
max-models wants to merge 4 commits into
Conversation
…s, control variate) Two bugs made total_energy at t=0 inconsistent with the later time steps: 1. The initial field e = -grad(phi) set in post_allocate did not have its ghost regions updated (grad.dot(..., out=e) does not fill them). The first VlasovAmpereCoupling push then evaluated a wrong field near the periodic boundary, so the Crank-Nicolson step did not conserve energy in step 1 only (full-f jump of ~2e-4 relative on 8 elements). Now synced in VlasovAmpereOneSpecies, VlasovMaxwellOneSpecies, ColdPlasmaVlasov and LinearVlasovAmpereOneSpecies. 2. With the control variate, post_allocate reset the weights to the full-f weights0 after the initial Poisson solve, so kinetic_energy at t=0 was the full-f energy and from step 1 on the delta-f energy (total_energy dropped from ~19 to ~0.1). The weights are now reset to weights0 only without the control variate (VlasovAmpereOneSpecies, VlasovMaxwellOneSpecies incl. its Gauss-law diagnostic). ColdPlasmaVlasov never reset them, so full-f runs used delta-f weights; it now follows the same rule. Adds test_energy_conservation (full-f and control variate) to the VlasovAmpereOneSpecies verification tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With loading="sobol_standard"/"sobol_antithetic" every process draws its markers on the whole unit cube (pseudo-random loading keeps only those on the process domain). They were only sent to their owning process when box sorting was enabled (SortingParameters(do_sort=True, boxes_per_dim=...)). Without it, the initial charge accumulation in the Poisson solve of VlasovAmpereOneSpecies wrote outside the local stencil arrays (fill_vec index -2 / 14 for a local range 0..13, seen with pyccel --debug), corrupting the heap. On Linux this aborted the 2-rank verification job (Fatal Python error: Aborted) in the new test_energy_conservation, which is the only MPI test loading Sobol markers without box sorting. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Found while writing the GPU end-to-end test of
VlasovAmpereOneSpecies(#734). Setup:VlasovAmpereOneSpecies(alpha=1, epsilon=-1, with_B0=False), Cuboidr1=12.56, 8x1x1 elements, degree (3,1,1),ppc=40sobol_standard, Maxwellian background,ModesCosdensity perturbation of amplitude 0.1,dt=0.1, 3 steps, solver tolerances 1e-12, initial Poisson withstab_eps=1e-6.total_energy(electric_energy + kinetic_energy):Full-f: the Crank-Nicolson coupling conserved the total to round-off from step 1 on, but step 1 itself jumped by 1.6e-4 relative. Control variate: the t=0 value was a different quantity than the later values.
Root causes and fix
Ghost regions of the initial E field.
post_allocatesetse = -grad(phi)withderham.grad.dot(-phi, out=e), which does not fill the ghost regions. The pusher inVlasovAmpereCouplingevaluatese_mid = (e^n + e^{n+1})/2at the markers, ghost cells included, so in the first step it pushed with a wrong field near the periodic boundary. The field solve itself does not read ghost cells, so the field and the velocities were no longer consistent and energy was not conserved. From step 2 on,ecomes fromupdate_feec_variables, which syncs the ghost regions. This changes the physics of the first step, not only the diagnostic. Fix:e.update_ghost_regions()after the initial Poisson solve inVlasovAmpereOneSpecies,VlasovMaxwellOneSpecies,ColdPlasmaVlasovandLinearVlasovAmpereOneSpecies. These are all the models that write-grad(phi)intoe_fieldthis way.Control-variate weights reset after the initial Poisson solve.
initialize_weightsalready sets delta-f weights whencontrol_variate=True.post_allocatecallsupdate_weights()so that the Poisson right-hand side is the charge off - f0, and then setweights = weights0without checking the control variate. That left full-f weights for the t=0 diagnostic. The firstPushEtathen calledupdate_weights(), so from step 1 on the weights were delta-f again. Fix: reset toweights0only without the control variate. The same change is applied to theVlasovMaxwellOneSpeciesGauss-law diagnostic, which restored full-f weights at every scalar update.ColdPlasmaVlasovhad the opposite problem: it never reset, so its full-f runs used delta-f weights for the whole run. It now follows the same rule.Sobol-loaded markers were not sent to their MPI process (
pic/base.py). Withloading="sobol_standard"/"sobol_antithetic"every process draws its markers on the whole unit cube, butmpi_sort_markerswas only called when box sorting was enabled. Without it, the first accumulation (the initial Poisson solve) wrote outside the local stencil arrays and corrupted the heap. The newtest_energy_conservationis the only MPI test that loads Sobol markers without box sorting, and it made the next test abort the 2-rank verification job on Linux (Fatal Python error: Aborted). On 2 ranks, half of each rank's 3200 markers were outside its domain before the fix and none after. Fix: callmpi_sort_markers()after a Sobol draw when there is an MPI communicator and box sorting is off.With the control variate,
kinetic_energyis now the kinetic energy off - f0at all times, including t=0. This is now stated in theKineticEnergyPICdocstring, and it matches whatLinearMHDVlasovCCandToyDriftalready did.Expected behaviour (no code change)
With the control variate, the delta-f total energy is not conserved to round-off, even after this fix. The Crank-Nicolson step conserves
E_E + α²/2 Σ w_δ |v|²with the weights frozen during the step. Afterwardsupdate_weightschangesw_δby-Δf0(v)/(s0 N), which adds two things per step:∝ Σ f0 (v·E)|v|²), andO(Δt²)term: the energy the field gives to the background markers.Measured on 16 elements with
ppc=4000up to T=0.3, the drift is 3.0e-3, 1.2e-3 and 2.7e-4 for dt = 0.1, 0.05 and 0.025. So it converges with dt, and it is a property of the scheme, not of the diagnostic.Affected models
VlasovAmpereOneSpecies,VlasovMaxwellOneSpeciesandColdPlasmaVlasov(ghost regions + weights),LinearVlasovAmpereOneSpecies(ghost regions only; its markers are delta-f by construction).Tests
test_energy_conservation[False/True]inmodels/tests/verification/test_verif_VlasovAmpereOneSpecies.py. It uses the setup above with 16 elements, so that the CI run on 4 ranks with clones gets at least p=3 elements per rank. ~2 s each.ppc=40: drift from t=0 must be< 1e-10. On devel it is 9.4e-7; after the fix it is round-off.ppc=400: drift must be< 0.1, for the reason above. On devel it is 0.99; after the fix it is 3.7e-3.mpirun -n 2 --with-mpi,mpirun -n 4 --with-mpiandmpirun -n 4 --with-mpi --nclones 2.test_weak_Landau(control variate),test_verif_LinearVlasovAmpereOneSpecies.py,test_verif_VlasovMaxwellOneSpecies.py, and the default-parameter runs (test_single_model) ofVlasovAmpereOneSpecies,VlasovMaxwellOneSpeciesandLinearVlasovAmpereOneSpecies.ColdPlasmaVlasovrun takes more than 5 minutes on devel as well. I stopped it, so CI covers it.Note for #734
The GPU e2e test of
VlasovAmpereOneSpeciesin #734 currently uses full-f weights and only a loose t=0 check (|en_tot[1] - en_tot[0]| < 1e-3) because of these two bugs. Once this is merged, it can checktotal_energyfrom t=0 at solver tolerance, and it can also use the control variate.🤖 Generated with Claude Code