Skip to content

Share common Scatterer between Vectors on same FunctionSpace - #4293

Open
chrisrichardson wants to merge 18 commits into
mainfrom
chris/vector-share-scatterer
Open

Share common Scatterer between Vectors on same FunctionSpace#4293
chrisrichardson wants to merge 18 commits into
mainfrom
chris/vector-share-scatterer

Conversation

@chrisrichardson

@chrisrichardson chrisrichardson commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Resolves #3065 by adding a Scatterer to FunctionSpace. All Functions created on a FunctionSpace now create a Vector with the same shared Scatterer.
Because all Vectors in the same FunctionSpace have the same parallel layout, a shared Scatterer should always work. The MPI_Request and the actual data, required during a scatter operation are external to the Scatterer so should not be affected by sharing the Scatterer itself.

@garth-wells

Copy link
Copy Markdown
Member

Can you describe the fix? Issue #3065 presents several options, with pros and cons.

@chrisrichardson
chrisrichardson marked this pull request as ready for review July 21, 2026 13:10
chrisrichardson and others added 10 commits July 21, 2026 14:23
- Function::collapse() built its new Vector via the two-argument
  (map, bs) constructor, which constructs a fresh Scatterer instead
  of reusing the one already built by the collapsed FunctionSpace.
  Use the FunctionSpace-based Vector constructor instead.
- Mark the new FunctionSpace-based Vector constructor explicit to
  match the existing converting-constructor convention and avoid
  accidental implicit conversions.
- Remove a stray blank line before the first FunctionSpace
  constructor's body.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The point-to-point scatter_fwd_begin/scatter_rev_begin overloads
share communicator _comm0 for both directions, but used a fixed send
tag (0) paired with MPI_ANY_TAG on the matching receive. That relies
on messages being posted in the same relative order on every rank to
avoid a forward message being matched against a reverse receive (or
vice versa) - an assumption that no longer holds now that a Scatterer
can be shared across sibling Vectors on the same FunctionSpace, which
allows independent scatters to be interleaved in either order.

Give forward and reverse each a fixed, distinct tag instead, so
message matching no longer depends on posting order.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
MPI neighbourhood collectives are the only communication mode now
used, simplifying the Scatterer interface and removing the unused
point-to-point code path and its dedicated MPI tags.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread cpp/dolfinx/la/Vector.h Outdated
@garth-wells

Copy link
Copy Markdown
Member

I'm not entirely sure that the FunctionSpace is the right place for the scatterer. The scatter is related to where the data in a vector is stored, which is really a property of the Vector.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Associate a common::Scatterer with fem::FunctionSpace

2 participants