PetscSFRegisterPersistent#

Register root and leaf data as memory regions that will be used for repeated PetscSF communications.

Synopsis#

#include "petscsf.h" 
PetscErrorCode PetscSFRegisterPersistent(PetscSF sf, MPI_Datatype unit, const void *rootdata, const void *leafdata)

Collective

Input Parameters#

sf - star forest
unit - the data type contained within rootdata and leafdata
rootdata - root data that will be used for multiple PetscSF communications
leafdata - leaf data that will be used for multiple PetscSF communications

Notes#

Implementations of PetscSF can make optimizations for repeated communication using the same memory regions, but these optimizations can be unsound if rootdata or leafdata is deallocated and the PetscSF is not informed. The intended pattern is

  PetscMalloc2(nroots, &rootdata, nleaves, &leafdata);

  PetscSFRegisterPersistent(sf, unit, rootdata, leafdata);
  // repeated use of rootdata and leafdata will now be optimized

  PetscSFBcastBegin(sf, unit, rootdata, leafdata, MPI_REPLACE);
  PetscSFBcastEnd(sf, unit, rootdata, leafdata, MPI_REPLACE);
  // ...
  PetscSFReduceBegin(sf, unit, leafdata, rootdata, MPI_SUM);
  PetscSFReduceEnd(sf, unit, leafdata, rootdata, MPI_SUM);
  // ... (other communications)

  // rootdata and leafdata must be deregistered before freeing
  // skipping this can lead to undefined behavior including
  // deadlocks
  PetscSFDeregisterPersistent(sf, unit, rootdata, leafdata);

  // it is now safe to free rootdata and leafdata
  PetscFree2(rootdata, leafdata);

If you do not register rootdata and leafdata it will not cause an error, but optimizations that reduce the setup time for each communication cannot be made. Currently, the only implementation of PetscSF that benefits from PetscSFRegisterPersistent() is PETSCSFWINDOW. For the default PETSCSFBASIC there is no benefit to using PetscSFRegisterPersistent().