PetscShmgetAllocateArray#

allocates shared memory accessible by all MPI processes in the server

Synopsis#

PetscErrorCode PetscShmgetAllocateArray(size_t sz, size_t asz, void **addr)

Not Collective, only called on the first MPI process

Input Parameters#

sz - the number of elements in the array
asz - the size of an entry in the array, for example sizeof(PetscScalar)

Output Parameters#

addr - the address of the array

Notes#

Uses PetscMalloc() if PETSC_HAVE_SHMGET is not defined or the MPI linear solver server is not running

Sometimes when a program crashes, shared memory IDs may remain, making it impossible to rerun the program. Use

  $PETSC_DIR/lib/petsc/bin/petscfreesharedmemory

will also free the memory.

Use the Unix command ipcs -m to see what memory IDs are currently allocated and ipcrm -m ID to remove a memory ID

Under Apple macOS the following file must be copied to /Library/LaunchDaemons/sharedmemory.plist (ensure this file is owned by root and not the user) and the machine rebooted before using shared memory

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
 <key>Label</key>
 <string>shmemsetup</string>
 <key>UserName</key>
 <string>root</string>
 <key>GroupName</key>
 <string>wheel</string>
 <key>ProgramArguments</key>
 <array>
 <string>/usr/sbin/sysctl</string>
 <string>-w</string>
 <string>kern.sysv.shmmax=4194304000</string>
 <string>kern.sysv.shmmni=2064</string>
 <string>kern.sysv.shmseg=2064</string>
 <string>kern.sysv.shmall=131072000</string>
  </array>
 <key>KeepAlive</key>
 <false/>
 <key>RunAtLoad</key>
 <true/>
</dict>
</plist>

Use the command

  /usr/sbin/sysctl -a | grep shm

to confirm that the shared memory limits you have requested are available.

Fortran Note#

The calling sequence is PetscShmgetAllocateArray[Scalar,Int](PetscInt start, PetscInt len, Petsc[Scalar,Int], pointer :: d1(:), ierr)

Developer Note#

More specifically this uses PetscMalloc() if !PCMPIServerUseShmget || !PCMPIServerActive || PCMPIServerInSolve where PCMPIServerInSolve indicates that the solve is nested inside a MPI linear solver server solve and hence should not allocate the vector and matrix memory in shared memory.

Level#

developer

Location#

src/sys/utils/server.c

Index of all Sys routines
Table of Contents for all manual pages
Index of all manual pages