Changes: Development

General:

Configure/Build:

• Add --download-cutest and --download-sifdecode for the CUTEst optimization testing environment and SIF problem decoder

• Add Meson package builds, --download-meson, and --download-package-meson-arguments for additional Meson setup arguments

• Add --download-ninja and --with-ninja-exec for the Ninja backend used by Meson package builds

• Add --with-meson-exec to select an existing Meson executable

• Add --with-cmake-debug-find to control whether --debug-find is passed to the CMake configure of downloaded packages; it is on by default and needs CMake 3.17 or newer

• Change OpenMP target offload for CUDA/HIP to require --with-openmp-target-offload instead of being enabled automatically

• Build petsc4py, mpi4py, and slepc4py with -O0 to shorten build time; the binding code is no longer optimized

• Fix --with-shared-libraries=1 on Microsoft Windows with Intel ifx, where Fortran code failed to link

Sys:

• Add PETSC_TLS to declare a variable thread-local when configured with --with-threadsafety=1

• Maintain a separate PetscStack for each thread when configured with --with-debugging=1 --with-threadsafety=1

• Add topology-aware GPU binding using hwloc; Requires hwloc version 2.0 or later

• Add PetscCommCreateNonempty() and PetscCommDestroyNonempty() for external libraries, such as ParMETIS and PT-SCOTCH, that require every process of their communicator to own data

• Change PETSC_PI, PETSC_PHI, PETSC_SQRT2, PETSC_E, PETSC_MAX_REAL, PETSC_REAL_MIN, PETSC_MACHINE_EPSILON, PETSC_SQRT_MACHINE_EPSILON, and PETSC_SMALL from macros to static const PetscReal objects, constexpr in C++ so they can be used in constant expressions and device code, and likewise PETSC_MIN_REAL, PETSC_INFINITY, and PETSC_NINFINITY except in C with the Microsoft, Intel classic, and NVHPC compilers, where they remain macros

• Change PETSC_REAL_MIN in double precision from the largest subnormal value to DBL_MIN, its documented smallest positive normalized value

• Change PETSC_SQRT_MACHINE_EPSILON in double precision to the exactly representable 1.4901161193847656e-08, one ulp below the previous value; finite-difference step sizes change accordingly

• Reduce the include dependencies of many public headers

• Several public headers now include only the corresponding petsc*types.h instead of the full API header: petscdmproduct.h, petscdmredundant.h and petscspace.h no longer provide the DM API, petsctaolinesearch.h no longer provides the Tao API, petscdmda.h, petscfeceed.h and petscfvceed.h no longer provide the PetscFE/PetscFV APIs, and petsclayouthdf5.h no longer provides the IS API; code that obtained those APIs through one of these headers must now include the API header as well

• petscsys.h no longer includes petscpkg_version.h; include it directly to use the PETSC_PKG_*_VERSION_*() macros

• petscviewer.h, petscoptions.h, petsclog.h, petscstring.h, petsctime.h, and petscdevicetypes.h no longer include petscsys.h (just include petscsys.h instead of these individual files), and petscdstypes.h no longer provides the DMLabel API

• PetscCitationsList is no longer declared in a public header; register citations with PetscCitationsRegister()

• Move the PETSc memory-management API (PetscMalloc(), PetscFree(), and related routines) into the new header petscmem.h; the API remains available through petscsys.h

• Move the PetscPreLoad* functionality into petscpreload.h

• Remove the --with-serialize-functions configure option and the associated function-pointer serialization feature (PETSC_SERIALIZE_FUNCTIONS)

Event Logging:

• Change the CpuToGpu Count and GpuToCpu Count columns of -log_view to report the maximum over the processes instead of the average, which rounded to 0 whenever the total number of copies was less than half the number of processes; the size columns remain averages

PetscViewer:

PetscDraw:

AO:

IS:

VecScatter / PetscSF:

PF:

Vec:

• Add a device implementation of VecSqrtAbs() for VECKOKKOS; previously it copied the vector to the host

• Change VecGetState() to return a VecState and add VecStateCompare(), VecStateCompareUpdate(), and VecStateInvalidate()

• Make VecStrideMin() and VecStrideMax() propagate NaNs with the smallest global NaN index, handle IEEE infinities with empty ranks, and return the same value with or without an index

PetscSection:

PetscPartitioner:

Mat:

• Add device implementations of MatNorm() with NORM_1, NORM_FROBENIUS, and NORM_INFINITY for MATAIJKOKKOS; previously all norms copied the matrix values to the host

• Add -mat_spd to set MAT_SPD from the options database in MatSetFromOptions()

• Change MatSetOption() to error when a symmetry-related option contradicts an already known property

• Add support for MatZeroRows() and MatZeroRowsColumns() with a nonzero diagonal value on a MATIS whose local matrix does not implement MatSetValues(), such as MATSHELL

• Change MatSetValuesCOO() for MATAIJ and its device subtypes to generate an error when the nonzero pattern has changed since MatSetPreallocationCOO(), for example because MatSetValues() inserted a new nonzero; values were previously written into wrong locations in such a sequence of events

• Change MatAXPY() with DIFFERENT_NONZERO_PATTERN for MATSEQAIJ and MATMPIAIJ to add in place when the nonzeros of X are a subset of those of Y, keeping the nonzero pattern and nonzero state of Y, as well as the options that survive assembly, such as MAT_SYMMETRY_ETERNAL

• Add MatGetRowSumAbs() for MATHYPRE, on the host and the device, and device implementations of MatGetRowSumAbs() for MATAIJCUSPARSE, MATAIJHIPSPARSE, and MATAIJKOKKOS; previously the AIJ device types copied the matrix values to the host

• Change MatSetRandom() to no longer assemble at the interface; MATOP_SETRANDOM implementations must leave the matrix assembled with increased state

• Change MatCreateSubMatricesMPI() to return parallel submatrices of the same MATAIJ subtype as the input matrix, for example MATAIJCUSPARSE, instead of always MATMPIAIJ

• Add device support for MATPRODUCT_AtB with a MATMPIAIJCUSPARSE, MATMPIAIJHIPSPARSE, or MATMPIAIJKOKKOS matrix and a MATMPIDENSECUDA or MATMPIDENSEHIP matrix, also with -matproduct_batch_size; previously a MATMAIJ product copied the dense matrices to the host

• Change MATPRODUCT_AtB of a MATDIAGONAL matrix and a MATDENSECUDA or MATDENSEHIP matrix to scale the rows on the device; previously it looped MatMultTranspose() over the columns

MatCoarsen:

PC:

• Add PCHPDDMSetScaling() to supply a diagonal scaling Vec for the deflation space set with PCHPDDMSetDeflationMat(). If the fine level is a PCASM, its default PCASMType is set to PC_ASM_WEIGHTED, in which case the scaling is forwarded accordingly

• Change interface to PCPatchSetDiscretisationInfo() to address subspace DoFs using ISs instead of with offsets and block sizes

KSP:

• Add KSPGMRESSetVectorAllocationSize() to indicate how many additional vectors are allocated each time more restart vectors are allocated for GMRES implementations

• Add KSPCGTRNormType, KSPCGSetTRNormType(), and KSPCGGetTRNormType() to select the trust-region norm in KSPCG, KSPSTCG, KSPNASH, and KSPGLTR

• Deprecate KSPCGGetNormD() in favor of KSPCGGetTRSolutionNorm() and -ksp_cg_dtype in favor of -ksp_cg_tr_norm_type

• Implement KSPSTCG as a specialization of KSPCG, sharing trust-region norms, feasible Euclidean initial guesses, and objective targets

• Add -ksp_cg_radius radius and KSPCGGetTRSolutionNorm() support to KSPCG

• Add KSPBuildSolutionFlexibleDefault() for use by flexible KSPType implementations as their buildsolution operation

SNES:

• Change interface to SNESPatchSetDiscretisationInfo() to address subspace DoFs using ISs instead of with offsets and block sizes

SNESLineSearch:

TS:

TAO:

• Add support for CUTEst unconstrained problems in TAO tutorials and tests

• Map TAOSNES convergence and divergence reasons with the public TaoConvergedReasonFromSNES(), and add TaoSNESGetSNES() to access the underlying solver

TaoTerm:

PetscRegressor:

PetscDA:

DM:

DMSwarm:

DMPlex:

FE/FV:

DMNetwork:

DMStag:

DT:

Fortran:

• PETSC_PI, PETSC_MAX_REAL, PETSC_MIN_REAL, PETSC_MACHINE_EPSILON, PETSC_SQRT_MACHINE_EPSILON, PETSC_INFINITY, and PETSC_NINFINITY are now parameter constants and can be used in constant expressions

• Add PETSC_PHI, PETSC_SQRT2, and PETSC_E

• PETSC_SMALL is now protected and cannot be modified by user code