Changes: Development

General:

• Change behavior of -options_left when set to true: it no longer triggers a call to PetscOptionsView()

• Change \(PETSC_DIR/lib/petsc/bin/petscfreesharedmemory to \)PETSC_DIR/lib/petsc/bin/petscfreesharedmemory.sh

Configure/Build:

• Make SYCL a devicePackage, i.e., builds --with-sycl now have PETSC_HAVE_DEVICE defined

• Add the option --with-devicelanguage to compile PetscDevice code using either a C or C++ compiler

• Add Caliper, an instrumentation and performance profiling library that can be used to profile Hypre.

• Add typing stubs for the Python extension module petsc4py.PETSc.

• Remove --with-mumps-serial option, rely on --with-mpi=0

Sys:

• Add PetscCallHYPRE() to check HYPRE error codes and print error messages on failure

• Add PetscBTCountSet() to count set bits in PetscBT

• Add the option -hypre_umpire_device_pool_size <n> to set the Umpire device memory pool size (in MiB), which is used by HYPRE and 4 Gib by default

• Change PetscStackCallExternalVoid() to PetscCallExternalVoid()

Event Logging:

• Add two approaches for GPU energy monitoring: -log_view_gpu_energy and -log_view_gpu_energy_meter

• Add API PetscLogGpuEnergy(), PetscLogGpuEnergyMeter(), PetscLogGpuEnergyMeterBegin() and PetscLogGpuEnergyMeterEnd() for GPU energy monitoring

• Remove TAO_ObjectiveEval, TAO_GradientEval, TAO_ObjGradEval, TAO_HessianEval log events; objective, gradient, objective-and-gradient, and Hessian evaluations are now logged as TAOTERM_ObjectiveEval, TAOTERM_GradientEval, TAOTERM_ObjGradEval, TAOTERM_HessianEval under TAOTERM_CLASSID. These log events reflect the actual user callback invoked, not the TaoCompute*() function called by the solver; for example, calling TaoComputeGradient() when only an objective-and-gradient callback is provided logs TAOTERM_ObjGradEval

• Add TAO_ResidualEval log event for TaoComputeResidual(); previously TaoComputeResidual() reused TAO_ObjectiveEval

PetscViewer:

• Change the final argument of PetscViewerGLVisSetFields() to PetscCtxDestroyFn *. This means the destroy function must dereference the argument before operating on it

PetscDraw:

AO:

IS:

VecScatter / PetscSF:

PF:

Vec:

• Add VecNestGetSubVecsRead() and VecNestRestoreSubVecsRead() for read-only access to subvectors

• Add VecPointwiseSign() and VecSignMode

PetscSection:

• Add PetscSectionMigrateData(), akin to DMPlexDistributeData()

PetscPartitioner:

Mat:

• Add MatCreateDenseWithMemType() and MatDenseReplaceArrayWithMemType()

• Change the destroy() function argument of MatShellSetMatProductOperation() to type PetscCtxDestroyFn *. This means the destroy function must dereference the argument before operating on it

• Remove MatMissingDiagonal(). Developers should use MatGetDiagonalMarkers_SeqXXX() when the functionality is needed

• Change MatSetOption(A, MAT_HERMITIAN, PETSC_TRUE) for MatSBAIJ to no longer automatically set the option MAT_SYMMETRIC to PETSC_FALSE. It is now the duty of the user to call MatSetOption(A, MAT_SYMMETRIC, PETSC_FALSE) if a MatSBAIJ is Hermitian but not symmetric

• Deprecate -matmatmult_Bbn in favor of -matproduct_batch_size

• Add MatADot() and MatANorm()

• Add -mat_vec_type option for MatSetFromOptions()

MatCoarsen:

PC:

• Add multi-precision support for MUMPS. One could use -pc_precision <single, double> to set the precision to be used by MUMPS, which can be different from PetscScalar’s precision

• Add support for MUMPS out-of-core facility with the option -mat_mumps_ooc_tmpdir <dir> and new functions MatMumpsSetOocTmpDir(), MatMumpsGetOocTmpDir()

• Add PCPatchSetComputeOperatorExteriorFacets() and PCPatchSetComputeFunctionExteriorFacets() to support exterior (boundary) facet integral callbacks in the patch preconditioner

KSP:

• Change KSPSetDMActive() to take a KSPDMActive argument

• Remove KSPHPDDMPrecision in favor of PetscPrecision

• Deprecate KSPPIPEGCRSetModifyPC(), KSPGCRSetModifyPC(), and KSPFGMRESSetModifyPC() in favor of KSPFlexibleSetModifyPC()

• Deprecate KSPFGMRESModifyPCNoChange() and KSPFGMRESModifyPCKSP() in favor of KSPFlexibleModifyPCNoChange() and KSPFlexibleModifyPCKSP()

SNES:

• Change the destroy() function argument of SNESSetConvergenceTest() to type PetscCtxDestroyFn *. This means the destroy function must dereference the argument before operating on it

• Add SNESSetObjectiveDomainError()

• Change SNES_DIVERGED_FNORM_NAN to SNES_DIVERGED_FUNCTION_NANORINF

• Add SNES_DIVERGED_OBJECTIVE_NANORINF

• Add SNES_DIVERGED_OBJECTIVE_DOMAIN

• Add developer functions SNESCheckFunctionDomainError(), SNESLineSearchCheckFunctionDomainError(), SNESCheckObjectiveDomainError(), SNESLineSearchCheckObjectiveDomainError(), SNESCheckJacobianDomainError(), and SNESLineSearchCheckJacobianDomainError()

• Add SNESNewtonALSetDiagonalScaling to allow for per-DoF scaling of solution vector when computing arc length

SNESLineSearch:

TS:

• Add TSPseudoComputeFunction() to get nonlinear residual while avoiding recalculation if possible

• Remove unused TSPseudoVerifyTimeStepDefault()

• Remove TSPseudoComputeTimeStep() and TSPseudoVerifyTimeStep()

• Change the destroy() function argument of TSTrajectorySetTransform() to type PetscCtxDestroyFn *. This means the destroy function must dereference the argument before operating on it

• Correct option -ts_max_reject to -ts_max_step_rejections

• Correct option -ts_dt to -ts_time_step

• Change TSAdaptCheckStage() to call function set by TSAdaptSetCheckStage() before other checks

• Fix -ts_ssp_nstages to -ts_ssp_num_stages

TAO:

• Add TaoTerm object to allow easily constructing and using objective functions, and their derivatives that are the sum of two or more terms. This will allow the easy implementation of Tao solvers that utilize the sum structure

• Add TaoGetTerm() and TaoAddTerm() for manipulating the objective, gradient, and Hessian evaluation of a Tao using TaoTerm

• Add TaoGetHessianMatrices() to get the Hessian and preconditioner matrices from a Tao

• Fix TAOBRGN such that TAOBRGN subsolver properly appends TAOBRGN’s prefix

• Change TaoComputeResidual() to increment a new residual evaluation counter instead of the objective function evaluation counter

• Change -tao_view to report residual evaluations separately and to include matrix-free finite-difference gradient evaluations in the gradient evaluation count

TaoTerm:

• Add TAOTERMCALLBACKS implementation of TaoTerm constructed from the function callbacks passed to a Tao object

• Add TAOTERMSHELL implementation of TaoTerm for user-defined objective routines

• Add TAOTERMSUM implementation of TaoTerm for scaled, mapped sums of terms

• Add TAOTERMHALFL2SQUARED implementation of TaoTerm for a squared-norm penalty function

• Add TAOTERML1 implementation of TaoTerm for a 1-norm penalty function

• Add TAOTERMQUADRATIC implementation of TaoTerm for a quadratic penalty function

• Add finite difference utilities: TaoTermComputeGradientFD(), TaoTermComputeHessianFD(), TaoTermCreateHessianMFFD(), and set/get functions TaoTermComputeGradientSetUseFD()/TaoTermComputeGradientGetUseFD(), TaoTermComputeHessianSetUseFD()/TaoTermComputeHessianGetUseFD()

PetscRegressor:

PetscDA:

• Add data assimilation object PetscDA with two ensemble-based implementations PETSCDAETKF and PETSCDALETKF and three tutorials: Lorenz-96, traveling wave, and dam break

• Move localization PetscDALETKFGetLocalizationMatrix() utility out of DMPLEX and into PETSCDALETKF

DM:

• Change the final argument of DMShellSetDestroyContext() to PetscCtxDestroyFn *. This means the destroy function must dereference the argument before operating on it

• Add DMLabelGetValueISGlobal() to get globally consistent IS of values in a DMLabel

DMSwarm:

DMPlex:

• Add DMPlexVecGetClosureAtDepth()

• Add an extra communicator argument to DMPlexFilter() to allow extracting local meshes

• Change verticesAdjSaved parameter in DMPlexCreateFromCell*Parallel*() functions to be allocated by function rather than by user

• Add DMPlexCreateColoring()

FE/FV:

DMNetwork:

DMStag:

DT:

Fortran:

• Replace ./configure option --with-mpi-f90module-visibility with --with-mpi-ftn-module=<mpi or mpi_f08>

• Add PETSC_INT_KIND and PETSC_MPIINT_KIND

• Fortran code should now use MPIU_Comm instead of MPI_Comm, and similarly for other MPI types, see section “Fortran and MPI” in the users guide

• Fortran interface definitions are now automatically generated for all functions that take context variable arguments, represented in the C source code with a type of PetscCtx, allowing the use of any Fortran derived type (or PETSc object) as the context

• For all PETSc functions XXXGetYYY() that return a context variable as an argument, represented in the C source code with an argument type of PetscCtxRt, a macro is generated used with Interface_XXXGetYYY(AppCtx) which tells the Fortran compiler that a pointer to that derived type type(AppCtx) is returned from the Fortran version of XXXGetYYY(). See src/snes/tutorials/ex5f90.F90`