=============
Changes: 3.21
=============

..
   STYLE GUIDELINES:
   * Capitalize sentences
   * Use imperative, e.g., Add, Improve, Change, etc.
   * Don't use a period (.) at the end of entries
   * If multiple sentences are needed, use a period or semicolon to divide sentences, but not at the end of the final sentence

.. rubric:: General:

- Add single precision support for using HYPRE and MFEM
- Require Fortran 2003 compiler with GET_COMMAND_ARGUMENT() support for building PETSc with Fortran bindings

.. rubric:: Configure/Build:

- Add ``--download-blis-use-openmp=0`` to force ``download-blis`` to not build with OpenMP when ``with-openmp`` is provided
- Add ``PetscBLASSetNumThreads()`` and ``PetscBLASGetNumThreads()`` for controlling how many threads the BLAS routines use
- Change ``win_cl`` and similar ``win32fe`` compiler wrappers to ``win32fe_cl``
- Add build support for Intel oneAPI compilers ``icx`` and ``ifx`` on Microsoft Windows with compiler wrappers ``win32fe_icx`` and ``win32fe_ifx`` (only static library build with ``ifx``)
- Add lib/petsc/bin/maint/runjobs.py list-of-jobs script to allow submitting a subset of the jobs to the CI

.. rubric:: Sys:

- Add ``PetscBench`` an object class for managing benchmarks in PETSc
- Deprecate ``PetscVoidFunction``, ``PetscVoidStarFunction``, and ``PetscErrorCodeFunction`` typedefs in favor of
  ``PetscVoidFn`` and ``PetscErrorCodeFn``
- Add ``PetscOptionsBoundedReal()`` and ``PetscOptionsRangeReal()``
- Rename Petsc stream types to ``PETSC_STREAM_DEFAULT``, ``PETSC_STREAM_NONBLOCKING``, ``PETSC_STREAM_DEFAULT_WITH_BARRIER`` and ``PETSC_STREAM_NONBLOCKING_WITH_BARRIER``. The root device context uses ``PETSC_STREAM_DEFAULT`` by default

.. rubric:: Event Logging:

.. rubric:: PetscViewer:

- Change ``PetscViewerRestoreSubViewer()`` to no longer need a call to ``PetscViewerFlush()`` after it
- Introduce ``PetscOptionsRestoreViewer()`` that must be called after ``PetscOptionsGetViewer()`` and ``PetscOptionsGetViewers()``
  to ensure thread safety
- Add ``PetscViewerASCIIWORLDSetFileUnit()``

.. rubric:: PetscDraw:

.. rubric:: AO:

.. rubric:: IS:

- Add ``ISLocalToGlobalMappingGetNodeInfo()`` and ``ISLocalToGlobalMappingRestoreNodeInfo()`` to access neighboring information of local indices
- Add support to load an ``ISLocalToGlobalMapping`` via ``ISLocalToGlobalMappingLoad()`` from data previously stored using ``ISLocalToGlobalMappingView()``
- Add ``ISLocalToGlobalMappingGetBlockMultiLeavesSF()`` to share local block data

.. rubric:: VecScatter / PetscSF:

- Add MPI-4.0 persistent neighborhood collectives support. Use -sf_neighbor_persistent along with -sf_type neighbor to enable it
- Add ``PetscSFCreateStridedSF()`` to communicate strided blocks of data

.. rubric:: PF:

.. rubric:: Vec:

- Add ``VecGhostGetGhostIS()`` to get the ghost indices of a ghosted vector
- Add ``-vec_mdot_use_gemv`` to let ``VecMDot()``, ``VecMTDot()``  use BLAS2 ``gemv()`` instead of custom unrolled kernel. Default is on
- Add ``-vec_maxpy_use_gemv`` to let ``VecMAXPY()`` use BLAS2 ``gemv()`` instead of custom unrolled kernel. Default is off
- ``VecReplaceArray()`` on the first Vec obtained from ``VecDuplicateVecs()`` with either of the two above \*_use_gemv options won't work anymore. If needed, turn them off or use ``VecDuplicateVec()`` instead
- ``VecScale()`` is now a logically collective operation
- Add ``VecISShift()`` to shift a part of the vector
- ``VecISSet()`` does no longer accept NULL as index set
- ``VecLoad()`` automatically determines whether the file was written using 32-bit or 64-bit indices, and files can read with PETSc built either way

.. rubric:: PetscSection:

- Add ``PetscSectionGetBlockStarts()`` and ``PetscSectionSetBlockStarts()``
- Add argument to ``PetscSectionCreateGlobalSection()`` that can ignore the local section permutation
- Add ``PetscSectionCreateComponentSubsection()``

.. rubric:: PetscPartitioner:

- Add edge weights to ``PetscPartitionerPartition()``

.. rubric:: Mat:

- Reset ``MATLMVM`` history vectors if size is changed
- Add specific support for ``MatMultHermitianTranspose()`` and ``MatMultHermitianTransposeAdd()`` in ``MATSHELL``, ``MATDENSE``, ``MATNEST``, and ``MATSCALAPACK``
- Add function ``MatProductGetAlgorithm()``
- ``MATTRANSPOSEVIRTUAL``, ``MATHERMITIANTRANSPOSEVIRTUAL``, ``MATNORMAL``, ``MATNORMALHERMITIAN``, and ``MATCOMPOSITE`` now derive from ``MATSHELL``. This implies a new behavior for those ``Mat``, as calling ``MatAssemblyBegin()``/``MatAssemblyEnd()`` destroys scalings and shifts for ``MATSHELL``, but it was not previously the case for other ``MatType``
- Add function ``MatGetRowSumAbs()`` to compute vector of L1 norms of rows ([B]AIJ only)
- Add partial support for ``MatBackwardSolve()``/``MatForwardSolve()`` with ``MATSOLVERMKL_PARDISO`` and ``MATSOLVERMKL_CPARDISO``
- Deprecate ``MATIS`` options ``-matis_xxx``. Use ``-mat_is_xxx``
- Add support for repeated entries in the local part of the local to global map for ``MATIS`` via the routines ``MatISSetAllowRepeated()`` and ``MatISGetAllowRepeated()``
- Add support to dump and load a matrix of ``MATIS`` type
- Add dense representations of symmetric Broyden matrices ``MATLMVMDBFGS``, ``MATLMVMDDFP``, and ``MATLMVMDQN``
- Add ``MATSELLHIP`` and the corresponding HIP kernels for matrix-vector multiplication

.. rubric:: MatCoarsen:

- Add ``MatCoarsenSetMaximumIterations()`` with corresponding option ``-mat_coarsen_max_it <4>``. The number of iteration of the coarsening method. Used for the HEM coarsener
- Add ``MatCoarsenSetThreshold()`` with corresponding option ``-mat_coarsen_threshold <-1>``. Threshold for filtering graph for HEM. Like GAMG < 0 means no filtering
- Change API for several PetscCD methods used internally in ``PCGAMG`` and ``MatCoarsen`` (eg, change ``PetscCDSetChuckSize()`` to ``PetscCDSetChunckSize()``), remove ``Mat`` argument from``PetscCDGetASMBlocks()``

.. rubric:: PC:

- Add ``PCGAMGSetLowMemoryFilter()`` with corresponding option ``-pc_gamg_low_memory_threshold_filter``. Use the system ``MatFilter`` graph/matrix filter, without a temporary copy of the graph, otherwise use method that can be faster
- Add ``PCGAMGASMSetHEM()`` with corresponding option ``-pc_gamg_asm_hem_aggs N``. Use ASM smoother constructed from N applications of heavy edge matching
- ``PCMAT`` use ``MatSolve()`` if implemented by the matrix type
- Add ``PCLMVMSetUpdateVec()`` for the automatic update of the LMVM preconditioner inside a SNES solve
- Add ``PCGAMGSetInjectionIndex()`` with corresponding option ``-pc_gamg_injection_index i,j,k...``. Inject provided indices of fine grid operator as first coarse grid restriction (sort of p-multigrid for C1 elements)
- Add ``PC_JACOBI_ROWL1`` to ``PCJacobiType`` to use (scaled) l1 row norms for diagonal approximation with scaling of off-diagonal elements
- Add ``PCJacobiSetRowl1Scale()`` and ``-pc_jacobi_rowl1_scale scale`` to access new scale member of PC_Jacobi class, for new row l1 Jacobi
- Add ``-mg_fine_...`` prefix alias for fine grid options to override ``-mg_levels_...`` options, like ``-mg_coarse_...``
- The generated sub-matrices in ``PCFIELDSPLIT``, ``PCASM``, and ``PCBJACOBI`` now retain any null space or near null space attached to them even if the non-zero structure of the outer matrix changes
- Add support for multiple subdomain per MPI process in ``PCBDDC``
- Add `PCJacobiGetDiagonal()` to access the Jacobi inverted matrix diagonal and its square root variant

.. rubric:: KSP:

.. rubric:: SNES:

- Add support for Quasi-Newton models in ``SNESNEWTONTR`` via ``SNESNewtonTRSetQNType``
- Add support for trust region norm customization in ``SNESNEWTONTR`` via ``SNESNewtonTRSetNormType``
- Remove default of ``KSPPREONLY`` and ``PCLU`` for ``SNESNASM`` subdomain solves: for ``SNESASPIN`` use ``-npc_sub_ksp_type preonly -npc_sub_pc_type lu``
- Add function typedefs ``SNESInitialGuessFn``, ``SNESFunctionFn``, ``SNESObjectiveFn``, ``SNESJacobianFn``, and ``SNESNGSFn``
- Deprecate ``DMDASNESFunction``, ``DMDASNESJacobian``, ``DMDASNESObjective``, ``DMDASNESFunctionVec``, ``DMDASNESJacobianVec``, and ``DMDASNESObjectiveVec``
  in favor of ``DMDASNESFunctionFn``, ``DMDASNESJacobianFn``, ``DMDASNESObjectiveFn``, ``DMDASNESFunctionVecFn``, ``DMDASNESJacobianVecFn``, and ``DMDASNESObjectiveVecFn``

.. rubric:: SNESLineSearch:

- Deprecate ``SNESLineSearchShellSetUserFunc()`` and ``SNESLineSearchShellGetUserFunc()`` in favor of ``SNESLineSearchShellSetApply()`` and ``SNESLineSearchShellGetApply()``

.. rubric:: TS:

- Add support for custom predictor callbacks in the second-order generalized-alpha method using ``TSAlpha2SetPredictor()``
- Allow adaptivity to change time step size in first step of second-order generalized-alpha method
- Add ``TSSetPostEventStep()`` to control the first step after event
- Rename ``TSSetPostEventIntervalStep()`` to ``TSSetPostEventSecondStep()``, controlling the second step after event
- Rename option ``-ts_event_post_eventinterval_step`` to ``-ts_event_post_event_second_step``
- Change the (event) indicator functions type from ``PetscScalar[]`` to ``PetscReal[]`` in the user ``indicator()`` callback set by ``TSSetEventHandler()``
- Add ``TSGetStepRollBack()`` to access the internal rollback flag
- Add boolean flag to ``TSSetResize()`` to control when to resize

.. rubric:: TAO:

- Deprecate ``TaoCancelMonitors()`` (resp. ``-tao_cancelmonitors``) in favor of ``TaoMonitorCancel()`` (resp. ``-tao_monitor_cancel``)
- Deprecate ``-tao_view_gradient``, ``-tao_view_ls_residual``, ``-tao_view_solution``, and ``-tao_view_stepdirection`` in favor of
  ``-tao_monitor_gradient``, ``-tao_monitor_ls_residual``, ``-tao_monitor_solution``, and ``-tao_monitor_step``
- Deprecate ``-tao_draw_solution``, ``-tao_draw_gradient``, and ``-tao_draw_step`` in favor of ``-tao_monitor_solution_draw``, ``-tao_monitor_gradient_draw``, and ``-tao_monitor_step_draw``
- Deprecate ``TaoSetMonitor()`` in favor of ``TaoMonitorSet()``
- Deprecate all of the provided ``Tao`` monitor routine names in favor of the standard PETSc naming conventions

.. rubric:: DM/DA:

- Add MPI reduction inside ``SNESComputeObjective_DMDA()``. No need to call reduction in local callback
- Deprecate ``PetscSimplePointFunc`` in favor of ``PetscSimplePointFn``
- Move ``DMPlexReorderDefaultFlag`` to ``DMReorderDefaultFlag``
- Add ``DMCreateSectionPermutation()``, ``DMReorderSectionGetType()``, and ``DMReorderSectionSetType()``
- Add ``DMReorderSectionGetDefault()`` and ``DMReorderSectionSetDefault()`` to allow point permutations when sections are built automatically
- Change interface to ``DMCreateSectionSubDM()`` to add component specification
- Add ``DMDAGetBoundaryType()``

.. rubric:: DMSwarm:

- Add continuous ``DM`` argument to ``DMSwarmProjectFields()``
- Add ``DMSwarmGetFieldInfo()``
- Add ``DMSwarmVectorGetField()``

.. rubric:: DMPlex:

- Drop support for MED, i.e. remove ``DMPlexCreateMedFromFile()`` and ``--with-med``
- Change protototype of ``DMPlexSetSNESLocalFEM()``. Now it accepts a single context and a Boolean indicating to use the objective function callback
- Replace ``DMProjectCoordinates()`` with ``DMSetCoordinateDisc()``
- Add argument to ``DMPlexCreateCoordinateSpace()``
- Add ``DMPlexCoordMap`` and some default maps
- Add Boolean argument to ``DMPlexPartitionLabelCreateSF()`` to sort ranks
- Add ``DMClearAuxiliaryVec()`` to clear the auxiliary data
- Add ignoreLabelHalo, sanitizeSubmesh, and ownershipTransferSF arguments to ``DMPlexFilter()``
- Change ``DMPlexSetIsoperiodicFaceSF()``, ``DMPlexGetIsoperiodicFaceSF()``, and ``DMPlexSetIsoperiodicFaceTransform()`` to accept multiple values for different periodic pairings

.. rubric:: FE/FV:

- Add Jacobian type argument to ``PetscFEIntegrateBdJacobian()``
- Add ``PetscFVClone()``
- Add ``PetscFVCreateDualSpace()``

.. rubric:: DMNetwork:

.. rubric:: DMStag:

- Add support for ``DMLocalToLocalBegin()`` and ``DMLocalToLocalEnd()``
- Add ``DMStagSetRefinementFactor()`` and ``DMStagGetRefinementFactor()`` to set and get the refinement ratio
- Add support for arbitrary refinement ratio and degree of freedom in interpolation and restriction

.. rubric:: DT:

- Add ``PetscDSUpdateBoundaryLabels()``

.. rubric:: Fortran: