From: Fackler, P. <fac...@or...> - 2023-12-14 19:06:10
Attachments:
image.png
|
I'm using the following sequence of functions related to the Jacobian matrix: DMDACreate1d(..., &da); DMSetFromOptions(da); DMSetUp(da); DMSetMatType(da, MATAIJKOKKOS); DMSetMatrixPreallocateSkip(da, PETSC_TRUE); Mat J; DMCreateMatrix(da, &J); MatSetPreallocationCOO(J, ...); I recently added the call to DMSetMatrixPreallocateSkip, hoping the allocation would be delayed to MatSetPreallocationCOO, and that it would require less memory. The documentation<https://petsc.org/release/manualpages/DM/DMSetMatrixPreallocateSkip/> says that the data structures will not be preallocated. The following data from heaptrack shows that the allocation is still happening in the call to DMCreateMatrix. [cid:bda9ef12-a46f-47b2-9b9b-a4b2808b6b13] Can someone help me understand this? Thanks, Philip Fackler Research Software Engineer, Application Engineering Group Advanced Computing Systems Research Section Computer Science and Mathematics Division Oak Ridge National Laboratory |
From: Jed B. <je...@je...> - 2023-12-14 20:49:40
|
17 GB for a 1D DMDA, wow. :-) Could you try applying this diff to make it work for DMDA (it's currently handled by DMPlex)? diff --git i/src/dm/impls/da/fdda.c w/src/dm/impls/da/fdda.c index cad4d926504..bd2a3bda635 100644 --- i/src/dm/impls/da/fdda.c +++ w/src/dm/impls/da/fdda.c @@ -675,19 +675,21 @@ PetscErrorCode DMCreateMatrix_DA(DM da, Mat *J) specialized setting routines depend only on the particular preallocation details of the matrix, not the type itself. */ - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); - if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); - if (!aij) { - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); - if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); - if (!baij) { - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); - if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); - if (!sbaij) { - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); + if (!dm->prealloc_skip) { // Flag is likely set when user intends to use MatSetPreallocationCOO() + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); + if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); + if (!aij) { + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); + if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); + if (!baij) { + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); + if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); + if (!sbaij) { + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); + } + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); } - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); } } if (aij) { "Fackler, Philip via petsc-users" <pet...@mc...> writes: > I'm using the following sequence of functions related to the Jacobian matrix: > > DMDACreate1d(..., &da); > DMSetFromOptions(da); > DMSetUp(da); > DMSetMatType(da, MATAIJKOKKOS); > DMSetMatrixPreallocateSkip(da, PETSC_TRUE); > Mat J; > DMCreateMatrix(da, &J); > MatSetPreallocationCOO(J, ...); > > I recently added the call to DMSetMatrixPreallocateSkip, hoping the allocation would be delayed to MatSetPreallocationCOO, and that it would require less memory. The documentation<https://petsc.org/release/manualpages/DM/DMSetMatrixPreallocateSkip/> says that the data structures will not be preallocated. The following data from heaptrack shows that the allocation is still happening in the call to DMCreateMatrix. > > [cid:bda9ef12-a46f-47b2-9b9b-a4b2808b6b13] > > Can someone help me understand this? > > Thanks, > > Philip Fackler > Research Software Engineer, Application Engineering Group > Advanced Computing Systems Research Section > Computer Science and Mathematics Division > Oak Ridge National Laboratory |
From: Matthew K. <kn...@gm...> - 2023-12-14 21:19:27
Attachments:
image.png
|
On Thu, Dec 14, 2023 at 2:06 PM Fackler, Philip via petsc-users < pet...@mc...> wrote: > I'm using the following sequence of functions related to the Jacobian > matrix: > > DMDACreate1d(..., &da); > DMSetFromOptions(da); > DMSetUp(da); > DMSetMatType(da, MATAIJKOKKOS); > DMSetMatrixPreallocateSkip(da, PETSC_TRUE); > Mat J; > DMCreateMatrix(da, &J); > MatSetPreallocationCOO(J, ...); > > I recently added the call to DMSetMatrixPreallocateSkip, hoping the > allocation would be delayed to MatSetPreallocationCOO, and that it would > require less memory. The documentation > <https://petsc.org/release/manualpages/DM/DMSetMatrixPreallocateSkip/> says > that the data structures will not be preallocated. > You are completely correct. DMDA is just ignoring this flag. We will fix it. Thanks for catching this. Matt > The following data from heaptrack shows that the allocation is still > happening in the call to DMCreateMatrix. > > > Can someone help me understand this? > > Thanks, > > > *Philip Fackler * > Research Software Engineer, Application Engineering Group > Advanced Computing Systems Research Section > Computer Science and Mathematics Division > *Oak Ridge National Laboratory* > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/> |
From: Jed B. <je...@je...> - 2023-12-14 21:28:07
|
I had a one-character typo in the diff above. This MR to release should work now. https://gitlab.com/petsc/petsc/-/merge_requests/7120 Jed Brown <je...@je...> writes: > 17 GB for a 1D DMDA, wow. :-) > > Could you try applying this diff to make it work for DMDA (it's currently handled by DMPlex)? > > diff --git i/src/dm/impls/da/fdda.c w/src/dm/impls/da/fdda.c > index cad4d926504..bd2a3bda635 100644 > --- i/src/dm/impls/da/fdda.c > +++ w/src/dm/impls/da/fdda.c > @@ -675,19 +675,21 @@ PetscErrorCode DMCreateMatrix_DA(DM da, Mat *J) > specialized setting routines depend only on the particular preallocation > details of the matrix, not the type itself. > */ > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); > - if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); > - if (!aij) { > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); > - if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); > - if (!baij) { > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); > - if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); > - if (!sbaij) { > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); > - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); > + if (!dm->prealloc_skip) { // Flag is likely set when user intends to use MatSetPreallocationCOO() > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); > + if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); > + if (!aij) { > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); > + if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); > + if (!baij) { > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); > + if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); > + if (!sbaij) { > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); > + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); > + } > + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); > } > - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); > } > } > if (aij) { > > > "Fackler, Philip via petsc-users" <pet...@mc...> writes: > >> I'm using the following sequence of functions related to the Jacobian matrix: >> >> DMDACreate1d(..., &da); >> DMSetFromOptions(da); >> DMSetUp(da); >> DMSetMatType(da, MATAIJKOKKOS); >> DMSetMatrixPreallocateSkip(da, PETSC_TRUE); >> Mat J; >> DMCreateMatrix(da, &J); >> MatSetPreallocationCOO(J, ...); >> >> I recently added the call to DMSetMatrixPreallocateSkip, hoping the allocation would be delayed to MatSetPreallocationCOO, and that it would require less memory. The documentation<https://petsc.org/release/manualpages/DM/DMSetMatrixPreallocateSkip/> says that the data structures will not be preallocated. The following data from heaptrack shows that the allocation is still happening in the call to DMCreateMatrix. >> >> [cid:bda9ef12-a46f-47b2-9b9b-a4b2808b6b13] >> >> Can someone help me understand this? >> >> Thanks, >> >> Philip Fackler >> Research Software Engineer, Application Engineering Group >> Advanced Computing Systems Research Section >> Computer Science and Mathematics Division >> Oak Ridge National Laboratory |
From: Fackler, P. <fac...@or...> - 2023-12-18 18:54:43
|
Jed, That seems to have worked (ridiculously well). It's now 55MB, and it's happening in the call to MatSetPreallocationCOO. Thank you, Philip Fackler Research Software Engineer, Application Engineering Group Advanced Computing Systems Research Section Computer Science and Mathematics Division Oak Ridge National Laboratory ________________________________ From: Jed Brown <je...@je...> Sent: Thursday, December 14, 2023 16:27 To: Fackler, Philip <fac...@or...>; pet...@mc... <pet...@mc...>; xol...@li... <xol...@li...> Subject: [EXTERNAL] Re: [petsc-users] Call to DMSetMatrixPreallocateSkip not changing allocation behavior I had a one-character typo in the diff above. This MR to release should work now. https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_merge-5Frequests_7120&d=DwIBAg&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=v9sHqomCGBRWotign4NcwYwOpszOJehUGs_EO3eGn4SSZqxnfK7Iv15-X8nO1lii&s=h_jIP-6WcIjR6LssfGrV6Z2DojlN_w7Me4-a4rBE074&e= Jed Brown <je...@je...> writes: > 17 GB for a 1D DMDA, wow. :-) > > Could you try applying this diff to make it work for DMDA (it's currently handled by DMPlex)? > > diff --git i/src/dm/impls/da/fdda.c w/src/dm/impls/da/fdda.c > index cad4d926504..bd2a3bda635 100644 > --- i/src/dm/impls/da/fdda.c > +++ w/src/dm/impls/da/fdda.c > @@ -675,19 +675,21 @@ PetscErrorCode DMCreateMatrix_DA(DM da, Mat *J) > specialized setting routines depend only on the particular preallocation > details of the matrix, not the type itself. > */ > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); > - if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); > - if (!aij) { > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); > - if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); > - if (!baij) { > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); > - if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); > - if (!sbaij) { > - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); > - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); > + if (!dm->prealloc_skip) { // Flag is likely set when user intends to use MatSetPreallocationCOO() > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); > + if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); > + if (!aij) { > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); > + if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); > + if (!baij) { > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); > + if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); > + if (!sbaij) { > + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); > + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); > + } > + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); > } > - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); > } > } > if (aij) { > > > "Fackler, Philip via petsc-users" <pet...@mc...> writes: > >> I'm using the following sequence of functions related to the Jacobian matrix: >> >> DMDACreate1d(..., &da); >> DMSetFromOptions(da); >> DMSetUp(da); >> DMSetMatType(da, MATAIJKOKKOS); >> DMSetMatrixPreallocateSkip(da, PETSC_TRUE); >> Mat J; >> DMCreateMatrix(da, &J); >> MatSetPreallocationCOO(J, ...); >> >> I recently added the call to DMSetMatrixPreallocateSkip, hoping the allocation would be delayed to MatSetPreallocationCOO, and that it would require less memory. The documentation<https://urldefense.us/v2/url?u=https-3A__petsc.org_release_manualpages_DM_DMSetMatrixPreallocateSkip_&d=DwIBAg&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=v9sHqomCGBRWotign4NcwYwOpszOJehUGs_EO3eGn4SSZqxnfK7Iv15-X8nO1lii&s=IMLBs0ydxDPvuXeD6jmsq1BN_8oieHVSyG6VA9c0DyM&e= > says that the data structures will not be preallocated. The following data from heaptrack shows that the allocation is still happening in the call to DMCreateMatrix. >> >> [cid:bda9ef12-a46f-47b2-9b9b-a4b2808b6b13] >> >> Can someone help me understand this? >> >> Thanks, >> >> Philip Fackler >> Research Software Engineer, Application Engineering Group >> Advanced Computing Systems Research Section >> Computer Science and Mathematics Division >> Oak Ridge National Laboratory |
From: Jed B. <je...@je...> - 2023-12-18 19:47:25
|
Great, thanks for letting us know. It'll merge to release shortly and thus be in petsc >= 3.20.3. "Fackler, Philip" <fac...@or...> writes: > Jed, > > That seems to have worked (ridiculously well). It's now 55MB, and it's happening in the call to MatSetPreallocationCOO. > > Thank you, > > Philip Fackler > Research Software Engineer, Application Engineering Group > Advanced Computing Systems Research Section > Computer Science and Mathematics Division > Oak Ridge National Laboratory > ________________________________ > From: Jed Brown <je...@je...> > Sent: Thursday, December 14, 2023 16:27 > To: Fackler, Philip <fac...@or...>; pet...@mc... <pet...@mc...>; xol...@li... <xol...@li...> > Subject: [EXTERNAL] Re: [petsc-users] Call to DMSetMatrixPreallocateSkip not changing allocation behavior > > I had a one-character typo in the diff above. This MR to release should work now. > > https://urldefense.us/v2/url?u=https-3A__gitlab.com_petsc_petsc_-2D_merge-5Frequests_7120&d=DwIBAg&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=v9sHqomCGBRWotign4NcwYwOpszOJehUGs_EO3eGn4SSZqxnfK7Iv15-X8nO1lii&s=h_jIP-6WcIjR6LssfGrV6Z2DojlN_w7Me4-a4rBE074&e= > > Jed Brown <je...@je...> writes: > >> 17 GB for a 1D DMDA, wow. :-) >> >> Could you try applying this diff to make it work for DMDA (it's currently handled by DMPlex)? >> >> diff --git i/src/dm/impls/da/fdda.c w/src/dm/impls/da/fdda.c >> index cad4d926504..bd2a3bda635 100644 >> --- i/src/dm/impls/da/fdda.c >> +++ w/src/dm/impls/da/fdda.c >> @@ -675,19 +675,21 @@ PetscErrorCode DMCreateMatrix_DA(DM da, Mat *J) >> specialized setting routines depend only on the particular preallocation >> details of the matrix, not the type itself. >> */ >> - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); >> - if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); >> - if (!aij) { >> - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); >> - if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); >> - if (!baij) { >> - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); >> - if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); >> - if (!sbaij) { >> - PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); >> - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); >> + if (!dm->prealloc_skip) { // Flag is likely set when user intends to use MatSetPreallocationCOO() >> + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIAIJSetPreallocation_C", &aij)); >> + if (!aij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqAIJSetPreallocation_C", &aij)); >> + if (!aij) { >> + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPIBAIJSetPreallocation_C", &baij)); >> + if (!baij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqBAIJSetPreallocation_C", &baij)); >> + if (!baij) { >> + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISBAIJSetPreallocation_C", &sbaij)); >> + if (!sbaij) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSBAIJSetPreallocation_C", &sbaij)); >> + if (!sbaij) { >> + PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatMPISELLSetPreallocation_C", &sell)); >> + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatSeqSELLSetPreallocation_C", &sell)); >> + } >> + if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); >> } >> - if (!sell) PetscCall(PetscObjectQueryFunction((PetscObject)A, "MatISSetPreallocation_C", &is)); >> } >> } >> if (aij) { >> >> >> "Fackler, Philip via petsc-users" <pet...@mc...> writes: >> >>> I'm using the following sequence of functions related to the Jacobian matrix: >>> >>> DMDACreate1d(..., &da); >>> DMSetFromOptions(da); >>> DMSetUp(da); >>> DMSetMatType(da, MATAIJKOKKOS); >>> DMSetMatrixPreallocateSkip(da, PETSC_TRUE); >>> Mat J; >>> DMCreateMatrix(da, &J); >>> MatSetPreallocationCOO(J, ...); >>> >>> I recently added the call to DMSetMatrixPreallocateSkip, hoping the allocation would be delayed to MatSetPreallocationCOO, and that it would require less memory. The documentation<https://urldefense.us/v2/url?u=https-3A__petsc.org_release_manualpages_DM_DMSetMatrixPreallocateSkip_&d=DwIBAg&c=v4IIwRuZAmwupIjowmMWUmLasxPEgYsgNI-O7C4ViYc&r=DAkLCjn8leYU-uJ-kfNEQMhPZWx9lzc4d5KgIR-RZWQ&m=v9sHqomCGBRWotign4NcwYwOpszOJehUGs_EO3eGn4SSZqxnfK7Iv15-X8nO1lii&s=IMLBs0ydxDPvuXeD6jmsq1BN_8oieHVSyG6VA9c0DyM&e= > says that the data structures will not be preallocated. The following data from heaptrack shows that the allocation is still happening in the call to DMCreateMatrix. >>> >>> [cid:bda9ef12-a46f-47b2-9b9b-a4b2808b6b13] >>> >>> Can someone help me understand this? >>> >>> Thanks, >>> >>> Philip Fackler >>> Research Software Engineer, Application Engineering Group >>> Advanced Computing Systems Research Section >>> Computer Science and Mathematics Division >>> Oak Ridge National Laboratory |