Re: [Libmesh-users] Variable groups to use blocked matrix (*BAIJ)

[Adriano C. <adr...@gm...>]

Dear John, Roy, David and Paul,

Many thanks for your return.
We found that using the option --enable-blocked-storage during the
configure step would automatically cast the BAIJ matrix representation, but
that is not the case.
Later we discovered that using the method
flow_system.identify_variable_groups(true);
after the .add_variables() made the job, that is confirmed by a -ksp_view
showing bs=4.
If we use a fixed grid with no AMR, then the code works pretty fine, but
with AMR turned on we are having the same issue as reported by David in the
mailing list thread pointed by Roy.
That is, there is a MatSetValue in a position that is not preallocated in
the BAIJ data structure.
Let us know if we could be of any help to fix this issue. The reported
error is bellow.

Best,
Adriano and Linda

---------------------------------------------------------------------
 Nonlinear solver converged at step 2
*** Warning, This code is deprecated, and likely to be removed in future
library versions!
/home1/05293/lindag/local/libmesh-block/include/libmesh/elem.h, line 2136,
compiled May 11 2018 at 06:08:00 ***
*** Warning, This code is deprecated, and likely to be removed in future
library versions!
/home1/05293/lindag/local/libmesh-block/include/libmesh/elem.h, line 1922,
compiled May 11 2018 at 06:08:00 ***

****************************************************
Mesh Refinement:
  Considering Transport Variable
  Number of elements before AMR step: 76800
  Number of elements after AMR step: 104226
  Nelem variation = 35.7109 %.

*********************************************************************
 Solving Navier-Stokes equation...
---------------------------------------------------------------------
Step L. Iter.  |b-AX|      |du|        |du|/|u|    C. R.
---------------------------------------------------------------------
[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Argument out of range
[0]PETSC ERROR: New nonzero at (4372,4385) caused a malloc
Use MatSetOption(A, MAT_NEW_NONZERO_ALLOCATION_ERR, PETSC_FALSE) to turn
off this check
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for
trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.9.1, unknown
[0]PETSC ERROR: ./sediment-opt on a  named
c455-011.stampede2.tacc.utexas.edu by lindag Fri May 11 07:29:20 2018
[0]PETSC ERROR: Configure options --prefix=/home1/05293/lindag/local/petsc
--with-blaslapack-dir=/opt/intel/compilers_and_libraries/linux/mkl
--with-debugging=0 --download-hypre --download-ml --download-mumps
--download-scalapack
[0]PETSC ERROR: #1 MatSetValues_MPIBAIJ() line 213 in
/home1/05293/lindag/src/petsc/src/mat/impls/baij/mpi/mpibaij.c
[0]PETSC ERROR: #2 MatSetValues() line 1312 in
/home1/05293/lindag/src/petsc/src/mat/interface/matrix.c
application called MPI_Abort(MPI_COMM_WORLD, 1) - process 0

2018-05-10 11:30 GMT-03:00 Paul T. Bauman <ptb...@gm...>:

> The last time I talked to @benkirk (which was a long time ago...) he'd
> indicated they used this feature very regularly. It may have been for only
> a single variable group, but I can't say for sure. We definitely need to
> exercise this more I think (and I'm also guilty of not really doing it when
> really I should be).
>
> On Thu, May 10, 2018 at 9:44 AM, David Knezevic <
> dav...@ak...> wrote:
>
>> On Thu, May 10, 2018 at 9:38 AM, Roy Stogner <roy...@ic...>
>> wrote:
>>
>> >
>> > On Wed, 9 May 2018, John Peterson wrote:
>> >
>> > On Wed, May 9, 2018 at 11:56 AM, Linda G <li...@na...> wrote:
>> >>
>> >> Dear Libmesh Users,
>> >>>
>> >>> We are trying to figure out how to set up a Stabilized NS FEM
>> formulation
>> >>> to work
>> >>> in a blocked matrix fashion. After reading the documentation we
>> realized
>> >>> that this boils down to define a
>> >>> variable group, but we could not find any example (even the Elasticity
>> >>> ones) that uses the
>> >>> block matrix (*BAIJ) feature.
>> >>> Please remember that our formulation is stabilized, so we are using
>> P1P1
>> >>> and P2P2 for velocity and pressure.
>> >>> Is the function to define variables groups a member of System class or
>> >>> DofMap class?
>> >>> Please if anyone has a working example, or teach us how to do it, we
>> >>> really
>> >>> appreciate.
>> >>>
>> >>>
>> >> If I understand correctly, this should happen automatically if you
>> have a
>> >> System with 1 variable group that contains more than 1 var, provided
>> that
>> >> you configure libmesh with --enable-blocked-storage. The default is
>> >> --disable-blocked-storage since I believe we ran into some issues while
>> >> using this option in the past.
>> >>
>> >
>> > https://github.com/libMesh/libmesh/issues/703 is the only concrete
>> > issue I can find, thanks to SourceForge mailing list changes breaking
>> > old archive links.
>> >
>> > We should probably track this down soon so we can at least upgrade
>> > blocked-storage from "enable only explicitly" to "enable with
>> > --enable-everything".  I haven't been testing --enable-blocked-storage
>> > for a long time so it may have regressed even further.
>> > ---
>> > Roy
>>
>>
>>
>> Should we re-open #703 then? I agree that we never got to the bottom of
>> what was going on there, we just avoided the issue by disabling some
>> examples.
>>
>> David
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Libmesh-users mailing list
>> Lib...@li...
>> https://lists.sourceforge.net/lists/listinfo/libmesh-users
>>
>
>


-- 
Adriano Côrtes
=================================================
*NUMPEX-COMP / Campus Duque de Caxias and*
*High-performance Computing Center (NACAD/COPPE)*
Federal University of Rio de Janeiro (UFRJ)