Re: [Libmesh-users] problems with libmesh 1.4.0

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 422-6466

Thank you.

Could you, please, tell when 1.4.1 is going to be released?

Thank you,

Michael.

On 3/27/2019 9:53 AM, John Peterson wrote:

On Tue, Mar 26, 2019 at 7:09 PM Povolotskyi, Mykhailo <mpo...@pu...<mailto:mpo...@pu...>> wrote:
Dear Libmesh developers,

I upgrarded from 1.2.1 to 1.4.0

My code stopped working.

The first error is as follows:

[0]PETSC ERROR: --------------------- Error Message
--------------------------------------------------------------
[0]PETSC ERROR: Object is in wrong state
[0]PETSC ERROR: Not for unassembled vector
[0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html
for trouble shooting.
[0]PETSC ERROR: Petsc Release Version 3.8.4, Mar, 24, 2018
[0]PETSC ERROR: /home/mpovolot/NEMO5/prototype/bug1/Bell/nemo on a linux
named brown-a302.rcac.purdue.edu<http://brown-a302.rcac.purdue.edu> by mpovolot Tue Mar 26 18:58:29 2019
[0]PETSC ERROR: Configure options --with-scalar-type=real --with-x=0
--with-hdf5 --download-hdf5=1 --with-single-library=1 --with-pic=1
--with-shared-libraries=0 --with-log=0 --with-clanguage=C++
--CXXFLAGS="-fopenmp -fPIC" --CFLAGS="-fopenmp -fPIC" --with-fortran=0
--FFLAGS="-fopenmp -fPIC" --with-debugging=0 --with-cc=mpicc
--with-fc=mpif90 --with-cxx=mpicxx COPTFLAGS= CXXOPTFLAGS= FOPTFLAGS=
--download-metis=1 --download-parmetis=1
--with-valgrind-dir=/apps/brown/valgrind/3.13.0_gcc-4.8.5
--download-mumps=1 --with-fortran-kernels=0 --download-superlu_dist=1
--with-blaslapack-lib="-L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64
-lmkl_intel_lp64 -lmkl_gnu_thread -lmkl_core "
--with-blacs-lib=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64/libmkl_blacs_intelmpi_lp64.so
--with-blacs-include=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/include
--with-scalapack-lib="-Wl,-rpath,/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64
-L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64
-lmkl_gf_lp64 -lmkl_gnu_thread -lmkl_core  -lpthread
-L/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/lib/intel64
-lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_lp64"
--with-scalapack-include=/apps/cent7/intel/compilers_and_libraries_2017.1.132/linux/mkl/include
[0]PETSC ERROR: #1 VecCopy() line 1562 in
/depot/kildisha/apps/brown/nemo5/libs/petsc/build-real/src/vec/vec/interface/vector.c
terminate called after throwing an instance of
'libMesh::PetscSolverException'
   what():  Object is in wrong state

Program received signal SIGABRT, Aborted.
0x00002aaab14ea207 in raise () from /usr/lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install
glibc-2.17-260.el7.x86_64
libibumad-43.1.1.MLNX20180612.87b4d9b-0.1.45101.x86_64
libibverbs-41mlnx1-OFED.4.5.0.1.0.45101.x86_64
libicu-50.1.2-17.el7.x86_64 libmlx4-41mlnx1-OFED.4.5.0.0.3.45101.x86_64
libmlx5-41mlnx1-OFED.4.5.0.3.8.45101.x86_64 libnl3-3.2.28-4.el7.x86_64
librdmacm-41mlnx1-OFED.4.2.0.1.3.45101.x86_64
numactl-libs-2.0.9-7.el7.x86_64
opensm-libs-5.3.0.MLNX20181108.33944a2-0.1.45101.x86_64
python-libs-2.7.5-76.el7.x86_64 zlib-1.2.7-18.el7.x86_64
(gdb) up
#1  0x00002aaab14eb8f8 in abort () from /usr/lib64/libc.so.6
(gdb)
#2  0x00002aaab087086d in __gnu_cxx::__verbose_terminate_handler () at
/tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/vterminate.cc:95
95      /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/vterminate.cc: No such
file or directory.
(gdb)
#3  0x00002aaab086e8b6 in __cxxabiv1::__terminate (handler=<optimized
out>) at /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
47      /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc: No
such file or directory.
(gdb)
#4  0x00002aaab086e901 in std::terminate () at
/tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
57      in /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc
(gdb)

#5  0x00002aaab086eb18 in __cxxabiv1::__cxa_throw (obj=0x5f95400,
tinfo=0x2aaab00a41e0 <typeinfo for libMesh::PetscSolverException>, dest=
     0x2aaaad2c0a42
<libMesh::PetscSolverException::~PetscSolverException()>) at
/tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc:87
87      /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc: No such
file or directory.
(gdb)
#6  0x00002aaabe208304 in libMesh::PetscVector<double>::operator=
(this=0x373fc60, v=...) at src/numerics/petsc_vector.C:581
581           LIBMESH_CHKERR(ierr);
(gdb)
#7  0x00002aaabe208cb3 in libMesh::PetscVector<double>::operator=
(this=0x373fc60, v_in=...) at src/numerics/petsc_vector.C:552
552       *this = *v;
(gdb)
#8  0x00002aaabe20c6ae in libMesh::PetscVector<double>::localize
(this=0x373fba0, v_local_in=..., send_list=...) at
src/numerics/petsc_vector.C:723
723           v_local_in = *this;
(gdb)
#9  0x00002aaabe43f570 in libMesh::System::update (this=0x3734120) at
src/systems/system.C:424
424       solution->localize (*current_local_solution, send_list);
(gdb)
#10 0x00002aaabe3bbb33 in libMesh::libmesh_petsc_snes_residual_helper
(snes=0x58daf60, x=0x3628680, ctx=0x373fdb0) at
src/solvers/petsc_nonlinear_solver.C:98
98        sys.update();
(gdb)
#11 0x00002aaabe3bbe56 in libMesh::libmesh_petsc_snes_residual
(snes=0x58daf60, x=0x3628680, r=0x36b70d0, ctx=0x373fdb0) at
src/solvers/petsc_nonlinear_solver.C:152
152         ResidualContext rc =
libmesh_petsc_snes_residual_helper(snes, x, ctx);
(gdb)
#12 0x00002aaaaeae4dff in SNESComputeFunction () from
/home/mpovolot/NEMO5/prototype/lib/libnemo.so
(gdb)
#13 0x00002aaaaefd4fe6 in SNESSolve_NEWTONTR(_p_SNES*) () from
/home/mpovolot/NEMO5/prototype/lib/libnemo.so
(gdb)
#14 0x00002aaaaeaeed40 in SNESSolve () from
/home/mpovolot/NEMO5/prototype/lib/libnemo.so
(gdb)
#15 0x00002aaabe3bf889 in libMesh::PetscNonlinearSolver<double>::solve
(this=0x373fdb0, pre_in=..., x_in=..., r_in=...) at
src/solvers/petsc_nonlinear_solver.C:926
926       ierr = SNESSolve (_snes, PETSC_NULL, x->vec());
(gdb)
#16 0x00002aaabe4367ad in libMesh::NonlinearImplicitSystem::solve
(this=0x3734120) at src/systems/nonlinear_implicit_system.C:183
183 nonlinear_solver->max_linear_iterations);
(gdb)
#17 0x00002aaaadb45bb2 in NonlinearPoisson::execute_for_single_replica
(this=0x2cbaa90, my_system=0x3734120) at NonlinearPoisson.cpp:1359
1359        system->solve();
(gdb)
#18 0x00002aaaadb66dc4 in NonlinearPoisson::execute_solver
(this=0x2cbaa90) at NonlinearPoisson.cpp:1247
1247          execute_for_single_replica(my_system);
(gdb)

In order to solve it, I had to compile with disable-ghosted

After that, I have got another similar error:

#1  0x00002aaab14eb8f8 in abort () from /usr/lib64/libc.so.6
(gdb)
#2  0x00002aaab087086d in __gnu_cxx::__verbose_terminate_handler () at
/tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/vterminate.cc:95
95      in /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/vterminate.cc
(gdb)
#3  0x00002aaab086e8b6 in __cxxabiv1::__terminate (handler=<optimized
out>) at /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:47
47      /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc: No
such file or directory.
(gdb)
#4  0x00002aaab086e901 in std::terminate () at
/tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc:57
57      in /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_terminate.cc
(gdb)
#5  0x00002aaab086eb18 in __cxxabiv1::__cxa_throw (obj=0x5fe1060,
tinfo=0x2aaab0059a20 <typeinfo for libMesh::LogicError>,
dest=0x2aaaacc4de4a <libMesh::LogicError::~LogicError()>)
     at /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc:87
87      /tmp/aai/gcc-5.2.0/libstdc++-v3/libsupc++/eh_throw.cc: No such
file or directory.
(gdb)
#6  0x00002aaabda5c5b4 in libMesh::DofMap::enforce_constraints_exactly
(this=0x382c3e0, system=..., v=0x382ab70, homogeneous=false) at
src/base/dof_map_constraints.C:2108
2108          libmesh_assert (v_local->closed());
(gdb)
#7  0x00002aaabe3bbb0b in libMesh::libmesh_petsc_snes_residual_helper
(snes=0x58f21f0, x=0x3713590, ctx=0x382ac80) at
src/solvers/petsc_nonlinear_solver.C:105
105       sys.get_dof_map().enforce_constraints_exactly(sys,
sys.current_local_solution.get());
(gdb)
#8  0x00002aaabe3bbde2 in libMesh::libmesh_petsc_snes_residual
(snes=0x58f21f0, x=0x3713590, r=0x37a0560, ctx=0x382ac80) at
src/solvers/petsc_nonlinear_solver.C:152
152         ResidualContext rc =
libmesh_petsc_snes_residual_helper(snes, x, ctx);
(gdb)
#9  0x00002aaaaeae4dff in SNESComputeFunction () from
/home/mpovolot/NEMO5/prototype/lib/libnemo.so
(gdb)
#10 0x00002aaaaefd4fe6 in SNESSolve_NEWTONTR(_p_SNES*) () from
/home/mpovolot/NEMO5/prototype/lib/libnemo.so
(gdb)
#11 0x00002aaaaeaeed40 in SNESSolve () from
/home/mpovolot/NEMO5/prototype/lib/libnemo.so
(gdb)
#12 0x00002aaabe3bf815 in libMesh::PetscNonlinearSolver<double>::solve
(this=0x382ac80, pre_in=..., x_in=..., r_in=...) at
src/solvers/petsc_nonlinear_solver.C:926
926       ierr = SNESSolve (_snes, PETSC_NULL, x->vec());
(gdb)
#13 0x00002aaabe4365bd in libMesh::NonlinearImplicitSystem::solve
(this=0x381f030) at src/systems/nonlinear_implicit_system.C:183
183 nonlinear_solver->max_linear_iterations);

Please advise.

This could be related to the bug that was fixed in

3d86f9946 Close residual/vector before applying constraints

Unfortunately that fix did not make it into the 1.4.0 release because it came out after it was tagged, but it will be in an eventual 1.4.1 bugfix release. If your code works with 1.4.0, it shouldn't be too hard to also try it with master or the "v1.4.1_branch" from the upstream repository to see if the problem goes away.

Re: compiling with --disable-ghosted I'm not sure why one would need to do this, it probably masks a real bug/issue with your code but that would require some investigation to determine.

--
John