libmesh-users — General information for libmesh users

Re: [Libmesh-users] Suggestion: turn some macros into functions

[edgar <edg...@cr...>]

> Hi Edgar,
> 
> I don't think this mailing list --8<-- accept{s} attachments.

I live in a bubble :P .

> Second best would be to paste a diff into
> the body of an email (assuming it's not too long, in which case the 
> first
> option should be used).

Third best: create a fork on notabug.org or GNU Savannah and send you 
the link to the corresponding branch, right? ;D

In any case, below is the original suggestion. Let me know if you like 
the idea, and I will create the branch on notabug.org :D .

   Hello,

   I would kindly and respectfully like to suggest turning some macros in 
=libmesh_common.h= into functions. As I was exploring libMesh, I naïvely 
tried this:

   #+CAPTION: Using the =libmesh_error_msg= macro
   #+begin_src cpp :libs $(libmesh-config --cxxflags --include --ldflags 
--libs)
     #include <stdio.h>
     #include <iostream>
     #include "libmesh/libmesh.h"
     #include "libmesh/mesh.h"

     int main (int argc, char * argv[]){
       libMesh::LibMeshInit init(argc, argv);

       if (argc < 4){
         libMesh::libmesh_error_msg("Usage: ");
       }

     }
   #+end_src

   #+caption: Result from using =libmesh_error_msg= (does not compile).
   #+begin_example
     In file included from /usr/include/libmesh/libmesh.h:25,
                      from /tmp/babel-hlBzed/C-src-YQnbkN.cpp:3:
     /tmp/babel-hlBzed/C-src-YQnbkN.cpp: In function 'int main(int, 
char**)':
     /tmp/babel-hlBzed/C-src-YQnbkN.cpp:15:14: error: expected 
unqualified-id before 'do'
        15 |     libMesh::libmesh_error_msg("Usage: " // << argv[0] <<
           |              ^~~~~~~~~~~~~~~~~
   #+end_example

   This does not work, because =libmesh_error_msg= is actually a macro 
defined in =libmesh_common.h=, and does not need =libMesh::=.

   If, however, the macro is replaced with the code, it should produce:
   #+caption: Using the raw text from =libmesh_error_msg=
   #+begin_src cpp :libs $(libmesh-config --cxxflags --include --ldflags 
--libs)
     #include <stdio.h>
     #include <iostream>
     #include "libmesh/libmesh.h"
     #include "libmesh/mesh.h"

     int main (int argc, char * argv[]){
       libMesh::LibMeshInit init(argc, argv);

       if (argc < 4){
         char msg[] = "Usage: ";
         do {
           libMesh::err << msg << std::endl;
           std::stringstream msg_stream;
           msg_stream << msg;
           libMesh::MacroFunctions::report_error(__FILE__, __LINE__, 
LIBMESH_DATE, LIBMESH_TIME);
           LIBMESH_THROW(libMesh::LogicError(msg_stream.str()));
         } while (0);
       }

     }
   #+end_src

   #+caption: Expected output (error message from libMesh) with raw text 
from =libmesh_error_msg=
   #+begin_example
     Usage:
     Stack frames: 5
     0: libMesh::print_trace(std::ostream&)
     1: libMesh::MacroFunctions::report_error(char const*, int, char 
const*, char const*)
     2: /tmp/babel-hlBzed/C-bin-tb49Ls(+0xcdee) [0x563818bc5dee]
     3: __libc_start_main
     4: /tmp/babel-hlBzed/C-bin-tb49Ls(+0xceae) [0x563818bc5eae]
     [0] /tmp/babel-hlBzed/C-src-APc5Dz.cpp, line 22, compiled Mar 18 
2021 at 17:20:53
     
--------------------------------------------------------------------------
     MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
     with errorcode 1.

     NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
     You may or may not see output from other processes, depending on
     exactly when Open MPI kills them.
     
--------------------------------------------------------------------------
   #+end_example

   This makes me think that we can have a function (or a template) like 
this:
   #+caption: Fuction to replace =libmesh_error_msg= macro
   #+begin_src cpp :libs $(libmesh-config --cxxflags --include --ldflags 
--libs)
     #include <stdio.h>
     #include <iostream>
     #include "libmesh/libmesh.h"
     #include "libmesh/mesh.h"

     void libmesh_error_msg_fun(std::string &msg) {
       do {
         libMesh::err << msg << std::endl;
         std::stringstream msg_stream;
         msg_stream << msg;
         libMesh::MacroFunctions::report_error(__FILE__, __LINE__, 
LIBMESH_DATE, LIBMESH_TIME);
         LIBMESH_THROW(libMesh::LogicError(msg_stream.str()));
       } while (0);
     }

     template<typename X, typename Y>
     int libmesh_example_requires(X condition, Y option){
       do {
         if (!(condition)) {
           libMesh::out <<
             "Configuring libMesh with " <<
             option <<
             " is required to run this example." << std::endl;
           return 77;
         }
       } while (0);
     }

     int main (int argc, char * argv[]){
       libMesh::LibMeshInit init(argc, argv);

       if (argc < 4){
         std::string msg = {"Usage: "};
         libmesh_error_msg_fun(msg);
       }
     }
   #+end_src

   #+RESULTS:

   #+begin_example
     Usage:
     Stack frames: 6
     0: libMesh::print_trace(std::ostream&)
     1: libMesh::MacroFunctions::report_error(char const*, int, char 
const*, char const*)
     2: /tmp/babel-CpgWD6/C-bin-mEeZoe(+0xcf4f) [0x55567f87ef4f]
     3: /tmp/babel-CpgWD6/C-bin-mEeZoe(+0xcd76) [0x55567f87ed76]
     4: __libc_start_main
     5: /tmp/babel-CpgWD6/C-bin-mEeZoe(+0xcdbe) [0x55567f87edbe]
     [0] /tmp/babel-CpgWD6/C-src-HfTgSq.cpp, line 18, compiled Mar 18 
2021 at 20:48:30
     
--------------------------------------------------------------------------
     MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
     with errorcode 1.

     NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
     You may or may not see output from other processes, depending on
     exactly when Open MPI kills them.
     
--------------------------------------------------------------------------
   #+end_example

   I was expecting =libMesh::= to work as well, because =libmesh_*_msg= 
look very much like functions, and they are inside a =namespace=. I am 
sure that there is a solid reason for which they were kept as macros, 
and that is why this is a humble suggestion. Thank you.

Re: [Libmesh-users] Suggestion: turn some macros into functions

[John P. <jwp...@gm...>]

On Mon, Mar 22, 2021 at 1:10 PM <ed...@in...> wrote:

> > Hi Edgar,
> >
> > I don't think this mailing list --8<-- accept{s} attachments.
>
> I live in a bubble :P .
>
> > Second best would be to paste a diff into
> > the body of an email (assuming it's not too long, in which case the
> > first
> > option should be used).
>
> Third best: create a fork on notabug.org or GNU Savannah and send you
> the link to the corresponding branch, right? ;D
>
> In any case, below is the original suggestion. Let me know if you like
> the idea, and I will create the branch on notabug.org :D .
>
>    Hello,
>
>    I would kindly and respectfully like to suggest turning some macros in
> =libmesh_common.h= into functions. As I was exploring libMesh, I naïvely
> tried this:
>
>    #+CAPTION: Using the =libmesh_error_msg= macro
>    #+begin_src cpp :libs $(libmesh-config --cxxflags --include --ldflags
> --libs)
>      #include <stdio.h>
>      #include <iostream>
>      #include "libmesh/libmesh.h"
>      #include "libmesh/mesh.h"
>
>      int main (int argc, char * argv[]){
>        libMesh::LibMeshInit init(argc, argv);
>
>        if (argc < 4){
>          libMesh::libmesh_error_msg("Usage: ");
>        }
>
>      }
>    #+end_src
>
>    #+caption: Result from using =libmesh_error_msg= (does not compile).
>    #+begin_example
>      In file included from /usr/include/libmesh/libmesh.h:25,
>                       from /tmp/babel-hlBzed/C-src-YQnbkN.cpp:3:
>      /tmp/babel-hlBzed/C-src-YQnbkN.cpp: In function 'int main(int,
> char**)':
>      /tmp/babel-hlBzed/C-src-YQnbkN.cpp:15:14: error: expected
> unqualified-id before 'do'
>         15 |     libMesh::libmesh_error_msg("Usage: " // << argv[0] <<
>            |              ^~~~~~~~~~~~~~~~~
>    #+end_example
>
>    This does not work, because =libmesh_error_msg= is actually a macro
> defined in =libmesh_common.h=, and does not need =libMesh::=.
>
>    If, however, the macro is replaced with the code, it should produce:
>    #+caption: Using the raw text from =libmesh_error_msg=
>    #+begin_src cpp :libs $(libmesh-config --cxxflags --include --ldflags
> --libs)
>      #include <stdio.h>
>      #include <iostream>
>      #include "libmesh/libmesh.h"
>      #include "libmesh/mesh.h"
>
>      int main (int argc, char * argv[]){
>        libMesh::LibMeshInit init(argc, argv);
>
>        if (argc < 4){
>          char msg[] = "Usage: ";
>          do {
>            libMesh::err << msg << std::endl;
>            std::stringstream msg_stream;
>            msg_stream << msg;
>            libMesh::MacroFunctions::report_error(__FILE__, __LINE__,
> LIBMESH_DATE, LIBMESH_TIME);
>            LIBMESH_THROW(libMesh::LogicError(msg_stream.str()));
>          } while (0);
>        }
>
>      }
>    #+end_src
>
>    #+caption: Expected output (error message from libMesh) with raw text
> from =libmesh_error_msg=
>    #+begin_example
>      Usage:
>      Stack frames: 5
>      0: libMesh::print_trace(std::ostream&)
>      1: libMesh::MacroFunctions::report_error(char const*, int, char
> const*, char const*)
>      2: /tmp/babel-hlBzed/C-bin-tb49Ls(+0xcdee) [0x563818bc5dee]
>      3: __libc_start_main
>      4: /tmp/babel-hlBzed/C-bin-tb49Ls(+0xceae) [0x563818bc5eae]
>      [0] /tmp/babel-hlBzed/C-src-APc5Dz.cpp, line 22, compiled Mar 18
> 2021 at 17:20:53
>
> --------------------------------------------------------------------------
>      MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
>      with errorcode 1.
>
>      NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
>      You may or may not see output from other processes, depending on
>      exactly when Open MPI kills them.
>
> --------------------------------------------------------------------------
>    #+end_example
>
>    This makes me think that we can have a function (or a template) like
> this:
>    #+caption: Fuction to replace =libmesh_error_msg= macro
>    #+begin_src cpp :libs $(libmesh-config --cxxflags --include --ldflags
> --libs)
>      #include <stdio.h>
>      #include <iostream>
>      #include "libmesh/libmesh.h"
>      #include "libmesh/mesh.h"
>
>      void libmesh_error_msg_fun(std::string &msg) {
>        do {
>          libMesh::err << msg << std::endl;
>          std::stringstream msg_stream;
>          msg_stream << msg;
>          libMesh::MacroFunctions::report_error(__FILE__, __LINE__,
> LIBMESH_DATE, LIBMESH_TIME);
>          LIBMESH_THROW(libMesh::LogicError(msg_stream.str()));
>        } while (0);
>      }
>
>      template<typename X, typename Y>
>      int libmesh_example_requires(X condition, Y option){
>        do {
>          if (!(condition)) {
>            libMesh::out <<
>              "Configuring libMesh with " <<
>              option <<
>              " is required to run this example." << std::endl;
>            return 77;
>          }
>        } while (0);
>      }
>
>      int main (int argc, char * argv[]){
>        libMesh::LibMeshInit init(argc, argv);
>
>        if (argc < 4){
>          std::string msg = {"Usage: "};
>          libmesh_error_msg_fun(msg);
>        }
>      }
>    #+end_src
>
>    #+RESULTS:
>
>    #+begin_example
>      Usage:
>      Stack frames: 6
>      0: libMesh::print_trace(std::ostream&)
>      1: libMesh::MacroFunctions::report_error(char const*, int, char
> const*, char const*)
>      2: /tmp/babel-CpgWD6/C-bin-mEeZoe(+0xcf4f) [0x55567f87ef4f]
>      3: /tmp/babel-CpgWD6/C-bin-mEeZoe(+0xcd76) [0x55567f87ed76]
>      4: __libc_start_main
>      5: /tmp/babel-CpgWD6/C-bin-mEeZoe(+0xcdbe) [0x55567f87edbe]
>      [0] /tmp/babel-CpgWD6/C-src-HfTgSq.cpp, line 18, compiled Mar 18
> 2021 at 20:48:30
>
> --------------------------------------------------------------------------
>      MPI_ABORT was invoked on rank 0 in communicator MPI_COMM_WORLD
>      with errorcode 1.
>
>      NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
>      You may or may not see output from other processes, depending on
>      exactly when Open MPI kills them.
>
> --------------------------------------------------------------------------
>    #+end_example
>
>    I was expecting =libMesh::= to work as well, because =libmesh_*_msg=
> look very much like functions, and they are inside a =namespace=. I am
> sure that there is a solid reason for which they were kept as macros,
> and that is why this is a humble suggestion. Thank you.
>

The main reason that libmesh_error_msg() and friends are macros is that
they use the __FILE__ and __LINE__ preprocessor defines to help direct the
user to the exact place in the code where the error was triggered. I'm not
sure if it is possible to do this same trick with inline functions, IIRC it
will always report the line where the inline function is defined in the
source rather than the line where it is called.

-- 
John

Re: [Libmesh-users] Setting output directories in libmesh

[edgar <edg...@cr...>]

Out of frustration, I got a new e-mail address (wish me luck)

> --- 8< snip ---

Thank you for your kind answers, John. I will keep adapting the 
directory structure for the package through a PKGBUiLD.

> Since your attachment did not come through, I'm not sure where you'd 
> like
> to add this line. My suggestion would be to pull the branch from the
> original PR, make your proposed change, and then either push that to 
> your
> own libmesh fork so that we can try it out, or paste the diff (output 
> of
> git diff or git log -p for the commit) so that someone can recreate it.

As ridiculous as it sounds, the sed line fixes a patch (I did not know 
how to achieve my goal with the netcdf.m4 file). The end goal is to have 
an autoconf that allows for a pre-installed netcdf. I am attaching the 
patch that I prepared (and needs further fixing).

The steps that I use to build are here: 
https://notabug.org/broncodev/libmesh-pkgbuild.git. This is not a 
libMesh fork, but a recipe used by pacman (package manager) to build 
libMesh. It also holds the netcdf.m4.patch. In the PKGBUILD file, you 
will find:

     (from 
https://notabug.org/broncodev/libmesh-pkgbuild/src/master/PKGBUILD#L85)
     $ patch -d "${srcdir}"/libmesh/m4 -i "${srcdir}"/netcdf.m4.patch
     $ cd "${srcdir}/${realname}"
     $ autoconf
     $ [[ -f /usr/include/netcdf.h ]] && \
       sed -i "s-\(ac_subdirs_all='\)contrib/netcdf/v4-\1-g; 
s-\(subdirs=\"\$subdirs\) contrib/netcdf/v4-\1-g" configure

I hope that these commands and the patch that I am sending 
(netcdf.m4.patch) suffice to show that the configure file is modified 
such that it finds a local netcdf.

Thanks again.

Re: [Libmesh-users] Suggestion: turn some macros into functions

[John P. <jwp...@gm...>]

On Fri, Mar 19, 2021 at 11:40 AM <ed...@op...> wrote:

> I am sorry, but my e-mail provider refuses to send my mesage as plain
> text. I am thus attaching a compressed file in the hope that it reaches
> you.
>

Hi Edgar,

I don't think this mailing list requires plain text messages (though I
could be wrong about that). It definitely does not accept attachments. If
you have a patch you'd like us to consider, the best approach is definitely
to make a PR for it on GitHub. Second best would be to paste a diff into
the body of an email (assuming it's not too long, in which case the first
option should be used).

-- 
John

Re: [Libmesh-users] AMR repartition in parallel?

[John P. <jwp...@gm...>]

On Wed, Mar 17, 2021 at 7:57 AM Li Luo <li...@ka...> wrote:

>
> Dear developers,
>
> I am using libmesh's AMR in parallel. It seems that when the mesh is
> changed significantly, for example, when some local region is refined then
> coarsened as an object moves, the partition remains the same.
>
> As in the below two pictures, a refined 'circle' moves upward, but the
> partition doesn't change even though the mesh changes a lot.
>
> For load-balancing purposes, isn't it feasible to have more subdomains
> near the refined part?
>

That's true, but we don't repartition every time the EquationSystems is
reinit'd since that would also incur some non-trivial cost that might not
be required in all simulations. That said, it should be *possible* to
repartition the mesh "on demand" and then reinit()... I'm not sure if we
have an example of this, but it would be good if you could try it out and
report back on whether it works and/or speeds up the simulation.

-- 
John

Re: [Libmesh-users] Setting output directories in libmesh

[John P. <jwp...@gm...>]

On Wed, Mar 17, 2021 at 11:17 AM <ed...@op...> wrote:

> Hello,
>
> I would like to know if someone can help me with some things regarding
> compilation, please.
>
> 1. Is there a way to specify the directory for the examples? (as in
> =/usr/share/libmesh/examples=)
>

Not currently. There's a --prefix, and everything goes in there.



>
> 2. Is there a way to specify the directory for the contrib directory (as
> in =/usr/share/libmesh/contrib=)? or is it safe to move
> =contrib/libtool= without affecting any of the configuration files, i.e.
> =Make.common=?
>

Same answer as above regarding the contrib directory. libtool is just a
shell script, so it should be safe to move it, other than the issue of
other scripts/Makefiles which are expecting it to be in a certain place.



>
> 3. All the Makefiles from the examples have
>
>         LIBMESH_DIR ?= /usr
>         ...
>         include $(LIBMESH_DIR)/Make.common
>
>     I would prefer to have =Make.common= in =/usr/share/libmesh/=, and I
> can move it there. In fact, I can run a recursive =sed= for all the
> Makefiles, but I would like to know if there is a way to set the
> location for =Make.common= so that the Makefiles know it.
>

I think it might be easier for you to specify a --prefix during configure,
then you can just install everything to some directory other than /usr. It
will be difficult (as you've seen) to move around individual pieces of the
installation like this, because they are all interdependent on one
another...



>
> 4. Some time ago, there was a pull request with =netcdf.m4.patch= to
> enable netcdf with libMesh[1]. It did not work, because it still needs
> this =sed= command after the patch is applied:
>
>         sed -i "s-\(ac_subdirs_all='\)contrib/netcdf/v4-\1-g;
> s-\(subdirs=\"\$subdirs\) contrib/netcdf/v4-\1-g" configure
>
>     Can someone help me to modify the patch so that there is no need for
> such rule? (I am attaching the file)
>

Since your attachment did not come through, I'm not sure where you'd like
to add this line. My suggestion would be to pull the branch from the
original PR, make your proposed change, and then either push that to your
own libmesh fork so that we can try it out, or paste the diff (output of
git diff or git log -p for the commit) so that someone can recreate it.

-- 
John

[Libmesh-users] Suggestion: turn some macros into functions

[<ed...@op...>]

I am sorry, but my e-mail provider refuses to send my mesage as plain 
text. I am thus attaching a compressed file in the hope that it reaches 
you.

-------------------------------------------------
This free account was provided by VFEmail.net - report spam to ab...@vf...
 
ONLY AT VFEmail! - Use our Metadata Mitigator to keep your email out of the NSA's hands!
$24.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!  

[Libmesh-users] Setting output directories in libmesh

[<ed...@op...>]

Hello,

I would like to know if someone can help me with some things regarding 
compilation, please.

1. Is there a way to specify the directory for the examples? (as in 
=/usr/share/libmesh/examples=)

2. Is there a way to specify the directory for the contrib directory (as 
in =/usr/share/libmesh/contrib=)? or is it safe to move 
=contrib/libtool= without affecting any of the configuration files, i.e. 
=Make.common=?

3. All the Makefiles from the examples have

        LIBMESH_DIR ?= /usr
        ...
        include $(LIBMESH_DIR)/Make.common

    I would prefer to have =Make.common= in =/usr/share/libmesh/=, and I 
can move it there. In fact, I can run a recursive =sed= for all the 
Makefiles, but I would like to know if there is a way to set the 
location for =Make.common= so that the Makefiles know it.

4. Some time ago, there was a pull request with =netcdf.m4.patch= to 
enable netcdf with libMesh[1]. It did not work, because it still needs 
this =sed= command after the patch is applied:

        sed -i "s-\(ac_subdirs_all='\)contrib/netcdf/v4-\1-g; 
s-\(subdirs=\"\$subdirs\) contrib/netcdf/v4-\1-g" configure

    Can someone help me to modify the patch so that there is no need for 
such rule? (I am attaching the file)

Many thanks!

[1] https://sourceforge.net/p/libmesh/mailman/message/36889108/



-------------------------------------------------
This free account was provided by VFEmail.net - report spam to ab...@vf...
 
ONLY AT VFEmail! - Use our Metadata Mitigator to keep your email out of the NSA's hands!
$24.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!  

Re: [Libmesh-users] Error while mesh partitioning by subdomain_partitioner

[Pilhwa L. <jpi...@gm...>]

Dear John,

Thanks for the clarification. 

With best regards,
Pilhwa

Re: [Libmesh-users] Error while mesh partitioning by subdomain_partitioner

[John P. <jwp...@gm...>]

On Wed, Mar 10, 2021 at 8:01 AM Pilhwa Lee <jpi...@gm...> wrote:

> Dear John,
>
> I didn’t point out the actual reason of the problem in the previous
> correspondence.
>
> The issue was “distributed mesh”. In TACC, I need to use “allgather” to
> fetch all elements of the mesh. That wasn’t the case in my local
> workstation with the same code.
>
> Could you clarify how the usage of “distributed mesh” is determined in
> libMesh? Thanks.


Based on the error message you reported above, it looks like you are using
a libmesh build provided by TACC:

> /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/
src/partitioning/mapped_subdomain_partitioner.C

It is possible that TACC configured libmesh with the --enable-parmesh flag,
which causes all instances of "Mesh" in your code to be treated as
DistributedMesh. You can check whether this is the case by looking at the
libmesh_config.h file of the libmesh installation you are using and
checking whether LIBMESH_ENABLE_PARMESH is defined. The default is for
"Mesh" to be treated as ReplicatedMesh, which may be how you compiled it on
your local workstation. Unfortunately, as you found, you can't generally
switch a code that was developed with ReplicatedMesh to use DistributedMesh
without finding and fixing a few bugs first. Also, if you find that you
have to use allgather() then this may defeat the purpose of using a
DistributedMesh, as you have now "serialized" the Mesh, but with a less
efficient data structure than would have been employed by ReplicatedMesh.

A possible fix is to update your code so that it explicitly uses
ReplicatedMesh and avoids "Mesh". This will make it more portable between
your workstation and systems like TACC.

-- 
John

Re: [Libmesh-users] Error while mesh partitioning by subdomain_partitioner

[Pilhwa L. <jpi...@gm...>]

Dear John,

I didn’t point out the actual reason of the problem in the previous correspondence. 

The issue was “distributed mesh”. In TACC, I need to use “allgather” to fetch all elements of the mesh. That wasn’t the case in my local workstation with the same code.

Could you clarify how the usage of “distributed mesh” is determined in libMesh? Thanks.

With best regards,
Pilhwa

Re: [Libmesh-users] Error while mesh partitioning by subdomain_partitioner

[Pilhwa L. <jpi...@gm...>]

Dear John,

The problem is solved. As you mentioned, the subdomain id was not well assigned. Technically, the problem was related to not typecasting PetscScalar to double in my code, not directly involving with libMesh.

Thanks!

With best regards,
Pilhwa

Re: [Libmesh-users] Error while mesh partitioning by subdomain_partitioner

[John P. <jwp...@gm...>]

On Tue, Mar 9, 2021 at 12:59 PM Pilhwa Lee <jpi...@gm...> wrote:

> Dear All,
>
> In TACC, I’m trying to do mesh partitioning by subdomain_partitioner.
> There comes the following error:
>
> map_find() error: key "22484" not found in file
> /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/src/partitioning/mapped_subdomain_partitioner.Cmap_find()
> error: key "12288" not found in file
> /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/src/partitioning/mapped_subdomain_partitioner.C
> on line 54map_find() error: key "19121" not found in file
> /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/src/partitioning/mapped_subdomain_partitioner.C
> on line 54
>  on line 54
>
> [0] [2] ./include/libmesh/utility.h, line 135, compiled [3]
> ./include/libmesh/utility.h, line 135, compiled Feb  3 2021 at
> ./include/libmesh/utility.h, line 135, compiled Feb  3 2021 at 17:00:42
>
> When I test in a local workstation based on GCC, there is no problem. I
> wonder whether there are some issues of configuration depending on GCC
> versus Intel64 compilers and permission.


The key which is not found in this case, 22484, seems pretty large to be a
subdomain id. Are you sure you're using the subdomain_to_proc map
correctly? From the documentation:

  /**
   * Before calling partition() or partition_range(), the user must
   * assign all the Mesh subdomains to certain processors by adding
   * them to this std::map.  For example:
   * subdomain_to_proc[1] = 0;
   * subdomain_to_proc[2] = 0;
   * subdomain_to_proc[3] = 0;
   * subdomain_to_proc[4] = 1;
   * subdomain_to_proc[5] = 1;
   * subdomain_to_proc[6] = 2;
   * Would partition the mesh onto three processors, with subdomains
   * 1, 2, and 3 on processor 0, subdomains 4 and 5 on processor 1,
   * and subdomain 6 on processor 2.
   */

What are the contents of the subdomain_to_proc map when the error occurs?
Note that this map will generally need to be changed as you vary the number
of processors that you run on, and it is up to the user to set it up
correctly. I don't really see how the error could be due to different
compiler toolchains, but I guess anything is possible.

-- 
John

[Libmesh-users] Error while mesh partitioning by subdomain_partitioner

[Pilhwa L. <jpi...@gm...>]

Dear All,

In TACC, I’m trying to do mesh partitioning by subdomain_partitioner. There comes the following error:

map_find() error: key "22484" not found in file /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/src/partitioning/mapped_subdomain_partitioner.Cmap_find() error: key "12288" not found in file /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/src/partitioning/mapped_subdomain_partitioner.C on line 54map_find() error: key "19121" not found in file /admin/build/admin/rpms/stampede2/BUILD/libmesh-1.6.0/src/partitioning/mapped_subdomain_partitioner.C on line 54
 on line 54

[0] [2] ./include/libmesh/utility.h, line 135, compiled [3] ./include/libmesh/utility.h, line 135, compiled Feb  3 2021 at ./include/libmesh/utility.h, line 135, compiled Feb  3 2021 at 17:00:42

When I test in a local workstation based on GCC, there is no problem. I wonder whether there are some issues of configuration depending on GCC versus Intel64 compilers and permission. 

Any opinions will be greatly appreciated. Thanks.

With best regards,
Pilhwa

[Libmesh-users] Reload data from .xda files in parallel

[Li L. <li...@ka...>]

Dear Developers,

I tried writing data to ".xda" like:

equation_systems.write("./data/saved_solution.xda",WRITE);

Since I am using parallel mesh "--enable-parmesh", it writes as many files
as the number of processors used, like  "saved_solution.xda.031" ...

I want to reload the saved solution to my program and continue to run it
further, how can I read these data files?

Thank you!

Best,
Li Luo

-- 

This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

[Libmesh-users] Reload .xda data in parallel

[Li L. <li...@ka...>]

Dear Developers,

I tried writing data to ".xda" like:

equation_systems.write("./data/saved_solution.xda",WRITE);

Since I am using parallel mesh "--enable-parmesh", it writes as many files
as the number of processors used, like  "saved_solution.xda.031" ...

I want to reload the saved solution to my program and run it further, how
can I read these data files?

Best,
Li Luo

-- 

This message and its contents, including attachments are intended solely 
for the original recipient. If you are not the intended recipient or have 
received this message in error, please notify me immediately and delete 
this message from your computer system. Any unauthorized use or 
distribution is prohibited. Please consider the environment before printing 
this email.

Re: [Libmesh-users] problem with installation

[John P. <jwp...@gm...>]

On Thu, Feb 4, 2021 at 1:11 AM Michael Povolotskyi <mpo...@gm...>
wrote:

> Hello,
>
> I have a problem with libmesh installation:
>
> <<< C++ compiler is unknown but accepted gcc version >>>
> checking for a sed that does not truncate output... /usr/bin/sed
> checking for C++ compiler vendor... gnu
> configure: error: libMesh now requires C++11
>

Hi Michael,

You passed '--disable-cxx11' to configure, which as the error message says,
libmesh no longer allows. It looks like you are using GCC 9.2 which is a
relatively new compiler, so you should be OK if you just remove this flag
from configure.

-- 
John

[Libmesh-users] problem with installation

[Michael P. <mpo...@gm...>]

Hello,

I have a problem with libmesh installation:

<<< C++ compiler is unknown but accepted gcc version >>>
checking for a sed that does not truncate output... /usr/bin/sed
checking for C++ compiler vendor... gnu
configure: error: libMesh now requires C++11


The log file is attached.

Thank you,

Michael.

Re: [Libmesh-users] Build error

[Jed B. <je...@je...>]

John Peterson <jwp...@gm...> writes:

> On Tue, Jan 26, 2021 at 10:11 AM Pedro Rodrigues <pjs...@gm...> wrote:
>
>> Yes I let PETSc to build MPI with default options Just --download MPI.
>> Maybe that is the problem. I will try to build it as you suggest
>>
>
> OK, well if using static libs wasn't intentional, I'd suggest you start
> over and reconfigure/rebuild PETSc with --with-shared-libraries=1. PETSc
> built with shared libraries is more versatile and useful (due to the
> ability to dlopen) and also you won't have to be restricted to building a
> static libmesh in that case.

Note that shared libraries have been default since PETSc 3.4 (2013). If you're using such an old PETSc, we strongly encourage you to upgrade.

Re: [Libmesh-users] Build error

[John P. <jwp...@gm...>]

On Tue, Jan 26, 2021 at 10:11 AM Pedro Rodrigues <pjs...@gm...> wrote:

> Yes I let PETSc to build MPI with default options Just --download MPI.
> Maybe that is the problem. I will try to build it as you suggest
>

OK, well if using static libs wasn't intentional, I'd suggest you start
over and reconfigure/rebuild PETSc with --with-shared-libraries=1. PETSc
built with shared libraries is more versatile and useful (due to the
ability to dlopen) and also you won't have to be restricted to building a
static libmesh in that case.

--
John

Re: [Libmesh-users] Build error

[John P. <jwp...@gm...>]

On Tue, Jan 26, 2021 at 4:50 AM Pedro Rodrigues <pjs...@gm...> wrote:

> Hello
>
> I tried to build under Linux but got this error message:
>
>
> /usr/bin/ld:
>
> /home/ubuntu/Dev/petsc/arch-linux2-c-debug/lib/libmpi.a(lib_libmpi_la-initthread.o):
> relocation R_X86_64_TPOFF32 against hidden symbol `MPIR_Per_thread' can not
> be used when making a shared object
> /usr/bin/ld:
> /home/ubuntu/Dev/petsc/arch-linux2-c-debug/lib/libpetsc.a(str.o):
> relocation R_X86_64_PC32 against symbol `petscstack' can not be used when
> making a shared object; recompile with -fPIC
> /usr/bin/ld: final link failed: bad value
> collect2: error: ld returned 1 exit status
> make[1]: *** [Makefile:12332: libmesh_opt.la] Error 1
> make[1]: Leaving directory '/home/ubuntu/Dev/libmesh'
> make: *** [Makefile:32014: all-recursive] Error 1
>
> Can you help?
>

Hmm... did you request PETSc to download/build its own MPI? And only static
libs?

Just wondering because it looks like your MPI and PETSc
/home/ubuntu/Dev/petsc/arch-linux2-c-debug/lib/libmpi.a
/home/ubuntu/Dev/petsc/arch-linux2-c-debug/lib/libpetsc.a

are both static (foo.a) libs. If this was intentional, you'll probably need
to configure libmesh with --disable-shared --enable-static (yes, you need
both) to be consistent with PETSc and MPI.

-- 
John

[Libmesh-users] Build error

[Pedro R. <pjs...@gm...>]

Hello

I tried to build under Linux but got this error message:


/usr/bin/ld:
/home/ubuntu/Dev/petsc/arch-linux2-c-debug/lib/libmpi.a(lib_libmpi_la-initthread.o):
relocation R_X86_64_TPOFF32 against hidden symbol `MPIR_Per_thread' can not
be used when making a shared object
/usr/bin/ld:
/home/ubuntu/Dev/petsc/arch-linux2-c-debug/lib/libpetsc.a(str.o):
relocation R_X86_64_PC32 against symbol `petscstack' can not be used when
making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:12332: libmesh_opt.la] Error 1
make[1]: Leaving directory '/home/ubuntu/Dev/libmesh'
make: *** [Makefile:32014: all-recursive] Error 1

Can you help?

thanks in advance

-- 
Pedro Rodrigues

Re: [Libmesh-users] Setting the linear solver

[Roy S. <roy...@od...>]

On Wed, 9 Dec 2020, Laura Scarabosio wrote:

> Thank you for your explanations. I built libmesh by disabling mpi,
> so is the default solver still PETSc?

If you configured with PETSc (which can be configured with "mpiuni"
rather than a real MPI), it is.  If you didn't, it isn't.

You can look for HAVE_PETSC in libmesh_config.h to check for sure, but
if you don't know whether you configured with a no-MPI PETSc build,
then you probably didn't.

> Also, for some reasons I am not able to pass command line arguments,
> they don't work.

No, they depend on PETSc.

> Therefore I tried to do the modifications that you suggested in the
> code itself, and assumed that the default solver was Eigen. I tried

> LinearImplicitSystem * ls = dynamic_cast<LinearImplicitSystem*>(& eq_sys.get_system("Diffusion")); 

This cast isn't really useful unless you test the results - if you have a
LinearImplicitSystem then get_linear_solver() will give you what you want even
when called from the base class; if you don't then the dynamic_cast here will
just return nullptr.

> EigenSparseLinearSolver<double> * linearsolver = dynamic_cast< EigenSparseLinearSolver<double> *>(ls->get_linear_solver());                      

This cast shouldn't be necessary or helpful; set_solver_type() is a
method in the base class, not just the derived.

> linearsolver->set_solver_type(CG);
>
> and small variants, but I am not able to use the enum, CG gives me a
> compilation error.

The important thing is *which* compilation error, though, right?  Did
you include enum_solver_type.h?

> >> something as specific as SuperLU_Dist you have to get at through the
> >> PETSc interface, either by extracting the internal PETSc object from
> >> one of our solver shims and using their APIs

> Can I still use PETSc since I have no mpi or should I use Eigen?

You can use either.  I still like PETSc better even in serial but the
improvements for you might not be worth the hassle of downloading and
configuring it.

> Also, would you have an example to do what you said, since I cannot
> use the command line?

We have a few; see `grep set_solver_type examples/*/*/*`.
Look at miscellaneous_ex12.C, maybe?
---
Roy

Re: [Libmesh-users] Setting the linear solver

[Roy S. <roy...@od...>]

On Tue, 8 Dec 2020, Laura Scarabosio wrote:

> >> LinearSolver::set_solver_type(foo) and set_preconditioner_type(bar),
> 
> But shouldn't I get the linear solver from the equation system or
> the system? If I declare a linear solver as you said, how does
> libmesh know that it has to be used to solve a particular system of
> equations?

I'm afraid the answer is "it depends".

If you have a LinearImplicitSystem or a DifferentiableSystem with a
NewtonSolver then get_linear_solver() will return a pointer to the
solver used in the linear solve or the quasi-Newton step of the
nonlinear solve, which in those cases is a persistent solver that you
can change the attributes of for subsequent uses.

But if you have a plain System or ExplicitSystem, then it doesn't even
have a linear solver associated with it; they're more simple than
that, intended for visualization or non-stiff problems or such.

And many of our other system types aren't simple *enough* - e.g.
there's a nonlinear solver attached, and we can't easily extract (or
even guarantee the existance of) a linear solver at the core of it, so
get_linear_solver() just generates a temporary for use in
linearizations (adjoint solves and sensitivity solves) of the system,
not a reference to a fixed solver you can modify.

This is where my second suggestion comes in:

>       but most of our users instead rely on the PETSc command line flags,
>       which we pass through to them and which can be a lot more flexible.

Likewise,

> >> I like using a PETSc configured with MUMPs support; I think more of
> our users prefer SuperLU_Dist.

> This means that to use the direct solver I should use set_solver_type(SuperLU_Dist)?

Nope.  We don't even bother trying to keep up with the N-factorial
different combinations of solvers that each linear algebra package
might support.  Something as generic as GMRES we have an enum for, but
something as specific as SuperLU_Dist you have to get at through the
PETSc interface, either by extracting the internal PETSc object from
one of our solver shims and using their APIs, or by using the PETSc
command line flags, e.g.:

./yourapp -ksp_type preonly -pc_type lu -pc_factor_mat_solver_package superlu_dist
---
Roy

Re: [Libmesh-users] Setting the linear solver

[Roy S. <roy...@od...>]

On Tue, 8 Dec 2020, Laura Scarabosio wrote:

> I would like to ask what are the default linear solver and preconditioner
> in Libmesh,

GMRES and ILU, if you've configured with a linear algebra package
supporting them.

> and how to specify in the code which ones to use (e.g. as flags
> or parameters).

LinearSolver::set_solver_type(foo) and set_preconditioner_type(bar),
but most of our users instead rely on the PETSc command line flags,
which we pass through to them and which can be a lot more flexible.

> Also, is there the possibility to use a direct solver?

I like using a PETSc configured with MUMPs support; I think more of
our users prefer SuperLU_Dist.

> Apologies if I overlooked this info in the Libmesh examples.

No, it's deliberately stripped out of the examples, most of which are
intended to work seamlessly even if libMesh is configured without the
most advanced linear algebra packages.
---
Roy