Menu
▾
▴
libmesh-users
|
From: Povolotskyi, M. <mpo...@pu...> - 2018-06-13 01:49:04
|
Dear Libmesh develepers,
I want to create different instances of mesh on several MPI processes. Then I want to output the mesh from one MPI process.
I am facing problems with the following code:
#include "libmesh/libmesh.h"
#include "libmesh/mesh_generation.h"
#include "libmesh/mesh.h"
int main(int argc, char ** argv)
{
MPI_Init(&argc, &argv);
{
libMesh::LibMeshInit init (argc, argv,MPI_COMM_WORLD);
libMesh::Mesh mesh(libMesh::Parallel::Communicator(MPI_COMM_SELF));
libMesh::MeshTools::Generation::build_cube (mesh,
10, 10, 5, 0.0, 2.0, 0.0, 3.0, 0.0, 4.0,libMesh::HEX8);
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
if (rank == 0)
{
mesh.write("mesh1.vtu");
}
}
MPI_Finalize();
return 0;
}
The problems are as follows:
Problem #1) If I run the code in serial, I'm getting warning:
Warning: This MeshOutput subclass only supports meshes which have been serialized!
Warning: This MeshOutput subclass only supports meshes which have been serialized!
The .pvtu extension should be used when writing VTK files in libMesh.
My question:
a) how to avoid the first waning about the mesh not being serialized?
b) I tried to change the filename to mesh1.pvtu
In this case I'm getting an error message:
ERROR: Unrecognized file extension: mesh1.pvtu
I understand the following:
*.dat -- Tecplot ASCII file
*.e -- Sandia's ExodusII format
*.exd -- Sandia's ExodusII format
*.fro -- ACDL's surface triangulation file
*.gmv -- LANL's GMV (General Mesh Viewer) format
*.mesh -- MEdit mesh format
*.mgf -- MGF binary mesh format
*.msh -- GMSH ASCII file
*.n -- Sandia's Nemesis format
*.nem -- Sandia's Nemesis format
*.plt -- Tecplot binary file
*.poly -- TetGen ASCII file
*.ucd -- AVS's ASCII UCD format
*.ugrid -- Kelly's DIVA ASCII format
*.unv -- I-deas Universal format
*.vtu -- VTK (paraview-readable) format
*.xda -- libMesh ASCII format
*.xdr -- libMesh binary format,
Exiting without writing output
Problem #2) If I run the code in serial, namely mpiexec -n 2 a.out, then the program gets frozen.
It is frozen in the output.
Thank you for your help,
Michael.
|
|
From: John P. <jwp...@gm...> - 2018-06-13 16:01:54
|
On Tue, Jun 12, 2018 at 7:48 PM, Povolotskyi, Mykhailo <mpo...@pu...>
wrote:
> Dear Libmesh develepers,
>
> I want to create different instances of mesh on several MPI processes.
> Then I want to output the mesh from one MPI process.
>
>
> I am facing problems with the following code:
>
>
> #include "libmesh/libmesh.h"
> #include "libmesh/mesh_generation.h"
> #include "libmesh/mesh.h"
> int main(int argc, char ** argv)
> {
>
> MPI_Init(&argc, &argv);
>
MPI is already initialized in LibMeshInit, so no need to do this manually
unless your real code does MPI communication before LibMeshInit...
> {
> libMesh::LibMeshInit init (argc, argv,MPI_COMM_WORLD);
> libMesh::Mesh mesh(libMesh::Parallel::Communicator(MPI_COMM_SELF));
> libMesh::MeshTools::Generation::build_cube (mesh,
> 10, 10, 5, 0.0, 2.0, 0.0,
> 3.0, 0.0, 4.0,libMesh::HEX8);
>
> int rank;
> MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>
Again, I'd just use init.comm().rank() to find out the rank.
>
> if (rank == 0)
> {
> mesh.write("mesh1.vtu");
> }
> }
> MPI_Finalize();
>
MPI_Finalize is called in the LibMeshInit destructor, no need to call it
manually.
> return 0;
> }
>
The problems are as follows:
>
> Problem #1) If I run the code in serial, I'm getting warning:
>
> Warning: This MeshOutput subclass only supports meshes which have been
> serialized!
> Warning: This MeshOutput subclass only supports meshes which have been
> serialized!
> The .pvtu extension should be used when writing VTK files in libMesh.
>
>
> My question:
>
> a) how to avoid the first waning about the mesh not being serialized?
>
Unless you have configured libmesh with --enable-parmesh (and therefore
Mesh == DistributedMesh) this warning can be safely ignored.
>
> b) I tried to change the filename to mesh1.pvtu
>
> In this case I'm getting an error message:
>
>
>
> ERROR: Unrecognized file extension: mesh1.pvtu
> I understand the following:
>
This is just a (possibly unnecessary) limitation of the NamebasedIO class,
it should work (as in misc_ex11 and misc_ex4 if you explicitly construct a
VTKIO object and then call the write() method.
--
John
|
|
From: Michael P. <mpo...@pu...> - 2018-06-13 16:49:33
|
Thank you John,
I do have some MPI code before LibMeshInit in my real application.
Also, If I do not call MPI_Finalize at the very end I'm getting warning
messages from MPI.
Thank you for you explanation with .pvtu, I have changed my code to
libMesh::VTKIO out(mesh); out.write("mesh1.pvtu"); and this worked.
Do you have any suggestions for the Problem #2 that I reported?
It seems to me that the problem is that the communicator of the Mesh
object is smaller than the communicator of the VTK. This causes a
problem for me.
Thank you,
Michael.
On 06/13/2018 12:01 PM, John Peterson wrote:
>
>
> On Tue, Jun 12, 2018 at 7:48 PM, Povolotskyi, Mykhailo
> <mpo...@pu... <mailto:mpo...@pu...>> wrote:
>
> Dear Libmesh develepers,
>
> I want to create different instances of mesh on several MPI
> processes. Then I want to output the mesh from one MPI process.
>
>
> I am facing problems with the following code:
>
>
> #include "libmesh/libmesh.h"
> #include "libmesh/mesh_generation.h"
> #include "libmesh/mesh.h"
> int main(int argc, char ** argv)
> {
>
> MPI_Init(&argc, &argv);
>
>
> MPI is already initialized in LibMeshInit, so no need to do this
> manually unless your real code does MPI communication before
> LibMeshInit...
>
> {
> libMesh::LibMeshInit init (argc, argv,MPI_COMM_WORLD);
> libMesh::Mesh
> mesh(libMesh::Parallel::Communicator(MPI_COMM_SELF));
> libMesh::MeshTools::Generation::build_cube (mesh,
> 10, 10, 5, 0.0,
> 2.0, 0.0, 3.0, 0.0, 4.0,libMesh::HEX8);
>
> int rank;
> MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>
>
> Again, I'd just use init.comm().rank() to find out the rank.
>
>
> if (rank == 0)
> {
> mesh.write("mesh1.vtu");
> }
> }
> MPI_Finalize();
>
>
> MPI_Finalize is called in the LibMeshInit destructor, no need to call
> it manually.
>
> return 0;
> }
>
>
>
> The problems are as follows:
>
> Problem #1) If I run the code in serial, I'm getting warning:
>
> Warning: This MeshOutput subclass only supports meshes which have
> been serialized!
> Warning: This MeshOutput subclass only supports meshes which have
> been serialized!
> The .pvtu extension should be used when writing VTK files in libMesh.
>
>
> My question:
>
> a) how to avoid the first waning about the mesh not being serialized?
>
>
> Unless you have configured libmesh with --enable-parmesh (and
> therefore Mesh == DistributedMesh) this warning can be safely ignored.
>
>
> b) I tried to change the filename to mesh1.pvtu
>
> In this case I'm getting an error message:
>
>
>
> ERROR: Unrecognized file extension: mesh1.pvtu
> I understand the following:
>
>
> This is just a (possibly unnecessary) limitation of the NamebasedIO
> class, it should work (as in misc_ex11 and misc_ex4 if you explicitly
> construct a VTKIO object and then call the write() method.
>
> --
> John
|
|
From: John P. <jwp...@gm...> - 2018-06-13 16:28:39
|
On Wed, Jun 13, 2018 at 10:14 AM, Michael Povolotskyi <mpo...@pu...>
wrote:
> Thank you John,
>
> I do have some MPI code before LibMeshInit in my real application.
>
> Also, If I do not call MPI_Finalize at the very end I'm getting warning
> messages from MPI.
>
> Thank you for you explanation with .pvtu, I have changed my code to
> libMesh::VTKIO out(mesh); out.write("mesh1.pvtu"); and this worked.
>
> Do you have any suggestions for the Problem #2 that I reported?
>
> It seems to me that the problem is that the communicator of the Mesh
> object is smaller than the communicator of the VTK. This causes a problem
> for me.
>
Hmm, I don't think we have considered this use case in the past, the
vtkMPIController is initialized in libmesh.C, and it must be assuming
MPI_COMM_WORLD...
_vtk_mpi_controller = vtkMPIController::New();
_vtk_mpi_controller->Initialize(&argc, const_cast<char ***>(&argv),
/*initialized_externally=*/1);
_vtk_mpi_controller->SetGlobalController(_vtk_mpi_controller);
In order for the parallel VTK writer to work, the Mesh probably therefore
also needs to be using MPI_COMM_WORLD.
--
John
|
|
From: Michael P. <mpo...@pu...> - 2018-06-13 16:35:10
|
Hi John,
It would be absolutely impossible for me to use Mesh on MPI_COMM_WORLD.
Do you thing, will it possible either to reset the vtkMPIController or
to split his communicator?
Michael.
On 06/13/2018 12:28 PM, John Peterson wrote:
>
>
> On Wed, Jun 13, 2018 at 10:14 AM, Michael Povolotskyi
> <mpo...@pu... <mailto:mpo...@pu...>> wrote:
>
> Thank you John,
>
> I do have some MPI code before LibMeshInit in my real application.
>
> Also, If I do not call MPI_Finalize at the very end I'm getting
> warning messages from MPI.
>
> Thank you for you explanation with .pvtu, I have changed my code
> to libMesh::VTKIO out(mesh); out.write("mesh1.pvtu"); and this
> worked.
>
> Do you have any suggestions for the Problem #2 that I reported?
>
> It seems to me that the problem is that the communicator of the
> Mesh object is smaller than the communicator of the VTK. This
> causes a problem for me.
>
>
> Hmm, I don't think we have considered this use case in the past, the
> vtkMPIController is initialized in libmesh.C, and it must be assuming
> MPI_COMM_WORLD...
>
> _vtk_mpi_controller = vtkMPIController::New();
> _vtk_mpi_controller->Initialize(&argc, const_cast<char ***>(&argv),
> /*initialized_externally=*/1);
> _vtk_mpi_controller->SetGlobalController(_vtk_mpi_controller);
>
> In order for the parallel VTK writer to work, the Mesh probably
> therefore also needs to be using MPI_COMM_WORLD.
>
> --
> John
|
|
From: Michael P. <mpo...@pu...> - 2018-06-13 17:03:11
|
Hello John,
I played with it, and this is what worked for me.
#include <vector>
#include <sstream>
using namespace std;
#define private public//to get access to
libMesh::LibMeshInit._vtk_mpi_controller
#include "libmesh/mesh_generation.h"
#include "libmesh/mesh.h"
#include "libmesh/vtk_io.h"
#include "libmesh/libmesh.h"
#include "vtkMPIController.h"
int main(int argc, char ** argv)
{
MPI_Init(&argc, &argv);
{
libMesh::LibMeshInit init (argc, argv,MPI_COMM_WORLD);
int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank);
vtkMPICommunicator* mcomm =
vtkMPICommunicator::GetWorldCommunicator()->NewInstance ();
mcomm->SplitInitialize (mcomm, rank, 0);
init._vtk_mpi_controller->SetCommunicator (mcomm);
libMesh::Mesh mesh(libMesh::Parallel::Communicator(MPI_COMM_SELF));
libMesh::MeshTools::Generation::build_cube (mesh,
10, 10, 5, 0.0, 2.0,
0.0, 3.0, 0.0, 4.0,libMesh::HEX8);
if (rank == 0)
{
libMesh::VTKIO out(mesh);
out.write("mesh1.pvtu");
}
}
MPI_Finalize();
return 0;
}
What do you think? Can you provide an access to
libMesh::LibMeshInit._vtk_mpi_controller?
Or there is a better solution?
Michael.
On 06/13/2018 12:28 PM, John Peterson wrote:
>
>
> On Wed, Jun 13, 2018 at 10:14 AM, Michael Povolotskyi
> <mpo...@pu... <mailto:mpo...@pu...>> wrote:
>
> Thank you John,
>
> I do have some MPI code before LibMeshInit in my real application.
>
> Also, If I do not call MPI_Finalize at the very end I'm getting
> warning messages from MPI.
>
> Thank you for you explanation with .pvtu, I have changed my code
> to libMesh::VTKIO out(mesh); out.write("mesh1.pvtu"); and this
> worked.
>
> Do you have any suggestions for the Problem #2 that I reported?
>
> It seems to me that the problem is that the communicator of the
> Mesh object is smaller than the communicator of the VTK. This
> causes a problem for me.
>
>
> Hmm, I don't think we have considered this use case in the past, the
> vtkMPIController is initialized in libmesh.C, and it must be assuming
> MPI_COMM_WORLD...
>
> _vtk_mpi_controller = vtkMPIController::New();
> _vtk_mpi_controller->Initialize(&argc, const_cast<char ***>(&argv),
> /*initialized_externally=*/1);
> _vtk_mpi_controller->SetGlobalController(_vtk_mpi_controller);
>
> In order for the parallel VTK writer to work, the Mesh probably
> therefore also needs to be using MPI_COMM_WORLD.
>
> --
> John
|
|
From: Roy S. <roy...@ic...> - 2018-06-13 17:29:45
|
On Wed, 13 Jun 2018, John Peterson wrote: > MPI is already initialized in LibMeshInit, so no need to do this manually > unless your real code does MPI communication before LibMeshInit... > >> MPI_Finalize(); > > MPI_Finalize is called in the LibMeshInit destructor, no need to call it > manually. Just to clarify: there's no need to call MPI_Finalize manually *unless* you also called MPI_Init manually. If we see that you've already initialized MPI yourself then we leave you with the responsibility to finalize yourself too. --- Roy |
|
From: Michael P. <mpo...@pu...> - 2018-06-13 17:57:04
|
Thank you, now I see. On 06/13/2018 01:29 PM, Roy Stogner wrote: > > On Wed, 13 Jun 2018, John Peterson wrote: > >> MPI is already initialized in LibMeshInit, so no need to do this >> manually >> unless your real code does MPI communication before LibMeshInit... >> >>> MPI_Finalize(); >> >> MPI_Finalize is called in the LibMeshInit destructor, no need to call it >> manually. > > Just to clarify: there's no need to call MPI_Finalize manually > *unless* you also called MPI_Init manually. If we see that you've > already initialized MPI yourself then we leave you with the > responsibility to finalize yourself too. > --- > Roy |
|
From: John P. <jwp...@gm...> - 2018-06-13 19:15:33
|
On Wed, Jun 13, 2018 at 11:03 AM, Michael Povolotskyi <mpo...@pu...>
wrote:
> Hello John,
>
> I played with it, and this is what worked for me.
>
> #include <vector>
> #include <sstream>
> using namespace std;
>
> #define private public //to get access to libMesh::LibMeshInit._vtk_mpi_
> controller
>
> #include "libmesh/mesh_generation.h"
> #include "libmesh/mesh.h"
> #include "libmesh/vtk_io.h"
> #include "libmesh/libmesh.h"
> #include "vtkMPIController.h"
> int main(int argc, char ** argv)
> {
>
> MPI_Init(&argc, &argv);
>
> {
> libMesh::LibMeshInit init (argc, argv,MPI_COMM_WORLD);
> int rank;
> MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>
> vtkMPICommunicator* mcomm = vtkMPICommunicator::
> GetWorldCommunicator()->NewInstance ();
> mcomm->SplitInitialize (mcomm, rank, 0);
>
> init._vtk_mpi_controller->SetCommunicator (mcomm);
>
> libMesh::Mesh mesh(libMesh::Parallel::Communicator(MPI_COMM_SELF));
> libMesh::MeshTools::Generation::build_cube (mesh,
> 10, 10, 5, 0.0, 2.0, 0.0,
> 3.0, 0.0, 4.0,libMesh::HEX8);
>
>
> if (rank == 0)
> {
> libMesh::VTKIO out(mesh);
> out.write("mesh1.pvtu");
> }
> }
> MPI_Finalize();
> return 0;
>
>
>
> }
>
> What do you think? Can you provide an access to
> libMesh::LibMeshInit._vtk_mpi_controller?
> Or there is a better solution?
>
I think it would be reasonable to provide a public accessor for
_vtk_mpi_controller, as long as we can still get away with a forward
declaration for vtkMPIController.
Would it be possible to do the same algorithm from within VTKIO::read()? At
that point, we have a Communicator (mesh.comm()) for the Mesh we are
currently writing, so we could probably do the same New/Split/Set series of
steps there, but unfortunately we don't have access to the LibMeshInit at
that point...
--
John
|
|
From: Michael P. <mpo...@pu...> - 2018-06-13 19:27:38
|
On 06/13/2018 03:15 PM, John Peterson wrote:
>
>
> On Wed, Jun 13, 2018 at 11:03 AM, Michael Povolotskyi
> <mpo...@pu... <mailto:mpo...@pu...>> wrote:
>
> Hello John,
>
> I played with it, and this is what worked for me.
>
> #include <vector>
> #include <sstream>
> using namespace std;
>
> #define private public//to get access to
> libMesh::LibMeshInit._vtk_mpi_controller
>
> #include "libmesh/mesh_generation.h"
> #include "libmesh/mesh.h"
> #include "libmesh/vtk_io.h"
> #include "libmesh/libmesh.h"
> #include "vtkMPIController.h"
> int main(int argc, char ** argv)
> {
>
> MPI_Init(&argc, &argv);
>
> {
> libMesh::LibMeshInit init (argc, argv,MPI_COMM_WORLD);
> int rank;
> MPI_Comm_rank(MPI_COMM_WORLD, &rank);
>
> vtkMPICommunicator* mcomm =
> vtkMPICommunicator::GetWorldCommunicator()->NewInstance ();
> mcomm->SplitInitialize (mcomm, rank, 0);
>
> init._vtk_mpi_controller->SetCommunicator (mcomm);
>
> libMesh::Mesh
> mesh(libMesh::Parallel::Communicator(MPI_COMM_SELF));
> libMesh::MeshTools::Generation::build_cube (mesh,
> 10, 10, 5, 0.0, 2.0, 0.0, 3.0, 0.0, 4.0,libMesh::HEX8);
>
>
> if (rank == 0)
> {
> libMesh::VTKIO out(mesh);
> out.write("mesh1.pvtu");
> }
> }
> MPI_Finalize();
> return 0;
>
>
>
> }
>
>
> What do you think? Can you provide an access to
> libMesh::LibMeshInit._vtk_mpi_controller?
> Or there is a better solution?
>
>
> I think it would be reasonable to provide a public accessor for
> _vtk_mpi_controller, as long as we can still get away with a forward
> declaration for vtkMPIController.
>
> Would it be possible to do the same algorithm from within
> VTKIO::read()? At that point, we have a Communicator (mesh.comm()) for
> the Mesh we are currently writing, so we could probably do the same
> New/Split/Set series of steps there, but unfortunately we don't have
> access to the LibMeshInit at that point...
>
> --
> John
Would you add the access to _vtk_mpi_controller for me?
I can do it myself, but I prefer to be consistent with you.
It looks to me that for a regular solution you need to reset the
vtkMPICommunicator before any read/write event. Can you store the
vtkMPICommunicator inside the mesh class?
Michael.
|
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X