From: Yujie <rec...@gm...> - 2007-08-28 19:26:31
|
Dear libmesh developers: Roy told me that METIS can work well in libmesh and ParMETIS can't work well. I want to know whether the problem has been solved. If not, where is the key problem? In addition, I want to confirm that libmesh will call METIS for mesh partitioning after libmesh reads the mesh data. thanks a lot. best regards, Yujie |
From: Roy S. <roy...@ic...> - 2007-08-28 19:51:44
|
On Tue, 28 Aug 2007, Yujie wrote: > Roy told me that METIS can work well in libmesh and ParMETIS can't work > well. I didn't say that ParMETIS can't work well, just that I thought it wasn't used by default. > I want to know whether the problem has been solved. If not, where is > the key problem? In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there doesn't seem to be an easy way for the user to choose a ParmetisPartitioner or something else without having to edit library code. We ought to add an API to MeshBase to let users supply their own instance of a Partitioner subclass. > In addition, I want to confirm that libmesh will call METIS for mesh > partitioning after libmesh reads the mesh data. The partitioner is called in MeshBase::prepare_for_use(), which gets called as soon as the mesh has been fully generated or loaded from a file. --- Roy |
From: Yujie <rec...@gm...> - 2007-08-28 20:51:46
|
Dear Roy: I am very sorry for misunderstanding what you said. However, it is likely difficult to call ParMETIS in libmesh? Whether will it affect the implementation of dynamic repartitioning in adaptive mesh refinement. I have found that a function (ParMETIS_V3_AdaptiveRepart) in ParMETIS is used to deal with the adaptively refined mesh. METIS uses the methods (coarsen, partition and refine) to partition the mesh, which most likely affects the element distribution. I don't confirm this. Could you give me some information about it? thanks a lot. Regards, Yujie On 8/28/07, Roy Stogner <roy...@ic...> wrote: > > On Tue, 28 Aug 2007, Yujie wrote: > > > Roy told me that METIS can work well in libmesh and ParMETIS can't work > > well. > > I didn't say that ParMETIS can't work well, just that I thought it > wasn't used by default. > > > I want to know whether the problem has been solved. If not, where is > > the key problem? > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there > doesn't seem to be an easy way for the user to choose a > ParmetisPartitioner or something else without having to edit library > code. We ought to add an API to MeshBase to let users supply their own > instance of a Partitioner subclass. > > > In addition, I want to confirm that libmesh will call METIS for mesh > > partitioning after libmesh reads the mesh data. > > The partitioner is called in MeshBase::prepare_for_use(), which gets > called as soon as the mesh has been fully generated or loaded from a > file. > --- > Roy > |
From: John P. <pet...@cf...> - 2007-08-28 20:12:39
|
Roy Stogner writes: > On Tue, 28 Aug 2007, Yujie wrote: > > > Roy told me that METIS can work well in libmesh and ParMETIS can't work > > well. > > I didn't say that ParMETIS can't work well, just that I thought it > wasn't used by default. > > > I want to know whether the problem has been solved. If not, where is > > the key problem? > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there > doesn't seem to be an easy way for the user to choose a > ParmetisPartitioner or something else without having to edit library > code. We ought to add an API to MeshBase to let users supply their own > instance of a Partitioner subclass. The old way was to build a Partitioner p outside the Mesh and call p.partition(mesh,10); This isn't sufficient in general since as you mentioned the Mesh repartitions itself during each call to prepare_for_use(). I agree the Mesh should contain AutoPtr<Partitioner> and by default this will be a MetisPartitioner. Do you think that would work? -J > > In addition, I want to confirm that libmesh will call METIS for mesh > > partitioning after libmesh reads the mesh data. > > The partitioner is called in MeshBase::prepare_for_use(), which gets > called as soon as the mesh has been fully generated or loaded from a > file. > --- > Roy |
From: Roy S. <roy...@ic...> - 2007-08-28 20:22:50
|
On Tue, 28 Aug 2007, John Peterson wrote: > This isn't sufficient in general since as you mentioned the Mesh > repartitions itself during each call to prepare_for_use(). I agree > the Mesh should contain AutoPtr<Partitioner> and by default this will > be a MetisPartitioner. Do you think that would work? The member variable sounds right, but I'd say that the default should depend on libmesh_config.h: a ParmetisPartitioner if HAVE_PARMETIS is true, or a MetisPartitioner if HAVE_METIS is true, or some type of SFCPartitioner if HAVE_SFCURVES is true, or a CentroidPartitioner as a last-ditch fallback. --- Roy |
From: John P. <pet...@cf...> - 2007-08-28 20:45:30
|
Roy Stogner writes: > On Tue, 28 Aug 2007, John Peterson wrote: > > > This isn't sufficient in general since as you mentioned the Mesh > > repartitions itself during each call to prepare_for_use(). I agree > > the Mesh should contain AutoPtr<Partitioner> and by default this will > > be a MetisPartitioner. Do you think that would work? > > The member variable sounds right, but I'd say that the default should > depend on libmesh_config.h: a ParmetisPartitioner if HAVE_PARMETIS is > true, or a MetisPartitioner if HAVE_METIS is true, or some type of > SFCPartitioner if HAVE_SFCURVES is true, or a CentroidPartitioner as a > last-ditch fallback. Up to this time, the ParMetis code has gone largely untested. That would be my only qualm against making it a default. As long as we don't have a truly parallel mesh, there really is no reason to use it anyway. -J |
From: Yujie <rec...@gm...> - 2007-08-28 21:02:10
|
Dear John: "Up to this time, the ParMetis code has gone largely untested. That would be my only qualm against making it a default. As long as we don't have a truly parallel mesh, there really is no reason to use it anyway." You mean that ParMETIS have been thoroughly tested by its authors or you? Why do you mention "As long as we don't have a truly parallel mesh"? You mean what the truly parallel mesh is? Thanks a lot. Yujie On 8/28/07, John Peterson <pet...@cf...> wrote: > > Roy Stogner writes: > > On Tue, 28 Aug 2007, Yujie wrote: > > > > > Roy told me that METIS can work well in libmesh and ParMETIS can't > work > > > well. > > > > I didn't say that ParMETIS can't work well, just that I thought it > > wasn't used by default. > > > > > I want to know whether the problem has been solved. If not, where is > > > the key problem? > > > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there > > doesn't seem to be an easy way for the user to choose a > > ParmetisPartitioner or something else without having to edit library > > code. We ought to add an API to MeshBase to let users supply their own > > instance of a Partitioner subclass. > The old way was to build a Partitioner p outside the Mesh and call > > p.partition(mesh,10); > > This isn't sufficient in general since as you mentioned the Mesh > repartitions itself during each call to prepare_for_use(). I agree > the Mesh should contain AutoPtr<Partitioner> and by default this will > be a MetisPartitioner. Do you think that would work? > > -J > > > > > In addition, I want to confirm that libmesh will call METIS for mesh > > > partitioning after libmesh reads the mesh data. > > > > The partitioner is called in MeshBase::prepare_for_use(), which gets > > called as soon as the mesh has been fully generated or loaded from a > > file. > > --- > > Roy > |
From: Derek G. <fri...@gm...> - 2007-08-28 21:35:28
|
So.... ParMetis is for partitioning a mesh that is already split up and each piece _only_ lives on one processor. So if you have 4 elements and you are running on 2 processors... each processor only has a copy of 2 elements. So repartitioning the mesh is a difficult problem because no one processor holds an entire copy of the mesh. This is what ParMetis is good at. In libMesh this isn't the case. In the example above each processor actually holds a copy of all 4 elements. This means that each processor calls METIS when the mesh changes and they all repartition their own copy of the mesh in exactly the same way. There is no parallel communication necessary to repartition the mesh This is why we prefer using METIS right now... because it's fast... and it works for the way we have our mesh setup. But like I mentioned we are planning on transitioning to a mesh that is "truly parallel" in that each processor only has a copy of a couple of elements and no one processor owns a full copy of the mesh.... and when we do so we'll have to switch over to using parmetis. I hope that clears things up. Derek On 8/28/07, Yujie <rec...@gm...> wrote: > Dear John: > > "Up to this time, the ParMetis code has gone largely untested. That would > be my only qualm against making it a default. As long as we don't have a > truly parallel mesh, there really is no reason to use it anyway." > > You mean that ParMETIS have been thoroughly tested by its authors or you? > Why do you mention "As long as we don't have a > truly parallel mesh"? You mean what the truly parallel mesh is? > > Thanks a lot. > > Yujie > > > > On 8/28/07, John Peterson <pet...@cf...> wrote: > > Roy Stogner writes: > > > On Tue, 28 Aug 2007, Yujie wrote: > > > > > > > Roy told me that METIS can work well in libmesh and ParMETIS can't > work > > > > well. > > > > > > I didn't say that ParMETIS can't work well, just that I thought it > > > wasn't used by default. > > > > > > > I want to know whether the problem has been solved. If not, where is > > > > the key problem? > > > > > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there > > > doesn't seem to be an easy way for the user to choose a > > > ParmetisPartitioner or something else without having to edit library > > > code. We ought to add an API to MeshBase to let users supply their own > > > instance of a Partitioner subclass. > > The old way was to build a Partitioner p outside the Mesh and call > > > > p.partition(mesh,10); > > > > This isn't sufficient in general since as you mentioned the Mesh > > repartitions itself during each call to prepare_for_use(). I agree > > the Mesh should contain AutoPtr<Partitioner> and by default this will > > be a MetisPartitioner. Do you think that would work? > > > > -J > > > > > > > > In addition, I want to confirm that libmesh will call METIS for mesh > > > > partitioning after libmesh reads the mesh data. > > > > > > The partitioner is called in MeshBase::prepare_for_use(), which gets > > > called as soon as the mesh has been fully generated or loaded from a > > > file. > > > --- > > > Roy > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > > |
From: Derek G. <fri...@gm...> - 2007-08-28 21:05:00
|
Forgot to copy my response to the list. ---------- Forwarded message ---------- From: Derek Gaston <fri...@gm...> Date: Aug 28, 2007 2:58 PM Subject: Re: [Libmesh-users] Is ParMETIS available in libmesh? To: Yujie <rec...@gm...> The deal is that the mesh isn't truly parallel in libMesh.... it is actually copied to every node and only the calculations are actually done in parallel. Because of this there isn't much difference between using METIS and PARMETIS. I have used some of the parmetis partitioners in libMesh when I was doing some testing... and if I remember correctly didn't find any difference between them and METIS for libMesh. We are now looking at parallelizing the mesh in libMesh at which point we will have to switch to using PARMETIS... and taking advantage of things like Adaptive Repartitioning... but right now there is no reason to. Derek On 8/28/07, Yujie <rec...@gm...> wrote: > Dear Roy: > > I am very sorry for misunderstanding what you said. However, it is likely > difficult to call ParMETIS in libmesh? Whether will it affect the > implementation of dynamic repartitioning in adaptive mesh refinement. I have > found that a function (ParMETIS_V3_AdaptiveRepart) in ParMETIS is used to > deal with the adaptively refined mesh. METIS uses the methods (coarsen, > partition and refine) to partition the mesh, which most likely affects the > element distribution. I don't confirm this. Could you give me some > information about it? > > thanks a lot. > > Regards, > Yujie > > > > On 8/28/07, Roy Stogner <roy...@ic...> wrote: > > On Tue, 28 Aug 2007, Yujie wrote: > > > > > Roy told me that METIS can work well in libmesh and ParMETIS can't work > > > well. > > > > I didn't say that ParMETIS can't work well, just that I thought it > > wasn't used by default. > > > > > I want to know whether the problem has been solved. If not, where is > > > the key problem? > > > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there > > doesn't seem to be an easy way for the user to choose a > > ParmetisPartitioner or something else without having to edit library > > code. We ought to add an API to MeshBase to let users supply their own > > instance of a Partitioner subclass. > > > > > In addition, I want to confirm that libmesh will call METIS for mesh > > > partitioning after libmesh reads the mesh data. > > > > The partitioner is called in MeshBase::prepare_for_use(), which gets > > called as soon as the mesh has been fully generated or loaded from a > > file. > > --- > > Roy > > > > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > > |
From: Yujie <rec...@gm...> - 2007-08-28 21:11:42
|
Dear Derek: What is the truly parallel mesh? Currently, is it difficult to realize the truly parallel mesh in libmesh? In the paper about ParMETIS, the partitionning speed is improved because of the parallel utilization. In addition, dynamic repartitioning is another key problem to call ParMETIS in libmesh, I think. thanks, Yujie On 8/28/07, Derek Gaston <fri...@gm...> wrote: > > The deal is that the mesh isn't truly parallel in libMesh.... it is > actually copied to every node and only the calculations are actually > done in parallel. > > Because of this there isn't much difference between using METIS and > PARMETIS. I have used some of the parmetis partitioners in libMesh > when I was doing some testing... and if I remember correctly didn't > find any difference between them and METIS for libMesh. > > We are now looking at parallelizing the mesh in libMesh at which point > we will have to switch to using PARMETIS... and taking advantage of > things like Adaptive Repartitioning... but right now there is no > reason to. > > Derek > > On 8/28/07, Yujie <rec...@gm...> wrote: > > Dear Roy: > > > > I am very sorry for misunderstanding what you said. However, it is > likely > > difficult to call ParMETIS in libmesh? Whether will it affect the > > implementation of dynamic repartitioning in adaptive mesh refinement. I > have > > found that a function (ParMETIS_V3_AdaptiveRepart) in ParMETIS is used > to > > deal with the adaptively refined mesh. METIS uses the methods (coarsen, > > partition and refine) to partition the mesh, which most likely affects > the > > element distribution. I don't confirm this. Could you give me some > > information about it? > > > > thanks a lot. > > > > Regards, > > Yujie > > > > > > > > On 8/28/07, Roy Stogner <roy...@ic...> wrote: > > > On Tue, 28 Aug 2007, Yujie wrote: > > > > > > > Roy told me that METIS can work well in libmesh and ParMETIS can't > work > > > > well. > > > > > > I didn't say that ParMETIS can't work well, just that I thought it > > > wasn't used by default. > > > > > > > I want to know whether the problem has been solved. If not, where is > > > > the key problem? > > > > > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and there > > > doesn't seem to be an easy way for the user to choose a > > > ParmetisPartitioner or something else without having to edit library > > > code. We ought to add an API to MeshBase to let users supply their > own > > > instance of a Partitioner subclass. > > > > > > > In addition, I want to confirm that libmesh will call METIS for mesh > > > > partitioning after libmesh reads the mesh data. > > > > > > The partitioner is called in MeshBase::prepare_for_use(), which gets > > > called as soon as the mesh has been fully generated or loaded from a > > > file. > > > --- > > > Roy > > > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Libmesh-users mailing list > > Lib...@li... > > https://lists.sourceforge.net/lists/listinfo/libmesh-users > > > > > |
From: Roy S. <roy...@ic...> - 2007-08-28 22:06:38
|
On Tue, 28 Aug 2007, Yujie wrote: > What is the truly parallel mesh? A "truly parallel" mesh would be one where each processor only stores it's own "local" elements and a few neighboring "ghost" elements, and no single processor knows about the entire mesh. > Currently, is it difficult to realize the truly parallel mesh in libmesh? It depends on your definition of difficult, and it's always hard to predict in advance how long software development will take, but as a wild guess I'd say that getting the core functionality working with a truly parallel mesh would take 2-4 solid man-months of effort. It would take even more if the developers weren't already familiar with libMesh internals, which would probably be the case since all the primary libMesh developers have too much else to do right now! ;-) At the moment we scale well to dozens of processors, and for our own research the time we'd save by running on hundreds or thousands of CPUs would be vastly outweighed by the time we'd have to spend coding it. Currently the only experienced developer who would greatly benefit from a fully parallel mesh class is Derek, because his time isn't just wasted waiting for simulations to finish, it's wasted working with another finite element library that does have fully parallel meshes but also fully unpleasant APIs. > In the paper about ParMETIS, the partitionning speed is improved > because of the parallel utilization. I'm sure it is, but as Derek said the partitioning doesn't take much of our time in typical applications. > In addition, dynamic repartitioning is another key problem to call > ParMETIS in libmesh, I think. I believe we do dynamic repartitioning by simply throwing a refined/coarsened mesh at a partitioner which has no knowledge of the previous partitioning. Obviously this won't be as efficient, but it works. --- Roy |
From: li p. <li...@ya...> - 2007-08-29 13:02:25
|
hi, at this point I would like to ask, how many system matrix and solution will be generated. I mean if I run parallel code in a cluster, does it mean there is one system matrix and system solution? And each node manages one part of the matrix and solution vector? thanx pan --- Roy Stogner <roy...@ic...> wrote: > On Tue, 28 Aug 2007, Yujie wrote: > > > What is the truly parallel mesh? > > A "truly parallel" mesh would be one where each > processor only stores > it's own "local" elements and a few neighboring > "ghost" elements, and > no single processor knows about the entire mesh. > > > Currently, is it difficult to realize the truly > parallel mesh in libmesh? > > It depends on your definition of difficult, and it's > always hard to > predict in advance how long software development > will take, but as a > wild guess I'd say that getting the core > functionality working with a > truly parallel mesh would take 2-4 solid man-months > of effort. It > would take even more if the developers weren't > already familiar with > libMesh internals, which would probably be the case > since all the > primary libMesh developers have too much else to do > right now! ;-) > > At the moment we scale well to dozens of processors, > and for our own > research the time we'd save by running on hundreds > or thousands of > CPUs would be vastly outweighed by the time we'd > have to spend coding > it. Currently the only experienced developer who > would greatly > benefit from a fully parallel mesh class is Derek, > because his time > isn't just wasted waiting for simulations to finish, > it's wasted > working with another finite element library that > does have fully > parallel meshes but also fully unpleasant APIs. > > > In the paper about ParMETIS, the partitionning > speed is improved > > because of the parallel utilization. > > I'm sure it is, but as Derek said the partitioning > doesn't take much > of our time in typical applications. > > > In addition, dynamic repartitioning is another key > problem to call > > ParMETIS in libmesh, I think. > > I believe we do dynamic repartitioning by simply > throwing a > refined/coarsened mesh at a partitioner which has no > knowledge of the > previous partitioning. Obviously this won't be as > efficient, but it > works. > --- > Roy > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? > Stop. > Now Search log events and configuration files using > AJAX and a browser. > Download your FREE copy of Splunk now >> > http://get.splunk.com/ > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > ____________________________________________________________________________________ Looking for a deal? Find great prices on flights and hotels with Yahoo! FareChase. http://farechase.yahoo.com/ |
From: David K. <dav...@gm...> - 2007-08-29 13:07:38
|
> hi, > at this point I would like to ask, how many system > matrix and solution will be generated. > I mean if I run parallel code in a cluster, does it > mean there is one system matrix and system solution? > And each node manages one part of the matrix and > solution vector? Yes, exactly. And a point that has attracted a lot of attention on the list lately is that by contrast the mesh is stored in full on every node. - Dave > --- Roy Stogner <roy...@ic...> wrote: > >> On Tue, 28 Aug 2007, Yujie wrote: >> >>> What is the truly parallel mesh? >> >> A "truly parallel" mesh would be one where each >> processor only stores >> it's own "local" elements and a few neighboring >> "ghost" elements, and >> no single processor knows about the entire mesh. >> >>> Currently, is it difficult to realize the truly >> parallel mesh in libmesh? >> >> It depends on your definition of difficult, and it's >> always hard to >> predict in advance how long software development >> will take, but as a >> wild guess I'd say that getting the core >> functionality working with a >> truly parallel mesh would take 2-4 solid man-months >> of effort. It >> would take even more if the developers weren't >> already familiar with >> libMesh internals, which would probably be the case >> since all the >> primary libMesh developers have too much else to do >> right now! ;-) >> >> At the moment we scale well to dozens of processors, >> and for our own >> research the time we'd save by running on hundreds >> or thousands of >> CPUs would be vastly outweighed by the time we'd >> have to spend coding >> it. Currently the only experienced developer who >> would greatly >> benefit from a fully parallel mesh class is Derek, >> because his time >> isn't just wasted waiting for simulations to finish, >> it's wasted >> working with another finite element library that >> does have fully >> parallel meshes but also fully unpleasant APIs. >> >>> In the paper about ParMETIS, the partitionning >> speed is improved >>> because of the parallel utilization. >> >> I'm sure it is, but as Derek said the partitioning >> doesn't take much >> of our time in typical applications. >> >>> In addition, dynamic repartitioning is another key >> problem to call >>> ParMETIS in libmesh, I think. >> >> I believe we do dynamic repartitioning by simply >> throwing a >> refined/coarsened mesh at a partitioner which has no >> knowledge of the >> previous partitioning. Obviously this won't be as >> efficient, but it >> works. >> --- >> Roy >> >> > ---------------------------------------------------------------------- > --- >> This SF.net email is sponsored by: Splunk Inc. >> Still grepping through log files to find problems? >> Stop. >> Now Search log events and configuration files using >> AJAX and a browser. >> Download your FREE copy of Splunk now >> >> http://get.splunk.com/ >> _______________________________________________ >> Libmesh-users mailing list >> Lib...@li... >> > https://lists.sourceforge.net/lists/listinfo/libmesh-users >> > > > > > ______________________________________________________________________ > ______________ > Looking for a deal? Find great prices on flights and hotels with > Yahoo! FareChase. > http://farechase.yahoo.com/ > > ---------------------------------------------------------------------- > --- > This SF.net email is sponsored by: Splunk Inc. > Still grepping through log files to find problems? Stop. > Now Search log events and configuration files using AJAX and a > browser. > Download your FREE copy of Splunk now >> http://get.splunk.com/ > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users |
From: Yujie <rec...@gm...> - 2007-08-28 21:29:33
|
Dear Derek: That is, after finishing an adaptive mesh refinement, all the processors need to call METIS for repartitioning. How to guarantee the identical repartitioning mesh is obtained in each processor, the same parameters setup? Whether can the element distribution be guaruateed after adaptive mesh refinement? Truly parallel mesh is a difficult problem, especially with the increase of mesh data. Is there a better method to solve it? thanks, Yujie On 8/28/07, Derek Gaston <fri...@gm...> wrote: > > So.... ParMetis is for partitioning a mesh that is already split up > and each piece _only_ lives on one processor. So if you have 4 > elements and you are running on 2 processors... each processor only > has a copy of 2 elements. So repartitioning the mesh is a difficult > problem because no one processor holds an entire copy of the mesh. > This is what ParMetis is good at. > > In libMesh this isn't the case. In the example above each processor > actually holds a copy of all 4 elements. This means that each > processor calls METIS when the mesh changes and they all repartition > their own copy of the mesh in exactly the same way. There is no > parallel communication necessary to repartition the mesh > > This is why we prefer using METIS right now... because it's fast... > and it works for the way we have our mesh setup. But like I mentioned > we are planning on transitioning to a mesh that is "truly parallel" in > that each processor only has a copy of a couple of elements and no one > processor owns a full copy of the mesh.... and when we do so we'll > have to switch over to using parmetis. > > I hope that clears things up. > > Derek > > On 8/28/07, Yujie <rec...@gm...> wrote: > > Dear John: > > > > "Up to this time, the ParMetis code has gone largely untested. That > would > > be my only qualm against making it a default. As long as we don't have > a > > truly parallel mesh, there really is no reason to use it anyway." > > > > You mean that ParMETIS have been thoroughly tested by its authors or > you? > > Why do you mention "As long as we don't have a > > truly parallel mesh"? You mean what the truly parallel mesh is? > > > > Thanks a lot. > > > > Yujie > > > > > > > > On 8/28/07, John Peterson <pet...@cf...> wrote: > > > Roy Stogner writes: > > > > On Tue, 28 Aug 2007, Yujie wrote: > > > > > > > > > Roy told me that METIS can work well in libmesh and ParMETIS can't > > work > > > > > well. > > > > > > > > I didn't say that ParMETIS can't work well, just that I thought it > > > > wasn't used by default. > > > > > > > > > I want to know whether the problem has been solved. If not, where > is > > > > > the key problem? > > > > > > > > In src/mesh/mesh_base.C, we build a MetisPartitioner object, and > there > > > > doesn't seem to be an easy way for the user to choose a > > > > ParmetisPartitioner or something else without having to edit library > > > > code. We ought to add an API to MeshBase to let users supply their > own > > > > instance of a Partitioner subclass. > > > The old way was to build a Partitioner p outside the Mesh and call > > > > > > p.partition(mesh,10); > > > > > > This isn't sufficient in general since as you mentioned the Mesh > > > repartitions itself during each call to prepare_for_use(). I agree > > > the Mesh should contain AutoPtr<Partitioner> and by default this will > > > be a MetisPartitioner. Do you think that would work? > > > > > > -J > > > > > > > > > > > In addition, I want to confirm that libmesh will call METIS for > mesh > > > > > partitioning after libmesh reads the mesh data. > > > > > > > > The partitioner is called in MeshBase::prepare_for_use(), which gets > > > > called as soon as the mesh has been fully generated or loaded from a > > > > file. > > > > --- > > > > Roy > > > > > > > > > > ------------------------------------------------------------------------- > > This SF.net email is sponsored by: Splunk Inc. > > Still grepping through log files to find problems? Stop. > > Now Search log events and configuration files using AJAX and a browser. > > Download your FREE copy of Splunk now >> http://get.splunk.com/ > > _______________________________________________ > > Libmesh-users mailing list > > Lib...@li... > > https://lists.sourceforge.net/lists/listinfo/libmesh-users > > > > > |
From: Roy S. <roy...@ic...> - 2007-08-28 22:12:57
|
On Tue, 28 Aug 2007, Yujie wrote: > That is, after finishing an adaptive mesh refinement, all the processors > need to call METIS for repartitioning. How to guarantee the identical > repartitioning mesh is obtained in each processor, the same parameters > setup? Whether can the element distribution be guaruateed after adaptive > mesh refinement? Yes, and yes. The process is deterministic, and gets the same results on each processor even with an adaptively refined mesh. > Truly parallel mesh is a difficult problem, especially with the > increase of mesh data. Is there a better method to solve it? Well, our method of running parallel calculations on serial meshes actually isn't bad as long as you've got a problem where the mesh doesn't take up too much RAM on every node. To me the only obvious sort of improvement to that other than fully parallelizing the mesh would be to use a "compressed" mesh, a block structured mesh, a two-level macroelement mesh, a coarse mesh and refinement tree with child elements created on the fly during calculations... something like that which would make serial meshes with more elements fit into memory. --- Roy |
From: Yujie <rec...@gm...> - 2007-08-28 22:45:45
|
Thanks so much, guys:). Thanks for your help. Regards, Yujie On 8/28/07, Roy Stogner <roy...@ic...> wrote: > > On Tue, 28 Aug 2007, Yujie wrote: > > > That is, after finishing an adaptive mesh refinement, all the processors > > need to call METIS for repartitioning. How to guarantee the identical > > repartitioning mesh is obtained in each processor, the same parameters > > setup? Whether can the element distribution be guaruateed after adaptive > > mesh refinement? > > Yes, and yes. The process is deterministic, and gets the same results > on each processor even with an adaptively refined mesh. > > > Truly parallel mesh is a difficult problem, especially with the > > increase of mesh data. Is there a better method to solve it? > > Well, our method of running parallel calculations on serial meshes > actually isn't bad as long as you've got a problem where the mesh > doesn't take up too much RAM on every node. To me the only obvious > sort of improvement to that other than fully parallelizing the mesh > would be to use a "compressed" mesh, a block structured mesh, a > two-level macroelement mesh, a coarse mesh and refinement tree with > child elements created on the fly during calculations... something > like that which would make serial meshes with more elements fit into > memory. > --- > Roy > |