From: luyi <lu...@ma...> - 2008-05-12 03:18:48
|
Hello, Now I write a DG programme on libmesh, when I use the FIRST or higher order basis, the first step is too slow, besides this everything is ok, I use XYZ basis and I check the code by perf_log, the time ratio is up to snuff. I want to know when you run the millions dof progamme, how long the first step cost? thank you! Luyi |
From: Roy S. <ro...@st...> - 2008-05-12 03:31:38
|
On Mon, 12 May 2008, luyi wrote: > Now I write a DG programme on libmesh, when I use the FIRST or higher > order basis, the first step is too slow, > besides this everything is ok, I use XYZ basis and I check the code by > perf_log, the time ratio is up to snuff. The first time step (especially in a non-adaptive run) has some overhead that later timesteps don't, but it's also possible you've found a bug. If we're not preallocating the sparse matrix with the correct sparsity pattern, for example, PETSc might take quite a long time on the first assembly. Could you run your code for only one time step and check the perf_log on that, to see what part of the code is taking too long? --- Roy |
From: Kirk, B. (JSC-EG) <ben...@na...> - 2008-05-12 17:34:53
|
Can you try running the code with --implicit_neighbor_dofs on the command line? By default the matrix sparsity is computed assuming continuous shape functions, but for a DG forumation this is not a good assumption. Let me know if that works for you, if so we can look at putting something more sophisticated in the library to detect this situation and handling it automatically. -Ben ________________________________ From: lib...@li... on behalf of Roy Stogner Sent: Sun 5/11/2008 10:31 PM To: luyi Cc: lib...@li... Subject: Re: [Libmesh-users] about the long time for the assemble of first step! On Mon, 12 May 2008, luyi wrote: > Now I write a DG programme on libmesh, when I use the FIRST or higher > order basis, the first step is too slow, > besides this everything is ok, I use XYZ basis and I check the code by > perf_log, the time ratio is up to snuff. The first time step (especially in a non-adaptive run) has some overhead that later timesteps don't, but it's also possible you've found a bug. If we're not preallocating the sparse matrix with the correct sparsity pattern, for example, PETSc might take quite a long time on the first assembly. Could you run your code for only one time step and check the perf_log on that, to see what part of the code is taking too long? --- Roy ------------------------------------------------------------------------- This SF.net email is sponsored by the 2008 JavaOne(SM) Conference Don't miss this year's exciting event. There's still time to save $100. Use priority code J8TL2D2. http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone _______________________________________________ Libmesh-users mailing list Lib...@li... https://lists.sourceforge.net/lists/listinfo/libmesh-users |
From: luyi <lu...@ma...> - 2008-05-13 00:57:44
|
Hello, There is another problem, my programme contains implicit backward euler and explicit runge-kuuta , for RK, there is no offset matrix but if I use high order(SECOND or higher) it is also need much time to finish the first step, I use the transientLinearImplicitSystem to construct RKDG, the perf_log is as follow: ------------------------------------------------------------------------------ | Matrix Assembly Performance: Alive time=747.555, Active time=746.16 | ------------------------------------------------------------------------------ | Event nCalls Total Avg Percent of | | Time Time Active Time | |------------------------------------------------------------------------------| | | | bounray 402 0.0045 0.000011 0.00 | | elem init 4410 0.1852 0.000042 0.02 | | interior 12828 0.4093 0.000032 0.05 | | mass matrix 30870 0.1772 0.000006 0.02 | | matrix insertion 4410 745.3840 0.169021 99.90 | ------------------------------------------------------------------------------ | Totals: 52920 746.1602 100.00 | ------------------------------------------------------------------------------ It is only Ke and Fe like the continuous programme, maybe for all discontinuous galerkin there is the same problem? Thank you very much! Luyi Kirk, Benjamin (JSC-EG) 写道: > Can you try running the code with --implicit_neighbor_dofs on the > command line? > By default the matrix sparsity is computed assuming continuous shape > functions, but for a DG forumation this is not a good assumption. > Let me know if that works for you, if so we can look at putting > something more sophisticated in the library to detect this situation > and handling it automatically. > -Ben > > ------------------------------------------------------------------------ > *From:* lib...@li... on behalf of Roy > Stogner > *Sent:* Sun 5/11/2008 10:31 PM > *To:* luyi > *Cc:* lib...@li... > *Subject:* Re: [Libmesh-users] about the long time for the assemble of > first step! > > > On Mon, 12 May 2008, luyi wrote: > > > Now I write a DG programme on libmesh, when I use the FIRST or higher > > order basis, the first step is too slow, > > besides this everything is ok, I use XYZ basis and I check the code by > > perf_log, the time ratio is up to snuff. > > The first time step (especially in a non-adaptive run) has some > overhead that later timesteps don't, but it's also possible you've > found a bug. If we're not preallocating the sparse matrix with the > correct sparsity pattern, for example, PETSc might take quite a long > time on the first assembly. > > Could you run your code for only one time step and check the perf_log > on that, to see what part of the code is taking too long? > --- > Roy > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > |
From: luyi <lu...@ma...> - 2008-05-20 02:14:34
|
Hello,Ben, I change my libmesh to SVN code and the implicit part works well with --implicit_neighbor_dofs on the command line, the first step is much faster than libmesh0.62. Thank you very much. Kirk, Benjamin (JSC-EG) 写道: > Can you try running the code with --implicit_neighbor_dofs on the > command line? > By default the matrix sparsity is computed assuming continuous shape > functions, but for a DG forumation this is not a good assumption. > Let me know if that works for you, if so we can look at putting > something more sophisticated in the library to detect this situation > and handling it automatically. > -Ben > > ------------------------------------------------------------------------ > *From:* lib...@li... on behalf of Roy > Stogner > *Sent:* Sun 5/11/2008 10:31 PM > *To:* luyi > *Cc:* lib...@li... > *Subject:* Re: [Libmesh-users] about the long time for the assemble of > first step! > > > On Mon, 12 May 2008, luyi wrote: > > > Now I write a DG programme on libmesh, when I use the FIRST or higher > > order basis, the first step is too slow, > > besides this everything is ok, I use XYZ basis and I check the code by > > perf_log, the time ratio is up to snuff. > > The first time step (especially in a non-adaptive run) has some > overhead that later timesteps don't, but it's also possible you've > found a bug. If we're not preallocating the sparse matrix with the > correct sparsity pattern, for example, PETSc might take quite a long > time on the first assembly. > > Could you run your code for only one time step and check the perf_log > on that, to see what part of the code is taking too long? > --- > Roy > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > |
From: luyi <lu...@ma...> - 2008-05-13 02:07:22
|
The CONSTANT basis for both implicit and RK is OK but higher order need much more time because I use elem-based basis, maybe the problem is this? Thank you very much and Best regards! Luyi Kirk, Benjamin (JSC-EG) 写道: > Can you try running the code with --implicit_neighbor_dofs on the > command line? > By default the matrix sparsity is computed assuming continuous shape > functions, but for a DG forumation this is not a good assumption. > Let me know if that works for you, if so we can look at putting > something more sophisticated in the library to detect this situation > and handling it automatically. > -Ben > > ------------------------------------------------------------------------ > *From:* lib...@li... on behalf of Roy > Stogner > *Sent:* Sun 5/11/2008 10:31 PM > *To:* luyi > *Cc:* lib...@li... > *Subject:* Re: [Libmesh-users] about the long time for the assemble of > first step! > > > On Mon, 12 May 2008, luyi wrote: > > > Now I write a DG programme on libmesh, when I use the FIRST or higher > > order basis, the first step is too slow, > > besides this everything is ok, I use XYZ basis and I check the code by > > perf_log, the time ratio is up to snuff. > > The first time step (especially in a non-adaptive run) has some > overhead that later timesteps don't, but it's also possible you've > found a bug. If we're not preallocating the sparse matrix with the > correct sparsity pattern, for example, PETSc might take quite a long > time on the first assembly. > > Could you run your code for only one time step and check the perf_log > on that, to see what part of the code is taking too long? > --- > Roy > > ------------------------------------------------------------------------- > This SF.net email is sponsored by the 2008 JavaOne(SM) Conference > Don't miss this year's exciting event. There's still time to save $100. > Use priority code J8TL2D2. > http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone > _______________________________________________ > Libmesh-users mailing list > Lib...@li... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > |
From: Roy S. <ro...@st...> - 2008-05-13 03:09:35
|
On Tue, 13 May 2008, luyi wrote: > The CONSTANT basis for both implicit and RK is OK but higher order need > much more time because > > I use elem-based basis, maybe the problem is this? You're probably just not getting the correct sparsity pattern preallocation because libMesh isn't expecting you to be doing a DG method, and so isn't expecting direct coupling between the elem-based DoFs in neighboring elements. I'm surprised CONSTANT worked okay if that's the problem, though, unless we have some half-working autodetection code in there that I don't know about. But my important quesiton: Did you try Ben's command line option suggestion yet? There ought to be a better way to autodetect DG-like DoF coupling, but unless we do a "fake" system assembly on a two-element mesh, I don't see how to make it seamless. For now, the problem is probably just "you need to explicitly tell libMesh you're doing DG". > Thank you very much and Best regards! If you're feeling especially grateful, you might think about contributing a simple DG libMesh example once your own code is straightened out. All the main developers are pretty swamped right now, but that's something we've needed for a while. --- Roy |
From: luyi <lu...@ma...> - 2008-05-14 00:17:25
|
Hello,Roy, I change the RKDG code to ExplicitSystem and I write a small Gassian elimination solver, it works well and I can get high order results. I have tried the command line "--implicit_neighbor_dofs" and it no improvment. I ask Ben yesterday, can I write a GMRES+ILU(0) solver running in each processor and only exchange the solution between partitioned mesh, I think it is the same as the PETSC solver because its preconditioner also make ILU(0) in each processor, if that I can set the sparse matrix pattern by myself and I think the the programme can be runned faster. Can libmesh run without PETSC but only with mpi? I'd like to send a DG example after several days, libmesh is a so excellent library and thank you for your kindly reply. Best regards! Luyi 2008-5-14 Roy Stogner 写道: > > On Tue, 13 May 2008, luyi wrote: > >> The CONSTANT basis for both implicit and RK is OK but higher order need >> much more time because >> >> I use elem-based basis, maybe the problem is this? > > You're probably just not getting the correct sparsity pattern > preallocation because libMesh isn't expecting you to be doing a DG > method, and so isn't expecting direct coupling between the elem-based > DoFs in neighboring elements. I'm surprised CONSTANT worked okay if > that's the problem, though, unless we have some half-working > autodetection code in there that I don't know about. > > But my important quesiton: > > Did you try Ben's command line option suggestion yet? There ought to > be a better way to autodetect DG-like DoF coupling, but unless we do a > "fake" system assembly on a two-element mesh, I don't see how to make > it seamless. For now, the problem is probably just "you need to > explicitly tell libMesh you're doing DG". > >> Thank you very much and Best regards! > > If you're feeling especially grateful, you might think about > contributing a simple DG libMesh example once your own code is > straightened out. All the main developers are pretty swamped right > now, but that's something we've needed for a while. > --- > Roy > > |
From: Roy S. <ro...@st...> - 2008-05-14 00:31:05
|
On Wed, 14 May 2008, luyi wrote: > I change the RKDG code to ExplicitSystem and I write a small Gassian > elimination solver, it works well and I can get high order results. > > > I have tried the command line "--implicit_neighbor_dofs" and it no > improvment. We may have a bug in sparsity pattern allocation for DG problems, then. If your DG example makes the slowdown reproduceable on our systems, hopefully one of us can find the time to dig into the DofMap code and figure out the problem. > Can libmesh run without PETSC but only with mpi? In theory, yes; that should give you a libMesh DistributedVector instead of a PetscVector as the default NumericVector implementation. None of us run that way right now, though, so I can't guarantee there haven't been any regressions in that code. There's also no linear solver implementation (or even sparse matrix implementation, IIRC) for that case, so you'll have to write your own. If that works well for you, I'd encourage you to switch from ExplicitSystem back to ImplicitSystem, but contribute your matrix/solver code to the library. PETSc has a thousand features and a million options, and we should be getting Trilinos support sooner or later, but having a lightweight solver internal to the library would be nice too. > I'd like to send a DG example after several days, libmesh is a so excellent > library and thank you for your kindly reply. Thanks! --- Roy |
From: Lorenzo B. <bot...@gm...> - 2008-05-14 07:50:26
|
2008/5/14 luyi <lu...@ma...>: > I have tried the command line "--implicit_neighbor_dofs" and it no > improvment. This sound strange to me... I have some DG code and without this option the assembly is more than ten time slower because I can't get the right sparsity pattern. The difference could be that I use MONOMIAL basis, I have never tried XYZ. I have already send this short example to Roy some time ago and I hope it could be useful for you. Can I ask you what pde are you solving. Lorenzo |
From: luyi <lu...@ma...> - 2008-05-16 05:33:57
|
Thank you very much for your hint. There is another strange problem, I change my RKDG to MOMOMIAL but I can only ran succesful when I use CONSTANT, SECOND and FOURTH basis, FIRST and THIRD is unsteady, can your epplitic code works well with FIRST or THIRD? It is so strange, when I use XYZ basis, CONSTANT and FIRST is OK but SECOND can not work. I will change my implicit code to monomial later and connect you. My code solve euler equation and I am trying to solve NS equaiton later. Best Regards! Luyi Lorenzo Botti 写道: > 2008/5/14 luyi <lu...@ma...>: > > >> I have tried the command line "--implicit_neighbor_dofs" and it no >> improvment. >> > > This sound strange to me... > I have some DG code and without this option the assembly > is more than ten time slower because I can't get the right sparsity pattern. > The difference could be that I use MONOMIAL basis, I have never tried XYZ. > I have already send this short example to Roy some time ago and I hope > it could be useful for you. > Can I ask you what pde are you solving. > > Lorenzo > |
From: Lorenzo B. <bot...@gm...> - 2008-05-16 08:11:23
|
> There is another strange problem, I change my RKDG to MOMOMIAL but I can > only > > ran succesful when I use CONSTANT, SECOND and FOURTH basis, FIRST and THIRD > > is unsteady, can your epplitic code works well with FIRST or THIRD? Yes, it works. I have tested it till P8 and as you can see libMesh does both h, p, and hp refinement. > My code solve euler equation and I am trying to solve NS equaiton later. I'm interested in solving NS too. Actually I have also some code for the Stokes equation but I miss the convective flux treatment and so I would like to implement an Euler solver in the near future. I wish I had time to take a look at the matrix free approach. My Stokes code uses an artificial compressibility flux approach but I'm not happy of the convergence I can achive with GMRES in large 3D problems. I add a couple of screenshot of the stokes solution of the lid-driven cavity with P4 and h_ref, Re is 400. I'm very interested to know what kind of scheme you are using and how it performs with libMesh. Thank you very much for the reply. Lorenzo |