Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project!

## libmesh-users

 [Libmesh-users] Associating more than one mesh to EquationSystems From: Vijay S. Mahadevan - 2008-02-17 05:40:25 ```Hi all, Well, the subject line says it all. But here is more to give a better context to the problem. Assume that a multi-physics problem or a problem with single system on a staggerred grid (velocity, pressure on different meshes) needs to be solved. Since the Mesh is always associated with EquationSystems and the association of a LinearImplicitSystem or NonlinearImplicitSystem to the EquationSystems is how the description of the problem mesh is made, you can quickly see that without a constructor for the System class to take its own mesh object, there is an inherent difficulty in associating 2 meshes to the same problem. I hope that wasn't confusing. I could have possibly overlooked something important. I would appreciate it if someone can point out ways to do this elegantly with the existing design intact. Thanks in advance, Vijay ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Benjamin Kirk - 2008-02-18 17:05:37 ```> Assume that a multi-physics problem or a problem with single system on > a staggerred grid (velocity, pressure on different meshes) needs to be > solved. Since the Mesh is always associated with EquationSystems and > the association of a LinearImplicitSystem or NonlinearImplicitSystem > to the EquationSystems is how the description of the problem mesh is > made, you can quickly see that without a constructor for the System > class to take its own mesh object, there is an inherent difficulty in > associating 2 meshes to the same problem. I hope that wasn't > confusing. Splitting the discrete operator into two systems will force you to solve them iteratively in a decoupled fashion -- so long as you are OK with this then you could effectively accomplish the same thing with two EquationSystems objects, each one containing its own distinct mesh and System. I have not tried it, but I would think that should work fine. If it does not work then that should probably be considered a bug and get fixed. -Ben ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Vijay S. Mahadevan - 2008-02-18 19:42:01 ```Ben, The penalty of using operator-splitting is that you end up with a discrete system that has only conditional stability in time integration since the coupling is explicit. If you do iterate between the different operators at each time step, such an issue can be avoided but at the increased cost of coupled physics iterations/time step. My goal is to perform fully implicit coupling (without operator splitting) of 2 different physics and use the SNES Petsc object to solve the nonlinear system. I guess from your previous answer I glean that this is currently not possible due to the issue with the association of the mesh to EquationSystems. I was thinking about a "hack" to simulate such a behavior by creating a dummy EquationSystems object/System (Nonlinear or Linear) and associate a corresponding mesh to the EquationSystems object. But my question then is whether this would entail excessive overhead say in creating 2-3 such Systems and managing them manually to interface and control the interactions. On a sidenote, it seems to be an odd design decision in Libmesh IMHO to associate the mesh to EquationSystems rather than System. Since EquationSystems is a collection of different Systems that can reside on different meshes individually, and no one other than the system managing the solution needs to know about the computational mesh for the different variables. If this was the case, you could create meshes for each system, associate it with the unknown variables and add them to a collection, the EquationSystems object. And that makes a lot of sense to me. Also I would like to know the reason for such a design and if it is something that is necessary in the solution procedure. If I have naively overlooked important aspects in my statement, feel free to set me straight ! Thanks for all the help. Vijay On Feb 18, 2008 11:05 AM, Benjamin Kirk wrote: > > > Assume that a multi-physics problem or a problem with single system on > > a staggerred grid (velocity, pressure on different meshes) needs to be > > solved. Since the Mesh is always associated with EquationSystems and > > the association of a LinearImplicitSystem or NonlinearImplicitSystem > > to the EquationSystems is how the description of the problem mesh is > > made, you can quickly see that without a constructor for the System > > class to take its own mesh object, there is an inherent difficulty in > > associating 2 meshes to the same problem. I hope that wasn't > > confusing. > > Splitting the discrete operator into two systems will force you to solve > them iteratively in a decoupled fashion -- so long as you are OK with this > then you could effectively accomplish the same thing with two > EquationSystems objects, each one containing its own distinct mesh and > System. I have not tried it, but I would think that should work fine. > > If it does not work then that should probably be considered a bug and get > fixed. > > -Ben > > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Benjamin Kirk - 2008-02-19 00:22:40 ```> The penalty of using operator-splitting is that you end up with a > discrete system that has only conditional stability in time > integration since the coupling is explicit. If you do iterate between > the different operators at each time step, such an issue can be > avoided but at the increased cost of coupled physics iterations/time > step. Right -- but I assumed you were going to do this anyway since you mentioned separate systems. In the end each system ends up with its own linear system, so the two system solution is not what you want. > My goal is to perform fully implicit coupling (without operator > splitting) of 2 different physics and use the SNES Petsc object to > solve the nonlinear system. I guess from your previous answer I glean > that this is currently not possible due to the issue with the > association of the mesh to EquationSystems. I was thinking about a > "hack" to simulate such a behavior by creating a dummy EquationSystems > object/System (Nonlinear or Linear) and associate a corresponding mesh > to the EquationSystems object. But my question then is whether this > would entail excessive overhead say in creating 2-3 such Systems and > managing them manually to interface and control the interactions. Probably not the easiest solution... > On a sidenote, it seems to be an odd design decision in Libmesh IMHO > to associate the mesh to EquationSystems rather than System. Since > EquationSystems is a collection of different Systems that can reside > on different meshes individually, and no one other than the system > managing the solution needs to know about the computational mesh for > the different variables. If this was the case, you could create meshes > for each system, associate it with the unknown variables and add them > to a collection, the EquationSystems object. And that makes a lot of > sense to me. I guess it depends on what you mean by a 'mesh.' I can easily conceive of a single mesh which meets your needs. It can contain overlapping elements of different types if need be, and of different sizes. You would want to use the element subdomain flag or something like that to associate a set of elements with a set of physics, but the point is that all the elements for the multiple sets of physics lie in the same 'mesh.' This would require some work in the DofMap to allocate dofs only to a subset of the active elements, but that is do-able. > Also I would like to know the reason for such a design > and if it is something that is necessary in the solution procedure. If > I have naively overlooked important aspects in my statement, feel free > to set me straight ! I can't say for sure without knowing the PDE(s) you want to solve and the proposed discretization, but it may very well fall into the existing framework. In your example you mention velocity/pressure on a staggered grid. This is exactly the way ex11/ex13 work, for example, but in disguise. The velocity is interpolated in a quadratic basis, the pressure interpolated linearly. The end result is that the 'effective pressure mesh' is a factor of 2 coarser in each dimension than the 'velocity mesh.' But in typical finite element fashion this is done by using separate element types for the different variables rather than a separate mesh. If that sounds more like what you are talking about then everything should work as-is. > Thanks for all the help. No problem! Again, point us to the PDEs and we might have some more insight. -Ben ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Roy Stogner - 2008-02-19 01:22:27 ```On Mon, 18 Feb 2008, Benjamin Kirk wrote: > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > single mesh which meets your needs. It can contain overlapping elements of > different types if need be, and of different sizes. You would want to use > the element subdomain flag or something like that to associate a set of > elements with a set of physics, but the point is that all the elements for > the multiple sets of physics lie in the same 'mesh.' > > This would require some work in the DofMap to allocate dofs only to a subset > of the active elements, but that is do-able. We could get DofMap to allocate certain dofs only to certain elements, yes: we'd been thinking of using the p refinement level as a way of handling transient element deletions/recreations for certain problems, and using an optional per-variable flag instead would work just as well. But that's not quite the whole of it; in the staggered grids situation, there's also the question of getting the sparsity pattern right when none of the elements in one grid are neighbors of elements in the other. Could you share QUAD9 nodes such that an interior node on one grid is a vertex node in the other? That'll probably work okay on uniform grids with the Lagrange basis, but try and do anything fancier like hierarchics or adaptive h-refinement and I'll bet a dozen assert() statements (or worse, hidden assumptions) in the library start failing. --- Roy ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Vijay S. Mahadevan - 2008-02-20 21:48:11 Attachments: PDEs.pdf ```Ben, Sorry about the late reply. I got caught with some other things. > > But my question then is whether this > > would entail excessive overhead say in creating 2-3 such Systems and > > managing them manually to interface and control the interactions. > > Probably not the easiest solution... Do you see a different route to doing the same thing ? I would be very interested in knowing the alternatives since although possible, this approach might require quite a few modifications and hacks to the design of my code based on Libmesh. > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > single mesh which meets your needs. It can contain overlapping elements of > different types if need be, and of different sizes. You would want to use > the element subdomain flag or something like that to associate a set of > elements with a set of physics, but the point is that all the elements for > the multiple sets of physics lie in the same 'mesh.' I would define mesh as the actual discretization closely related to a single physics. Since different physics have different characteristic length scales, imposing the same mesh or discretization for all physics is far from ideal. I understand that you can associate different FE basis with different variables in the system but isn't the type (triangle, quad) of the element still the same in each case ? Or maybe I have misunderstood you here since i have only been using the build cube function for all my preliminary work. When you say "all the elements for the multiple sets of physics lie in the same 'mesh.' ", do you mean that the 'mesh' is the union of all the meshes for individual physics ? Then do you create elements that lie only on elements that you are interested in ? For example in 1-D, i could create a fine mesh and make this my base mesh with size h. Maybe 1 physics will then use an element with size h and another physics could use elements of size 2h and another possibly uses elements of size 4h. If this is what you suggested, I can buy this idea. I have attached a pdf document along with the mail showing the PDEs that am dealing with. I just copied and pasted few things in there and wrote a short explanation. So pardon me if certain things aren't very clear. I'll be glad to give you any details to understand this better. Hope this helps clarify the context of the original problem. Thanks and awaiting your reply. Vijay On Mon, Feb 18, 2008 at 6:22 PM, Benjamin Kirk wrote: > > The penalty of using operator-splitting is that you end up with a > > discrete system that has only conditional stability in time > > integration since the coupling is explicit. If you do iterate between > > the different operators at each time step, such an issue can be > > avoided but at the increased cost of coupled physics iterations/time > > step. > > Right -- but I assumed you were going to do this anyway since you mentioned > separate systems. In the end each system ends up with its own linear > system, so the two system solution is not what you want. > > > > My goal is to perform fully implicit coupling (without operator > > splitting) of 2 different physics and use the SNES Petsc object to > > solve the nonlinear system. I guess from your previous answer I glean > > that this is currently not possible due to the issue with the > > association of the mesh to EquationSystems. I was thinking about a > > "hack" to simulate such a behavior by creating a dummy EquationSystems > > object/System (Nonlinear or Linear) and associate a corresponding mesh > > to the EquationSystems object. But my question then is whether this > > would entail excessive overhead say in creating 2-3 such Systems and > > managing them manually to interface and control the interactions. > > Probably not the easiest solution... > > > > On a sidenote, it seems to be an odd design decision in Libmesh IMHO > > to associate the mesh to EquationSystems rather than System. Since > > EquationSystems is a collection of different Systems that can reside > > on different meshes individually, and no one other than the system > > managing the solution needs to know about the computational mesh for > > the different variables. If this was the case, you could create meshes > > for each system, associate it with the unknown variables and add them > > to a collection, the EquationSystems object. And that makes a lot of > > sense to me. > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > single mesh which meets your needs. It can contain overlapping elements of > different types if need be, and of different sizes. You would want to use > the element subdomain flag or something like that to associate a set of > elements with a set of physics, but the point is that all the elements for > the multiple sets of physics lie in the same 'mesh.' > > This would require some work in the DofMap to allocate dofs only to a subset > of the active elements, but that is do-able. > > > > Also I would like to know the reason for such a design > > and if it is something that is necessary in the solution procedure. If > > I have naively overlooked important aspects in my statement, feel free > > to set me straight ! > > I can't say for sure without knowing the PDE(s) you want to solve and the > proposed discretization, but it may very well fall into the existing > framework. In your example you mention velocity/pressure on a staggered > grid. This is exactly the way ex11/ex13 work, for example, but in disguise. > The velocity is interpolated in a quadratic basis, the pressure interpolated > linearly. The end result is that the 'effective pressure mesh' is a factor > of 2 coarser in each dimension than the 'velocity mesh.' But in typical > finite element fashion this is done by using separate element types for the > different variables rather than a separate mesh. > > If that sounds more like what you are talking about then everything should > work as-is. > > > > Thanks for all the help. > > No problem! Again, point us to the PDEs and we might have some more insight. > > -Ben > > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: John Peterson - 2008-02-20 22:24:45 ```Hi, Hmm... you didn't mention before that you wanted to pose simultaneous problems in different space dimensions, or you did I didn't understand that's what you were doing... So there's no flow at all in the porous material? It seems that the two sets of equations are only coupled through the boundary, and thus a decoupled, iterative approach to solving the two is a natural way to handle it. Why not declare two EquationSystems each containing a single System, one for the 3D heat conduction and one for the 1D fluid flow? Then, depending on whatever decoupled scheme you will use to solve the two, you can define custom functions which pull values out of one EquationSystem/System/Mesh and use them in the other. The idea behind an EquationSystems object was to hold multiple Systems of PDEs all posed on the same **domain**. Here you have 2 domains so I would use two EquationSystems. My \$0.02, John Vijay S. Mahadevan writes: > Ben, > > Sorry about the late reply. I got caught with some other things. > > > > But my question then is whether this > > > would entail excessive overhead say in creating 2-3 such Systems and > > > managing them manually to interface and control the interactions. > > > > Probably not the easiest solution... > > Do you see a different route to doing the same thing ? I would be very > interested in knowing the alternatives since although possible, this > approach might require quite a few modifications and hacks to the > design of my code based on Libmesh. > > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > > single mesh which meets your needs. It can contain overlapping elements of > > different types if need be, and of different sizes. You would want to use > > the element subdomain flag or something like that to associate a set of > > elements with a set of physics, but the point is that all the elements for > > the multiple sets of physics lie in the same 'mesh.' > > I would define mesh as the actual discretization closely related to a > single physics. Since different physics have different characteristic > length scales, imposing the same mesh or discretization for all > physics is far from ideal. I understand that you can associate > different FE basis with different variables in the system but isn't > the type (triangle, quad) of the element still the same in each case ? > Or maybe I have misunderstood you here since i have only been using > the build cube function for all my preliminary work. > > When you say "all the elements for the multiple sets of physics lie in > the same 'mesh.' ", do you mean that the 'mesh' is the union of all > the meshes for individual physics ? Then do you create elements that > lie only on elements that you are interested in ? For example in 1-D, > i could create a fine mesh and make this my base mesh with size h. > Maybe 1 physics will then use an element with size h and another > physics could use elements of size 2h and another possibly uses > elements of size 4h. If this is what you suggested, I can buy this > idea. > > I have attached a pdf document along with the mail showing the PDEs > that am dealing with. I just copied and pasted few things in there and > wrote a short explanation. So pardon me if certain things aren't very > clear. I'll be glad to give you any details to understand this better. > Hope this helps clarify the context of the original problem. > > Thanks and awaiting your reply. > > Vijay > > On Mon, Feb 18, 2008 at 6:22 PM, Benjamin Kirk wrote: > > > The penalty of using operator-splitting is that you end up with a > > > discrete system that has only conditional stability in time > > > integration since the coupling is explicit. If you do iterate between > > > the different operators at each time step, such an issue can be > > > avoided but at the increased cost of coupled physics iterations/time > > > step. > > > > Right -- but I assumed you were going to do this anyway since you mentioned > > separate systems. In the end each system ends up with its own linear > > system, so the two system solution is not what you want. > > > > > > > My goal is to perform fully implicit coupling (without operator > > > splitting) of 2 different physics and use the SNES Petsc object to > > > solve the nonlinear system. I guess from your previous answer I glean > > > that this is currently not possible due to the issue with the > > > association of the mesh to EquationSystems. I was thinking about a > > > "hack" to simulate such a behavior by creating a dummy EquationSystems > > > object/System (Nonlinear or Linear) and associate a corresponding mesh > > > to the EquationSystems object. But my question then is whether this > > > would entail excessive overhead say in creating 2-3 such Systems and > > > managing them manually to interface and control the interactions. > > > > Probably not the easiest solution... > > > > > > > On a sidenote, it seems to be an odd design decision in Libmesh IMHO > > > to associate the mesh to EquationSystems rather than System. Since > > > EquationSystems is a collection of different Systems that can reside > > > on different meshes individually, and no one other than the system > > > managing the solution needs to know about the computational mesh for > > > the different variables. If this was the case, you could create meshes > > > for each system, associate it with the unknown variables and add them > > > to a collection, the EquationSystems object. And that makes a lot of > > > sense to me. > > > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > > single mesh which meets your needs. It can contain overlapping elements of > > different types if need be, and of different sizes. You would want to use > > the element subdomain flag or something like that to associate a set of > > elements with a set of physics, but the point is that all the elements for > > the multiple sets of physics lie in the same 'mesh.' > > > > This would require some work in the DofMap to allocate dofs only to a subset > > of the active elements, but that is do-able. > > > > > > > Also I would like to know the reason for such a design > > > and if it is something that is necessary in the solution procedure. If > > > I have naively overlooked important aspects in my statement, feel free > > > to set me straight ! > > > > I can't say for sure without knowing the PDE(s) you want to solve and the > > proposed discretization, but it may very well fall into the existing > > framework. In your example you mention velocity/pressure on a staggered > > grid. This is exactly the way ex11/ex13 work, for example, but in disguise. > > The velocity is interpolated in a quadratic basis, the pressure interpolated > > linearly. The end result is that the 'effective pressure mesh' is a factor > > of 2 coarser in each dimension than the 'velocity mesh.' But in typical > > finite element fashion this is done by using separate element types for the > > different variables rather than a separate mesh. > > > > If that sounds more like what you are talking about then everything should > > work as-is. > > > > > > > Thanks for all the help. > > > > No problem! Again, point us to the PDEs and we might have some more insight. > > > > -Ben > > > > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Vijay S. Mahadevan - 2008-02-20 23:03:50 ```John, Thanks for the reply. I think that maybe I might have led you down a different route now by posting the PDE document. Although that particular problem is something I do need to tackle soon, it is not what is bugging me currently. For simplification, consider 2 physics on the same domain: Consider the 3-D heat conduction and a neutron diffusion model (both are nonlinear diffusion-reaction equations) which are described over the same 3D domain. Now, can I get away with using a single EquationSystems object with a single mesh even though the element sizes used for different physics could be different ? And since I would define 'mesh' to be the representation of the domain using different elements, and since the elements used in terms of size and type could be different for different physics, these would be different meshes. Does that make sense ? I can see how coupling 2 physics described on different dimensions will affect EquationSystems since Mesh(dim) is different. Like I said before, the 1-D fluid flow will be a later addition and I would want to have a solution that is extensible to accomodate that but currently, coupling 2 coupled diffusion-reaction systems consistently without any loss of accuracy in space-time is a bigger priority to me now ! ~Vijay On Wed, Feb 20, 2008 at 4:24 PM, John Peterson wrote: > Hi, > > Hmm... you didn't mention before that you wanted to pose simultaneous > problems in different space dimensions, or you did I didn't understand > that's what you were doing... > > So there's no flow at all in the porous material? > > It seems that the two sets of equations are only coupled through the > boundary, and thus a decoupled, iterative approach to solving the two > is a natural way to handle it. > > Why not declare two EquationSystems each containing a single System, > one for the 3D heat conduction and one for the 1D fluid flow? Then, > depending on whatever decoupled scheme you will use to solve the two, > you can define custom functions which pull values out of one > EquationSystem/System/Mesh and use them in the other. > > The idea behind an EquationSystems object was to hold multiple Systems > of PDEs all posed on the same **domain**. Here you have 2 domains so > I would use two EquationSystems. > > My \$0.02, > John > > > > > > > > > Vijay S. Mahadevan writes: > > Ben, > > > > Sorry about the late reply. I got caught with some other things. > > > > > > But my question then is whether this > > > > would entail excessive overhead say in creating 2-3 such Systems and > > > > managing them manually to interface and control the interactions. > > > > > > Probably not the easiest solution... > > > > Do you see a different route to doing the same thing ? I would be very > > interested in knowing the alternatives since although possible, this > > approach might require quite a few modifications and hacks to the > > design of my code based on Libmesh. > > > > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > > > single mesh which meets your needs. It can contain overlapping elements of > > > different types if need be, and of different sizes. You would want to use > > > the element subdomain flag or something like that to associate a set of > > > elements with a set of physics, but the point is that all the elements for > > > the multiple sets of physics lie in the same 'mesh.' > > > > I would define mesh as the actual discretization closely related to a > > single physics. Since different physics have different characteristic > > length scales, imposing the same mesh or discretization for all > > physics is far from ideal. I understand that you can associate > > different FE basis with different variables in the system but isn't > > the type (triangle, quad) of the element still the same in each case ? > > Or maybe I have misunderstood you here since i have only been using > > the build cube function for all my preliminary work. > > > > When you say "all the elements for the multiple sets of physics lie in > > the same 'mesh.' ", do you mean that the 'mesh' is the union of all > > the meshes for individual physics ? Then do you create elements that > > lie only on elements that you are interested in ? For example in 1-D, > > i could create a fine mesh and make this my base mesh with size h. > > Maybe 1 physics will then use an element with size h and another > > physics could use elements of size 2h and another possibly uses > > elements of size 4h. If this is what you suggested, I can buy this > > idea. > > > > I have attached a pdf document along with the mail showing the PDEs > > that am dealing with. I just copied and pasted few things in there and > > wrote a short explanation. So pardon me if certain things aren't very > > clear. I'll be glad to give you any details to understand this better. > > Hope this helps clarify the context of the original problem. > > > > Thanks and awaiting your reply. > > > > Vijay > > > > On Mon, Feb 18, 2008 at 6:22 PM, Benjamin Kirk wrote: > > > > The penalty of using operator-splitting is that you end up with a > > > > discrete system that has only conditional stability in time > > > > integration since the coupling is explicit. If you do iterate between > > > > the different operators at each time step, such an issue can be > > > > avoided but at the increased cost of coupled physics iterations/time > > > > step. > > > > > > Right -- but I assumed you were going to do this anyway since you mentioned > > > separate systems. In the end each system ends up with its own linear > > > system, so the two system solution is not what you want. > > > > > > > > > > My goal is to perform fully implicit coupling (without operator > > > > splitting) of 2 different physics and use the SNES Petsc object to > > > > solve the nonlinear system. I guess from your previous answer I glean > > > > that this is currently not possible due to the issue with the > > > > association of the mesh to EquationSystems. I was thinking about a > > > > "hack" to simulate such a behavior by creating a dummy EquationSystems > > > > object/System (Nonlinear or Linear) and associate a corresponding mesh > > > > to the EquationSystems object. But my question then is whether this > > > > would entail excessive overhead say in creating 2-3 such Systems and > > > > managing them manually to interface and control the interactions. > > > > > > Probably not the easiest solution... > > > > > > > > > > On a sidenote, it seems to be an odd design decision in Libmesh IMHO > > > > to associate the mesh to EquationSystems rather than System. Since > > > > EquationSystems is a collection of different Systems that can reside > > > > on different meshes individually, and no one other than the system > > > > managing the solution needs to know about the computational mesh for > > > > the different variables. If this was the case, you could create meshes > > > > for each system, associate it with the unknown variables and add them > > > > to a collection, the EquationSystems object. And that makes a lot of > > > > sense to me. > > > > > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > > > single mesh which meets your needs. It can contain overlapping elements of > > > different types if need be, and of different sizes. You would want to use > > > the element subdomain flag or something like that to associate a set of > > > elements with a set of physics, but the point is that all the elements for > > > the multiple sets of physics lie in the same 'mesh.' > > > > > > This would require some work in the DofMap to allocate dofs only to a subset > > > of the active elements, but that is do-able. > > > > > > > > > > Also I would like to know the reason for such a design > > > > and if it is something that is necessary in the solution procedure. If > > > > I have naively overlooked important aspects in my statement, feel free > > > > to set me straight ! > > > > > > I can't say for sure without knowing the PDE(s) you want to solve and the > > > proposed discretization, but it may very well fall into the existing > > > framework. In your example you mention velocity/pressure on a staggered > > > grid. This is exactly the way ex11/ex13 work, for example, but in disguise. > > > The velocity is interpolated in a quadratic basis, the pressure interpolated > > > linearly. The end result is that the 'effective pressure mesh' is a factor > > > of 2 coarser in each dimension than the 'velocity mesh.' But in typical > > > finite element fashion this is done by using separate element types for the > > > different variables rather than a separate mesh. > > > > > > If that sounds more like what you are talking about then everything should > > > work as-is. > > > > > > > > > > Thanks for all the help. > > > > > > No problem! Again, point us to the PDEs and we might have some more insight. > > > > > > -Ben > > > > > > > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Roy Stogner - 2008-02-20 23:24:59 ```On Wed, 20 Feb 2008, Vijay S. Mahadevan wrote: > For simplification, consider 2 physics on the same domain: Consider > the 3-D heat conduction and a neutron diffusion model (both are > nonlinear diffusion-reaction equations) which are described over the > same 3D domain. Now, can I get away with using a single > EquationSystems object with a single mesh even though the element > sizes used for different physics could be different ? And since I > would define 'mesh' to be the representation of the domain using > different elements, and since the elements used in terms of size and > type could be different for different physics, these would be > different meshes. Does that make sense ? This makes sense, but I don't think it's a situation that we're currently set up to handle well. If you want to do explicit or decoupled implicit solves then you're probably okay with using multiple EquationSystems with a different Mesh on each (although you might have to pay more attention to quadrature than usual). But, if you want to mix physics from different meshes and solve a linear system for all variables at once, the problem is more complex than just storing a different mesh for each variable. Just getting the sparsity pattern right would be a non-trivial task, if you're concerned with memory use then you'll want to use a complex data structure which "shares" common elements between meshes yet allows indpendent adaptivity of each, and if you're concerned with performance then you'll need to create (and maintain through adaptivity) lookup tables to quickly find elements in one mesh which overlap an element in another. libMesh does none of those things. I think I've only seen one code designed with fully coupled multiphysics on indpendently refined meshes in mind; a presenter at last year's Finite Element Rodeo called it "MultiMesh" if I remember correctly. I don't know if it's public or what it's other drawbacks were, but you might see what you can find online. --- Roy ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: John Peterson - 2008-02-20 23:58:23 ```Vijay S. Mahadevan writes: > John, > > Thanks for the reply. I think that maybe I might have led you down a > different route now by posting the PDE document. Although that > particular problem is something I do need to tackle soon, it is not > what is bugging me currently. > > For simplification, consider 2 physics on the same domain: Consider > the 3-D heat conduction and a neutron diffusion model (both are > nonlinear diffusion-reaction equations) which are described over the > same 3D domain. Now, can I get away with using a single > EquationSystems object with a single mesh even though the element > sizes used for different physics could be different ? And since I > would define 'mesh' to be the representation of the domain using > different elements, and since the elements used in terms of size and > type could be different for different physics, these would be > different meshes. Does that make sense ? You basically want to have two (or more) completely unstructured, overlapping grids with a different variable defined on each one, if I understand correctly... The problem is that it's easier to describe/conceptualize than it is to implement, at least in a completely general sense, and while in the LibMesh framework. The approaches that Ben described for Taylor-Hood elements where you have a linear pressure sitting on top of a quadratic velocity field are, I imagine, too specialized for what you want to do. (Another issue which arises in these mixed methods is the compatibility of the spaces, and that can be tricky to work out.) I would still use two EquationSystems and two meshes, and decouple the equations. If the variables u and v are associated with meshes umesh and vmesh I would do something like: do solve u system on umesh, treating v as a constant compute updated nodal u DOF values on vmesh solve v system on vmesh, treating u as a constant compute updated nodal v DOF values on umesh until converged I don't know if this is equivalent to the operator splitting problem which introduces conditional stability constraints. I'd expect the overall convergence rate for this process to be linear, with quadratic convergence during each of the intermediate solves. This scheme would work in the current libmesh AFAIK. -J > > I can see how coupling 2 physics described on different dimensions > will affect EquationSystems since Mesh(dim) is different. Like I said > before, the 1-D fluid flow will be a later addition and I would want > to have a solution that is extensible to accomodate that but > currently, coupling 2 coupled diffusion-reaction systems consistently > without any loss of accuracy in space-time is a bigger priority to me > now ! > > ~Vijay > > On Wed, Feb 20, 2008 at 4:24 PM, John Peterson > wrote: > > Hi, > > > > Hmm... you didn't mention before that you wanted to pose simultaneous > > problems in different space dimensions, or you did I didn't understand > > that's what you were doing... > > > > So there's no flow at all in the porous material? > > > > It seems that the two sets of equations are only coupled through the > > boundary, and thus a decoupled, iterative approach to solving the two > > is a natural way to handle it. > > > > Why not declare two EquationSystems each containing a single System, > > one for the 3D heat conduction and one for the 1D fluid flow? Then, > > depending on whatever decoupled scheme you will use to solve the two, > > you can define custom functions which pull values out of one > > EquationSystem/System/Mesh and use them in the other. > > > > The idea behind an EquationSystems object was to hold multiple Systems > > of PDEs all posed on the same **domain**. Here you have 2 domains so > > I would use two EquationSystems. > > > > My \$0.02, > > John > > > > > > > > > > > > > > > > > > Vijay S. Mahadevan writes: > > > Ben, > > > > > > Sorry about the late reply. I got caught with some other things. > > > > > > > > But my question then is whether this > > > > > would entail excessive overhead say in creating 2-3 such Systems and > > > > > managing them manually to interface and control the interactions. > > > > > > > > Probably not the easiest solution... > > > > > > Do you see a different route to doing the same thing ? I would be very > > > interested in knowing the alternatives since although possible, this > > > approach might require quite a few modifications and hacks to the > > > design of my code based on Libmesh. > > > > > > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > > > > single mesh which meets your needs. It can contain overlapping elements of > > > > different types if need be, and of different sizes. You would want to use > > > > the element subdomain flag or something like that to associate a set of > > > > elements with a set of physics, but the point is that all the elements for > > > > the multiple sets of physics lie in the same 'mesh.' > > > > > > I would define mesh as the actual discretization closely related to a > > > single physics. Since different physics have different characteristic > > > length scales, imposing the same mesh or discretization for all > > > physics is far from ideal. I understand that you can associate > > > different FE basis with different variables in the system but isn't > > > the type (triangle, quad) of the element still the same in each case ? > > > Or maybe I have misunderstood you here since i have only been using > > > the build cube function for all my preliminary work. > > > > > > When you say "all the elements for the multiple sets of physics lie in > > > the same 'mesh.' ", do you mean that the 'mesh' is the union of all > > > the meshes for individual physics ? Then do you create elements that > > > lie only on elements that you are interested in ? For example in 1-D, > > > i could create a fine mesh and make this my base mesh with size h. > > > Maybe 1 physics will then use an element with size h and another > > > physics could use elements of size 2h and another possibly uses > > > elements of size 4h. If this is what you suggested, I can buy this > > > idea. > > > > > > I have attached a pdf document along with the mail showing the PDEs > > > that am dealing with. I just copied and pasted few things in there and > > > wrote a short explanation. So pardon me if certain things aren't very > > > clear. I'll be glad to give you any details to understand this better. > > > Hope this helps clarify the context of the original problem. > > > > > > Thanks and awaiting your reply. > > > > > > Vijay > > > > > > On Mon, Feb 18, 2008 at 6:22 PM, Benjamin Kirk wrote: > > > > > The penalty of using operator-splitting is that you end up with a > > > > > discrete system that has only conditional stability in time > > > > > integration since the coupling is explicit. If you do iterate between > > > > > the different operators at each time step, such an issue can be > > > > > avoided but at the increased cost of coupled physics iterations/time > > > > > step. > > > > > > > > Right -- but I assumed you were going to do this anyway since you mentioned > > > > separate systems. In the end each system ends up with its own linear > > > > system, so the two system solution is not what you want. > > > > > > > > > > > > > My goal is to perform fully implicit coupling (without operator > > > > > splitting) of 2 different physics and use the SNES Petsc object to > > > > > solve the nonlinear system. I guess from your previous answer I glean > > > > > that this is currently not possible due to the issue with the > > > > > association of the mesh to EquationSystems. I was thinking about a > > > > > "hack" to simulate such a behavior by creating a dummy EquationSystems > > > > > object/System (Nonlinear or Linear) and associate a corresponding mesh > > > > > to the EquationSystems object. But my question then is whether this > > > > > would entail excessive overhead say in creating 2-3 such Systems and > > > > > managing them manually to interface and control the interactions. > > > > > > > > Probably not the easiest solution... > > > > > > > > > > > > > On a sidenote, it seems to be an odd design decision in Libmesh IMHO > > > > > to associate the mesh to EquationSystems rather than System. Since > > > > > EquationSystems is a collection of different Systems that can reside > > > > > on different meshes individually, and no one other than the system > > > > > managing the solution needs to know about the computational mesh for > > > > > the different variables. If this was the case, you could create meshes > > > > > for each system, associate it with the unknown variables and add them > > > > > to a collection, the EquationSystems object. And that makes a lot of > > > > > sense to me. > > > > > > > > I guess it depends on what you mean by a 'mesh.' I can easily conceive of a > > > > single mesh which meets your needs. It can contain overlapping elements of > > > > different types if need be, and of different sizes. You would want to use > > > > the element subdomain flag or something like that to associate a set of > > > > elements with a set of physics, but the point is that all the elements for > > > > the multiple sets of physics lie in the same 'mesh.' > > > > > > > > This would require some work in the DofMap to allocate dofs only to a subset > > > > of the active elements, but that is do-able. > > > > > > > > > > > > > Also I would like to know the reason for such a design > > > > > and if it is something that is necessary in the solution procedure. If > > > > > I have naively overlooked important aspects in my statement, feel free > > > > > to set me straight ! > > > > > > > > I can't say for sure without knowing the PDE(s) you want to solve and the > > > > proposed discretization, but it may very well fall into the existing > > > > framework. In your example you mention velocity/pressure on a staggered > > > > grid. This is exactly the way ex11/ex13 work, for example, but in disguise. > > > > The velocity is interpolated in a quadratic basis, the pressure interpolated > > > > linearly. The end result is that the 'effective pressure mesh' is a factor > > > > of 2 coarser in each dimension than the 'velocity mesh.' But in typical > > > > finite element fashion this is done by using separate element types for the > > > > different variables rather than a separate mesh. > > > > > > > > If that sounds more like what you are talking about then everything should > > > > work as-is. > > > > > > > > > > > > > Thanks for all the help. > > > > > > > > No problem! Again, point us to the PDEs and we might have some more insight. > > > > > > > > -Ben > > > > > > > > > > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Benjamin Kirk - 2008-02-21 01:20:37 ```>> For simplification, consider 2 physics on the same domain: Consider >> the 3-D heat conduction and a neutron diffusion model (both are >> nonlinear diffusion-reaction equations) which are described over the >> same 3D domain. Now, can I get away with using a single >> EquationSystems object with a single mesh even though the element >> sizes used for different physics could be different ? And since I >> would define 'mesh' to be the representation of the domain using >> different elements, and since the elements used in terms of size and >> type could be different for different physics, these would be >> different meshes. Does that make sense ? The question makes sense, the motivation does not (to me anyway). > > You basically want to have two (or more) completely unstructured, > overlapping grids with a different variable defined on each one, > if I understand correctly... > > The problem is that it's easier to describe/conceptualize than it is > to implement, at least in a completely general sense, and while in the > LibMesh framework. > > The approaches that Ben described for Taylor-Hood elements where you have > a linear pressure sitting on top of a quadratic velocity field are, > I imagine, too specialized for what you want to do. (Another issue > which arises in these mixed methods is the compatibility of the spaces, > and that can be tricky to work out.) And this is a key point... Loosely coupling two separate systems on different meshes is not too hard and often done in practice. You could, for example, have a temperature field on a tetrahedral discretization of the same space as a hexahedral discretization of a velocity field. Each would give rise to its own implicit system. The only compatiblity you have between the discretizations is the interpolation error you are willing to take when you interpolate the solution of one system as the input data of another. If these truly need to be in the same implicit system, though, I cannot think of *why* you would want to have them on two separate discretizations?? -Ben ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Roy Stogner - 2008-02-21 01:31:37 ```On Wed, 20 Feb 2008, Benjamin Kirk wrote: > If these truly need to be in the same implicit system, though, I cannot > think of *why* you would want to have them on two separate discretizations?? To use separate refinement patterns. For a simple example, the solute layers you might want to adaptively resolve in a concentration variable for a transport problem won't necessarily have anything to do with the boundary layers and/or corner singularities in the velocity/pressure variables. It's not a bad idea, it's just very hard to do in such a way that the computational overhead of maintaining and integrating on separate meshes doesn't overwhelm the computational benefit of getting the same solution quality with fewer redundant degrees of freedom. --- Roy ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Vijay S. Mahadevan - 2008-02-21 03:13:46 ```Thanks for all the responses. It is always great to hear different opinions ! Roy, Thanks for the detailed response. I now understand that a Fully implicit coupled solve, if need be, has to be performed only by manually managing the libmesh objects (EquationSystems, Mesh and System). I also understand the problem with deriving the sparsity pattern for a given coupled equation system since this is something I've been working on for the past few weeks. >> if you're concerned with performance then you'll need to create (and maintain through >> adaptivity) lookup tables to quickly find elements in one mesh which overlap an element in another. Yes. This is something I have to attack soon since non-overlapping meshes in different physics is inevitable in the problem I am solving and performance of the solution procedure is quite an important consideration. Also, thanks very much for pointing me to the MultiMesh. I will try to find out more about it and see how useful it is for my case. John, What you suggested is a perfect Operator-Splitting algorithm :-) and hence will have the stability limit and a first order in time penalty unless I iterate over the solution for each physics. I've already tried this before although not with Libmesh, and can tell you that this is not an option for problems I will be dealing with. I do plan to have that option open in order to be used for physics-based preconditioning to accelerate convergence of the fully implicit nonlinear solve/time step. Again, thanks for confirming that I need to maintain my own equationsystems and mesh ! Ben, Multi-physics problems usually have physics with different length scales and different time scales. It is necessary to use appropriate meshes depending on the physics to resolve the evolution of solution and using a single mesh (union of all physics meshes) will lead to a very high DoF than needed and I'll literally be overkilling the problem. The other issue you raise is a very valid one and interpolation error would reduce the spatial convergence to first order, if not done properly. There are several ways I think will preserve the spatial accuracy when coupling solutions on different meshes and this is currently a hot research topic in the multi-physics community. I will also be dealing with this very soon when my coupled diffusion code is ready and will let you know my findings. Everyone, thanks for all the suggestions and help in making me understand the capability and limitations of the current implementation of LibMesh. I am going to go ahead with the idea to maintain separate meshes for each physics, compute the residuals, and solve the nonlinear system with SNES manually. I will definitely have a lot of questions when I dig in more into this. Appreciate all the help and feel free to suggest alternative implementations. Vijay On Wed, Feb 20, 2008 at 7:31 PM, Roy Stogner wrote: > > On Wed, 20 Feb 2008, Benjamin Kirk wrote: > > > If these truly need to be in the same implicit system, though, I cannot > > think of *why* you would want to have them on two separate discretizations?? > > To use separate refinement patterns. For a simple example, the solute > layers you might want to adaptively resolve in a concentration > variable for a transport problem won't necessarily have anything to do > with the boundary layers and/or corner singularities in the > velocity/pressure variables. It's not a bad idea, it's just very hard > to do in such a way that the computational overhead of maintaining and > integrating on separate meshes doesn't overwhelm the computational > benefit of getting the same solution quality with fewer redundant > degrees of freedom. > --- > Roy > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Libmesh-users mailing list > Libmesh-users@... > https://lists.sourceforge.net/lists/listinfo/libmesh-users > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Benjamin Kirk - 2008-02-21 03:58:01 ``` > Multi-physics problems usually have physics with different length > scales and different time scales. It is necessary to use appropriate > meshes depending on the physics to resolve the evolution of solution > and using a single mesh (union of all physics meshes) will lead to a > very high DoF than needed and I'll literally be overkilling the > problem. I don't disagree that the optimal mesh for each component would be great to have, but I just hope you aren't underestimating the overhead involved in seeking that composite mesh. This is *especially* true in the transient case. When you add it all together the transient+adaptive+nonlinear+linear nested looping really can get out of control. My experience with a number of transient multiphysics problems has shown this repeatedly. The "optimal" mesh in a transient reactive problem is elusive -- it will be different at each timestep! Sure, the DOF count may be lower, but when you roll it all together my tried and true approach is to throw more mesh than you need at the current timestep but then go a while before refining (and reallocating (and repartitioning (and projecting ...))). I can almost guarantee much faster walltime with this approach. If the implicit system gets big you can run on a bigger machine, right? ;-) -Ben ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Vijay S. Mahadevan - 2008-02-21 04:57:31 ```> I don't disagree that the optimal mesh for each component would be great to > have, but I just hope you aren't underestimating the overhead involved in > seeking that composite mesh. This is *especially* true in the transient > case. When you add it all together the transient+adaptive+nonlinear+linear > nested looping really can get out of control. Yes, I understand the complications this is going to involve. That is why I am planning to attack this one step at a time. I already have the code in place to solve a single physics (diffusion-reaction) with transient+nonlinear+linear loops in place, where I manually create and use my SNES object and application context. I personally think that the nonlinear solver in Libmesh needs more work in terms of providing routines to set different options that PETSc SNES object exposes. But then again, this might be specific to my current implementation and hence don't want to generalize this. Anyway, the next step would be to design a custom interface to deal with the different physics and implement the coupling in a consistent way, so as to not lose the spatial accuracy, while using different meshes. If all those work according to plan, then adaptivity in space will be included and then later adaptivity in time using higher order IRK schemes. That way, I will be able to test all the solution procedure thoroughly and have confidence in the final solution. I am curious as to what kind of multiphysics problems you have solved with Libmesh before and what kind of approach you took for those. I gather you used a single mesh for both the physics but where you able to preserve the accuracy of the coupled solution in space and time ? And did you use Operator-splitting with iterations over the coupled physics ? ~Vijay On Wed, Feb 20, 2008 at 9:57 PM, Benjamin Kirk wrote: > > > Multi-physics problems usually have physics with different length > > scales and different time scales. It is necessary to use appropriate > > meshes depending on the physics to resolve the evolution of solution > > and using a single mesh (union of all physics meshes) will lead to a > > very high DoF than needed and I'll literally be overkilling the > > problem. > > I don't disagree that the optimal mesh for each component would be great to > have, but I just hope you aren't underestimating the overhead involved in > seeking that composite mesh. This is *especially* true in the transient > case. When you add it all together the transient+adaptive+nonlinear+linear > nested looping really can get out of control. > > My experience with a number of transient multiphysics problems has shown > this repeatedly. The "optimal" mesh in a transient reactive problem is > elusive -- it will be different at each timestep! Sure, the DOF count may > be lower, but when you roll it all together my tried and true approach is to > throw more mesh than you need at the current timestep but then go a while > before refining (and reallocating (and repartitioning (and projecting > ...))). I can almost guarantee much faster walltime with this approach. If > the implicit system gets big you can run on a bigger machine, right? ;-) > > -Ben > > ```
 Re: [Libmesh-users] Associating more than one mesh to EquationSystems From: Benjamin Kirk - 2008-02-25 21:36:37 ```> I am curious as to what kind of multiphysics problems you have solved > with Libmesh before and what kind of approach you took for those. I > gather you used a single mesh for both the physics but where you able > to preserve the accuracy of the coupled solution in space and time ? > And did you use Operator-splitting with iterations over the coupled > physics ? Actually most recently I have been working the conjugate heat transfer problem. The "external" application is hypersonic flight and the "internal" application is solid-body conduction. This uses two meshes, one for the fluid flow and one for the heat transfer. An operator-split approach is natural in this problem because of the disparate time scales. The external flow characteristic times are ~microseconds while the heat transfer problem is ~seconds. The approach couples the problem at the boundary. The temperature is interpolated from the heat transfer part to provide an isothermal boundary condition for the flow. The heat transfer is then interpolated from the fluid part and transformed into a heat transfer coefficient boundary condition for the heat transfer problem. It works well because the heat transfer coefficient is approximately constant for a small range in wall temperature, so the time coupling here is especially loose. -Ben ```