You can subscribe to this list here.
2003 
_{Jan}
(4) 
_{Feb}
(1) 
_{Mar}
(9) 
_{Apr}
(2) 
_{May}
(7) 
_{Jun}
(1) 
_{Jul}
(1) 
_{Aug}
(4) 
_{Sep}
(12) 
_{Oct}
(8) 
_{Nov}
(3) 
_{Dec}
(4) 

2004 
_{Jan}
(1) 
_{Feb}
(21) 
_{Mar}
(31) 
_{Apr}
(10) 
_{May}
(12) 
_{Jun}
(15) 
_{Jul}
(4) 
_{Aug}
(6) 
_{Sep}
(5) 
_{Oct}
(11) 
_{Nov}
(43) 
_{Dec}
(13) 
2005 
_{Jan}
(25) 
_{Feb}
(12) 
_{Mar}
(49) 
_{Apr}
(19) 
_{May}
(104) 
_{Jun}
(60) 
_{Jul}
(10) 
_{Aug}
(42) 
_{Sep}
(15) 
_{Oct}
(12) 
_{Nov}
(6) 
_{Dec}
(4) 
2006 
_{Jan}
(1) 
_{Feb}
(6) 
_{Mar}
(31) 
_{Apr}
(17) 
_{May}
(5) 
_{Jun}
(95) 
_{Jul}
(38) 
_{Aug}
(44) 
_{Sep}
(6) 
_{Oct}
(8) 
_{Nov}
(21) 
_{Dec}

2007 
_{Jan}
(5) 
_{Feb}
(46) 
_{Mar}
(9) 
_{Apr}
(23) 
_{May}
(17) 
_{Jun}
(51) 
_{Jul}
(41) 
_{Aug}
(4) 
_{Sep}
(28) 
_{Oct}
(71) 
_{Nov}
(193) 
_{Dec}
(20) 
2008 
_{Jan}
(46) 
_{Feb}
(46) 
_{Mar}
(18) 
_{Apr}
(38) 
_{May}
(14) 
_{Jun}
(107) 
_{Jul}
(50) 
_{Aug}
(115) 
_{Sep}
(84) 
_{Oct}
(96) 
_{Nov}
(105) 
_{Dec}
(34) 
2009 
_{Jan}
(89) 
_{Feb}
(93) 
_{Mar}
(119) 
_{Apr}
(73) 
_{May}
(39) 
_{Jun}
(51) 
_{Jul}
(27) 
_{Aug}
(8) 
_{Sep}
(91) 
_{Oct}
(90) 
_{Nov}
(77) 
_{Dec}
(67) 
2010 
_{Jan}
(25) 
_{Feb}
(36) 
_{Mar}
(98) 
_{Apr}
(45) 
_{May}
(25) 
_{Jun}
(60) 
_{Jul}
(17) 
_{Aug}
(36) 
_{Sep}
(48) 
_{Oct}
(45) 
_{Nov}
(65) 
_{Dec}
(39) 
2011 
_{Jan}
(26) 
_{Feb}
(48) 
_{Mar}
(151) 
_{Apr}
(108) 
_{May}
(61) 
_{Jun}
(108) 
_{Jul}
(27) 
_{Aug}
(50) 
_{Sep}
(43) 
_{Oct}
(43) 
_{Nov}
(27) 
_{Dec}
(37) 
2012 
_{Jan}
(56) 
_{Feb}
(120) 
_{Mar}
(72) 
_{Apr}
(57) 
_{May}
(82) 
_{Jun}
(66) 
_{Jul}
(51) 
_{Aug}
(75) 
_{Sep}
(166) 
_{Oct}
(232) 
_{Nov}
(284) 
_{Dec}
(105) 
2013 
_{Jan}
(168) 
_{Feb}
(151) 
_{Mar}
(30) 
_{Apr}
(145) 
_{May}
(26) 
_{Jun}
(53) 
_{Jul}
(76) 
_{Aug}
(33) 
_{Sep}
(23) 
_{Oct}
(72) 
_{Nov}
(125) 
_{Dec}
(38) 
2014 
_{Jan}
(47) 
_{Feb}
(62) 
_{Mar}
(27) 
_{Apr}
(8) 
_{May}
(12) 
_{Jun}
(2) 
_{Jul}
(22) 
_{Aug}
(22) 
_{Sep}

_{Oct}
(17) 
_{Nov}
(20) 
_{Dec}
(12) 
2015 
_{Jan}
(25) 
_{Feb}
(2) 
_{Mar}
(16) 
_{Apr}

_{May}

_{Jun}

_{Jul}

_{Aug}

_{Sep}

_{Oct}

_{Nov}

_{Dec}

From: Dmitry Karpeyev <karpeev@mc...>  20150313 17:06:35

Apologies in advance for multipleposting. There is a postdoctoral position available at the University of Chicago that may be of interest to the members of this list. The focus is on the development of new and existing PETSc solvers required by a range of applications. Specifically, we are interested in the development of effective solvers and preconditioners for variational inequalities with general constraints, improvements to linear/nonlinear multigrid preconditioners, nonlinear splitting and domaindecomposition methods. Relevant applications include nonlinear elasticity with frictional contact, stronglycoupled multiphysics models, problems with moving meshes, mortar finiteelement formulations, etc. Knowledge of PETSc, MPI programming and numerical algorithms is required. Familiarity with modern highperformance hardware as well as with FEM libraries libMesh and MOOSE is a big plus. Please forward this information to your colleagues that might be interested in this opportunity. Feel free to contact me for more details. Thanks! Dmitry. Dmitry Karpeyev Computation Institute University of Chicago and Argonne National Laboratory 
From: David Knezevic <david.knezevic@ak...>  20150311 17:33:27

On Wed, Mar 11, 2015 at 1:31 PM, Roy Stogner <roystgnr@...> wrote: > > On Wed, 11 Mar 2015, David Knezevic wrote: > > Changing "200" to "250" in: >> >> if (!is_planar_xy) >> _tree = new Trees::OctTree (this>_mesh, 250, _build_type); >> >> works for me. Would you be happy with a "hacky" fix like that in the >> short term? Or would you prefer to keep the code asis until we have >> a more thorough fix? >> > > I'm not happy with the fix, but I'm definitely not happy with the > current code either. > > Turn it from a magic number into a userexposed parameter, and I'm > satisfied. Go ahead and default the new parameter to 250 unless > anyone objects. > OK. Yeah, the current code has a bunch of magic numbers in it. At least this will be a (small) improvement on that. I'll make a PR using this approach. Thanks, David 
From: Roy Stogner <roystgnr@ic...>  20150311 17:31:22

On Wed, 11 Mar 2015, David Knezevic wrote: > Changing "200" to "250" in: > > if (!is_planar_xy) > _tree = new Trees::OctTree (this>_mesh, 250, _build_type); > > works for me. Would you be happy with a "hacky" fix like that in the > short term? Or would you prefer to keep the code asis until we have > a more thorough fix? I'm not happy with the fix, but I'm definitely not happy with the current code either. Turn it from a magic number into a userexposed parameter, and I'm satisfied. Go ahead and default the new parameter to 250 unless anyone objects.  Roy 
From: David Knezevic <david.knezevic@ak...>  20150311 17:25:19

On Wed, Mar 11, 2015 at 1:20 PM, Roy Stogner <roystgnr@...> wrote: > > On Wed, 11 Mar 2015, Roy Stogner wrote: > > On Wed, 11 Mar 2015, David Knezevic wrote: >> >> On Wed, Mar 11, 2015 at 12:29 PM, Roy Stogner <roystgnr@...> >>> wrote: >>> >>> We need some kind of fully general intersection test for cartesian >>> boxes with libMesh elements... >>> >> >> Hmm, OK. Do you have an idea about how to implement that? I'll be happy >>> to help out, if needed. >>> >> >> No great ideas. I've only seen intersection tests for cases involving >> polyhedra, spheres, and cones, not arbitrary quadratic parametric >> volumes. >> >> We could probably handle the (tri,bi,)linear nonaxisaligned >> anisotropic cases with standard algorithms, though: orthogonalize the >> vectors given by dx/dxi to get a more appropriate coordinate system, >> find the "axisaligned bounding box" by translating the nodes into >> those coordinates, and do the intersection test for that bounding box >> with the coordinatealigned BB that our trees use. The trilinear case >> has curved faces, but I believe each face is still always bounded by >> any bounding box of its four nodes. >> >> http://www.realtimerendering.com/intersections.html >> >> We'll need to figure out something smarter before we get to play >> safely with point locators for general quadratic geometry or NURBS, >> though. >> > > To clarify: although an exact intersection test would be *great*, all > we need for now is something like the above: a test that doesn't give > us any false negatives or *too* many false positives. > OK. Also, I'd also like to provide a quick for this current test mesh. Changing "200" to "250" in: if (!is_planar_xy) _tree = new Trees::OctTree (this>_mesh, 250, _build_type); works for me. Would you be happy with a "hacky" fix like that in the short term? Or would you prefer to keep the code asis until we have a more thorough fix? David 
From: Roy Stogner <roystgnr@ic...>  20150311 17:20:19

On Wed, 11 Mar 2015, Roy Stogner wrote: > On Wed, 11 Mar 2015, David Knezevic wrote: > >> On Wed, Mar 11, 2015 at 12:29 PM, Roy Stogner <roystgnr@...> wrote: >> >> We need some kind of fully general intersection test for cartesian >> boxes with libMesh elements... > >> Hmm, OK. Do you have an idea about how to implement that? I'll be happy to help out, if needed. > > No great ideas. I've only seen intersection tests for cases involving > polyhedra, spheres, and cones, not arbitrary quadratic parametric > volumes. > > We could probably handle the (tri,bi,)linear nonaxisaligned > anisotropic cases with standard algorithms, though: orthogonalize the > vectors given by dx/dxi to get a more appropriate coordinate system, > find the "axisaligned bounding box" by translating the nodes into > those coordinates, and do the intersection test for that bounding box > with the coordinatealigned BB that our trees use. The trilinear case > has curved faces, but I believe each face is still always bounded by > any bounding box of its four nodes. > > http://www.realtimerendering.com/intersections.html > > We'll need to figure out something smarter before we get to play > safely with point locators for general quadratic geometry or NURBS, > though. To clarify: although an exact intersection test would be *great*, all we need for now is something like the above: a test that doesn't give us any false negatives or *too* many false positives.  Roy 
From: Roy Stogner <roystgnr@ic...>  20150311 17:15:18

On Wed, 11 Mar 2015, David Knezevic wrote: > On Wed, Mar 11, 2015 at 12:29 PM, Roy Stogner <roystgnr@...> wrote: > > We need some kind of fully general intersection test for cartesian > boxes with libMesh elements... > Hmm, OK. Do you have an idea about how to implement that? I'll be happy to help out, if needed. No great ideas. I've only seen intersection tests for cases involving polyhedra, spheres, and cones, not arbitrary quadratic parametric volumes. We could probably handle the (tri,bi,)linear nonaxisaligned anisotropic cases with standard algorithms, though: orthogonalize the vectors given by dx/dxi to get a more appropriate coordinate system, find the "axisaligned bounding box" by translating the nodes into those coordinates, and do the intersection test for that bounding box with the coordinatealigned BB that our trees use. The trilinear case has curved faces, but I believe each face is still always bounded by any bounding box of its four nodes. http://www.realtimerendering.com/intersections.html We'll need to figure out something smarter before we get to play safely with point locators for general quadratic geometry or NURBS, though.  Roy 
From: David Knezevic <david.knezevic@ak...>  20150311 16:59:20

On Wed, Mar 11, 2015 at 12:29 PM, Roy Stogner <roystgnr@...> wrote: > > > On Wed, 11 Mar 2015, Roy Stogner wrote: > > My idea for a fix. After we do the cheap bounding box test: >> Let's also do a contains_point test? >> > > Gah. This is a stupid idea. Do a contains_point test *with what > point*? If we just test the bounding box corners then we'll have the > converse of the bug that got me screwing around with PointLocator in > the first place. > > We need some kind of fully general intersection test for cartesian > boxes with libMesh elements... > Hmm, OK. Do you have an idea about how to implement that? I'll be happy to help out, if needed. David 
From: Roy Stogner <roystgnr@ic...>  20150311 16:30:00

On Wed, 11 Mar 2015, Roy Stogner wrote: > My idea for a fix. After we do the cheap bounding box test: > Let's also do a contains_point test? Gah. This is a stupid idea. Do a contains_point test *with what point*? If we just test the bounding box corners then we'll have the converse of the bug that got me screwing around with PointLocator in the first place. We need some kind of fully general intersection test for cartesian boxes with libMesh elements...  Roy 
From: Roy Stogner <roystgnr@ic...>  20150311 16:07:43

On Wed, 11 Mar 2015, David Knezevic wrote: > On Wed, Mar 11, 2015 at 11:34 AM, Roy Stogner <roystgnr@...> wrote: > > On Wed, 11 Mar 2015, David Knezevic wrote: > > The issue seems to be in TreeNode<N>::insert(). > > In particular, it seems that the call to this>refine(); inside TreeNode<N>::insert is causing an infinite loop. > > > Yeah, next time a little more warning. > > "This seems to hang" and "this is going to suck up 27GB of RAM within > minutes so have fun on your laptop Roy!" are two very distinct issues. > ;) > > Sorry about that :) I ctrlC'd after about 30 seconds... My code seems to think you've got more than 200 elements whose bounding boxes all intersect a tiny bounding box around {0.0063477736470429497, 2.7432003055871306, 1.113949336408939}}, {0.0063477736468616191, 2.7432003055871306, 1.1139493364084094} That's... actually not insane, if you've got a very anisotropic mesh that's aligned diagonally. My idea for a fix. After we do the cheap bounding box test: bool intersects = true; for (unsigned int d=0; d<elem>dim(); d++) { if (max_coord(d) < this>bounding_box.first(d)  min_coord(d) > this>bounding_box.second(d)) intersects = false; } // If not, we should not care about this element. if (!intersects) return false; Let's also do a contains_point test?  Roy 
From: David Knezevic <david.knezevic@ak...>  20150311 15:43:56

On Wed, Mar 11, 2015 at 11:34 AM, Roy Stogner <roystgnr@...> wrote: > > On Wed, 11 Mar 2015, David Knezevic wrote: > > The issue seems to be in TreeNode<N>::insert(). >> >> In particular, it seems that the call to this>refine(); >> inside TreeNode<N>::insert is causing an infinite loop. >> > > Yeah, next time a little more warning. > > "This seems to hang" and "this is going to suck up 27GB of RAM within > minutes so have fun on your laptop Roy!" are two very distinct issues. > ;) > Sorry about that :) I ctrlC'd after about 30 seconds... David 
From: Roy Stogner <roystgnr@ic...>  20150311 15:34:48

On Wed, 11 Mar 2015, David Knezevic wrote: > The issue seems to be in TreeNode<N>::insert(). > > In particular, it seems that the call to this>refine(); inside TreeNode<N>::insert is causing an infinite loop. Yeah, next time a little more warning. "This seems to hang" and "this is going to suck up 27GB of RAM within minutes so have fun on your laptop Roy!" are two very distinct issues. ;)  Roy 
From: David Knezevic <david.knezevic@ak...>  20150311 15:18:46

> > > On Wed, Mar 11, 2015 at 10:55 AM, Roy Stogner <roystgnr@...> > wrote: > >> >> On Wed, 11 Mar 2015, David Knezevic wrote: >> >> Let me know if anyone else can reproduce this? >>> >> >> How many processors are you running on? SerialMesh or ParallelMesh? >> > One processor. SerialMesh. > > The issue seems to be in TreeNode<N>::insert(). > In particular, it seems that the call to this>refine(); inside TreeNode<N>::insert is causing an infinite loop. David 
From: David Knezevic <david.knezevic@ak...>  20150311 14:57:31

One processor. SerialMesh. The issue seems to be in TreeNode<N>::insert(). On Wed, Mar 11, 2015 at 10:55 AM, Roy Stogner <roystgnr@...> wrote: > > On Wed, 11 Mar 2015, David Knezevic wrote: > > Let me know if anyone else can reproduce this? >> > > How many processors are you running on? SerialMesh or ParallelMesh? >  > Roy > 
From: Cody Permann <codypermann@gm...>  20150311 14:55:53

Has anyone using libMesh seen invalid characters in string files output in the Exodus files? I'm seeing strings like this in several of our output files (output is from "ncdump") name_glo_var = "integral\000\340\000\000\277\367\220\276\f\"\203\254\277\367\220\276\f\"b>" ; Clearly there is garbage after the variable named "Integral" but the first escaped byte is always zero, or at least that's what it looks like to me. This may be a bug in the exodus writer or perhaps we need to clear some buffer that we aren't clearing. We don't always see it but is kind of nasty. Just seeing if anyone else has encountered this before. Cody 
From: Roy Stogner <roystgnr@ic...>  20150311 14:55:50

On Wed, 11 Mar 2015, David Knezevic wrote: > Let me know if anyone else can reproduce this? How many processors are you running on? SerialMesh or ParallelMesh?  Roy 
From: David Knezevic <david.knezevic@ak...>  20150311 14:17:47

From: John Peterson <jwpeterson@gm...>  20150209 22:24:45

On Mon, Feb 9, 2015 at 2:53 PM, Paul T. Bauman <ptbauman@...> wrote: > So it looks like gdb_backtrace() makes a system() call to print the stack > trace info from gdb. Does the system() call not inherit the environment > from that which launched libMeshbased application? > > I ask because I've noticed that this will hang if I have a local version > of gdb loaded in my environment, e.g. I've compiled a newer GDB than what's > on my system, made a module, and loaded that module such that it's in the > front of my $PATH. If I don't have that newer gdb version loaded, > everything is fine  the stack trace prints as it's supposed to and all is > well. > > So, if I'm understanding the issue correctly, would be there interest in > me adding a withgdb configure option that would call a user supplied gdb > version? > > My motivation is that this hang can be really annoying because if it's > hanging during a make check, I have to manually kill the process, ctrlc is > insufficient. > gdb_backtrace() isn't the most robust thing in the world... we have also had some issues with it on our clusters here. There are better ways of launching external programs than by using the system() command. I saw some examples using execvp with fork but they were more complicated, so I went with the simplest solution. I think the gdb_backtrac() capability should probably be disabled by default... while it can be useful for providing line numbers where the backtrace() call does not, the potential benefit does not outweigh a hung code during a crash in my opinion. That said, the ability to specify what gdb to use would also be a useful enhancement, especially if it fixes the problem for you...  John 
From: Paul T. Bauman <ptbauman@gm...>  20150209 21:53:10

So it looks like gdb_backtrace() makes a system() call to print the stack trace info from gdb. Does the system() call not inherit the environment from that which launched libMeshbased application? I ask because I've noticed that this will hang if I have a local version of gdb loaded in my environment, e.g. I've compiled a newer GDB than what's on my system, made a module, and loaded that module such that it's in the front of my $PATH. If I don't have that newer gdb version loaded, everything is fine  the stack trace prints as it's supposed to and all is well. So, if I'm understanding the issue correctly, would be there interest in me adding a withgdb configure option that would call a user supplied gdb version? My motivation is that this hang can be really annoying because if it's hanging during a make check, I have to manually kill the process, ctrlc is insufficient. Thanks, Paul 
From: Jed Brown <jed@je...>  20150123 02:40:27

Roy Stogner <roystgnr@...> writes: >> Interesting. Can you include the full unpreconditioned output? And >> please compare with and without ksp_gmres_modifiedgramschmidt. > > From "./mainopt ksp_monitor_true_residual ksp_norm_type unpreconditioned": > 0 KSP unpreconditioned resid norm 1.137340546775e01 true resid norm 1.137340546775e01 r(i)/b 1.000000000000e+00 > 1 KSP unpreconditioned resid norm 1.851221771429e02 true resid norm 1.850940946785e02 r(i)/b 1.627428963148e01 > 2 KSP unpreconditioned resid norm 8.058732139350e04 true resid norm 8.487837216519e04 r(i)/b 7.462881052281e03 > 3 KSP unpreconditioned resid norm 1.410067346760e04 true resid norm 3.750759001371e04 r(i)/b 3.297832836441e03 > 4 KSP unpreconditioned resid norm 3.036297998627e06 true resid norm 3.296862054294e04 r(i)/b 2.898746609924e03 > 5 KSP unpreconditioned resid norm 1.628004374290e07 true resid norm 4.048903454605e04 r(i)/b 3.559974596954e03 > 6 KSP unpreconditioned resid norm 6.839537794328e09 true resid norm 2.859920502301e04 r(i)/b 2.514568315013e03 > 7 KSP unpreconditioned resid norm 3.580827398197e10 true resid norm 3.424888565589e04 r(i)/b 3.011313168514e03 > 8 KSP unpreconditioned resid norm 1.371032175862e11 true resid norm 3.818671006476e04 r(i)/b 3.357544068312e03 > 9 KSP unpreconditioned resid norm 1.130911552385e12 true resid norm 2.475276237056e04 r(i)/b 2.176372102511e03 > 10 KSP unpreconditioned resid norm 4.683505266751e14 true resid norm 3.820485023471e04 r(i)/b 3.359139032108e03 > And after adding "ksp_gmres_modifiedgramschmidt" too: > 0 KSP unpreconditioned resid norm 1.137340546775e01 true resid norm 1.137340546775e01 r(i)/b 1.000000000000e+00 > 1 KSP unpreconditioned resid norm 1.851221771429e02 true resid norm 1.850940946785e02 r(i)/b 1.627428963148e01 > 2 KSP unpreconditioned resid norm 8.058732139350e04 true resid norm 8.290754235563e04 r(i)/b 7.289596998076e03 > 3 KSP unpreconditioned resid norm 1.409277416992e04 true resid norm 3.366737464418e04 r(i)/b 2.960184154134e03 > 4 KSP unpreconditioned resid norm 3.016318237424e06 true resid norm 3.904608063041e04 r(i)/b 3.433103720879e03 > 5 KSP unpreconditioned resid norm 1.684322222043e07 true resid norm 1.987083874964e04 r(i)/b 1.747131833643e03 > 6 KSP unpreconditioned resid norm 7.043704865544e09 true resid norm 3.021936047142e04 r(i)/b 2.657019531846e03 > 7 KSP unpreconditioned resid norm 3.538690849000e10 true resid norm 3.937474047247e04 r(i)/b 3.462000944582e03 > 8 KSP unpreconditioned resid norm 2.298664686412e11 true resid norm 2.516665836236e04 r(i)/b 2.212763664649e03 > 9 KSP unpreconditioned resid norm 8.918329988448e13 true resid norm 3.883206450876e04 r(i)/b 3.414286479003e03 > 10 KSP unpreconditioned resid norm 3.074816936477e14 true resid norm 2.987382118533e04 r(i)/b 2.626638192935e03 This seems to be a peculiar linear algebra phenomenon, seemingly due to a bad shift strategy for the indefinite matrix (see below). FGMRES, GCR, and and BiCG manage anyway while GMRES and BiCGStab do not. >> How big is this matrix? Can you write it with "ksp_view_mat binary" >> and send it to me (or post somewhere)? > > Not even 100x100; it's a verywellsimplified version of the real > problem. > > http://users.ices.utexas.edu/~roystgnr/binaryoutput src/ksp/ksp/examples/tutorials$ ./ex10 f binaryoutput rhs [0]PETSC ERROR:  Error Message  [0]PETSC ERROR: Zero pivot in LU factorization: http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot [0]PETSC ERROR: Zero pivot row 4 value 0 tolerance 2.22045e14 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2939geeb4e01 GIT Date: 20141105 15:13:20 0600 [0]PETSC ERROR: ./ex10 on a mpichg named batura by jed Fri Jan 9 11:43:38 2015 [0]PETSC ERROR: Configure options downloadchaco downloadctetgen downloadgenerator downloadhypre downloadml downloadsundials downloadsuperlu downloadsuperlu_dist downloadtriangle withc2html withexodusii withhdf5 withlgrind withmetis withmpidir=/home/jed/usr/ccache/mpich/ withnetcdf withparmetis withsowing withsuitesparse withx PETSC_ARCH=mpichg COPTFLAGS="Og g" [0]PETSC ERROR: #1 MatPivotCheck_none() line 634 in /home/jed/petsc/include/petscprivate/matimpl.h [0]PETSC ERROR: #2 MatPivotCheck() line 653 in /home/jed/petsc/include/petscprivate/matimpl.h [0]PETSC ERROR: #3 MatLUFactorNumeric_SeqAIJ_Inode() line 1424 in /home/jed/petsc/src/mat/impls/aij/seq/inode.c [0]PETSC ERROR: #4 MatLUFactorNumeric() line 2930 in /home/jed/petsc/src/mat/interface/matrix.c [0]PETSC ERROR: #5 PCSetUp_ILU() line 232 in /home/jed/petsc/src/ksp/pc/impls/factor/ilu/ilu.c [0]PETSC ERROR: #6 PCSetUp() line 902 in /home/jed/petsc/src/ksp/pc/interface/precon.c [0]PETSC ERROR: #7 KSPSetUp() line 306 in /home/jed/petsc/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #8 main() line 312 in /home/jed/petsc/src/ksp/ksp/examples/tutorials/ex10.c [0]PETSC ERROR: PETSc Option Table entries: [0]PETSC ERROR: f binaryoutput [0]PETSC ERROR: malloc_test [0]PETSC ERROR: rhs [0]PETSC ERROR: End of Error Message send entire error message to petscmaint@... $ ./ex10 f binaryoutput rhs pc_type lu [0]PETSC ERROR:  Error Message  [0]PETSC ERROR: Zero pivot in LU factorization: http://www.mcs.anl.gov/petsc/documentation/faq.html#ZeroPivot [0]PETSC ERROR: Zero pivot row 12 value 0 tolerance 2.22045e14 [0]PETSC ERROR: See http://www.mcs.anl.gov/petsc/documentation/faq.html for trouble shooting. [0]PETSC ERROR: Petsc Development GIT revision: v3.5.2939geeb4e01 GIT Date: 20141105 15:13:20 0600 [0]PETSC ERROR: ./ex10 on a mpichg named batura by jed Fri Jan 9 11:44:29 2015 [0]PETSC ERROR: Configure options downloadchaco downloadctetgen downloadgenerator downloadhypre downloadml downloadsundials downloadsuperlu downloadsuperlu_dist downloadtriangle withc2html withexodusii withhdf5 withlgrind withmetis withmpidir=/home/jed/usr/ccache/mpich/ withnetcdf withparmetis withsowing withsuitesparse withx PETSC_ARCH=mpichg COPTFLAGS="Og g" [0]PETSC ERROR: #1 MatPivotCheck_none() line 634 in /home/jed/petsc/include/petscprivate/matimpl.h [0]PETSC ERROR: #2 MatPivotCheck() line 653 in /home/jed/petsc/include/petscprivate/matimpl.h [0]PETSC ERROR: #3 MatLUFactorNumeric_SeqAIJ_Inode() line 1424 in /home/jed/petsc/src/mat/impls/aij/seq/inode.c [0]PETSC ERROR: #4 MatLUFactorNumeric() line 2930 in /home/jed/petsc/src/mat/interface/matrix.c [0]PETSC ERROR: #5 PCSetUp_LU() line 152 in /home/jed/petsc/src/ksp/pc/impls/factor/lu/lu.c [0]PETSC ERROR: #6 PCSetUp() line 902 in /home/jed/petsc/src/ksp/pc/interface/precon.c [0]PETSC ERROR: #7 KSPSetUp() line 306 in /home/jed/petsc/src/ksp/ksp/interface/itfunc.c [0]PETSC ERROR: #8 main() line 312 in /home/jed/petsc/src/ksp/ksp/examples/tutorials/ex10.c [0]PETSC ERROR: PETSc Option Table entries: [0]PETSC ERROR: f binaryoutput [0]PETSC ERROR: malloc_test [0]PETSC ERROR: pc_type lu [0]PETSC ERROR: rhs [0]PETSC ERROR: End of Error Message send entire error message to petscmaint@... A "positive definite" shift works, but converges very slowly. $ ./ex10 f binaryoutput rhs ksp_converged_reason ksp_monitor_true_residual ksp_rtol 1e10 pc_factor_shift_type positive_definite 0 KSP preconditioned resid norm 5.126361879870e+01 true resid norm 9.899494936612e+00 r(i)/b 1.000000000000e+00 1 KSP preconditioned resid norm 1.742253255361e+01 true resid norm 9.945020941036e+00 r(i)/b 1.004598820921e+00 2 KSP preconditioned resid norm 1.656435081546e+01 true resid norm 1.017851736308e+01 r(i)/b 1.028185521408e+00 3 KSP preconditioned resid norm 1.381919181644e+01 true resid norm 1.317793534482e+01 r(i)/b 1.331172492052e+00 [...] 92 KSP preconditioned resid norm 1.209846782924e08 true resid norm 7.516470758142e09 r(i)/b 7.592782062390e10 93 KSP preconditioned resid norm 1.010690565657e08 true resid norm 5.111314393418e09 r(i)/b 5.163207240517e10 94 KSP preconditioned resid norm 6.900556652393e09 true resid norm 3.274216395272e09 r(i)/b 3.307458023098e10 95 KSP preconditioned resid norm 4.665959835776e09 true resid norm 3.435941544160e09 r(i)/b 3.470825093766e10 Linear solve converged due to CONVERGED_RTOL iterations 95 Number of iterations = 95 Residual norm 3.43594e09 The "nonzero" shift does not: $ ./ex10 f binaryoutput rhs ksp_converged_reason ksp_monitor_true_residual ksp_rtol 1e10 pc_factor_shift_type nonzero 0 KSP preconditioned resid norm 7.686671891138e+01 true resid norm 9.899494936612e+00 r(i)/b 1.000000000000e+00 1 KSP preconditioned resid norm 1.125291740842e+01 true resid norm 2.677042946539e+00 r(i)/b 2.704221744322e01 2 KSP preconditioned resid norm 4.169562114751e01 true resid norm 1.053544866606e01 r(i)/b 1.064241027802e02 3 KSP preconditioned resid norm 3.884939867076e02 true resid norm 2.597689725056e02 r(i)/b 2.624062885722e03 4 KSP preconditioned resid norm 1.870022102589e03 true resid norm 2.366328914521e02 r(i)/b 2.390353174251e03 5 KSP preconditioned resid norm 1.006640708018e04 true resid norm 2.362168570871e02 r(i)/b 2.386150592527e03 6 KSP preconditioned resid norm 6.695456376917e06 true resid norm 2.362181124027e02 r(i)/b 2.386163273129e03 7 KSP preconditioned resid norm 2.568374375496e07 true resid norm 2.362165514116e02 r(i)/b 2.386147504737e03 8 KSP preconditioned resid norm 1.798370685705e08 true resid norm 2.362165522405e02 r(i)/b 2.386147513111e03 9 KSP preconditioned resid norm 8.573160013296e10 true resid norm 2.362165478290e02 r(i)/b 2.386147468548e03 Linear solve converged due to CONVERGED_RTOL iterations 9 Number of iterations = 9 Residual norm 0.0236217 Same story with "inblocks", but convergence appears to be fast if you use FGMRES, GCR, BiCG, or CGS. Now back to the failing case, I'm finding that if I write out the explicit preconditioned operator, then read that in with ex10 and execute with no preconditioner, I get nice convergence with either right or leftpreconditioned GMRES. (See reproducible test case below. Now that's weird and I don't have an explanation. I've had this email halfwritten for a while and have tinkered with the code trying to explain it, but haven't had an extended block of time so I figure I should post this in case anyone else has ideas. The next thing I have in mind is to create the same RHS vector (after preconditioning) and compare the iterates which should be identical (though the true residual won't because the operator used to compute the true residual is different). $ ./ex10 f binaryoutput rhs ksp_converged_reason ksp_monitor_true_residual ksp_rtol 1e16 pc_factor_shift_type nonzero ksp_norm_type preconditioned ksp_type gmres ksp_view_preconditioned_operator_explicit binary:pciluleft 0 KSP preconditioned resid norm 7.686671891138e+01 true resid norm 9.899494936612e+00 r(i)/b 1.000000000000e+00 1 KSP preconditioned resid norm 1.125291740842e+01 true resid norm 2.677042946539e+00 r(i)/b 2.704221744322e01 2 KSP preconditioned resid norm 4.169562114751e01 true resid norm 1.053544866606e01 r(i)/b 1.064241027802e02 3 KSP preconditioned resid norm 3.884939867076e02 true resid norm 2.597689725056e02 r(i)/b 2.624062885722e03 4 KSP preconditioned resid norm 1.870022102589e03 true resid norm 2.366328914521e02 r(i)/b 2.390353174251e03 5 KSP preconditioned resid norm 1.006640708018e04 true resid norm 2.362168570871e02 r(i)/b 2.386150592527e03 6 KSP preconditioned resid norm 6.695456376917e06 true resid norm 2.362181124027e02 r(i)/b 2.386163273129e03 7 KSP preconditioned resid norm 2.568374375496e07 true resid norm 2.362165514116e02 r(i)/b 2.386147504737e03 8 KSP preconditioned resid norm 1.798370685705e08 true resid norm 2.362165522405e02 r(i)/b 2.386147513111e03 9 KSP preconditioned resid norm 8.573160013296e10 true resid norm 2.362165478290e02 r(i)/b 2.386147468548e03 10 KSP preconditioned resid norm 4.225844037925e11 true resid norm 2.362165479751e02 r(i)/b 2.386147470024e03 11 KSP preconditioned resid norm 1.874852209366e12 true resid norm 2.362165479842e02 r(i)/b 2.386147470116e03 12 KSP preconditioned resid norm 6.799500125012e14 true resid norm 2.362165479837e02 r(i)/b 2.386147470110e03 13 KSP preconditioned resid norm 7.912052053829e15 true resid norm 2.362165479837e02 r(i)/b 2.386147470110e03 14 KSP preconditioned resid norm 5.551931114634e15 true resid norm 2.362165479837e02 r(i)/b 2.386147470110e03 Linear solve converged due to CONVERGED_RTOL iterations 14 Number of iterations = 14 Residual norm 0.0236217 $ ./ex10 f pciluleft rhs ksp_converged_reason ksp_monitor_true_residual ksp_rtol 1e16 pc_type none ksp_norm_type preconditioned ksp_type gmres 0 KSP preconditioned resid norm 9.899494936612e+00 true resid norm 9.899494936612e+00 r(i)/b 1.000000000000e+00 1 KSP preconditioned resid norm 2.184329742133e+00 true resid norm 2.184329742133e+00 r(i)/b 2.206506247157e01 2 KSP preconditioned resid norm 1.849697585610e01 true resid norm 1.849697585610e01 r(i)/b 1.868476722756e02 3 KSP preconditioned resid norm 7.233163029549e03 true resid norm 7.233163029549e03 r(i)/b 7.306598039460e04 4 KSP preconditioned resid norm 3.819911952092e04 true resid norm 3.819911952090e04 r(i)/b 3.858693778369e05 5 KSP preconditioned resid norm 1.940407608184e05 true resid norm 1.940407608155e05 r(i)/b 1.960107682846e06 6 KSP preconditioned resid norm 1.446966001367e06 true resid norm 1.446966000978e06 r(i)/b 1.461656387769e07 7 KSP preconditioned resid norm 4.828587794298e08 true resid norm 4.828587816553e08 r(i)/b 4.877610269485e09 8 KSP preconditioned resid norm 3.454073475625e09 true resid norm 3.454073609875e09 r(i)/b 3.489141246086e10 9 KSP preconditioned resid norm 2.149824726703e10 true resid norm 2.149827018270e10 r(i)/b 2.171653232853e11 10 KSP preconditioned resid norm 9.143524324851e12 true resid norm 9.143698087696e12 r(i)/b 9.236529889904e13 11 KSP preconditioned resid norm 1.841531740366e13 true resid norm 1.838833676326e13 r(i)/b 1.857502517149e14 12 KSP preconditioned resid norm 9.365474613992e15 true resid norm 9.795791624425e15 r(i)/b 9.895243835316e16 13 KSP preconditioned resid norm 3.696199042068e15 true resid norm 4.825320989910e15 r(i)/b 4.874310276239e16 14 KSP preconditioned resid norm 2.770629172379e15 true resid norm 5.677355330545e15 r(i)/b 5.734994933477e16 15 KSP preconditioned resid norm 2.303066887546e15 true resid norm 5.583218758734e15 r(i)/b 5.639902635926e16 16 KSP preconditioned resid norm 2.012645615368e15 true resid norm 5.463831520276e15 r(i)/b 5.519303313211e16 17 KSP preconditioned resid norm 1.810217685689e15 true resid norm 4.363896801037e15 r(i)/b 4.408201457731e16 18 KSP preconditioned resid norm 1.658676365430e15 true resid norm 5.310534564585e15 r(i)/b 5.364450003348e16 19 KSP preconditioned resid norm 1.539766535244e15 true resid norm 5.279107563524e15 r(i)/b 5.332703938259e16 20 KSP preconditioned resid norm 1.443237688687e15 true resid norm 5.579906258437e15 r(i)/b 5.636556505323e16 21 KSP preconditioned resid norm 1.362848936052e15 true resid norm 4.342661167469e15 r(i)/b 4.386750228446e16 22 KSP preconditioned resid norm 1.294551799862e15 true resid norm 5.338314359525e15 r(i)/b 5.392511833894e16 23 KSP preconditioned resid norm 1.235590344831e15 true resid norm 5.297753476103e15 r(i)/b 5.351539154296e16 24 KSP preconditioned resid norm 1.184014779259e15 true resid norm 5.576591790512e15 r(i)/b 5.633208387115e16 25 KSP preconditioned resid norm 1.138401595244e15 true resid norm 4.325597599836e15 r(i)/b 4.369513422184e16 26 KSP preconditioned resid norm 1.097683968914e15 true resid norm 5.338314359525e15 r(i)/b 5.392511833894e16 27 KSP preconditioned resid norm 1.061044509051e15 true resid norm 5.260395559362e15 r(i)/b 5.313801959641e16 28 KSP preconditioned resid norm 1.027844958802e15 true resid norm 5.482973286448e15 r(i)/b 5.538639417018e16 29 KSP preconditioned resid norm 9.975786885717e16 true resid norm 4.325597599836e15 r(i)/b 4.369513422184e16 30 KSP preconditioned resid norm 9.698377326135e16 true resid norm 5.338314359525e15 r(i)/b 5.392511833894e16 Linear solve converged due to CONVERGED_RTOL iterations 30 Number of iterations = 30 Residual norm < 1.e12 
From: Roy Stogner <roystgnr@ic...>  20150112 22:35:30

On Mon, 12 Jan 2015, Paul T. Bauman wrote: > On Mon, Jan 12, 2015 at 11:32 AM, Paul T. Bauman <ptbauman@...> wrote: > I'm going to try with OpenMPI and see if this is replicated or not. > > The introduction/ex4 and indeed make check all pass in dbg/devel/opt with LIBMESH_RUN="mpiexec np 2" using OpenMPI 1.8.4. :/ Want to hear something even weirder? introduction_ex4 works fine with MPICH when using enablecomplex. This ought to be such a simple problem to debug, too: mpirun np 2 ./exampledevel d 1 n 2 o FIRST Generates just 2 elements, 3 nodes. Hangs in devel and opt modes with MPICH, inside that (empty) send_receive_packed_range of constrained nodes.  Roy 
From: Paul T. Bauman <ptbauman@gm...>  20150112 16:32:14

Running ./exampledbg d 2 n 15 ksp_monitor PARMETIS ERROR: The sum of tpwgts for constraint #0 is not 1.0 PARMETIS ERROR: The sum of tpwgts for constraint #0 is not 1.0 Mesh Information: mesh_dimension()=2 spatial_dimension()=3 n_nodes()=961 n_local_nodes()=513 n_elem()=225 n_local_elem()=114 n_active_elem()=225 n_subdomains()=1 n_partitions()=2 n_processors()=2 n_threads()=1 processor_id()=0 EquationSystems n_systems()=1 System #0, "Poisson" Type "LinearImplicit" Variables="u" Finite Element Types="LAGRANGE", "JACOBI_20_00" Infinite Element Mapping="CARTESIAN" Approximation Orders="SECOND", "THIRD" n_dofs()=961 n_local_dofs()=513 n_constrained_dofs()=120 n_local_constrained_dofs()=62 n_vectors()=1 n_matrices()=1 DofMap Sparsity Average OnProcessor Bandwidth <= 14.7336 Average OffProcessor Bandwidth <= 0.720083 Maximum OnProcessor Bandwidth <= 26 Maximum OffProcessor Bandwidth <= 15 DofMap Constraints Number of DoF Constraints = 120 Number of Heterogenous Constraints= 118 Average DoF Constraint Length= 0 Number of Node Constraints = 0 Mesh Information: mesh_dimension()=2 spatial_dimension()=3 n_nodes()=961 n_local_nodes()=513 n_elem()=225 n_local_elem()=114 n_active_elem()=225 n_subdomains()=1 n_partitions()=2 n_processors()=2 n_threads()=1 processor_id()=0   Processor id: 0   Num Processors: 2   Time: Mon Jan 12 11:22:46 2015   OS: Linux   HostName: fry.eng.buffalo.edu   OS Release: 2.6.32504.3.3.el6.x86_64   OS Version: #1 SMP Fri Dec 12 16:05:43 EST 2014   Machine: x86_64   Username: pbauman   Configuration: ../../libMesh/configure 'prefix=/fry1/data/users/pbauman/software/libs/libmesh/master'  'enableeverything'   'withmetis=PETSc'   'enableparmesh'   'CXX=g++'   'CC=gcc'   'FC=gfortran'   'F77=gfortran'   'PETSC_DIR=/fry1/data/users/pbauman/software/libs/petsc/petsc3.5.2'   'PETSC_ARCH=gcc4.8.2mpich3.0.4openblas0.2.9.rc1cxxopt'   'VTK_INCLUDE=/fry1/data/users/pbauman/software/libs/vtk/5.10.1/gcc/4.8.2/include/vtk5.10'  'VTK_DIR=/fry1/data/users/pbauman/software/libs/vtk/5.10.1/gcc/4.8.2'     Matrix Assembly Performance: Alive time=0.032835, Active time=0.025345    Event nCalls Total Time Avg Time Total Time Avg Time % of Active Time   w/o Sub w/o Sub With Sub With Sub w/o S With S      Fe 114 0.0012 0.000010 0.0012 0.000010 4.65 4.65   Ke 114 0.0068 0.000059 0.0068 0.000059 26.71 26.71   elem init 114 0.0167 0.000147 0.0167 0.000147 66.06 66.06   matrix insertion 114 0.0007 0.000006 0.0007 0.000006 2.59 2.59    Totals: 456 0.0253 100.00   0 KSP Residual norm 6.281555363893e+00 1 KSP Residual norm 1.441578986136e+00 2 KSP Residual norm 8.657661935565e01 3 KSP Residual norm 6.298483075757e01 4 KSP Residual norm 4.584797396291e01 5 KSP Residual norm 3.672004215548e01 6 KSP Residual norm 2.854933103510e01 7 KSP Residual norm 2.207387889462e01 8 KSP Residual norm 1.507528183709e01 9 KSP Residual norm 7.609993007381e02 10 KSP Residual norm 3.870496969236e02 11 KSP Residual norm 1.694470333105e02 12 KSP Residual norm 1.056205718874e02 13 KSP Residual norm 7.962944594936e03 14 KSP Residual norm 6.247891445290e03 15 KSP Residual norm 5.142557174263e03 16 KSP Residual norm 4.064335778374e03 17 KSP Residual norm 2.986839198661e03 18 KSP Residual norm 2.212248591674e03 19 KSP Residual norm 1.711006399209e03 20 KSP Residual norm 1.423591127897e03 21 KSP Residual norm 1.228658042495e03 22 KSP Residual norm 1.021344123622e03 23 KSP Residual norm 8.969047514369e04 24 KSP Residual norm 7.995662891498e04 25 KSP Residual norm 6.652930701941e04 26 KSP Residual norm 5.304353279729e04 27 KSP Residual norm 4.030242592245e04 28 KSP Residual norm 2.730993578525e04 29 KSP Residual norm 1.645810058886e04 30 KSP Residual norm 1.103538427762e04 31 KSP Residual norm 8.610454940155e05 32 KSP Residual norm 6.498404359506e05 33 KSP Residual norm 4.556983417930e05 34 KSP Residual norm 3.231154166980e05 35 KSP Residual norm 2.577303633035e05 36 KSP Residual norm 2.113582798346e05 37 KSP Residual norm 1.752446809300e05 38 KSP Residual norm 1.484672477927e05 39 KSP Residual norm 1.302629184089e05 40 KSP Residual norm 1.172471062290e05 41 KSP Residual norm 1.006971964980e05 42 KSP Residual norm 8.230366349966e06 43 KSP Residual norm 6.171634470124e06 44 KSP Residual norm 4.117409795018e06 45 KSP Residual norm 2.611987673662e06 46 KSP Residual norm 1.745591647676e06 47 KSP Residual norm 1.190676355730e06 48 KSP Residual norm 9.028453572394e07 49 KSP Residual norm 7.224248002419e07 50 KSP Residual norm 5.339195753159e07 51 KSP Residual norm 3.748508543720e07 52 KSP Residual norm 2.677517396278e07 53 KSP Residual norm 1.840889093657e07 54 KSP Residual norm 1.207493202252e07 55 KSP Residual norm 7.670458622698e08 56 KSP Residual norm 4.744749826056e08 57 KSP Residual norm 2.888924255267e08 58 KSP Residual norm 1.855668558305e08 59 KSP Residual norm 1.225261973163e08 60 KSP Residual norm 7.274693375029e09 61 KSP Residual norm 5.678840027999e09 62 KSP Residual norm 4.272621003973e09 63 KSP Residual norm 3.297977189053e09 64 KSP Residual norm 2.507083460763e09 65 KSP Residual norm 1.834020100457e09 66 KSP Residual norm 1.337490896343e09 67 KSP Residual norm 1.047810076369e09 68 KSP Residual norm 7.828493744498e10 69 KSP Residual norm 5.516829101298e10 70 KSP Residual norm 3.616568657065e10 71 KSP Residual norm 2.576313931394e10 72 KSP Residual norm 2.089089652561e10 73 KSP Residual norm 1.754670989786e10 74 KSP Residual norm 1.516055264043e10 75 KSP Residual norm 1.314547789622e10 76 KSP Residual norm 1.151571443845e10 77 KSP Residual norm 1.002170334358e10 78 KSP Residual norm 8.399470485732e11 79 KSP Residual norm 6.720961716070e11 80 KSP Residual norm 4.385649839608e11 81 KSP Residual norm 2.932111979815e11 82 KSP Residual norm 1.808693464627e11 83 KSP Residual norm 1.155116888517e11 84 KSP Residual norm 7.787886447474e12 85 KSP Residual norm 5.539837062396e12 Warning: This MeshOutput subclass only supports meshes which have been serialized! Warning: This MeshOutput subclass only supports meshes which have been serialized!   Reference count information    N7libMesh10FEAbstractE reference count information:  Creations: 4  Destructions: 4  N7libMesh10Parameters5ValueE reference count information:  Creations: 2  Destructions: 2  N7libMesh12LinearSolverIdEE reference count information:  Creations: 1  Destructions: 1  N7libMesh12SparseMatrixIdEE reference count information:  Creations: 1  Destructions: 1  N7libMesh13NumericVectorIdEE reference count information:  Creations: 5  Destructions: 5  N7libMesh15EquationSystemsE reference count information:  Creations: 1  Destructions: 1  N7libMesh4ElemE reference count information:  Creations: 2632  Destructions: 2632  N7libMesh4NodeE reference count information:  Creations: 1333  Destructions: 1333  N7libMesh5QBaseE reference count information:  Creations: 5  Destructions: 5  N7libMesh6DofMapE reference count information:  Creations: 1  Destructions: 1  N7libMesh6SystemE reference count information:  Creations: 1  Destructions: 1  N7libMesh9DofObjectE reference count information:  Creations: 3965  Destructions: 3965    libMesh Performance: Alive time=1.62748, Active time=1.36299    Event nCalls Total Time Avg Time Total Time Avg Time % of Active Time   w/o Sub w/o Sub With Sub With Sub w/o S With S        DofMap   add_neighbors_to_send_list() 1 0.0050 0.005017 0.0073 0.007321 0.37 0.54   build_constraint_matrix_and_vector() 114 0.0018 0.000016 0.0018 0.000016 0.13 0.13   build_sparsity() 1 0.0313 0.031347 0.0378 0.037773 2.30 2.77   create_dof_constraints() 1 0.0072 0.007207 0.0170 0.017019 0.53 1.25   distribute_dofs() 1 0.1193 0.119323 0.6241 0.624148 8.75 45.79   dof_indices() 524 0.0257 0.000049 0.0257 0.000049 1.89 1.89   hetero_cnstrn_elem_mat_vec() 114 0.0011 0.000010 0.0011 0.000010 0.08 0.08   prepare_send_list() 1 0.0006 0.000619 0.0006 0.000619 0.05 0.05   reinit() 1 0.0148 0.014765 0.0148 0.014765 1.08 1.08     EquationSystems   build_solution_vector() 1 0.0019 0.001909 0.0077 0.007734 0.14 0.57     ExodusII_IO   write_nodal_data() 1 0.0057 0.005673 0.0060 0.005965 0.42 0.44     FE   compute_shape_functions() 144 0.0058 0.000040 0.0058 0.000040 0.43 0.43   init_shape_functions() 31 0.0002 0.000007 0.0002 0.000007 0.02 0.02   inverse_map() 90 0.0009 0.000010 0.0009 0.000010 0.07 0.07     FEMap   compute_affine_map() 144 0.0025 0.000018 0.0025 0.000018 0.19 0.19   compute_face_map() 30 0.0009 0.000028 0.0019 0.000063 0.06 0.14   init_face_shape_functions() 30 0.0002 0.000007 0.0002 0.000007 0.02 0.02   init_reference_to_physical_map() 31 0.0009 0.000029 0.0009 0.000029 0.07 0.07     Mesh   find_neighbors() 2 0.0289 0.014471 0.0317 0.015828 2.12 2.32     MeshCommunication   (all)gather() 1 0.0438 0.043773 0.0580 0.057992 3.21 4.25   compute_hilbert_indices() 4 0.0049 0.001213 0.0049 0.001213 0.36 0.36   delete_remote_elements() 3 0.0104 0.003473 0.0119 0.003965 0.76 0.87   find_global_indices() 4 0.0106 0.002642 0.0217 0.005420 0.78 1.59   parallel_sort() 4 0.0042 0.001053 0.0054 0.001358 0.31 0.40     MeshOutput   write_equation_systems() 1 0.0132 0.013242 0.1180 0.117971 0.97 8.66     MeshTools::Generation   build_cube() 1 0.0141 0.014134 0.0141 0.014134 1.04 1.04     Parallel   allgather() 24 0.0013 0.000052 0.0018 0.000074 0.09 0.13   broadcast() 1 0.0000 0.000009 0.0000 0.000009 0.00 0.00   max(bool) 8310 0.0544 0.000007 0.0544 0.000007 3.99 3.99   max(scalar) 19698 0.1212 0.000006 0.1212 0.000006 8.90 8.90   max(vector) 3763 0.0506 0.000013 0.1410 0.000037 3.71 10.34   max(vector<bool>) 3 0.0007 0.000224 0.0008 0.000252 0.05 0.06   min(bool) 8158 0.0525 0.000006 0.0525 0.000006 3.85 3.85   min(scalar) 61241 0.3972 0.000006 0.3972 0.000006 29.14 29.14   min(vector) 3763 0.0518 0.000014 0.1450 0.000039 3.80 10.64   probe() 67 0.0016 0.000024 0.0016 0.000024 0.12 0.12   receive() 67 0.0006 0.000009 0.0023 0.000035 0.04 0.17   send() 67 0.0003 0.000004 0.0003 0.000004 0.02 0.02   send_receive() 98 0.0011 0.000012 0.0042 0.000043 0.08 0.31   sum() 52 0.0008 0.000016 0.0016 0.000030 0.06 0.11     Parallel::Request   wait() 70 0.0002 0.000003 0.0002 0.000003 0.01 0.01     ParallelMesh   renumber_nodes_and_elements() 2 0.0722 0.036124 0.2182 0.109118 5.30 16.01     ParmetisPartitioner   repartition() 1 0.1673 0.167309 0.1818 0.181825 12.28 13.34     Partitioner   set_node_processor_ids() 1 0.0097 0.009723 0.0265 0.026458 0.71 1.94   set_parent_processor_ids() 1 0.0007 0.000657 0.0007 0.000657 0.05 0.05     PetscLinearSolver   solve() 1 0.0069 0.006875 0.0069 0.006875 0.50 0.50     System   assemble() 1 0.0158 0.015771 0.0331 0.033131 1.16 2.43    Totals: 106669 1.3630 100.00   
From: Roy Stogner <roystgnr@ic...>  20150112 15:31:11

On Mon, 12 Jan 2015, Paul T. Bauman wrote: > I should've asked: any special compile options? I just did > enableeverything and withmetis=PETSc and ran introduction/ex4 > as is. Was this ParallelMesh? Yes, that's probably critical: enableeverything enableparmesh The failure seems to be within a ParallelMeshonly code path.  Roy 
From: Paul T. Bauman <ptbauman@gm...>  20150112 15:08:20

I should've asked: any special compile options? I just did enableeverything and withmetis=PETSc and ran introduction/ex4 as is. Was this ParallelMesh? On Mon, Jan 12, 2015 at 10:05 AM, Paul T. Bauman <ptbauman@...> wrote: > I pulled the laster master and ran on my workstation with no problem. > Perhaps a conflicting MPI setup? > > On Sat, Jan 10, 2015 at 12:32 AM, Paul T. Bauman <ptbauman@...> > wrote: > >> I've been using mpich 3.1.x for months. Mostly the 0.9.4 branch. I'll >> give master a swing this weekend. >> >> >> >> > On Jan 9, 2015, at 6:06 PM, Roy Stogner <roystgnr@...> >> wrote: >> > >> > >> > I'm seeing the most astonishing bug: >> > >> > In devel mode, in introduction_ex4 of all places, miscommunications >> > are causing parallel runs to hang in devel mode. Running through gdb >> > I can actually watch as one process sends "0" and another receives >> > "1". >> > >> > I haven't yet ruled out a bug on our part (this is in >> > send_receive_packed_range code that does asynchronous I/O; maybe there >> > was a "1" already in the send queue?) but the same code (as well as >> > all other examples) is fine in dbg and opt modes, as well as with >> > openmpi. (I'm building mvapich2 to try that now) >> > >> > I haven't got any idea how it could be a bug on their part either; in >> > particular I see the same bug with MPICH2 1.4.1, 1.5, and MPICH 3.1.3. >> > >> > Switching compilers from GCC 4.8 to Intel 13.1 doesn't help either. >> > >> > So now I'm starting to wonder if there's merely some new MPI header or >> > linker conflict on the system I'm trying all this on. Are there any >> > other mpich/mpich2 users out there who are using the libMesh git >> > master? >> >  >> > Roy >> > >> > >>  >> > Dive into the World of Parallel Programming! The Go Parallel Website, >> > sponsored by Intel and developed in partnership with Slashdot Media, is >> your >> > hub for all things parallel software development, from weekly thought >> > leadership blogs to news, videos, case studies, tutorials and more. >> Take a >> > look and join the conversation now. http://goparallel.sourceforge.net >> > _______________________________________________ >> > Libmeshdevel mailing list >> > Libmeshdevel@... >> > https://lists.sourceforge.net/lists/listinfo/libmeshdevel >> > > 
From: Paul T. Bauman <ptbauman@gm...>  20150112 15:05:35

I pulled the laster master and ran on my workstation with no problem. Perhaps a conflicting MPI setup? On Sat, Jan 10, 2015 at 12:32 AM, Paul T. Bauman <ptbauman@...> wrote: > I've been using mpich 3.1.x for months. Mostly the 0.9.4 branch. I'll give > master a swing this weekend. > > > > > On Jan 9, 2015, at 6:06 PM, Roy Stogner <roystgnr@...> > wrote: > > > > > > I'm seeing the most astonishing bug: > > > > In devel mode, in introduction_ex4 of all places, miscommunications > > are causing parallel runs to hang in devel mode. Running through gdb > > I can actually watch as one process sends "0" and another receives > > "1". > > > > I haven't yet ruled out a bug on our part (this is in > > send_receive_packed_range code that does asynchronous I/O; maybe there > > was a "1" already in the send queue?) but the same code (as well as > > all other examples) is fine in dbg and opt modes, as well as with > > openmpi. (I'm building mvapich2 to try that now) > > > > I haven't got any idea how it could be a bug on their part either; in > > particular I see the same bug with MPICH2 1.4.1, 1.5, and MPICH 3.1.3. > > > > Switching compilers from GCC 4.8 to Intel 13.1 doesn't help either. > > > > So now I'm starting to wonder if there's merely some new MPI header or > > linker conflict on the system I'm trying all this on. Are there any > > other mpich/mpich2 users out there who are using the libMesh git > > master? > >  > > Roy > > > > >  > > Dive into the World of Parallel Programming! The Go Parallel Website, > > sponsored by Intel and developed in partnership with Slashdot Media, is > your > > hub for all things parallel software development, from weekly thought > > leadership blogs to news, videos, case studies, tutorials and more. Take > a > > look and join the conversation now. http://goparallel.sourceforge.net > > _______________________________________________ > > Libmeshdevel mailing list > > Libmeshdevel@... > > https://lists.sourceforge.net/lists/listinfo/libmeshdevel > 
From: Paul T. Bauman <ptbauman@gm...>  20150110 05:32:14

I've been using mpich 3.1.x for months. Mostly the 0.9.4 branch. I'll give master a swing this weekend. > On Jan 9, 2015, at 6:06 PM, Roy Stogner <roystgnr@...> wrote: > > > I'm seeing the most astonishing bug: > > In devel mode, in introduction_ex4 of all places, miscommunications > are causing parallel runs to hang in devel mode. Running through gdb > I can actually watch as one process sends "0" and another receives > "1". > > I haven't yet ruled out a bug on our part (this is in > send_receive_packed_range code that does asynchronous I/O; maybe there > was a "1" already in the send queue?) but the same code (as well as > all other examples) is fine in dbg and opt modes, as well as with > openmpi. (I'm building mvapich2 to try that now) > > I haven't got any idea how it could be a bug on their part either; in > particular I see the same bug with MPICH2 1.4.1, 1.5, and MPICH 3.1.3. > > Switching compilers from GCC 4.8 to Intel 13.1 doesn't help either. > > So now I'm starting to wonder if there's merely some new MPI header or > linker conflict on the system I'm trying all this on. Are there any > other mpich/mpich2 users out there who are using the libMesh git > master? >  > Roy > >  > Dive into the World of Parallel Programming! The Go Parallel Website, > sponsored by Intel and developed in partnership with Slashdot Media, is your > hub for all things parallel software development, from weekly thought > leadership blogs to news, videos, case studies, tutorials and more. Take a > look and join the conversation now. http://goparallel.sourceforge.net > _______________________________________________ > Libmeshdevel mailing list > Libmeshdevel@... > https://lists.sourceforge.net/lists/listinfo/libmeshdevel 