Re: [Libmesh-users] Block matrix nnz's

[Vijay S. M. <vi...@gm...>]

Yes, the runs were in serial. hmm. I just remembered that I also
supply my CouplingMatrix for the system matrix and so the nnz should
probably be divided by sum(CouplingMatrix(ivar,:)) where ivar would be
the variable number. I hope that makes sense. I'll check this again
tomorrow or later tonight and see if that fixes it.

Or I could also be creating an expanded stencil in my physics based
preconditioning that is triggering the extra mallocs. This should be
evident if I spy the matrix in matlab and compare with my sparsity for
matrix in the code.

On Fri, Dec 4, 2009 at 8:14 PM, Kirk, Benjamin (JSC-EG311)
<ben...@na...> wrote:
> That will be interesting.  Is this in serial?  In parallel you have 'on-processor' and 'off-processor' nonzeros, the total number in a given row is the sum of the two...
>
>
>
> ----- Original Message -----
> From: Vijay S. Mahadevan <vi...@gm...>
> To: Kirk, Benjamin (JSC-EG311)
> Cc: fri...@gm... <fri...@gm...>; lib...@li... <lib...@li...>
> Sent: Fri Dec 04 20:06:29 2009
> Subject: Re: [Libmesh-users] Block matrix nnz's
>
> Yes they are. I did divide the total number of nonzeros in the row by
> the number of variables but I still get lots of malloc hits during the
> first assembly. And that's why I posed this question. I'll have to
> print out the matrix and load it in matlab to see how my nnz
> allocation differs from the actual matrix sparsity. I'll do this
> tomorrow and let you know if there was something wrong in my
> assumption.
>
> On Fri, Dec 4, 2009 at 7:24 PM, Kirk, Benjamin (JSC-EG311)
> <ben...@na...> wrote:
>> Are all your variables of the same type?  If not, the concept of a block is more tedious...  If so, can't you just divide the total number of nonzeros in the row by the number of variables in the system?
>>
>> -Ben
>>
>>
>> ----- Original Message -----
>> From: Vijay S. Mahadevan <vi...@gm...>
>> To: Derek Gaston <fri...@gm...>
>> Cc: lib...@li... <lib...@li...>
>> Sent: Fri Dec 04 14:32:04 2009
>> Subject: Re: [Libmesh-users] Block matrix nnz's
>>
>>> Believe it or not... Dr. Ben Kirk has thought of everything... and there is
>>> no reason for us lesser beings to burn brain cycles over this kind of thing
>>> ;-)
>>
>> I have come to the same conclusion several times before too :-)
>>
>> Derek, I have used the CouplingMatrix option before but was thinking
>> along the lines of say calling DofMap with CouplingMatrix twice. One
>> with the right coupling and one with a reduced coupling strategy
>> (store only one variable block). This way, all true matvecs happen
>> using the true coupled matrix while the reduced size matrix is used
>> for performing preconditioning operations. There are some neat physics
>> based preconditioners that can use these single blocks effectively
>> (reduced storage and sometimes symmetric even if system matrix is
>> block unsymmetric) and depending on the problem, can work better than
>> ILU or some other numerical preconditioner.
>>
>> I got a reply from Rahul Sampath and his suggestion was to add a new
>> ImplicitSystem with one variable to get the right nnz distribution for
>> the single block. That seems reasonable and clean for now. Are there
>> other alternatives ?
>>
>> Vijay
>>
>> On Fri, Dec 4, 2009 at 1:32 PM, Derek Gaston <fri...@gm...> wrote:
>>> Believe it or not... Dr. Ben Kirk has thought of everything... and there is
>>> no reason for us lesser beings to burn brain cycles over this kind of thing
>>> ;-)
>>>
>>> In this case what you need to do is create a CouplingMatrix
>>> (http://libmesh.sourceforge.net/doxygen/classCouplingMatrix.php) and attach
>>> it to the DofMap _before_ compute_sparsity() gets called...
>>>
>>> You attach it to the DofMap by setting a public member:
>>>
>>> http://libmesh.sourceforge.net/doxygen/classDofMap.php#4d50b4c013a93b8036c126ced15ecbe6
>>>
>>> Derek
>>>
>>> On Dec 4, 2009, at 12:27 AM, Vijay S. Mahadevan wrote:
>>>
>>>> Hi,
>>>>
>>>> I'm trying to figure out a clean way to find the nnz sparsity
>>>> structure for a given variable. That is given an EquationSystem and
>>>> its corresponding Dofmap, can I find the correct nnz structure to
>>>> create a matrix ignoring the coupling between variables. It is useful
>>>> for creating block preconditioners where the entire matrix (with all
>>>> coupling between variables) is not needed but just individual blocks
>>>> would be enough to perform Block-Jacobi type operations for the
>>>> preconditioner.
>>>>
>>>> I can always sweep through all the elements, compute the dofs for the
>>>> variable of interest and create this nnz but that seems hackish. This
>>>> information should definitely be derived from DofMap but looking at
>>>> the DofMap code, I could not figure out how this can be done correctly
>>>> either. Is there a simpler and efficient way to do this ?
>>>>
>>>> Vijay
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Join us December 9, 2009 for the Red Hat Virtual Experience,
>>>> a free event focused on virtualization and cloud computing.
>>>> Attend in-depth sessions from your desk. Your couch. Anywhere.
>>>> http://p.sf.net/sfu/redhat-sfdev2dev
>>>> _______________________________________________
>>>> Libmesh-users mailing list
>>>> Lib...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/libmesh-users
>>>
>>>
>>
>> ------------------------------------------------------------------------------
>> Join us December 9, 2009 for the Red Hat Virtual Experience,
>> a free event focused on virtualization and cloud computing.
>> Attend in-depth sessions from your desk. Your couch. Anywhere.
>> http://p.sf.net/sfu/redhat-sfdev2dev
>> _______________________________________________
>> Libmesh-users mailing list
>> Lib...@li...
>> https://lists.sourceforge.net/lists/listinfo/libmesh-users
>>
>