pysparse-users

Re: [Pysparse-users] pysparse with ATLAS : modified setup.py needed

[Dominique O. <dom...@gm...>]

On Thu, Nov 26, 2009 at 2:05 AM, morovia morovia
<jal...@go...> wrote:

>        I could compile pysparse by enabling atlas
> library as stated in the comments of setup.py.   But
> I still get ImportError : undefined symbol "ATL_dcopy"
> when I try to import precon.
>
>  Can you please point out, where I need to make exactly
> the changes in the setup.py.

Hi Viswanath,

Could you show us the variables you changed in setup.py? Basically for
ATLAS support you need to link with libatlas, libf77blas and libcblas.
Those libraries must also be in your LD_LIBRARY_PATH.

We really need to be using the Numpy Distutils in Pysparse !

Thanks,

-- 
Dominique

[Pysparse-users] pysparse with ATLAS : modified setup.py needed

[morovia m. <jal...@go...>]

Hello,

       I could compile pysparse by enabling atlas
library as stated in the comments of setup.py.   But
I still get ImportError : undefined symbol "ATL_dcopy"
when I try to import precon.

 Can you please point out, where I need to make exactly
the changes in the setup.py.

Thanks in advance
Best regards

Viswanath.

Re: [Pysparse-users] Full matrix conversion to spmatrix.ll_mat

[Dominique O. <dom...@gm...>]

On Wed, Nov 18, 2009 at 11:47 AM, iza...@t-...
<iza...@t-...> wrote:
> Dear Sirs,
>
> Im using pysparse with the eigenvalue solver jdsym. It works great !!.  The
> only problem I have at the moment is the conversion time from the original
> format of my matrix. Here is a description of my process:
>
> - Read a file wit around 20000 points
> - Process this points and get a very big array (20000*20000) (this part is
> done with pyrex)
> - I reshape the array into a matrix (20000,20000)
> - I do some math ( matrix transpose, matrix scaling, dot product, transpose,
> etc.)
> - I get a matrix from which I would like to obtain a couple of eigenvectors
>
> In order to use jdsm I need the matrix in linked list format. I have tried
> several things:
>
> - write the matrix in matrix market format (using scipy) and the read with
> spmatrix.ll_mat_from_mtx
>   (it takes very very long)
> - convert directly to sparse coo format and then into  ll format
>  (I get an error that this format is not supported for conversion)
> - use the internal routine put(V,index1,index2)
> (this is faster but still takes a long time compared with the eigenvalue
> problem)
>  - I also use
>    for ii in range(N):
>     for jj in range(N):
>         Al[ii,jj]=K[ii,jj]
> (this is almost the same as using put)
>
>
> Is there any way to have it faster in the right format? This will improve a
> lot the all process of obtaining the eigenvalues. If would be great  to have
> something like:
>
> spmatrix.ll_mat(K)
>
> where K is a full matrix.
>
> I appreciate any help with this problem.

Hi Rodrigo,

Thanks for using PySparse! All comments can help us improve the
library. A (rough) design decision in PySparse is that matrix
operations should be "cheap", i.e., O(nnz), for matrices that are
indeed sparse. If your matrix is dense, I'm afraid any constructor of
the form spmatrix.ll_mat(K) will require O(n^2) operations. Please
correct me if I'm wrong but the Matrix Market format lists all the
nonzero elements of your matrix, and we *do* need all that information
to construct the matrix. I don't see how to lower this cost. Using a
different sparse format won't help either.

What could help is detect any exploitable structure in your matrix.
Yes it is dense but it probably isn't random and may have some pattern
to it. For instance, it could be (anti-)symmetric, (block-)circulant,
(block-)Toeplitz, or whatever. I feel that is where savings might be
found.

Alternatively, is there any chance to bypass the 2-dimensional array
that you process before building the PySparse matrix, construct the
PySparse matrix directly and operate on it instead? Or operate on
arrays of the form (irow,jcol,val) and then use put()?

I hope this helps. Good luck.

-- 
Dominique

Re: [Pysparse-users] Full matrix conversion to spmatrix.ll_mat

[<lu...@o2...>]

"iza...@t-..." <iza...@t-...> writes:

> Dear Sirs,
>
> Im using pysparse with the eigenvalue solver jdsym. It works great !!.  The
> only problem I have at the moment is the conversion time from the original
> format of my matrix. Here is a description of my process:
>
> - Read a file wit around 20000 points
> - Process this points and get a very big array (20000*20000) (this part is done
> with pyrex)
> - I reshape the array into a matrix (20000,20000)
> - I do some math ( matrix transpose, matrix scaling, dot product, transpose,
> etc.)
> - I get a matrix from which I would like to obtain a couple of eigenvectors
>
> In order to use jdsm I need the matrix in linked list format. I have tried
> several things:
>
> - write the matrix in matrix market format (using scipy) and the read with
> spmatrix.ll_mat_from_mtx
>   (it takes very very long)
> - convert directly to sparse coo format and then into  ll format
>  (I get an error that this format is not supported for conversion)
> - use the internal routine put(V,index1,index2)

I suggest you use nonzero() method of numpy arrays in case you do not.

> (this is faster but still takes a long time compared with the eigenvalue
> problem)
>  - I also use
>    for ii in range(N):
>     for jj in range(N):
>         Al[ii,jj]=K[ii,jj]
> (this is almost the same as using put)
>
>
> Is there any way to have it faster in the right format? This will improve a lot
> the all process of obtaining the eigenvalues. If would be great  to have
> something like:
>
> spmatrix.ll_mat(K)
>
> where K is a full matrix.
>
> I appreciate any help with this problem.
>
> Best regards
>
> Rodrigo
>
>
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july_______________________________________________
> Pysparse-users mailing list
> Pys...@li...
> https://lists.sourceforge.net/lists/listinfo/pysparse-users

Re: [Pysparse-users] transpose operation

[Dominique O. <dom...@gm...>]

On Tue, Nov 17, 2009 at 1:24 PM, Fergus Gallagher <fe...@go...> wrote:
> On Tue, Nov 17, 2009 at 01:09:24PM -0500, Dominique Orban wrote:

>>
>> If you really need A*A^T explicitly, I suggest building A^T and then using
>> dot().
>>
>
> I think we do.  Our matrices are  ~ 10 Million square, so A^T not
> cheap doing it either explictly as A[i,j]=A[j,i] (ouch) or as
> A^T = spmatrix.dot(A,I)

OK. What I meant was to construct B=A^T to start with, instead of A.
Then you can say dot(B,B) which will give you A*A^T.

-- 
Dominique

Re: [Pysparse-users] transpose operation

[Fergus G. <fe...@go...>]

On Tue, Nov 17, 2009 at 01:09:24PM -0500, Dominique Orban wrote:

Thanks for the reply

> 
> If you really need A*A^T explicitly, I suggest building A^T and then using
> dot().  
>

I think we do.  Our matrices are  ~ 10 Million square, so A^T not
cheap doing it either explictly as A[i,j]=A[j,i] (ouch) or as 
A^T = spmatrix.dot(A,I)

> Often you don't need this matrix explicitly though (e.g., it is easy
> to build matrix-vector products with A*A^T without assembling this matrix).
> Similarly, if you need to factorize it to solve A*A^T x = b, you can solve
> the equivalent system
> 
>   [ I  A^T ] [x]   [ 0]
>   [ A   0  ] [y] = [-b]
> 
> This last system is also much sparser, and it is symmetric. It is indefinite
> though.

Our application is more statistical (correlation) - we're not trying to solve/invert.

Regards.

-- 
Fergus Gallagher

"I take my children everywhere, but they always find their way back
home." - Robert Orben

Re: [Pysparse-users] transpose operation

[Dominique O. <dom...@gm...>]

On Tue, Nov 17, 2009 at 6:42 AM, Fergus Gallagher <fe...@go...>wrote:

> > The find/put method is surely the fastest in general. The more
> > fundamental question is do you really need the explicit transpose?
> > Often, algorithms that need the transpose operate directly on the
> > original matrix. If what you need is matrix-vector products of the form
> > A.T*x, you can always use the matvec_transp() method. If you're using
> > the higher-level PysparseMatrix objects, you can do A*x and x*A. The
> > latter actually computes A.T*x.
> >
> > I hope this helps.
> >
>
> Hi,
>
> thanks for all the replies.
>
> What we want to do is generate the "outer product"  A*B^T  (actually A*A^T
> in our case).
>

If you really need A*A^T explicitly, I suggest building A^T and then using
dot().  Often you don't need this matrix explicitly though (e.g., it is easy
to build matrix-vector products with A*A^T without assembling this matrix).
Similarly, if you need to factorize it to solve A*A^T x = b, you can solve
the equivalent system

  [ I  A^T ] [x]   [ 0]
  [ A   0  ] [y] = [-b]

This last system is also much sparser, and it is symmetric. It is indefinite
though.

-- 
Dominique

Re: [Pysparse-users] transpose operation

[Fergus G. <fe...@go...>]

> The find/put method is surely the fastest in general. The more
> fundamental question is do you really need the explicit transpose?
> Often, algorithms that need the transpose operate directly on the
> original matrix. If what you need is matrix-vector products of the form
> A.T*x, you can always use the matvec_transp() method. If you're using
> the higher-level PysparseMatrix objects, you can do A*x and x*A. The
> latter actually computes A.T*x.
>
> I hope this helps.
>

Hi, 

thanks for all the replies.

What we want to do is generate the "outer product"  A*B^T  (actually A*A^T
in our case). 

Regards

-- 
Fergus Gallagher

"Setting a good example for children takes all the fun out of middle
age." - William Feather

Re: [Pysparse-users] transpose operation

[Dominique O. <dom...@gm...>]

2009/11/16 Łukasz Pankowski <lu...@o2...>

> Toine Bogers <tb...@db...> writes:
>
> > Hi,
> >
> > What is the most efficient way of transposing a matrix using PySparse? A
> search
> > of the website doesn't yield any results and I don't see any transpose
> method
> > in the list of methods. I must be missing something here; I can't imagine
> there
> > is no fast transpose method available!
> >
> > I've tried two different ways of calculating the transpose myself:
> >
> > 1) looping through the entries of a matrix M with .items() and then
> simply
> > filling a new sparse matrix with the same values, but transposed.
> >
> >     def T_ver1( self, M ):
> >         (rows, cols) = M.shape
> >         t = spmatrix.ll_mat(cols, rows, M.nnz)
> >         for (x, y), value in M.items():
> >             t[y, x] = M[x,y]
> >         return t
> >
> > 2) I've noticed that using the .dot() method to multiply a matrix with
> the
> > identity matrix is a faster way of getting the transposed matrix M.
> >
> >     def T_ver2( self, M ):
> >         (rows, cols) = M.shape
> >         I = spmatrix.ll_mat(rows, rows, rows)
> >         for i in xrange(0, rows):
> >             I[i, i] = 1
> >         return spmatrix.dot(M, I)):
> >
> > Method 2 is faster, which I suspect is because the  .dot() function using
> > either the C or Fortran code directly. But why is there no direct
> M.transpose()
> > or M.T method to give me the transposed matrix?
>
> Hi,
>
> Mine is:
>
> def transpose(a):
>    b = ll_mat(a.shape[1], a.shape[0], a.nnz)
>    v, r, c = a.find()
>    b.put(v, c, r)
>    return b
>
> should be faster, though for the price of extra memory usage (v, r, c
> arrays).


Hi Toine,

The find/put method is surely the fastest in general. The more fundamental
question is do you really need the explicit transpose? Often, algorithms
that need the transpose operate directly on the original matrix. If what you
need is matrix-vector products of the form A.T*x, you can always use the
matvec_transp() method. If you're using the higher-level PysparseMatrix
objects, you can do A*x and x*A. The latter actually computes A.T*x.

I hope this helps.

-- 
Dominique

Re: [Pysparse-users] transpose operation

[<lu...@o2...>]

Toine Bogers <tb...@db...> writes:

> Hi,
>
> What is the most efficient way of transposing a matrix using PySparse? A search
> of the website doesn't yield any results and I don't see any transpose method
> in the list of methods. I must be missing something here; I can't imagine there
> is no fast transpose method available!
>
> I've tried two different ways of calculating the transpose myself:
>
> 1) looping through the entries of a matrix M with .items() and then simply
> filling a new sparse matrix with the same values, but transposed.
>
>     def T_ver1( self, M ):
>         (rows, cols) = M.shape
>         t = spmatrix.ll_mat(cols, rows, M.nnz)
>         for (x, y), value in M.items():
>             t[y, x] = M[x,y]
>         return t
>
> 2) I've noticed that using the .dot() method to multiply a matrix with the
> identity matrix is a faster way of getting the transposed matrix M.
>
>     def T_ver2( self, M ):
>         (rows, cols) = M.shape
>         I = spmatrix.ll_mat(rows, rows, rows)
>         for i in xrange(0, rows):
>             I[i, i] = 1
>         return spmatrix.dot(M, I)):
>
> Method 2 is faster, which I suspect is because the  .dot() function using
> either the C or Fortran code directly. But why is there no direct M.transpose()
> or M.T method to give me the transposed matrix?

Hi,

Mine is:

def transpose(a):
    b = ll_mat(a.shape[1], a.shape[0], a.nnz)
    v, r, c = a.find()
    b.put(v, c, r)
    return b

should be faster, though for the price of extra memory usage (v, r, c
arrays).

>
> Thanks in advance for the help!
>
>
> Kind regards,
> Toine Bogers
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now.  http://p.sf.net/sfu/bobj-july_______________________________________________
> Pysparse-users mailing list
> Pys...@li...
> https://lists.sourceforge.net/lists/listinfo/pysparse-users

[Pysparse-users] transpose operation

[Toine B. <tb...@db...>]

Hi,

What is the most efficient way of transposing a matrix using PySparse?  
A search of the website doesn't yield any results and I don't see any  
transpose method in the list of methods. I must be missing something  
here; I can't imagine there is no fast transpose method available!

I've tried two different ways of calculating the transpose myself:

1) looping through the entries of a matrix M with .items() and then  
simply filling a new sparse matrix with the same values, but transposed.

     def T_ver1( self, M ):
         (rows, cols) = M.shape
         t = spmatrix.ll_mat(cols, rows, M.nnz)
         for (x, y), value in M.items():
             t[y, x] = M[x,y]
         return t

2) I've noticed that using the .dot() method to multiply a matrix with  
the identity matrix is a faster way of getting the transposed matrix M.

     def T_ver2( self, M ):
         (rows, cols) = M.shape
         I = spmatrix.ll_mat(rows, rows, rows)
         for i in xrange(0, rows):
             I[i, i] = 1
         return spmatrix.dot(M, I)):

Method 2 is faster, which I suspect is because the  .dot() function  
using either the C or Fortran code directly. But why is there no  
direct M.transpose() or M.T method to give me the transposed matrix?

Thanks in advance for the help!


Kind regards,
Toine Bogers

Re: [Pysparse-users] PysparseSpDiagsMatrix usage from spdiags in matlab

[Dominique O. <dom...@gm...>]

On Thu, Nov 5, 2009 at 4:00 AM, zipeppe <zi...@gm...> wrote:

> Dear all,
>
> I don't know it this is the right place to ask, but unfortunately the
> official Python-forum doesn't help.
>

This is the right place for Pysparse questions.


> I am quite new to Python and I am unable to correctly translate the
> following MATLAB script in Python code, specially because of the *spdiags*proprietary function:
>
> function bd=BLC(b,A,B,s)
>    L=length(b);
>    Bs=-B/s;  As=-A/s;
>    bd=ones(L,1)*median(b);bd0=b;nm=norm(bd-bd0);nm0=RealMax;
>    M0=-ones(L,1)/As;
>    e=ones(L,1);
>    D0=spdiags([1*e -4*e 6*e -4*e 1*e],-2:2,L,L);
>    D0(1,1)=2; D0(L,L)=2;
>    D0(2,1)=-4; D0(1,2)=-4; D0(L,L-1)=-4; D0(L-1,L)=-4;
>    D0(2,2)=10;D0(L-1,L-1)=10;
>    I=0;
>    while nm>10 & I<30
>        %& nm<nm0;
>        I=I+1;
>        M=M0;D=D0;bd0=bd;nm0=nm;
>        for i=1:L
>            if bd(i)>b(i)
>                M(i)=M(i)+2*Bs*b(i)/As;
>                D(i,i)=D(i,i)+2*Bs/As;
>            end
>        end
>        bd=D\M;
>        nm=norm(bd0-bd);
>    end
>
> The algorithm receives *b* as (Nx1 data vector), *A* (1x1), *B* (1x1) and
> *s* (1x1) as local parameters, and creates *bd* (Nx1data vector) as
> output.
>

The Matlab command that you highlighted creates an L x L matrix with the
vectors e, -4*e, 6*e, -4*e and e on and around the main diagonal. The
position argument is -2:2 which means that the first vector (e) goes on the
sub-sub-diagonal, the second vector (-4e) goes on the sub-diagonal, 6e goes
on the main diagonal, -4e goes one above the main diagonal and e goes two
above the main diagonal like so.

Here is a code fragment that performs the same operation with Pysparse:

In [1]: from pysparse.pysparseMatrix import PysparseSpDiagsMatrix as spdiags
In [2]: import numpy as np
In [3]: L = 5  # The length of b in your Matlab code
In [4]: e = np.ones(L)
In [5]: D0 = spdiags(L, (e, -4*e, 6*e, -4*e, e), (-2,-1,0,1,2))
In [6]: print D0
 6.000000  -4.000000   1.000000      ---        ---
-4.000000   6.000000  -4.000000   1.000000      ---
 1.000000  -4.000000   6.000000  -4.000000   1.000000
    ---     1.000000  -4.000000   6.000000  -4.000000
    ---        ---     1.000000  -4.000000   6.000000

The first argument (L) specifies the matrix size, the second is the set of
vectors that should be laid out on the diagonals and the third gives the
indices of the diagonals that are concerned.

For a more Matlab-like notation, the last argument can also be written
np.r_[-2:2] (see the Numpy function r_ ... note the underscore).

I hope this helps.

-- 
Dominique

[Pysparse-users] PysparseSpDiagsMatrix usage from spdiags in matlab

[zipeppe <zi...@gm...>]

Dear all,

I don't know it this is the right place to ask, but unfortunately the
official Python-forum doesn't help.

I am quite new to Python and I am unable to correctly translate the
following MATLAB script in Python code, specially because of the
*spdiags*proprietary function:

function bd=BLC(b,A,B,s)
   L=length(b);
   Bs=-B/s;  As=-A/s;
   bd=ones(L,1)*median(b);bd0=b;nm=norm(bd-bd0);nm0=RealMax;
   M0=-ones(L,1)/As;
   e=ones(L,1);
   D0=spdiags([1*e -4*e 6*e -4*e 1*e],-2:2,L,L);
   D0(1,1)=2; D0(L,L)=2;
   D0(2,1)=-4; D0(1,2)=-4; D0(L,L-1)=-4; D0(L-1,L)=-4;
   D0(2,2)=10;D0(L-1,L-1)=10;
   I=0;
   while nm>10 & I<30
       %& nm<nm0;
       I=I+1;
       M=M0;D=D0;bd0=bd;nm0=nm;
       for i=1:L
           if bd(i)>b(i)
               M(i)=M(i)+2*Bs*b(i)/As;
               D(i,i)=D(i,i)+2*Bs/As;
           end
       end
       bd=D\M;
       nm=norm(bd0-bd);
   end

The algorithm receives *b* as (Nx1 data vector), *A* (1x1), *B* (1x1) and *s
* (1x1) as local parameters, and creates *bd* (Nx1data vector) as output.

It looks like the PySparse package is the best promising, despite that I am
not able to use *PysparseSpDiagsMatrix* function in any way... I got only
errors like:

*TypeError: unsubscriptable object  *
- or -
*IndexError: Not as many row indices as values*

Please, help me to understand what is going on in MATLAB and how to
translate this operation using the PySparse module:

Let have a B matrix as follow:

B =
   1    6   11
   2    7   12
   3    8   13
   4    9   14
   5   10   15

The spdiags MATLAB function works as follows:

A = full (spdiags (B, [-2 0 2], 5, 5))

 Matrix B                      Matrix A

1    6   11                 6    0   13    0    0
2    7   12                 0    7    0   14    0
3    8   13  == spdiags =>  1    0    8    0   15
4    9   14                 0    2    0    9    0
5   10   15                 0    0    3    0   10

A(3,1), A(4,2), and A(5,3) are taken from the upper part of B(:,1)

A(1,3), A(2,4), and A(3,5) are taken from the lower part of B(:,3).
How can I implement the same operation in python using *
PysparseSpDiagsMatrix* ?



Thanks for your help,
ZPP


-- 

-------------------------------------------------------------------
Coltivate Linux che Windows si pianta da solo!
-------------------------------------------------------------------

[Pysparse-users] PysparseSpDiagsMatrix usage from spdiags in matlab

[zipeppe <zi...@gm...>]

Dear all,

I don't know it this is the right place to ask, but unfortunately the
official Python-forum doesn't help.

I am quite new to Python and I am unable to correctly translate the
following MATLAB script in Python code, specially because of the
*spdiags*proprietary function:

function bd=BLC(b,A,B,s)
   L=length(b);
   Bs=-B/s;  As=-A/s;
   bd=ones(L,1)*median(b);bd0=b;nm=norm(bd-bd0);nm0=RealMax;
   M0=-ones(L,1)/As;
   e=ones(L,1);
   D0=spdiags([1*e -4*e 6*e -4*e 1*e],-2:2,L,L);
   D0(1,1)=2; D0(L,L)=2;
   D0(2,1)=-4; D0(1,2)=-4; D0(L,L-1)=-4; D0(L-1,L)=-4;
   D0(2,2)=10;D0(L-1,L-1)=10;
   I=0;
   while nm>10 & I<30
       %& nm<nm0;
       I=I+1;
       M=M0;D=D0;bd0=bd;nm0=nm;
       for i=1:L
           if bd(i)>b(i)
               M(i)=M(i)+2*Bs*b(i)/As;
               D(i,i)=D(i,i)+2*Bs/As;
           end
       end
       bd=D\M;
       nm=norm(bd0-bd);
   end

The algorithm receives *b* as (Nx1 data vector), *A* (1x1), *B* (1x1) and *s
* (1x1) as local parameters, and creates *bd* (Nx1data vector) as output.

It looks like the PySparse package is the best promising, despite that I am
not able to use *PysparseSpDiagsMatrix* function in any way... I got only
errors like:

*TypeError: unsubscriptable object  *
- or -
*IndexError: Not as many row indices as values*

Please, help me to understand what is going on in MATLAB and how to
translate this operation using the PySparse module:

Let have a B matrix as follow:

B =
   1    6   11
   2    7   12
   3    8   13
   4    9   14
   5   10   15

The spdiags function works as follows:

A = full (spdiags (B, [-2 0 2], 5, 5))

 Matrix B                      Matrix A

1    6   11                 6    0   13    0    0
2    7   12                 0    7    0   14    0
3    8   13  == spdiags =>  1    0    8    0   15
4    9   14                 0    2    0    9    0
5   10   15                 0    0    3    0   10

A(3,1), A(4,2), and A(5,3) are taken from the upper part of B(:,1)

A(1,3), A(2,4), and A(3,5) are taken from the lower part of B(:,3).
How can I implement the same operation in python using *
PysparseSpDiagsMatrix* ?



Thanks for your help,
ZPP

-- 

-------------------------------------------------------------------
Coltivate Linux che Windows si pianta da solo!
-------------------------------------------------------------------

Re: [Pysparse-users] Reshape Matrix

[<mv...@uv...>]

> On Sat, Oct 24, 2009 at 5:02 PM,  <mv...@uv...> wrote:
>> Hi,
>>
>> I was wondering if there is an easy way to extend or reshape a matrix.
>>
>> For example...
>> If I had a vector of length 10000 (shape=(1,10000)), then wanted to
>> insert
>> a new element at position 11000 without creating an entirely new vector
>> to
>> store it.
>>
>> I've looked at the Docs, and messed around with the package but was
>> unable
>> to find a simple way to do this.  Any tips would be appreaciated.
>>
>> Thanks,
>> Matt
>
> Hi Matt,
>
> Currently there is no provision for reshaping, clipping or expanding
> matrices in PySparse. I can only imagine doing this efficiently on
> matrices in linked-list format. There are essentially two ways to
> implement it: by adding methods at C level in ll_mat.c, or by
> expanding/subclassing the high-level PysparseMatrix class. If you
> decide to have a stab at it, I suggest the second avenue first,
> although I am not sure at this point that it is doable without getting
> your hands dirty in the C code.
>
> Are you in a situation where you don't know your problem size beforehand?
>
> --
> Dominique
>
>

Hey Dominique,

Thanks for the quick reply.  No, for most of this application I will not
know the size of the problem before hand.
I think I can still do it, it will just require a change in my thinking
and program flow.

If I do endup needing this feature(or it just makes it way simpler) I'll
gladly hack away at the c code to make it work and share any changes.


Thanks,
MV

Re: [Pysparse-users] Reshape Matrix

[Dominique O. <dom...@gm...>]

On Sat, Oct 24, 2009 at 5:02 PM,  <mv...@uv...> wrote:
> Hi,
>
> I was wondering if there is an easy way to extend or reshape a matrix.
>
> For example...
> If I had a vector of length 10000 (shape=(1,10000)), then wanted to insert
> a new element at position 11000 without creating an entirely new vector to
> store it.
>
> I've looked at the Docs, and messed around with the package but was unable
> to find a simple way to do this.  Any tips would be appreaciated.
>
> Thanks,
> Matt

Hi Matt,

Currently there is no provision for reshaping, clipping or expanding
matrices in PySparse. I can only imagine doing this efficiently on
matrices in linked-list format. There are essentially two ways to
implement it: by adding methods at C level in ll_mat.c, or by
expanding/subclassing the high-level PysparseMatrix class. If you
decide to have a stab at it, I suggest the second avenue first,
although I am not sure at this point that it is doable without getting
your hands dirty in the C code.

Are you in a situation where you don't know your problem size beforehand?

-- 
Dominique

[Pysparse-users] Reshape Matrix

[<mv...@uv...>]

Hi,

I was wondering if there is an easy way to extend or reshape a matrix.

For example...
If I had a vector of length 10000 (shape=(1,10000)), then wanted to insert
a new element at position 11000 without creating an entirely new vector to
store it.

I've looked at the Docs, and messed around with the package but was unable
to find a simple way to do this.  Any tips would be appreaciated.

Thanks,
Matt

Re: [Pysparse-users] build problem

[Dominique O. <dom...@gm...>]

That's good to hear. I'm copying the mailing list on this so this info
is accessible to other users in the future.

Enjoy,
Dominique

On Tue, Oct 20, 2009 at 8:56 PM, Ranjit Chacko <rjc...@gm...> wrote:
> Updating Xcode fixed the problem. I couldn't find the right version at first
> because for some reason you can only get it in the iPhone SDK.
>
> Thanks for your help,
>
> -Ranjit
>
> On Oct 20, 2009, at 10/20/09 12:05 PM, Dominique Orban wrote:
>
>> I have XCode 3.0 and I'm running OSX 10.5.8. I don't know if you can
>> still get it from Apple's website but it seems to be available from
>> here: http://www.versiontracker.com/dyn/moreinfo/macosx/21437
>>
>> XCode will be necessary to compile pretty much anything in OSX.
>>
>> Good luck,
>> Dominique
>>
>> On Tue, Oct 20, 2009 at 10:43 AM, Ranjit Chacko <rjc...@gm...>
>> wrote:
>>>
>>> I just tried getting a newer xcode, but the only one that seems to be
>>> available requires 10.6. Do you know what version of xcode I need and
>>> where I can get it?
>>>
>>> Thanks for your help,
>>>
>>> -Ranjit
>>>
>>> On Sun, Oct 18, 2009 at 2:59 PM, Dominique Orban
>>> <dom...@gm...> wrote:
>>>>
>>>> On Thu, Oct 15, 2009 at 4:21 PM, Ranjit Chacko <rjc...@gm...>
>>>> wrote:
>>>>>
>>>>> I just tried to build pysparse on my mac running 10.5.8 and it failed
>>>>> with
>>>>> the message below. Why did it fail?
>>>>>>>
>>>>>>> python setup.py install
>>>>
>>>> [snip]
>>>>
>>>>> gcc -g -L/usr/local/lib
>>>>> -L/Library/Frameworks/Python.framework/Versions/5.0.0/lib -bundle
>>>>> -undefined
>>>>> dynamic_lookup build/temp.macosx-10.3-i386-2.5/Src/spmatrixmodule.o -o
>>>>> build/lib.macosx-10.3-i386-2.5/pysparse/spmatrix.so
>>>>> /usr/bin/ld:
>>>>> /usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libSystem.dylib
>>>>> unknown flags (type) of section 6 (__TEXT,__literal16) in load command
>>>>> 0
>>>>> collect2: ld returned 1 exit status
>>>>> error: command 'gcc' failed with exit status 1
>>>>
>>>> Hi Ranjit,
>>>>
>>>> I am not entirely sure but my first guess would be that you upgraded
>>>> to 10.5.8 without upgrading/installing Xcode. Try installing the
>>>> latest Xcode and let us know if that did it.
>>>>
>>>> Thanks for trying out Pysparse!
>>>>
>>>> --
>>>> Dominique
>>>>
>>>
>>
>>
>>
>> --
>> Dominique
>
>



-- 
Dominique

Re: [Pysparse-users] build problem

[Dominique O. <dom...@gm...>]

On Thu, Oct 15, 2009 at 4:21 PM, Ranjit Chacko <rjc...@gm...> wrote:
> I just tried to build pysparse on my mac running 10.5.8 and it failed with
> the message below. Why did it fail?
>>>python setup.py install

[snip]

> gcc -g -L/usr/local/lib
> -L/Library/Frameworks/Python.framework/Versions/5.0.0/lib -bundle -undefined
> dynamic_lookup build/temp.macosx-10.3-i386-2.5/Src/spmatrixmodule.o -o
> build/lib.macosx-10.3-i386-2.5/pysparse/spmatrix.so
> /usr/bin/ld: /usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libSystem.dylib
> unknown flags (type) of section 6 (__TEXT,__literal16) in load command 0
> collect2: ld returned 1 exit status
> error: command 'gcc' failed with exit status 1

Hi Ranjit,

I am not entirely sure but my first guess would be that you upgraded
to 10.5.8 without upgrading/installing Xcode. Try installing the
latest Xcode and let us know if that did it.

Thanks for trying out Pysparse!

-- 
Dominique

[Pysparse-users] build problem

[Ranjit C. <rjc...@gm...>]

I just tried to build pysparse on my mac running 10.5.8 and it failed with
the message below. Why did it fail?
>>python setup.py install
running install
running build
running build_py
creating build
creating build/lib.macosx-10.3-i386-2.5
creating build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/__init__.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/__version__.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/directSolver.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/itsolvers_util.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/poisson.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/poisson_vec.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/pysparseMatrix.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/pysparseSuperLU.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/pysparseUmfpack.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/sparray.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/sparseMatrix.py -> build/lib.macosx-10.3-i386-2.5/pysparse
copying Lib/spmatrix_util.py -> build/lib.macosx-10.3-i386-2.5/pysparse
running build_ext
building 'pysparse.spmatrix' extension
creating build/temp.macosx-10.3-i386-2.5
creating build/temp.macosx-10.3-i386-2.5/Src
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd
-fno-common -dynamic -DNDEBUG -g -O3
-I/Library/Frameworks/Python.framework/Versions/5.0.0/include -DNUMPY=1
-IInclude
-I/Library/Frameworks/Python.framework/Versions/5.0.0/lib/python2.5/site-packages/numpy/core/include
-I/Library/Frameworks/Python.framework/Versions/5.0.0/include/python2.5 -c
Src/spmatrixmodule.c -o build/temp.macosx-10.3-i386-2.5/Src/spmatrixmodule.o
In file included from Src/spmatrixmodule.c:24:
Src/ll_mat.c:3146: warning: initialization from incompatible pointer type
In file included from Src/spmatrixmodule.c:26:
Src/sss_mat.c:216: warning: initialization from incompatible pointer type
gcc -g -L/usr/local/lib
-L/Library/Frameworks/Python.framework/Versions/5.0.0/lib -bundle -undefined
dynamic_lookup build/temp.macosx-10.3-i386-2.5/Src/spmatrixmodule.o -o
build/lib.macosx-10.3-i386-2.5/pysparse/spmatrix.so
/usr/bin/ld: /usr/lib/gcc/i686-apple-darwin8/4.0.1/../../../libSystem.dylib
unknown flags (type) of section 6 (__TEXT,__literal16) in load command 0
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

Re: [Pysparse-users] float elements?

[Dominique O. <dom...@gm...>]

On Thu, Oct 8, 2009 at 7:11 AM, Adler Perotte <ape...@gm...> wrote:
> Hello everyone,
>
> Thank you for developing this!
>
> I wanted to know if there is a way to store elements that are less memory
> intensive than doubles (ie floats)

Hi Adler,

At the moment Pysparse only supports the NPY_FLOAT64 data type, simply
because that's what the users and developers have needed so far. But
it would be great to generalize to other data types, including
complex, if there is sufficient interest. It is a substantial
development effort, though.

-- 
Dominique

[Pysparse-users] float elements?

[Adler P. <ape...@gm...>]

Hello everyone,

Thank you for developing this!

I wanted to know if there is a way to store elements that are less memory
intensive than doubles (ie floats)

Thanks,

-Adler

Re: [Pysparse-users] compiling problems with lapack

[Mag G. <mag...@gm...>]

so, it does not take LD_LIBRARY_CONFIG?



On Fri, Sep 18, 2009 at 9:47 AM, Dominique Orban
<dom...@gm...> wrote:
> On Thu, Sep 17, 2009 at 10:16 PM, Mag Gam<mag...@gm...> wrote:
>> I am trying to compile Pysparse 1.1 and I can't seem to find llapack.
>>
>> I keep getting ld: cannot find -llapack
>> collect2: ld returned 1 exist status
>>
>> I did compile LAPACK and placed it into my /usr/local/lib as liblapack.a
>>
>> and did a export LD_LIBRARY_PATH=/usr/local/lib
>>
>> I still get this error, any ideas?
>
> The best way to make sure your libraries are discovered is to edit
> setup.py and change the two following lines (towards the top of the
> file):
>
> library_dirs_list= ['/usr/local/lib']
> libraries_list = ['lapack', 'blas', 'g2c']
>
> You may not need libg2c any more if you are using a recent gfortran.
>
> --
> Dominique
>

Re: [Pysparse-users] compiling problems with lapack

[Dominique O. <dom...@gm...>]

On Thu, Sep 17, 2009 at 10:16 PM, Mag Gam<mag...@gm...> wrote:
> I am trying to compile Pysparse 1.1 and I can't seem to find llapack.
>
> I keep getting ld: cannot find -llapack
> collect2: ld returned 1 exist status
>
> I did compile LAPACK and placed it into my /usr/local/lib as liblapack.a
>
> and did a export LD_LIBRARY_PATH=/usr/local/lib
>
> I still get this error, any ideas?

The best way to make sure your libraries are discovered is to edit
setup.py and change the two following lines (towards the top of the
file):

library_dirs_list= ['/usr/local/lib']
libraries_list = ['lapack', 'blas', 'g2c']

You may not need libg2c any more if you are using a recent gfortran.

-- 
Dominique

[Pysparse-users] compiling problems with lapack

[Mag G. <mag...@gm...>]

I am trying to compile Pysparse 1.1 and I can't seem to find llapack.

I keep getting ld: cannot find -llapack
collect2: ld returned 1 exist status


I did compile LAPACK and placed it into my /usr/local/lib as liblapack.a

and did a export LD_LIBRARY_PATH=/usr/local/lib


I still get this error, any ideas?

TIA