pysparse-users

Re: [Pysparse-users] Has the keys() method changed behavior?

[Roman G. <rom...@gm...>]

- 1_1_x and branch_rel_1_0 are obsolete and should be removed.
- PSI can be deleted as well
- class_branch and python-2_4 I don't know

Thanks for your cleaning efforts!

Roman

On Aug 4, 2010, at 6:00 PM, Daniel Wheeler wrote:

> I reverted keys() back to its old behavior. I don't know why I changed
> it in the first place.  It's on branches/keys right now, so check it
> out and make sure it does what you want and I'll merge back to
> branches/version-1_1 and trunk.
> 
> Can we tidy up branches by removing some of the dead wood in there?
> Can I delete 1_1_x, PSI, branch_numpy, branch_rel_1_0, class_branch,
> python-2_4?
> 
> On Tue, Aug 3, 2010 at 5:21 AM, Toine Bogers <tb...@iv...> wrote:
>> Dear PySparse maintainers,
>> First of all: thanks for creating PySparse! It's been extremely useful in
>> writing my Python code for my collaborative filtering experiments. Up till
>> now I never really had any problems with it, but after I migrated to
>> v1.1.1 I ran into a peculiar 'bug/feature' with the keys() method.
>> I used to run v1.0 of PySparse on OS X 10.5 without any problems. I tried
>> installing the later versions, but they would never compile and the .dmg of
>> v1.0 on SourceForge worked fine for me. I recently migrated to Snow Leopard
>> (clean install) and to my surprise I could compile and install the latest
>> version of PySparse (v1.1.1) without any problems.
>> At first glance everything seemed to work just fine until I ran one of my
>> larger scripts. It returned a very peculiar error: the keys() method no
>> longer works as it should. Instead of returning a list of tuples of (row,
>> column)-indices it now returns a list containing two lists, one containing
>> all row indices, the other containing all column indices
>> The following example shows what I mean. I've created a small 3x3 sparse
>> matrix A:
>>>> print A
>> ll_mat(general, [3,3]):
>>  1.000000  2.000000  --------
>>  1.000000  1.000000  2.000000
>>  4.000000  5.000000  4.000000
>> Then when I print out the keys, it returns this:
>>>> print A.keys()
>> [[0, 0, 1, 1, 1, 2, 2, 2], [0, 1, 0, 1, 2, 0, 1, 2]]
>> whereas it used to return:
>> [[0, 0], [0, 1], [1, 0], [1, 1], [1, 2], [2, 0], [2, 1], [2, 2]]
>> And of course all of my code is expecting that method to work like that. The
>> online documentation still says that keys() returns "[...]a list of tuples
>> (i,j) of the nonzero matrix entries." Which is also what I expected it to
>> do, but not what it does anymore.
>> The strange thing is that the items() method still returns a list of
>> tuple-value pairs:
>>>> print A.items()
>> [((0, 0), 1.0), ((0, 1), 2.0), ((1, 0), 1.0), ((1, 1), 1.0), ((1, 2), 2.0),
>> ((2, 0), 4.0), ((2, 1), 5.0), ((2, 2), 4.0)]
>> In a way this is good, since it means I can, with some relatively minor
>> adjusments, use the items() method instead. But I would prefer not to have
>> to use items() when keys() is enough.
>> Do you have an explanation for the change in behavior I've observed? Is it
>> perhaps some sort of compilation option that I didn't set properly (which
>> makes the underlying Fortran/C-code return its results in a different format
>> to the keys() method)?
>> 
>> Kind regards,
>> Toine Bogers
>> ------------------------------------------------------------------------------
>> The Palm PDK Hot Apps Program offers developers who use the
>> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
>> of $1 Million in cash or HP Products. Visit us here for more details:
>> http://p.sf.net/sfu/dev2dev-palm
>> _______________________________________________
>> Pysparse-users mailing list
>> Pys...@li...
>> https://lists.sourceforge.net/lists/listinfo/pysparse-users
>> 
>> 
> 
> 
> 
> -- 
> Daniel Wheeler
> 
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> Pysparse-users mailing list
> Pys...@li...
> https://lists.sourceforge.net/lists/listinfo/pysparse-users

Re: [Pysparse-users] Has the keys() method changed behavior?

[Daniel W. <dan...@gm...>]

I reverted keys() back to its old behavior. I don't know why I changed
it in the first place.  It's on branches/keys right now, so check it
out and make sure it does what you want and I'll merge back to
branches/version-1_1 and trunk.

Can we tidy up branches by removing some of the dead wood in there?
Can I delete 1_1_x, PSI, branch_numpy, branch_rel_1_0, class_branch,
python-2_4?

On Tue, Aug 3, 2010 at 5:21 AM, Toine Bogers <tb...@iv...> wrote:
> Dear PySparse maintainers,
> First of all: thanks for creating PySparse! It's been extremely useful in
> writing my Python code for my collaborative filtering experiments. Up till
> now I never really had any problems with it, but after I migrated to
> v1.1.1 I ran into a peculiar 'bug/feature' with the keys() method.
> I used to run v1.0 of PySparse on OS X 10.5 without any problems. I tried
> installing the later versions, but they would never compile and the .dmg of
> v1.0 on SourceForge worked fine for me. I recently migrated to Snow Leopard
> (clean install) and to my surprise I could compile and install the latest
> version of PySparse (v1.1.1) without any problems.
> At first glance everything seemed to work just fine until I ran one of my
> larger scripts. It returned a very peculiar error: the keys() method no
> longer works as it should. Instead of returning a list of tuples of (row,
> column)-indices it now returns a list containing two lists, one containing
> all row indices, the other containing all column indices
> The following example shows what I mean. I've created a small 3x3 sparse
> matrix A:
>>> print A
> ll_mat(general, [3,3]):
>  1.000000  2.000000  --------
>  1.000000  1.000000  2.000000
>  4.000000  5.000000  4.000000
> Then when I print out the keys, it returns this:
>>> print A.keys()
> [[0, 0, 1, 1, 1, 2, 2, 2], [0, 1, 0, 1, 2, 0, 1, 2]]
> whereas it used to return:
> [[0, 0], [0, 1], [1, 0], [1, 1], [1, 2], [2, 0], [2, 1], [2, 2]]
> And of course all of my code is expecting that method to work like that. The
> online documentation still says that keys() returns "[...]a list of tuples
> (i,j) of the nonzero matrix entries." Which is also what I expected it to
> do, but not what it does anymore.
> The strange thing is that the items() method still returns a list of
> tuple-value pairs:
>>> print A.items()
> [((0, 0), 1.0), ((0, 1), 2.0), ((1, 0), 1.0), ((1, 1), 1.0), ((1, 2), 2.0),
> ((2, 0), 4.0), ((2, 1), 5.0), ((2, 2), 4.0)]
> In a way this is good, since it means I can, with some relatively minor
> adjusments, use the items() method instead. But I would prefer not to have
> to use items() when keys() is enough.
> Do you have an explanation for the change in behavior I've observed? Is it
> perhaps some sort of compilation option that I didn't set properly (which
> makes the underlying Fortran/C-code return its results in a different format
> to the keys() method)?
>
> Kind regards,
> Toine Bogers
> ------------------------------------------------------------------------------
> The Palm PDK Hot Apps Program offers developers who use the
> Plug-In Development Kit to bring their C/C++ apps to Palm for a share
> of $1 Million in cash or HP Products. Visit us here for more details:
> http://p.sf.net/sfu/dev2dev-palm
> _______________________________________________
> Pysparse-users mailing list
> Pys...@li...
> https://lists.sourceforge.net/lists/listinfo/pysparse-users
>
>



-- 
Daniel Wheeler

[Pysparse-users] Has the keys() method changed behavior?

[Toine B. <tb...@iv...>]

Dear PySparse maintainers,

First of all: thanks for creating PySparse! It's been extremely useful in writing my Python code for my collaborative filtering experiments. Up till now I never really had any problems with it, but after I migrated to v1.1.1 I ran into a peculiar 'bug/feature' with the keys() method. 

I used to run v1.0 of PySparse on OS X 10.5 without any problems. I tried installing the later versions, but they would never compile and the .dmg of v1.0 on SourceForge worked fine for me. I recently migrated to Snow Leopard (clean install) and to my surprise I could compile and install the latest version of PySparse (v1.1.1) without any problems. 

At first glance everything seemed to work just fine until I ran one of my larger scripts. It returned a very peculiar error: the keys() method no longer works as it should. Instead of returning a list of tuples of (row, column)-indices it now returns a list containing two lists, one containing all row indices, the other containing all column indices

The following example shows what I mean. I've created a small 3x3 sparse matrix A:

>> print A

ll_mat(general, [3,3]):
 1.000000  2.000000  -------- 
 1.000000  1.000000  2.000000 
 4.000000  5.000000  4.000000 

Then when I print out the keys, it returns this:

>> print A.keys()

[[0, 0, 1, 1, 1, 2, 2, 2], [0, 1, 0, 1, 2, 0, 1, 2]]

whereas it used to return:

[[0, 0], [0, 1], [1, 0], [1, 1], [1, 2], [2, 0], [2, 1], [2, 2]]

And of course all of my code is expecting that method to work like that. The online documentation still says that keys() returns "[...]a list of tuples (i,j) of the nonzero matrix entries." Which is also what I expected it to do, but not what it does anymore. 

The strange thing is that the items() method still returns a list of tuple-value pairs:

>> print A.items()

[((0, 0), 1.0), ((0, 1), 2.0), ((1, 0), 1.0), ((1, 1), 1.0), ((1, 2), 2.0), ((2, 0), 4.0), ((2, 1), 5.0), ((2, 2), 4.0)]

In a way this is good, since it means I can, with some relatively minor adjusments, use the items() method instead. But I would prefer not to have to use items() when keys() is enough.

Do you have an explanation for the change in behavior I've observed? Is it perhaps some sort of compilation option that I didn't set properly (which makes the underlying Fortran/C-code return its results in a different format to the keys() method)?


Kind regards,
Toine Bogers

Re: [Pysparse-users] pysparse with windows

[Dominique O. <dom...@gm...>]

On Tue, Jul 13, 2010 at 8:09 AM, AUGARDE C.E.
<cha...@du...> wrote:
> I am trying to use pysparse on a Windows (xp) python installation. When I
> try to test, by running testSuperLU as suggested at
> http://pysparse.sourceforge.net/introduction.html#testing-pysparse
> I encounter problems since python module resource is for Unix. Does anyone
> have any suggestions?
>
> I started with python 2.7 and have since tried going back to  python 2.5 as
> suggested in a reply to a post on another forum.

You are right that the 'resource' module is Unix specific and isn't
available on Windows. Switching Python version would not resolve the
issue.

I tried to correct the situation by adding a new "sptime" module to
Pysparse. The new module uses 'resource' if available, or time.clock()
if not. Could you grab the new branch and give it a try?

svn co https://pysparse.svn.sourceforge.net/svnroot/pysparse/branches/new-setup-py
pysparse

-- 
Dominique

[Pysparse-users] pysparse with windows

[AUGARDE C.E. <cha...@du...>]

I am trying to use pysparse on a Windows (xp) python installation. When I try to test, by running testSuperLU as suggested at http://pysparse.sourceforge.net/introduction.html#testing-pysparse <http://pysparse.sourceforge.net/introduction.html#testing-pysparse> 
I encounter problems since python module resource is for Unix. Does anyone have any suggestions?
 
I started with python 2.7 and have since tried going back to  python 2.5 as suggested in a reply to a post on another forum.
 
Thanks
 
Charles Augarde

Re: [Pysparse-users] build on snow leopard failing

[Jonathan G. <gu...@ni...>]

On Jun 22, 2010, at 8:03 PM, Ranjit Chacko wrote:

> I just tried to build pysparse 1.1.1 on my mac running snow leopard and it failed with the output below. Any ideas on how to fix this? Thanks,:
> 
> In file included from /Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/unicodeobject.h:4,
>                  from /Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:85,
>                  from Src/spmatrixmodule.c:1:
> /Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h: No such file or directory


Do you have a /Developer/SDKs/MacOSX10.4u.sdk ? I don't believe it's installed by default with Xcode.

What Python are you running?

[Pysparse-users] build on snow leopard failing

[Ranjit C. <rjc...@gm...>]

I just tried to build pysparse 1.1.1 on my mac running snow leopard and it
failed with the output below. Any ideas on how to fix this? Thanks,:

running build
running build_py
creating build/lib.macosx-10.3-fat-2.6
creating build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/__init__.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/__version__.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/directSolver.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/itsolvers_util.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/poisson.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/poisson_vec.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/pysparseMatrix.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/pysparseSuperLU.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/pysparseUmfpack.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/sparray.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/sparseMatrix.py -> build/lib.macosx-10.3-fat-2.6/pysparse
copying Lib/spmatrix_util.py -> build/lib.macosx-10.3-fat-2.6/pysparse
running build_ext
building 'pysparse.spmatrix' extension
creating build/temp.macosx-10.3-fat-2.6
creating build/temp.macosx-10.3-fat-2.6/Src
gcc-4.0 -arch ppc -arch i386 -isysroot /Developer/SDKs/MacOSX10.4u.sdk
-fno-strict-aliasing -fno-common -dynamic -DNDEBUG -g -O3 -DNUMPY=1
-DLENFUNC_OK=1 -IInclude
-I/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/numpy/core/include
-I/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6 -c
Src/spmatrixmodule.c -o build/temp.macosx-10.3-fat-2.6/Src/spmatrixmodule.o
In file included from
/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/unicodeobject.h:4,
                 from
/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:85,
                 from Src/spmatrixmodule.c:1:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h:
No such file or directory
In file included from
/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/unicodeobject.h:4,
                 from
/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/Python.h:85,
                 from Src/spmatrixmodule.c:1:
/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25: error: stdarg.h:
No such file or directory
lipo: can't figure out the architecture type of:
/var/folders/0U/0UxwF0nGHwaJh1DEleDsOE+++TM/-Tmp-//ccTcRNHm.out
error: command 'gcc-4.0' failed with exit status 1

Re: [Pysparse-users] dtypes

[Dominique O. <dom...@gm...>]

On Sat, Jun 19, 2010 at 4:17 AM, Ranjit Chacko <rjc...@gm...> wrote:
> Is there a way to use short integers with pysparse? I'm constructing a
> large adjacency matrix for a graph, and the entries are only going be
> 1's.

At the moment, only matrices of doubles can be stored. This question
has come up in the past and the ability to store matrices of ints and
complex numbers would be a big plus.

-- 
Dominique

[Pysparse-users] dtypes

[Ranjit C. <rjc...@gm...>]

Is there a way to use short integers with pysparse? I'm constructing a
large adjacency matrix for a graph, and the entries are only going be
1's.

Re: [Pysparse-users] memory leak

[Dominique O. <dom...@gm...>]

On Mon, Jun 7, 2010 at 7:15 PM, Daniel Wheeler
<dan...@gm...> wrote:
> Hi Dominique,
>
> I tried the new-setup-py branch against fipy and everything worked
> fine with the following small change:
>
> bunter[wd15]: svn diff
> Index: pysparse/__init__.py
> ===================================================================
> --- pysparse/__init__.py        (revision 177)
> +++ pysparse/__init__.py        (working copy)
> @@ -5,6 +5,7 @@
>  # Imports
>  from numpy._import_tools import PackageLoader
>  from version import version as __version__
> +from sparse import spmatrix
>
>  #from sparse import *
>  from misc import get_include
>
> The organization seems greatly improved. Thanks for all the hard work.
> Is the change above acceptable?

Hi Daniel,

I don't see any problem with the above change. The branch is updated.
Let's now make new-setup the default.

Cheers,

-- 
Dominique

Re: [Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

Hi Dominique,

I tried the new-setup-py branch against fipy and everything worked
fine with the following small change:

bunter[wd15]: svn diff
Index: pysparse/__init__.py
===================================================================
--- pysparse/__init__.py        (revision 177)
+++ pysparse/__init__.py        (working copy)
@@ -5,6 +5,7 @@
 # Imports
 from numpy._import_tools import PackageLoader
 from version import version as __version__
+from sparse import spmatrix

 #from sparse import *
 from misc import get_include

The organization seems greatly improved. Thanks for all the hard work.
Is the change above acceptable?

On Fri, Jun 4, 2010 at 11:06 AM, Dominique Orban
<dom...@gm...> wrote:
> On Fri, Jun 4, 2010 at 3:37 PM, Daniel Wheeler
> <dan...@gm...> wrote:
>> 2010/6/4 Dominique Orban <dom...@gm...>:
>>
>>> Daniel, should we be thinking about making new-setup the default in
>>> Pysparse? That is the one using the "standard" distutils layout and is
>>> the one in the Python Package Index. Have you tried it with FiPy?
>>
>> Net yet. I'll try it with fipy and then you can merge it back into
>> trunk. I'm going to make a release from the version 1_1_x branch
>> anyway (not from trunk) with the minor bug fixes we've been making so
>> this is irrelevant right now. Once you are happy with your merge we
>> can do a version 1.2 release from trunk. Does that sound reasonable?
>
> Sounds good. I'll wait for your comments on using the new-setup branch
> with Fipy. There will be some namespace adjustments required. Of
> course, the namespace can be adjusted too.
>
> --
> Dominique
>



-- 
Daniel Wheeler

Re: [Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

Just added version 1.1.1 to sourceforge and created tags/version-1_1_1.

On Fri, Jun 4, 2010 at 5:49 PM, Daniel Wheeler
<dan...@gm...> wrote:
> One more thing when I build pysparse I noticed that I'm getting a "-g"
> compiler argument? For example,
>
> gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
> -Wstrict-prototypes -fPIC -DUSE_VENDOR_BLAS=1 -DNUMPY=1 -Isuperlu
> -IInclude -I/users/wd15/usr/i686/5.0/lib/python2.5/site-packages/numpy/core/include
> -I/usr/include/python2.5 -c superlu/util.c -o
> build/temp.linux-i686-2.5/superlu/util.o
>
> How do I get rid of that? I'm using the setup.py from 1_1_x.
>
> 2010/6/4 Dominique Orban <dom...@gm...>:
>> 2010/6/4 Łukasz Pankowski <lu...@o2...>:
>>> Daniel Wheeler <dan...@gm...> writes:
>>>
>>>> Can I do a 1.1.1 release? Any objections? Find() method  is quite
>>>> important for fipy right now.
>>>
>>> Could you apply my old patch (2009-12-09) before the release:
>>>
>>> ll_mat: fix raising IndexError for invalid indices - ID: 2911619
>>> http://sourceforge.net/tracker/?func=detail&aid=2911619&group_id=101403&atid=629638
>>
>> Lukasz,
>>
>> Very sorry we missed this patch somehow. I fixed the new-setup branch.
>>
>> Daniel, should we be thinking about making new-setup the default in
>> Pysparse? That is the one using the "standard" distutils layout and is
>> the one in the Python Package Index. Have you tried it with FiPy?
>>
>> --
>> Dominique
>>
>
>
>
> --
> Daniel Wheeler
>



-- 
Daniel Wheeler

Re: [Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

One more thing when I build pysparse I noticed that I'm getting a "-g"
compiler argument? For example,

gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall
-Wstrict-prototypes -fPIC -DUSE_VENDOR_BLAS=1 -DNUMPY=1 -Isuperlu
-IInclude -I/users/wd15/usr/i686/5.0/lib/python2.5/site-packages/numpy/core/include
-I/usr/include/python2.5 -c superlu/util.c -o
build/temp.linux-i686-2.5/superlu/util.o

How do I get rid of that? I'm using the setup.py from 1_1_x.

2010/6/4 Dominique Orban <dom...@gm...>:
> 2010/6/4 Łukasz Pankowski <lu...@o2...>:
>> Daniel Wheeler <dan...@gm...> writes:
>>
>>> Can I do a 1.1.1 release? Any objections? Find() method  is quite
>>> important for fipy right now.
>>
>> Could you apply my old patch (2009-12-09) before the release:
>>
>> ll_mat: fix raising IndexError for invalid indices - ID: 2911619
>> http://sourceforge.net/tracker/?func=detail&aid=2911619&group_id=101403&atid=629638
>
> Lukasz,
>
> Very sorry we missed this patch somehow. I fixed the new-setup branch.
>
> Daniel, should we be thinking about making new-setup the default in
> Pysparse? That is the one using the "standard" distutils layout and is
> the one in the Python Package Index. Have you tried it with FiPy?
>
> --
> Dominique
>



-- 
Daniel Wheeler

Re: [Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

Dominque, You may run into quite a few conflicts when you try and
merge your new-setup branch back to trunk. If the bug fixes are being
duplicated you will have some reconciling to do. Ideally, you'd make
the bug fix on trunk and then merge trunk to your new-setup branch
keeping a record in the logs of the revision numbers. To merge
new-setup back in to trunk you would first merge trunk to new-setup
and reconcile and then merge back the other way. Anyway, we are quite
careful about this type of thing in fipy, see
<http://matforge.org/fipy/browser/trunk/documentation/ADMINISTRATA.txt>.
We may want to implement a similar policy for pysparse or add this
ADMINISTRATA file with suitable modifications. Cheers.

On Fri, Jun 4, 2010 at 11:06 AM, Dominique Orban
<dom...@gm...> wrote:
> On Fri, Jun 4, 2010 at 3:37 PM, Daniel Wheeler
> <dan...@gm...> wrote:
>> 2010/6/4 Dominique Orban <dom...@gm...>:
>>
>>> Daniel, should we be thinking about making new-setup the default in
>>> Pysparse? That is the one using the "standard" distutils layout and is
>>> the one in the Python Package Index. Have you tried it with FiPy?
>>
>> Net yet. I'll try it with fipy and then you can merge it back into
>> trunk. I'm going to make a release from the version 1_1_x branch
>> anyway (not from trunk) with the minor bug fixes we've been making so
>> this is irrelevant right now. Once you are happy with your merge we
>> can do a version 1.2 release from trunk. Does that sound reasonable?
>
> Sounds good. I'll wait for your comments on using the new-setup branch
> with Fipy. There will be some namespace adjustments required. Of
> course, the namespace can be adjusted too.
>
> --
> Dominique
>



-- 
Daniel Wheeler

Re: [Pysparse-users] memory leak

[Dominique O. <dom...@gm...>]

On Fri, Jun 4, 2010 at 3:37 PM, Daniel Wheeler
<dan...@gm...> wrote:
> 2010/6/4 Dominique Orban <dom...@gm...>:
>
>> Daniel, should we be thinking about making new-setup the default in
>> Pysparse? That is the one using the "standard" distutils layout and is
>> the one in the Python Package Index. Have you tried it with FiPy?
>
> Net yet. I'll try it with fipy and then you can merge it back into
> trunk. I'm going to make a release from the version 1_1_x branch
> anyway (not from trunk) with the minor bug fixes we've been making so
> this is irrelevant right now. Once you are happy with your merge we
> can do a version 1.2 release from trunk. Does that sound reasonable?

Sounds good. I'll wait for your comments on using the new-setup branch
with Fipy. There will be some namespace adjustments required. Of
course, the namespace can be adjusted too.

-- 
Dominique

Re: [Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

2010/6/4 Dominique Orban <dom...@gm...>:

> Daniel, should we be thinking about making new-setup the default in
> Pysparse? That is the one using the "standard" distutils layout and is
> the one in the Python Package Index. Have you tried it with FiPy?

Net yet. I'll try it with fipy and then you can merge it back into
trunk. I'm going to make a release from the version 1_1_x branch
anyway (not from trunk) with the minor bug fixes we've been making so
this is irrelevant right now. Once you are happy with your merge we
can do a version 1.2 release from trunk. Does that sound reasonable?

-- 
Daniel Wheeler

Re: [Pysparse-users] memory leak

[Dominique O. <dom...@gm...>]

2010/6/4 Łukasz Pankowski <lu...@o2...>:
> Daniel Wheeler <dan...@gm...> writes:
>
>> Can I do a 1.1.1 release? Any objections? Find() method  is quite
>> important for fipy right now.
>
> Could you apply my old patch (2009-12-09) before the release:
>
> ll_mat: fix raising IndexError for invalid indices - ID: 2911619
> http://sourceforge.net/tracker/?func=detail&aid=2911619&group_id=101403&atid=629638

Lukasz,

Very sorry we missed this patch somehow. I fixed the new-setup branch.

Daniel, should we be thinking about making new-setup the default in
Pysparse? That is the one using the "standard" distutils layout and is
the one in the Python Package Index. Have you tried it with FiPy?

-- 
Dominique

Re: [Pysparse-users] memory leak

[<lu...@o2...>]

Daniel Wheeler <dan...@gm...> writes:

> Can I do a 1.1.1 release? Any objections? Find() method  is quite
> important for fipy right now.

Could you apply my old patch (2009-12-09) before the release:

ll_mat: fix raising IndexError for invalid indices - ID: 2911619
http://sourceforge.net/tracker/?func=detail&aid=2911619&group_id=101403&atid=629638

The patch has unclear description (better below) and probably will fail
(another of my patches already applied is changing the same lines) -- I
can update the patch if you are willing to apply it.

Better description:

>>> from pysparse.spmatrix import ll_mat
>>> a = ll_mat(3,3)
>>> a[0,0]
0.0
>>> print a[0]
None
>>> print a[0,0,0]
None

But "a[0]" and "a[0,0,0]" should raise IndexError and that is what
happens in the C function, but the function returns None instead of NULL
and that is why the exception is not actually raised.

>
> On Thu, Jun 3, 2010 at 10:10 AM, Dominique Orban
> <dom...@gm...> wrote:
>> Yes please. Thanks for finding this bug!
>>
>> On Thu, Jun 3, 2010 at 3:47 PM, Daniel Wheeler
>> <dan...@gm...> wrote:
>>> Shall I do this on the version 1.1 branch and trunk?
>>>
>>> 2010/6/3 Dominique Orban <dom...@gm...>:
>>>> 2010/6/3 Łukasz Pankowski <lu...@o2...>:
>>>>> Daniel Wheeler <dan...@gm...> writes:
>>>>>
>>>>>> It seems that the find() method (lin 2986 in in Src/ll_mat.c) has a
>>>>>> memory leak. Calling PyDECREF on a_row, a_col and a_val fixes this
>>>>>> issue. I propose the following change:
>>>>>
>>>>> I propose simpler (`N' is like `O' but without PyINCREF, details below):
>>>>>
>>>>> $ git diff
>>>>> diff --git a/Src/ll_mat.c b/Src/ll_mat.c
>>>>> index 230829a..1adee2f 100644
>>>>> --- a/Src/ll_mat.c
>>>>> +++ b/Src/ll_mat.c
>>>>> @@ -3014,7 +3014,7 @@ static PyObject *LLMat_Find( LLMatObject *self, PyObject *args ) {
>>>>>     }
>>>>>   }
>>>>>
>>>>> -  return Py_BuildValue( "OOO",
>>>>> +  return Py_BuildValue( "NNN",
>>>>>                         PyArray_Return( a_val ),
>>>>>                         PyArray_Return( a_row ),
>>>>>                         PyArray_Return( a_col ) );
>>>>>
>>>>>
>>>>> from File: python2.5-api.info,  Node: Parsing arguments and building values
>>>>>
>>>>>    ``N' (object) {[PyObject * }]'
>>>>>          Same as `O', except it doesn't increment the reference count
>>>>>          on the object.  Useful when the object is created by a call
>>>>>          to an object constructor in the argument list.
>>>>
>>>> Thanks. I fixed it in the new-setup branch as well.
>>>>
>>>> --
>>>> Dominique
>>>>
>>>
>>>
>>>
>>> --
>>> Daniel Wheeler
>>>
>>
>>
>>
>> --
>> Dominique
>>

Re: [Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

Can I do a 1.1.1 release? Any objections? Find() method  is quite
important for fipy right now.

On Thu, Jun 3, 2010 at 10:10 AM, Dominique Orban
<dom...@gm...> wrote:
> Yes please. Thanks for finding this bug!
>
> On Thu, Jun 3, 2010 at 3:47 PM, Daniel Wheeler
> <dan...@gm...> wrote:
>> Shall I do this on the version 1.1 branch and trunk?
>>
>> 2010/6/3 Dominique Orban <dom...@gm...>:
>>> 2010/6/3 Łukasz Pankowski <lu...@o2...>:
>>>> Daniel Wheeler <dan...@gm...> writes:
>>>>
>>>>> It seems that the find() method (lin 2986 in in Src/ll_mat.c) has a
>>>>> memory leak. Calling PyDECREF on a_row, a_col and a_val fixes this
>>>>> issue. I propose the following change:
>>>>
>>>> I propose simpler (`N' is like `O' but without PyINCREF, details below):
>>>>
>>>> $ git diff
>>>> diff --git a/Src/ll_mat.c b/Src/ll_mat.c
>>>> index 230829a..1adee2f 100644
>>>> --- a/Src/ll_mat.c
>>>> +++ b/Src/ll_mat.c
>>>> @@ -3014,7 +3014,7 @@ static PyObject *LLMat_Find( LLMatObject *self, PyObject *args ) {
>>>>     }
>>>>   }
>>>>
>>>> -  return Py_BuildValue( "OOO",
>>>> +  return Py_BuildValue( "NNN",
>>>>                         PyArray_Return( a_val ),
>>>>                         PyArray_Return( a_row ),
>>>>                         PyArray_Return( a_col ) );
>>>>
>>>>
>>>> from File: python2.5-api.info,  Node: Parsing arguments and building values
>>>>
>>>>    ``N' (object) {[PyObject * }]'
>>>>          Same as `O', except it doesn't increment the reference count
>>>>          on the object.  Useful when the object is created by a call
>>>>          to an object constructor in the argument list.
>>>
>>> Thanks. I fixed it in the new-setup branch as well.
>>>
>>> --
>>> Dominique
>>>
>>
>>
>>
>> --
>> Daniel Wheeler
>>
>
>
>
> --
> Dominique
>



-- 
Daniel Wheeler

Re: [Pysparse-users] memory leak

[Dominique O. <dom...@gm...>]

2010/6/3 Łukasz Pankowski <lu...@o2...>:
> Daniel Wheeler <dan...@gm...> writes:
>
>> It seems that the find() method (lin 2986 in in Src/ll_mat.c) has a
>> memory leak. Calling PyDECREF on a_row, a_col and a_val fixes this
>> issue. I propose the following change:
>
> I propose simpler (`N' is like `O' but without PyINCREF, details below):
>
> $ git diff
> diff --git a/Src/ll_mat.c b/Src/ll_mat.c
> index 230829a..1adee2f 100644
> --- a/Src/ll_mat.c
> +++ b/Src/ll_mat.c
> @@ -3014,7 +3014,7 @@ static PyObject *LLMat_Find( LLMatObject *self, PyObject *args ) {
>     }
>   }
>
> -  return Py_BuildValue( "OOO",
> +  return Py_BuildValue( "NNN",
>                         PyArray_Return( a_val ),
>                         PyArray_Return( a_row ),
>                         PyArray_Return( a_col ) );
>
>
> from File: python2.5-api.info,  Node: Parsing arguments and building values
>
>    ``N' (object) {[PyObject * }]'
>          Same as `O', except it doesn't increment the reference count
>          on the object.  Useful when the object is created by a call
>          to an object constructor in the argument list.

Thanks. I fixed it in the new-setup branch as well.

-- 
Dominique

Re: [Pysparse-users] memory leak

[<lu...@o2...>]

Daniel Wheeler <dan...@gm...> writes:

> It seems that the find() method (lin 2986 in in Src/ll_mat.c) has a
> memory leak. Calling PyDECREF on a_row, a_col and a_val fixes this
> issue. I propose the following change:

I propose simpler (`N' is like `O' but without PyINCREF, details below):

$ git diff
diff --git a/Src/ll_mat.c b/Src/ll_mat.c
index 230829a..1adee2f 100644
--- a/Src/ll_mat.c
+++ b/Src/ll_mat.c
@@ -3014,7 +3014,7 @@ static PyObject *LLMat_Find( LLMatObject *self, PyObject *args ) {
     }
   }
 
-  return Py_BuildValue( "OOO",
+  return Py_BuildValue( "NNN",
                         PyArray_Return( a_val ),
                         PyArray_Return( a_row ),
                         PyArray_Return( a_col ) );


from File: python2.5-api.info,  Node: Parsing arguments and building values

    ``N' (object) {[PyObject * }]'
          Same as `O', except it doesn't increment the reference count
          on the object.  Useful when the object is created by a call
          to an object constructor in the argument list.

[Pysparse-users] memory leak

[Daniel W. <dan...@gm...>]

It seems that the find() method (lin 2986 in in Src/ll_mat.c) has a
memory leak. Calling PyDECREF on a_row, a_col and a_val fixes this
issue. I propose the following change:

$ svn diff
Index: Src/ll_mat.c
===================================================================
--- Src/ll_mat.c        (revision 168)
+++ Src/ll_mat.c        (working copy)
@@ -2987,6 +2987,7 @@

   /* Convert an LL matrix into coordinate format */

+  PyObject *returnTuple;
   PyArrayObject *a_row, *a_col, *a_val; /* Matrix in coordinate format */
   npy_intp       dmat[1];               /* Dimension descriptor */
   int           *pi, *pj;             /* Intermediate pointers to
matrix data */
@@ -3014,10 +3015,16 @@
     }
   }

-  return Py_BuildValue( "OOO",
-                        PyArray_Return( a_val ),
-                        PyArray_Return( a_row ),
-                        PyArray_Return( a_col ) );
+  returnTuple = Py_BuildValue( "OOO",
+                               PyArray_Return( a_val ),
+                               PyArray_Return( a_row ),
+                               PyArray_Return( a_col ) );
+
+  Py_DECREF(a_row);
+  Py_DECREF(a_col);
+  Py_DECREF(a_val);
+
+  return returnTuple;
 }

 /*********************************/

If no one has any particular objection, I'll add that changes to to
branches/1_1_x and merge them to trunk/.

Cheers

-- 
Daniel Wheeler

Re: [Pysparse-users] Trouble finding the correct ATLAS LAPACK libraries with pysparse 1.2a1

[Daniel W. <dan...@gm...>]

On Thu, May 13, 2010 at 11:58 AM, Eddie Schwalbach
<e-s...@no...> wrote:
> I removed the build directory, and reran the build and install steps with
> the correct library paths and everything seems to be working now.  I guess
> there was something in the build directory pointing to the wrong library
> location from my initial install attempt that wasn't removed by
>
> pythyon setup.py clean

"clean" does nothing to the installed directories, it will just clean
up the working copy. I would always delete the build directory rather
than using "clean" anyway.

> (Daniel, I now have FiPy up and running with Trilinos on our cluster, I'll
> share scaling info with you after I run some tests.)

Very good. Look forward to it.

-- 
Daniel Wheeler

Re: [Pysparse-users] Trouble finding the correct ATLAS LAPACK libraries with pysparse 1.2a1

[Eddie S. <e-s...@no...>]

I removed the build directory, and reran the build and install steps with
the correct library paths and everything seems to be working now.  I guess
there was something in the build directory pointing to the wrong library
location from my initial install attempt that wasn't removed by

pythyon setup.py clean

Thanks for the help.

(Daniel, I now have FiPy up and running with Trilinos on our cluster, I'll
share scaling info with you after I run some tests.)

Eddie

On Tue, May 11, 2010 at 12:48 PM, Dominique Orban <dom...@gm...
> wrote:

> On Tue, May 11, 2010 at 12:46:43PM -0500, Eddie Schwalbach wrote:
> > Hi guys, thanks for the response.  Our machine went down for maintenance
> > this morning, as soon as its back up I'll send the requested outputs.
> >
> > I haven't tested the ATLAS or LAPACK libraries, but I have been able to
> > compile other software against them (specifically Trilinos 9.0.3
> including
> > the PyTrilinos module).  Not sure if it matters, but there are other
> > versions of ATLAS (3.9.16 and 3.9.24) at /software/ATLAS/#.#.##  where
> #'s
> > are the version numbers.  These newer versions don't have full LAPACK
> > support or shared versions of the libraries.
> >
> > To answer your question Daniel, there was no previous install of PySparse
> > for the Python in my home directory, but I believe on the first attempt
> at
> > installing I used the default setup.py, perhaps this left something
> around
> > that's interfering with more recent attempts to install?  I did run
> >
> > python setup.py clean
> >
> > before modifying setup.py and rerunning the build/install steps.  Is
> there a
> > better way of removing/cleaning up before I try installing again?
>
> It is much safer to 'rm -rf build'. I find the Distutils aren't as
> reliable as Make.
>
> > Thanks again, and more info to follow when I can get access to the
> machine.
> >
> > Eddie
> >
> >
> > On Tue, May 11, 2010 at 9:01 AM, Daniel Wheeler
> > <dan...@gm...>wrote:
> > Was there a previous pysparse installation?
> >
> >
> > On Tue, May 11, 2010 at 2:01 AM, Dominique Orban
> > <dom...@gm...>wrote:
> >
> > > On Mon, May 10, 2010 at 5:59 PM, Eddie Schwalbach
> > > <e-s...@no...> wrote:
> > > > I'm trying to install pysparse on a Linux cluster with ATLAS 3.8.3
> (which
> > > > was built with full LAPACK support).  ATLAS is located at
> > > > /software/ATLAS/3.8.3
> > > > and I'm using Python2.6.4 which is located in the nonstandard
> location
> > > > /home/ejs698/software (which is at the beginning of my PATH and
> > > PYTHONPATH).
> > > >
> > > > I've modified the setup.py file to include the appropriate (I think)
> > > library
> > > > directories:
> > > >
> > > > library_dirs_list=
> > > >
> > >
> ['/software/ATLAS/3.8.3/lib/','/home/ejs698/software/lib/','/usr/bin/lib/']
> > > > libraries_list = ['f77blas', 'cblas', 'atlas', 'lapack', 'gfortran']
> > > >
> > > > I also have the environment variables:
> > > > LD_LIBRARY_PATH
> > > >
> > >
> /opt/moab/lib:/opt/torque/lib::/software/ATLAS/3.8.3/lib:/home/ejs698/software/lib
> > > > ATLAS  /software/ATLAS/3.8.3  (this was necessary for getting numpy
> built
> > > > against the correct ATLAS libraries)
> > > >
> > > > I don't see errors when I run python setup.py install, or when I try
> > > >      from pysparse import *
> > > > However, if I attempt to run any of the tests in the Examples
> directory,
> > > I
> > > > get:
> > > >
> > > > [ejs698@quser04 Examples]$ python jdsym_test.py
> > > > Traceback (most recent call last):
> > > >   File "jdsym_test.py", line 1, in <module>
> > > >     from pysparse import spmatrix, jdsym, itsolvers
> > > > ImportError: /software/ATLAS/lib/liblapack.so: cannot open shared
> object
> > > > file: No such file or directory
> > > >
> > > > This seems to indicate that pysparse is looking in the wrong place
> for
> > > the
> > > > lapack libraries (the correct location is
> > > > /software/ATLAS/3.8.3/lib/liblapack.so).  Have I missed some
> environment
> > > > variable or setting in the setup.py to properly specify these library
> > > > locations?
> > > >
> > > > Any suggestions are appreciated,  and thanks in advance,
> > >
> > > Hello Eddie,
> > >
> > > There are no environment variables to set in Pysparse, so no, you
> > > haven't missed any.
> > >
> > > Could you please post the complete output of 'python setup.py build'
> > > as well as the result of 'ls -l
> > > /software/ATLAS/3.8.3/lib/liblapack.so'? I don't doubt your word that
> > > the library is there but am just trying to get the obvious things out
> > > of the way first. Also I presume you can run the ATLAS and LAPACK
> > > tests with your libraries?
> > >
> > > Thanks,
> > >
> > > --
> > > Dominique
> > >
>

Re: [Pysparse-users] Trouble finding the correct ATLAS LAPACK libraries with pysparse 1.2a1

[Daniel W. <dan...@gm...>]

Perhaps.It shouldn't matter, but you can just simply remove the
pysparse directory in site-packages, wherever it is. Can't hurt.

On Tue, May 11, 2010 at 1:46 PM, Eddie Schwalbach
<e-s...@no...> wrote:
> Hi guys, thanks for the response.  Our machine went down for maintenance
> this morning, as soon as its back up I'll send the requested outputs.
>
> I haven't tested the ATLAS or LAPACK libraries, but I have been able to
> compile other software against them (specifically Trilinos 9.0.3 including
> the PyTrilinos module).  Not sure if it matters, but there are other
> versions of ATLAS (3.9.16 and 3.9.24) at /software/ATLAS/#.#.##  where #'s
> are the version numbers.  These newer versions don't have full LAPACK
> support or shared versions of the libraries.
>
> To answer your question Daniel, there was no previous install of PySparse
> for the Python in my home directory, but I believe on the first attempt at
> installing I used the default setup.py, perhaps this left something around
> that's interfering with more recent attempts to install?  I did run
>
> python setup.py clean
>
> before modifying setup.py and rerunning the build/install steps.  Is there a
> better way of removing/cleaning up before I try installing again?
>
> Thanks again, and more info to follow when I can get access to the machine.
>
> Eddie
>
>
> On Tue, May 11, 2010 at 9:01 AM, Daniel Wheeler <dan...@gm...>
> wrote:
> Was there a previous pysparse installation?
>
>
> On Tue, May 11, 2010 at 2:01 AM, Dominique Orban <dom...@gm...>
> wrote:
>>
>> On Mon, May 10, 2010 at 5:59 PM, Eddie Schwalbach
>> <e-s...@no...> wrote:
>> > I'm trying to install pysparse on a Linux cluster with ATLAS 3.8.3
>> > (which
>> > was built with full LAPACK support).  ATLAS is located at
>> > /software/ATLAS/3.8.3
>> > and I'm using Python2.6.4 which is located in the nonstandard location
>> > /home/ejs698/software (which is at the beginning of my PATH and
>> > PYTHONPATH).
>> >
>> > I've modified the setup.py file to include the appropriate (I think)
>> > library
>> > directories:
>> >
>> > library_dirs_list=
>> >
>> > ['/software/ATLAS/3.8.3/lib/','/home/ejs698/software/lib/','/usr/bin/lib/']
>> > libraries_list = ['f77blas', 'cblas', 'atlas', 'lapack', 'gfortran']
>> >
>> > I also have the environment variables:
>> > LD_LIBRARY_PATH
>> >
>> > /opt/moab/lib:/opt/torque/lib::/software/ATLAS/3.8.3/lib:/home/ejs698/software/lib
>> > ATLAS  /software/ATLAS/3.8.3  (this was necessary for getting numpy
>> > built
>> > against the correct ATLAS libraries)
>> >
>> > I don't see errors when I run python setup.py install, or when I try
>> >      from pysparse import *
>> > However, if I attempt to run any of the tests in the Examples directory,
>> > I
>> > get:
>> >
>> > [ejs698@quser04 Examples]$ python jdsym_test.py
>> > Traceback (most recent call last):
>> >   File "jdsym_test.py", line 1, in <module>
>> >     from pysparse import spmatrix, jdsym, itsolvers
>> > ImportError: /software/ATLAS/lib/liblapack.so: cannot open shared object
>> > file: No such file or directory
>> >
>> > This seems to indicate that pysparse is looking in the wrong place for
>> > the
>> > lapack libraries (the correct location is
>> > /software/ATLAS/3.8.3/lib/liblapack.so).  Have I missed some environment
>> > variable or setting in the setup.py to properly specify these library
>> > locations?
>> >
>> > Any suggestions are appreciated,  and thanks in advance,
>>
>> Hello Eddie,
>>
>> There are no environment variables to set in Pysparse, so no, you
>> haven't missed any.
>>
>> Could you please post the complete output of 'python setup.py build'
>> as well as the result of 'ls -l
>> /software/ATLAS/3.8.3/lib/liblapack.so'? I don't doubt your word that
>> the library is there but am just trying to get the obvious things out
>> of the way first. Also I presume you can run the ATLAS and LAPACK
>> tests with your libraries?
>>
>> Thanks,
>>
>> --
>> Dominique
>
>



-- 
Daniel Wheeler