Re: [Pysparse-users] pysparse and multiprocessing

[Dominique O. <dom...@gm...>]

On Thu, Apr 8, 2010 at 3:15 PM, Eduardo Lenz Cardoso
<le...@jo...> wrote:
> Hi,
>
> I am trying to use the pool method of
> multiprocessing. Each process has
> as input a number and a class. This class has
> a sparse matrix (pysparse ll_mat) inside it.
>
> The problem is that the class must be pickled
> and depickled when moving around processes and
> I am having the following error:
>
>
> Exception in thread Thread-1:
> Traceback (most recent call last):
>  File "/usr/lib/python2.6/threading.py", line 532, in __bootstrap_inner
>    self.run()
>  File "/usr/lib/python2.6/threading.py", line 484, in run
>    self.__target(*self.__args, **self.__kwargs)
>  File "/usr/lib/python2.6/multiprocessing/pool.py", line 225, in
> _handle_tasks
>    put(task)
> UnpickleableError: Cannot pickle <type 'll_mat'> objects
>
> I am searching all around and could not find any tip to
> solve this problem. Any clue ?

Hi Eduardo,

Unfortunately, Python objects implemented in C can't be pickled or
marshalled. A PysparseMatrix class, being a pure Python class, could
be pickled if you removed its 'matrix' member, but that would render
it useless. So: can't pickle in PySparse...

As an alternative I can suggest writing the matrix to file in mtx
format, and then reading it in again. Hopefully, that can help you. If
A is an ll_mat object, use A.export_mtx().

Good luck,

-- 
Dominique