numpy-discussion

[Numpy-discussion] question/request with Numeric compress and putmask

[Vineet J. <vi...@es...>]

I'm using the Numeric arrays for financial data elements. I'm interfacing
with an external c library which does not support invalid elements. To get
around this I maintain a separate mask array in my python class which
denotes which elements are valid. I then use the compress function with the
mask array to get an array with valid elements which I pass to the c
function.

I was then going to use the putmask to assign the returned values to their
corresponding place in the original array. This obviously (after reading the
documentation) is not going to work.

Example:

financial_data 	= [10, 11, 22, 33, INVALID, INVALID, 44, 55]
full_return_value = INVALID+zeros(lenfinancial_data)

my_mask 		= [1, 1, 1, 1, 0, 0, 1, 1]
compressed_data	= compress(financiali_data, my_mask)
			  [10, 11, 22, 33, 44, 55]

return_value 	= some_c_function(compressed_data)

Now the problem I have is copying the values in return_value to their
original place in the financial_data. I was happily going to use putmask
till about an hour back when I found out that this is not going to work.

What I would like to do is:

putmask(full_return_value, my_mask, return_value)

where return_value is treated like a list so that every 1 that is found in
my_mask the next element in return_value is used. Is their anything that
matches this?

I've looked at Masked Array. I don't like the fact that it return copies of
data and not references. I'm also unsure on how to pass it to c functions
(since it is a pure python package).

Any suggestions? and thanks for any help.

Re: [Numpy-discussion] question/request with Numeric compress and putmask

[Travis E. O. <oli...@ee...>]

Vineet Jain wrote:


> I'm using the Numeric arrays for financial data elements. I'm interfacing
> with an external c library which does not support invalid elements. To get
> around this I maintain a separate mask array in my python class which
> denotes which elements are valid. I then use the compress function with the
> mask array to get an array with valid elements which I pass to the c
> function.
> 
> What I would like to do is:
> 
> putmask(full_return_value, my_mask, return_value)
> 
> where return_value is treated like a list so that every 1 that is found in
> my_mask the next element in return_value is used. Is their anything that
> matches this?
> 

There are functions in SciPy to handle exactly this situation.


 >>> from scipy import *
 >>> info(insert)
  insert(arr, mask, vals)

Similar to putmask arr[mask] = vals but 1d array vals has the
same number of elements as the non-zero values of mask. Inverse of extract.
 >>> info(extract)
  extract(condition, arr)

Elements of ravel(condition) where ravel(condition) is true (1-d)

Equivalent of compress(ravel(condition), ravel(arr))



Thus, for your problem I would do:

financial_data 	= [10, 11, 22, 33, INVALID, INVALID, 44, 55]
my_mask 		= [1, 1, 1, 1, 0, 0, 1, 1]
compressed_data = extract(my_mask, financial_data)
return_value 	= some_c_function(compressed_data)
insert(financial_data, my_mask, return_value)


These functions are in scipy_base and so you only need to install 
scipy_base to get them.

Best regards,

-Travis Oliphant

RE: [Numpy-discussion] question/request with Numeric compress and putmask

[Vineet J. <vi...@es...>]

Thanks for the response. The format of the extract is actually extract(arry,
condition) and not extract(condition, arry). It worked after making that
change.

How is the compress in Numeric different than the extract?
Are the scipy.base classes implemented in c?
Will scipy.base support numarray module since I'll be upgrading to it in a
few weeks?

Thanks,

Vineet





-----Original Message-----
From: num...@li...
[mailto:num...@li...]On Behalf Of Travis
E. Oliphant
Sent: Sunday, February 01, 2004 11:32 PM
To: vi...@al...; num...@li...
Subject: Re: [Numpy-discussion] question/request with Numeric compress
and putmask


Vineet Jain wrote:


> I'm using the Numeric arrays for financial data elements. I'm interfacing
> with an external c library which does not support invalid elements. To get
> around this I maintain a separate mask array in my python class which
> denotes which elements are valid. I then use the compress function with
the
> mask array to get an array with valid elements which I pass to the c
> function.
>
> What I would like to do is:
>
> putmask(full_return_value, my_mask, return_value)
>
> where return_value is treated like a list so that every 1 that is found in
> my_mask the next element in return_value is used. Is their anything that
> matches this?
>

There are functions in SciPy to handle exactly this situation.


 >>> from scipy import *
 >>> info(insert)
  insert(arr, mask, vals)

Similar to putmask arr[mask] = vals but 1d array vals has the
same number of elements as the non-zero values of mask. Inverse of extract.
 >>> info(extract)
  extract(condition, arr)

Elements of ravel(condition) where ravel(condition) is true (1-d)

Equivalent of compress(ravel(condition), ravel(arr))



Thus, for your problem I would do:

financial_data 	= [10, 11, 22, 33, INVALID, INVALID, 44, 55]
my_mask 		= [1, 1, 1, 1, 0, 0, 1, 1]
compressed_data = extract(my_mask, financial_data)
return_value 	= some_c_function(compressed_data)
insert(financial_data, my_mask, return_value)


These functions are in scipy_base and so you only need to install
scipy_base to get them.

Best regards,

-Travis Oliphant








-------------------------------------------------------
The SF.Net email is sponsored by EclipseCon 2004
Premiere Conference on Open Tools Development and Integration
See the breadth of Eclipse activity. February 3-5 in Anaheim, CA.
http://www.eclipsecon.org/osdn
_______________________________________________
Numpy-discussion mailing list
Num...@li...
https://lists.sourceforge.net/lists/listinfo/numpy-discussion