#65 dmarrays do not properly convert to masked arrays.

Dan Welling

dmarrays do not convert to np.ma.masked_arrays properly. Specifically, the self.min/max methods fail once converted. This is especially problematic when trying to perform matplotlib contour plots using dmarrays, as the Z-variable handed to the contour function is converted to a masked array to weed out invalid values.

For example

import numpy as np
from spacepy.datamodel import dmarray
x = dmarray(np.arange(10))
x.min() # Works fine.

x2 = np.ma.masked_array(x)
x2.min() #Fails.

x3 = np.ma.masked_array( np.array(x) )
x3.min() # works.


  • Steve Morley

    Steve Morley - 2013-07-23

    This is an unforeseen side-effect of making the behaviour of dmarray more like numpy.array. The original problem was that ufuncs like max, min, etc. behave inconsistently between numpy arrays and subclasses of arrays (fix committed in [d4d970]).

    The original problem can be illustrated using the RealisticInfoArray subclass from http://docs.scipy.org/doc/numpy/user/basics.subclassing.html

    RealisticInfoArray(3) #output is a scalar (zero-dim) array
    3 #output is a python scalar

    The fix implemented there appears to be responsible for the breakage of numpy's conversion to masked_[subclass], where the masked array actually requires the output of the ufunc to remain as an array-type. I've not found where I can detect the fact that it's using a mask and add special case code for that (yet).

    I'll keep looking for how to fix this, while retaining the fix to min/max/mean/etc.
    The original problem should probably be elevated to numpy though, as it all arises through the fact that subclasses behave differently.



    Commit: [d4d970]

  • Jonathan Niehof

    Jonathan Niehof - 2014-03-17

    As I mentioned on the dev list, MaskedArray itself is a subclass of ndarray, so we can maybe try to steal some ideas from there...


Log in to post a comment.