Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: John Hunter - 2008-09-26 18:28 ```On Fri, Sep 26, 2008 at 12:39 PM, Tony S Yu wrote: > + if all(nonzero == False): > + raise ValueError('spy cannot plot sparse zeros > matrix') Is raising an exception the right choice here -- why can't we plot an all zeros image? JDH ```

[matplotlib-devel] spy: ignore zero values in sparse matrix Tony S Yu <tonyyu@MI...>
 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: John Hunter - 2008-09-26 18:28 ```On Fri, Sep 26, 2008 at 12:39 PM, Tony S Yu wrote: > + if all(nonzero == False): > + raise ValueError('spy cannot plot sparse zeros > matrix') Is raising an exception the right choice here -- why can't we plot an all zeros image? JDH ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Tony S Yu - 2008-09-26 19:37 ```On Sep 26, 2008, at 2:28 PM, John Hunter wrote: > On Fri, Sep 26, 2008 at 12:39 PM, Tony S Yu wrote: > >> + if all(nonzero == False): >> + raise ValueError('spy cannot plot sparse zeros >> matrix') > > Is raising an exception the right choice here -- why can't we plot an > all zeros image? > > JDH I guess you could plot sparse all-zero matrices with image mode. My only hesitation is that sparse arrays tend to be very large and (I imagine) this would lead to very slow performance. I assumed this was the reason image mode wasn't adapted to use sparse arrays. Actually, now that I think about it: you could plot a trivially small image and just adjust the coordinates so that they correspond to the original matrix shape. Is this what you were thinking? I should note that a dense zero array also fails to plot with spy *if marker mode is used*. -T ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: John Hunter - 2008-09-26 19:38 ```On Fri, Sep 26, 2008 at 2:36 PM, Tony S Yu wrote: > I guess you could plot sparse all-zero matrices with image mode. My only > hesitation is that sparse arrays tend to be very large and (I imagine) this > would lead to very slow performance. I assumed this was the reason image > mode wasn't adapted to use sparse arrays. > > Actually, now that I think about it: you could plot a trivially small image > and just adjust the coordinates so that they correspond to the original > matrix shape. Is this what you were thinking? This is something I considered, but I was thinking less about the implementation and more about the functionality. I don't want to raise an exception unless the input doesn't make sense. I would rather the user start at a boring image and figure out why it is blank that deal with an exception. > I should note that a dense zero array also fails to plot with spy *if marker > mode is used*. Can you fix this along with spy2? JDH ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Tony S Yu - 2008-09-26 20:02 ```On Sep 26, 2008, at 3:38 PM, John Hunter wrote: > On Fri, Sep 26, 2008 at 2:36 PM, Tony S Yu wrote: >> >> Actually, now that I think about it: you could plot a trivially >> small image >> and just adjust the coordinates so that they correspond to the >> original >> matrix shape. Is this what you were thinking? > > This is something I considered, but I was thinking less about the > implementation and more about the functionality. I don't want to > raise an exception unless the input doesn't make sense. I would > rather the user start at a boring image and figure out why it is blank > that deal with an exception. Yeah, I agree this is much friendlier. >> I should note that a dense zero array also fails to plot with spy >> *if marker >> mode is used*. > > Can you fix this along with spy2? I assume you mean spy, not spy2 (I just searched through the matplotlib files and saw that spy2 hasn't existed since 2006). I'll work on a patch to return a blank plot using the method described above (unless someone chimes in with a better suggestion). -Tony ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Eric Firing - 2008-09-26 21:02 ```Tony S Yu wrote: > On Sep 26, 2008, at 2:28 PM, John Hunter wrote: > >> On Fri, Sep 26, 2008 at 12:39 PM, Tony S Yu wrote: >> >>> + if all(nonzero == False): >>> + raise ValueError('spy cannot plot sparse zeros >>> matrix') >> Is raising an exception the right choice here -- why can't we plot an >> all zeros image? >> >> JDH > > I guess you could plot sparse all-zero matrices with image mode. My > only hesitation is that sparse arrays tend to be very large and (I > imagine) this would lead to very slow performance. I assumed this was > the reason image mode wasn't adapted to use sparse arrays. Also, if an image cannot be resolved by the output device, info is lost--one might not see anything at a location where there actually is a value--whereas with markers, a marker will always show up, and the only problem is that one can't necessarily distinguish a single point from a cluster. The real problem with all-zero values is that plot can't handle "plot([],[])". One can work around this by putting in bogus values to plot a single point, saving the line, and then setting the line data to empty; or, better, by not using the high-level plot command, but by generating the Line2D object and adding it to the axes. The Line2D initializer is happy with empty x and y sequences. I think if you use this approach it will kill two bugs (failure on all-zeros with sparse and full arrays) with one very simple stone. Eric > > Actually, now that I think about it: you could plot a trivially small > image and just adjust the coordinates so that they correspond to the > original matrix shape. Is this what you were thinking? > > I should note that a dense zero array also fails to plot with spy *if > marker mode is used*. > > -T > > ------------------------------------------------------------------------- > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge > Build the coolest Linux based applications with Moblin SDK & win great prizes > Grand prize is a trip for two to an Open Source event anywhere in the world > http://moblin-contest.org/redirect.php?banner_id=100&url=/ > _______________________________________________ > Matplotlib-devel mailing list > Matplotlib-devel@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-devel ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Tony S Yu - 2008-09-27 16:02 ```On Sep 26, 2008, at 5:01 PM, Eric Firing wrote: > Also, if an image cannot be resolved by the output device, info is > lost--one might not see anything at a location where there actually > is a value--whereas with markers, a marker will always show up, and > the only problem is that one can't necessarily distinguish a single > point from a cluster. > > The real problem with all-zero values is that plot can't handle > "plot([],[])". One can work around this by putting in bogus values > to plot a single point, saving the line, and then setting the line > data to empty; or, better, by not using the high-level plot command, > but by generating the Line2D object and adding it to the axes. The > Line2D initializer is happy with empty x and y sequences. I think if > you use this approach it will kill two bugs (failure on all-zeros > with sparse and full arrays) with one very simple stone. > > Eric Thanks for the tip Eric. Below is a patch for spy that implements Eric's suggestion. This patch seems to work for a couple simple tests on my end: sparse and dense arrays with non-zero and all-zero values. A couple of notes: * the call to `add_artist` isn't needed to show the correct plot, but it may be helpful for debugging. * the docstring for `spy` suggests that a Line2D instance is returned, but `spy` currently returns a list with a Line2D instance. I set all- zero arrays to return a list also, for consistency. -Tony Index: matplotlib/lib/matplotlib/axes.py =================================================================== --- matplotlib/lib/matplotlib/axes.py (revision 6123) +++ matplotlib/lib/matplotlib/axes.py (working copy) @@ -6723,9 +6723,9 @@ else: if hasattr(Z, 'tocoo'): c = Z.tocoo() - y = c.row - x = c.col - z = c.data + nonzero = c.data != 0. + y = c.row[nonzero] + x = c.col[nonzero] else: Z = np.asarray(Z) if precision is None: mask = Z!=0. @@ -6733,8 +6733,12 @@ y,x,z = mlab.get_xyz_where(mask, mask) if marker is None: marker = 's' if markersize is None: markersize = 10 - lines = self.plot(x, y, linestyle='None', - marker=marker, markersize=markersize, **kwargs) + if len(x) == 0: + lines = [mlines.Line2D([], [])] + self.add_artist(lines[0]) + else: + lines = self.plot(x, y, linestyle='None', + marker=marker, markersize=markersize, **kwargs) nr, nc = Z.shape self.set_xlim(xmin=-0.5, xmax=nc-0.5) self.set_ylim(ymin=nr-0.5, ymax=-0.5) Index: matplotlib/examples/pylab_examples/masked_demo.py =================================================================== --- matplotlib/examples/pylab_examples/masked_demo.py (revision 6123) +++ matplotlib/examples/pylab_examples/masked_demo.py (working copy) @@ -1,4 +1,4 @@ -#!/bin/env python +#!/usr/bin/env python ''' Plot lines with points masked out. Index: matplotlib/examples/misc/rec_groupby_demo.py =================================================================== --- matplotlib/examples/misc/rec_groupby_demo.py (revision 6123) +++ matplotlib/examples/misc/rec_groupby_demo.py (working copy) @@ -2,7 +2,7 @@ import matplotlib.mlab as mlab -r = mlab.csv2rec('data/aapl.csv') +r = mlab.csv2rec('../data/aapl.csv') r.sort() def daily_return(prices): ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Eric Firing - 2008-09-27 22:22 ```Tony S Yu wrote: > > On Sep 26, 2008, at 5:01 PM, Eric Firing wrote: >> Also, if an image cannot be resolved by the output device, info is >> lost--one might not see anything at a location where there actually is >> a value--whereas with markers, a marker will always show up, and the >> only problem is that one can't necessarily distinguish a single point >> from a cluster. >> >> The real problem with all-zero values is that plot can't handle >> "plot([],[])". One can work around this by putting in bogus values to >> plot a single point, saving the line, and then setting the line data >> to empty; or, better, by not using the high-level plot command, but by >> generating the Line2D object and adding it to the axes. The Line2D >> initializer is happy with empty x and y sequences. I think if you use >> this approach it will kill two bugs (failure on all-zeros with sparse >> and full arrays) with one very simple stone. >> >> Eric > > > Thanks for the tip Eric. Below is a patch for spy that implements Eric's > suggestion. This patch seems to work for a couple simple tests on my > end: sparse and dense arrays with non-zero and all-zero values. Tony, Thanks. I will take care of this shortly, along with fixing the failure of plot([],[]) and maybe a few other things. Eric > > A couple of notes: > > * the call to `add_artist` isn't needed to show the correct plot, but it > may be helpful for debugging. > > * the docstring for `spy` suggests that a Line2D instance is returned, > but `spy` currently returns a list with a Line2D instance. I set > all-zero arrays to return a list also, for consistency. > > > -Tony > > > > Index: matplotlib/lib/matplotlib/axes.py > =================================================================== > --- matplotlib/lib/matplotlib/axes.py (revision 6123) > +++ matplotlib/lib/matplotlib/axes.py (working copy) > @@ -6723,9 +6723,9 @@ > else: > if hasattr(Z, 'tocoo'): > c = Z.tocoo() > - y = c.row > - x = c.col > - z = c.data > + nonzero = c.data != 0. > + y = c.row[nonzero] > + x = c.col[nonzero] > else: > Z = np.asarray(Z) > if precision is None: mask = Z!=0. > @@ -6733,8 +6733,12 @@ > y,x,z = mlab.get_xyz_where(mask, mask) > if marker is None: marker = 's' > if markersize is None: markersize = 10 > - lines = self.plot(x, y, linestyle='None', > - marker=marker, markersize=markersize, **kwargs) > + if len(x) == 0: > + lines = [mlines.Line2D([], [])] > + self.add_artist(lines[0]) > + else: > + lines = self.plot(x, y, linestyle='None', > + marker=marker, markersize=markersize, > **kwargs) > nr, nc = Z.shape > self.set_xlim(xmin=-0.5, xmax=nc-0.5) > self.set_ylim(ymin=nr-0.5, ymax=-0.5) > Index: matplotlib/examples/pylab_examples/masked_demo.py > =================================================================== > --- matplotlib/examples/pylab_examples/masked_demo.py (revision 6123) > +++ matplotlib/examples/pylab_examples/masked_demo.py (working copy) > @@ -1,4 +1,4 @@ > -#!/bin/env python > +#!/usr/bin/env python > ''' > Plot lines with points masked out. > > Index: matplotlib/examples/misc/rec_groupby_demo.py > =================================================================== > --- matplotlib/examples/misc/rec_groupby_demo.py (revision 6123) > +++ matplotlib/examples/misc/rec_groupby_demo.py (working copy) > @@ -2,7 +2,7 @@ > import matplotlib.mlab as mlab > > > -r = mlab.csv2rec('data/aapl.csv') > +r = mlab.csv2rec('../data/aapl.csv') > r.sort() > > def daily_return(prices): > ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Eric Firing - 2008-09-28 00:56 ```Tony S Yu wrote: > > On Sep 26, 2008, at 5:01 PM, Eric Firing wrote: >> Also, if an image cannot be resolved by the output device, info is >> lost--one might not see anything at a location where there actually is >> a value--whereas with markers, a marker will always show up, and the >> only problem is that one can't necessarily distinguish a single point >> from a cluster. >> >> The real problem with all-zero values is that plot can't handle >> "plot([],[])". One can work around this by putting in bogus values to >> plot a single point, saving the line, and then setting the line data >> to empty; or, better, by not using the high-level plot command, but by >> generating the Line2D object and adding it to the axes. The Line2D >> initializer is happy with empty x and y sequences. I think if you use >> this approach it will kill two bugs (failure on all-zeros with sparse >> and full arrays) with one very simple stone. >> >> Eric > > > Thanks for the tip Eric. Below is a patch for spy that implements Eric's > suggestion. This patch seems to work for a couple simple tests on my > end: sparse and dense arrays with non-zero and all-zero values. > > A couple of notes: > > * the call to `add_artist` isn't needed to show the correct plot, but it > may be helpful for debugging. > > * the docstring for `spy` suggests that a Line2D instance is returned, > but `spy` currently returns a list with a Line2D instance. I set > all-zero arrays to return a list also, for consistency. Tony, Changes to spy and a few other things are in svn 6127. Regarding your last point, the docstring made more sense than the original implementation, so I changed the implementation to correspond to it. I hope this does not cause more trouble than it is worth; if it looks like it will, then I can easily change the behavior back and modify the docstring. It seems silly to always return a list with a single item, though, and I doubt that many people are making heavy use of the return value of the spy method anyway. Regarding your original idea, that sparse arrays should be handled like ordinary arrays, with only nonzero values plotted: I think this is going too far, and not far enough, so I did the following: 1) If "precision" is None or a non-zero value, the behavior for sparse and ordinary arrays is identical. Previously, the precision kwarg was silently ignored for sparse arrays. Now it is used. 2) If "precision" is 0, then one gets the old behavior: all locations with data are shown, regardless of value. It seems to me that one really wants to have this behavior available, to see how much of a sparse array is filled in. I am not entirely comfortable with the way the "precision" kwarg is being used to control this, but it seemed preferable to adding another kwarg. Alternatives could include swapping the roles of 0 and None, or letting precision take a string value to specify the old behavior. Actually, I think the most logical thing would be to let the default None give the old behavior, and require precision=0 to get the new behavior. What do you think? Is it OK if I make this change? It is more consistent with the old behavior. I also changed the behavior so that if a sparse array is input, with no marker specifications, it simply makes a default marker plot instead of raising an exception. Eric > > > -Tony > > > > Index: matplotlib/lib/matplotlib/axes.py > =================================================================== > --- matplotlib/lib/matplotlib/axes.py (revision 6123) > +++ matplotlib/lib/matplotlib/axes.py (working copy) > @@ -6723,9 +6723,9 @@ > else: > if hasattr(Z, 'tocoo'): > c = Z.tocoo() > - y = c.row > - x = c.col > - z = c.data > + nonzero = c.data != 0. > + y = c.row[nonzero] > + x = c.col[nonzero] > else: > Z = np.asarray(Z) > if precision is None: mask = Z!=0. > @@ -6733,8 +6733,12 @@ > y,x,z = mlab.get_xyz_where(mask, mask) > if marker is None: marker = 's' > if markersize is None: markersize = 10 > - lines = self.plot(x, y, linestyle='None', > - marker=marker, markersize=markersize, **kwargs) > + if len(x) == 0: > + lines = [mlines.Line2D([], [])] > + self.add_artist(lines[0]) > + else: > + lines = self.plot(x, y, linestyle='None', > + marker=marker, markersize=markersize, > **kwargs) > nr, nc = Z.shape > self.set_xlim(xmin=-0.5, xmax=nc-0.5) > self.set_ylim(ymin=nr-0.5, ymax=-0.5) > Index: matplotlib/examples/pylab_examples/masked_demo.py > =================================================================== > --- matplotlib/examples/pylab_examples/masked_demo.py (revision 6123) > +++ matplotlib/examples/pylab_examples/masked_demo.py (working copy) > @@ -1,4 +1,4 @@ > -#!/bin/env python > +#!/usr/bin/env python > ''' > Plot lines with points masked out. > > Index: matplotlib/examples/misc/rec_groupby_demo.py > =================================================================== > --- matplotlib/examples/misc/rec_groupby_demo.py (revision 6123) > +++ matplotlib/examples/misc/rec_groupby_demo.py (working copy) > @@ -2,7 +2,7 @@ > import matplotlib.mlab as mlab > > > -r = mlab.csv2rec('data/aapl.csv') > +r = mlab.csv2rec('../data/aapl.csv') > r.sort() > > def daily_return(prices): > ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Tony S Yu - 2008-10-02 01:27 ```Hi Eric, Sorry for the late reply. On Sep 27, 2008, at 8:56 PM, Eric Firing wrote: > Actually, I think the most logical thing would be to let the default > None give the old behavior, and require precision=0 to get the new > behavior. What do you think? Is it OK if I make this change? It > is more consistent with the old behavior. I'm ambivalent about this change. On one hand, I think it makes a lot more sense to have None give the old behavior and precision=0 to ignore zero values in the sparse array (then precision would be consistent for finite values and for zero). On the other hand, I think ignoring zero values should be the default behavior for sparse arrays (although, I definitely agree there should be the option to plot all assigned values). Would it be possible to make the change you suggest and also change the default precision value to 0? (see diff below) This change would also allow you to remove a lot of the special handling for precision=None, since precision=0 gives the same result (I didn't go this far in the diff below). > I also changed the behavior so that if a sparse array is input, with > no marker specifications, it simply makes a default marker plot > instead of raising an exception. Excellent idea. That behavior is much more user-friendly. Thanks, -Tony PS. Any comments on the small changes to the examples. Both changes are necessary for those examples to work on my computer (the shebang line throws an error when I run the code from my text editor). Index: matplotlib/lib/matplotlib/axes.py =================================================================== --- matplotlib/lib/matplotlib/axes.py (revision 6141) +++ matplotlib/lib/matplotlib/axes.py (working copy) @@ -6648,7 +6648,7 @@ return Pxx, freqs, bins, im - def spy(self, Z, precision=None, marker=None, markersize=None, + def spy(self, Z, precision=0., marker=None, markersize=None, aspect='equal', **kwargs): """ call signature:: @@ -6731,14 +6731,11 @@ else: if hasattr(Z, 'tocoo'): c = Z.tocoo() - if precision == 0: + if precision is None: y = c.row x = c.col else: - if precision is None: - nonzero = c.data != 0. - else: - nonzero = np.absolute(c.data) > precision + nonzero = np.absolute(c.data) > precision y = c.row[nonzero] x = c.col[nonzero] else: Index: matplotlib/examples/pylab_examples/masked_demo.py =================================================================== --- matplotlib/examples/pylab_examples/masked_demo.py (revision 6141) +++ matplotlib/examples/pylab_examples/masked_demo.py (working copy) @@ -1,4 +1,4 @@ -#!/bin/env python +#!/usr/bin/env python ''' Plot lines with points masked out. Index: matplotlib/examples/misc/rec_groupby_demo.py =================================================================== --- matplotlib/examples/misc/rec_groupby_demo.py (revision 6141) +++ matplotlib/examples/misc/rec_groupby_demo.py (working copy) @@ -2,7 +2,7 @@ import matplotlib.mlab as mlab -r = mlab.csv2rec('data/aapl.csv') +r = mlab.csv2rec('../data/aapl.csv') r.sort() def daily_return(prices): ```

 Re: [matplotlib-devel] spy: ignore zero values in sparse matrix From: Eric Firing - 2008-10-04 07:21 ```Tony S Yu wrote: > Hi Eric, > > Sorry for the late reply. > > On Sep 27, 2008, at 8:56 PM, Eric Firing wrote: > >> Actually, I think the most logical thing would be to let the default >> None give the old behavior, and require precision=0 to get the new >> behavior. What do you think? Is it OK if I make this change? It >> is more consistent with the old behavior. > > I'm ambivalent about this change. On one hand, I think it makes a lot > more sense to have None give the old behavior and precision=0 to > ignore zero values in the sparse array (then precision would be > consistent for finite values and for zero). > > On the other hand, I think ignoring zero values should be the default > behavior for sparse arrays (although, I definitely agree there should > be the option to plot all assigned values). > > Would it be possible to make the change you suggest and also change > the default precision value to 0? (see diff below) This change would > also allow you to remove a lot of the special handling for > precision=None, since precision=0 gives the same result (I didn't go > this far in the diff below). Good point. I made that change, but then made precision='present' be the value for sparse arrays to show all filled cells. precision=None is deprecated, but converted to 0. Eric ```