From: Markus Feldmann <feldmann_markus@gm...>  20090703 07:29:13

Michael Droettboom schrieb: > This should now be fixed on the maintenance branch and trunk. A Numpy > array allocation was not being NULLchecked in _path.cpp:affine_transform. > > I know a MemoryError doesn't help the user much more than a segfault, > but it always makes me feel better to get a real Python exception rather > than exploding ;) > > Mike Hi All, thanks for the bugfix. I also got the segfault, but forgot to wrote this. Is there a possibility to limit the maximum points shown ? And sorry that i can not post more of my code. My program is to big. Maybe this is a feature request. :) It would be nice to set up a maximum limit and compute a average. I doesnt know which way is the fastest. I only execute the plot command three times but every plot command plots round about 15 Mill. points. regards Markus 
From: Markus Feldmann <feldmann_markus@gm...>  20090630 17:22:43

Hi All, my program lets slow down my cpu. This only appears if i plot to much points. I am not sure how many point i need to get this, normally i plot 3*14e6 + 8e3, that is round about 50million points. My system is a dual core 2GHz cpu with 2Gbyte Ram. Here is my method to plot, def drawtransientall(self,min): self.subplot = self.figure.add_subplot(111) self.subplot.grid(True) list_t1,list_peaks,t2,list_samples = self.computetransientall(min,min+self.maxitems) offset = 0 color = ['green','red','blue','magenta','cyan'] markerPeaks = ['v','<','1','3','s'] markerSamples = ['^','>','2','4','p'] self.plots=[[],[]] for i in range(len(self.showBands)): self.plots[0] += self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i], linestyle='None') self.plots[1] += self.subplot.plot(t2,list_samples[i]+offset,color=color[i], marker=markerSamples[i],linestyle='None') offset +=1 self.subplot.set_xlim(t2[0]np.abs(t2[1]t2[0])/100,t2[1]+np.abs(t2[1]t2[0])/100) ymax = np.amax(list_samples) ymin = np.amin(list_samples) self.subplot.set_ylim([yminnp.abs(ymin)*0.1, ymax*1.2 + 2]) self.subplot.set_ylabel("abs(Sample(t)) und abs(Peak(t)+Offset)>",fontsize = 12) self.subplot.set_xlabel("Zeit in Sek. >",fontsize = 12) Any ideas how to avoid the slow down of my cpu ? regards Markus 
From: JaeJoon Lee <lee.joon@gm...>  20090701 17:34:20

A snippet of code does not help much. Please try to post a small concise standalone example that we can run and test. A general advise is to try to reduce the number of plot call, i.e., plot as may points as possible with a single plot call. However, 50million points seems to be awful a lot. 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels. My guess is that it makes little sense to plot 50 million points here. Anyhow, plotting 50million points with a single plot call dies with some segfault error in my machine. So, I feel that matplotlib may not be suitable for your task. But, John or others may have some insight how to deal with. Regards, JJ On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<feldmann_markus@...> wrote: > Hi All, > > my program lets slow down my cpu. This only appears if i plot to much > points. I am not sure how many point i need to get this, normally i plot > 3*14e6 + 8e3, that is round about 50million points. My system is a > dual core 2GHz cpu with 2Gbyte Ram. > > Here is my method to plot, > def drawtransientall(self,min): > self.subplot = self.figure.add_subplot(111) > self.subplot.grid(True) > list_t1,list_peaks,t2,list_samples = > self.computetransientall(min,min+self.maxitems) > offset = 0 > color = ['green','red','blue','magenta','cyan'] > markerPeaks = ['v','<','1','3','s'] > markerSamples = ['^','>','2','4','p'] > self.plots=[[],[]] > for i in range(len(self.showBands)): > self.plots[0] += > self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i], > linestyle='None') > self.plots[1] += > self.subplot.plot(t2,list_samples[i]+offset,color=color[i], > > marker=markerSamples[i],linestyle='None') > offset +=1 > > self.subplot.set_xlim(t2[0]np.abs(t2[1]t2[0])/100,t2[1]+np.abs(t2[1]t2[0])/100) > ymax = np.amax(list_samples) > ymin = np.amin(list_samples) > self.subplot.set_ylim([yminnp.abs(ymin)*0.1, ymax*1.2 + 2]) > self.subplot.set_ylabel("abs(Sample(t)) und > abs(Peak(t)+Offset)>",fontsize = 12) > self.subplot.set_xlabel("Zeit in Sek. >",fontsize = 12) > > Any ideas how to avoid the slow down of my cpu ? > > regards Markus > > >  > _______________________________________________ > Matplotlibusers mailing list > Matplotlibusers@... > https://lists.sourceforge.net/lists/listinfo/matplotlibusers > 
From: Michael Droettboom <mdroe@st...>  20090701 18:34:37
Attachments:
Message as HTML

I agree with JaeJoon here  try to reduce the number of points before passing it to matplotlib. However, I'm a little concerned about the segfault  I'd rather matplotlib give a MemoryError exception if that's in fact what is happening. JaeJoon  can you share your test that causes the segfault? The snippet below completely hogs my machine for a few minutes, but then, correctly, aborts with a MemoryError. This is on FC11 i586, Python 2.6, Numpy 1.3. ==== from matplotlib.pyplot import * import numpy as np points = np.random.random((50000000, 2)) plot(points) show() ==== Mike On 07/01/2009 01:34 PM, JaeJoon Lee wrote: > A snippet of code does not help much. > Please try to post a small concise standalone example that we can run and test. > > A general advise is to try to reduce the number of plot call, i.e., > plot as may points as possible with a single plot call. > > However, 50million points seems to be awful a lot. > 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels. > My guess is that it makes little sense to plot 50 million points here. > > Anyhow, plotting 50million points with a single plot call dies with > some segfault error in my machine. So, I feel that matplotlib may not > be suitable for your task. But, John or others may have some insight > how to deal with. > > Regards, > > JJ > > > > On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<feldmann_markus@...> wrote: > >> Hi All, >> >> my program lets slow down my cpu. This only appears if i plot to much >> points. I am not sure how many point i need to get this, normally i plot >> 3*14e6 + 8e3, that is round about 50million points. My system is a >> dual core 2GHz cpu with 2Gbyte Ram. >> >> Here is my method to plot, >> def drawtransientall(self,min): >> self.subplot = self.figure.add_subplot(111) >> self.subplot.grid(True) >> list_t1,list_peaks,t2,list_samples = >> self.computetransientall(min,min+self.maxitems) >> offset = 0 >> color = ['green','red','blue','magenta','cyan'] >> markerPeaks = ['v','<','1','3','s'] >> markerSamples = ['^','>','2','4','p'] >> self.plots=[[],[]] >> for i in range(len(self.showBands)): >> self.plots[0] += >> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i], >> linestyle='None') >> self.plots[1] += >> self.subplot.plot(t2,list_samples[i]+offset,color=color[i], >> >> marker=markerSamples[i],linestyle='None') >> offset +=1 >> >> self.subplot.set_xlim(t2[0]np.abs(t2[1]t2[0])/100,t2[1]+np.abs(t2[1]t2[0])/100) >> ymax = np.amax(list_samples) >> ymin = np.amin(list_samples) >> self.subplot.set_ylim([yminnp.abs(ymin)*0.1, ymax*1.2 + 2]) >> self.subplot.set_ylabel("abs(Sample(t)) und >> abs(Peak(t)+Offset)>",fontsize = 12) >> self.subplot.set_xlabel("Zeit in Sek. >",fontsize = 12) >> >> Any ideas how to avoid the slow down of my cpu ? >> >> regards Markus >> >> >>  >> _______________________________________________ >> Matplotlibusers mailing list >> Matplotlibusers@... >> https://lists.sourceforge.net/lists/listinfo/matplotlibusers >> >> > >  > _______________________________________________ > Matplotlibusers mailing list > Matplotlibusers@... > https://lists.sourceforge.net/lists/listinfo/matplotlibusers > 
From: JaeJoon Lee <lee.joon@gm...>  20090701 19:17:23

On Wed, Jul 1, 2009 at 2:34 PM, Michael Droettboom<mdroe@...> wrote: > I agree with JaeJoon here  try to reduce the number of points before > passing it to matplotlib. > > However, I'm a little concerned about the segfault  I'd rather matplotlib > give a MemoryError exception if that's in fact what is happening. JaeJoon >  can you share your test that causes the segfault? > > The snippet below completely hogs my machine for a few minutes, but then, > correctly, aborts with a MemoryError. > > This is on FC11 i586, Python 2.6, Numpy 1.3. > > ==== > > from matplotlib.pyplot import * > import numpy as np > > points = np.random.random((50000000, 2)) > plot(points) > show() > Yes, I also got MemoryError in this case during the plot() call. But I got segfault for the code below. x=random(50e6) y=random(50e6) plt.plot(x, y) plt.show() In this case, plot() runs fine, but segfault during show(). The segfault happens in the _path_module::affine_transform method of src/_path.cpp. I wonder if you can reproduce this. JJ > ==== > > Mike > > On 07/01/2009 01:34 PM, JaeJoon Lee wrote: > > A snippet of code does not help much. > Please try to post a small concise standalone example that we can run and > test. > > A general advise is to try to reduce the number of plot call, i.e., > plot as may points as possible with a single plot call. > > However, 50million points seems to be awful a lot. > 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels. > My guess is that it makes little sense to plot 50 million points here. > > Anyhow, plotting 50million points with a single plot call dies with > some segfault error in my machine. So, I feel that matplotlib may not > be suitable for your task. But, John or others may have some insight > how to deal with. > > Regards, > > JJ > > > > On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<feldmann_markus@...> > wrote: > > > Hi All, > > my program lets slow down my cpu. This only appears if i plot to much > points. I am not sure how many point i need to get this, normally i plot > 3*14e6 + 8e3, that is round about 50million points. My system is a > dual core 2GHz cpu with 2Gbyte Ram. > > Here is my method to plot, > def drawtransientall(self,min): > self.subplot = self.figure.add_subplot(111) > self.subplot.grid(True) > list_t1,list_peaks,t2,list_samples = > self.computetransientall(min,min+self.maxitems) > offset = 0 > color = ['green','red','blue','magenta','cyan'] > markerPeaks = ['v','<','1','3','s'] > markerSamples = ['^','>','2','4','p'] > self.plots=[[],[]] > for i in range(len(self.showBands)): > self.plots[0] += > self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i], > linestyle='None') > self.plots[1] += > self.subplot.plot(t2,list_samples[i]+offset,color=color[i], > > marker=markerSamples[i],linestyle='None') > offset +=1 > > self.subplot.set_xlim(t2[0]np.abs(t2[1]t2[0])/100,t2[1]+np.abs(t2[1]t2[0])/100) > ymax = np.amax(list_samples) > ymin = np.amin(list_samples) > self.subplot.set_ylim([yminnp.abs(ymin)*0.1, ymax*1.2 + 2]) > self.subplot.set_ylabel("abs(Sample(t)) und > abs(Peak(t)+Offset)>",fontsize = 12) > self.subplot.set_xlabel("Zeit in Sek. >",fontsize = 12) > > Any ideas how to avoid the slow down of my cpu ? > > regards Markus > > >  > _______________________________________________ > Matplotlibusers mailing list > Matplotlibusers@... > https://lists.sourceforge.net/lists/listinfo/matplotlibusers > > > >  > _______________________________________________ > Matplotlibusers mailing list > Matplotlibusers@... > https://lists.sourceforge.net/lists/listinfo/matplotlibusers > > 
From: JaeJoon Lee <lee.joon@gm...>  20090701 19:53:18

I tracked this down do line 962 of the _path.cpp. double* vertex_out = (double*)PyArray_DATA(result); My guess is that PyArray_SimpleNew at line 957 returns NULL for a memory error instead of raising an exception, which makes result=NULL and causes a segfault at line 962. Since I'm not an c++ expert, I'll leave it to you, Michael. Regards, JJ On Wed, Jul 1, 2009 at 3:16 PM, JaeJoon Lee<lee.j.joon@...> wrote: > On Wed, Jul 1, 2009 at 2:34 PM, Michael Droettboom<mdroe@...> wrote: >> I agree with JaeJoon here  try to reduce the number of points before >> passing it to matplotlib. >> >> However, I'm a little concerned about the segfault  I'd rather matplotlib >> give a MemoryError exception if that's in fact what is happening. JaeJoon >>  can you share your test that causes the segfault? >> >> The snippet below completely hogs my machine for a few minutes, but then, >> correctly, aborts with a MemoryError. >> >> This is on FC11 i586, Python 2.6, Numpy 1.3. >> >> ==== >> >> from matplotlib.pyplot import * >> import numpy as np >> >> points = np.random.random((50000000, 2)) >> plot(points) >> show() >> > > Yes, I also got MemoryError in this case during the plot() call. > > But I got segfault for the code below. > > x=random(50e6) > y=random(50e6) > plt.plot(x, y) > plt.show() > > In this case, plot() runs fine, but segfault during show(). > > The segfault happens in the _path_module::affine_transform method of > src/_path.cpp. > > I wonder if you can reproduce this. > > JJ > > >> ==== >> >> Mike >> >> On 07/01/2009 01:34 PM, JaeJoon Lee wrote: >> >> A snippet of code does not help much. >> Please try to post a small concise standalone example that we can run and >> test. >> >> A general advise is to try to reduce the number of plot call, i.e., >> plot as may points as possible with a single plot call. >> >> However, 50million points seems to be awful a lot. >> 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels. >> My guess is that it makes little sense to plot 50 million points here. >> >> Anyhow, plotting 50million points with a single plot call dies with >> some segfault error in my machine. So, I feel that matplotlib may not >> be suitable for your task. But, John or others may have some insight >> how to deal with. >> >> Regards, >> >> JJ >> >> >> >> On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<feldmann_markus@...> >> wrote: >> >> >> Hi All, >> >> my program lets slow down my cpu. This only appears if i plot to much >> points. I am not sure how many point i need to get this, normally i plot >> 3*14e6 + 8e3, that is round about 50million points. My system is a >> dual core 2GHz cpu with 2Gbyte Ram. >> >> Here is my method to plot, >> def drawtransientall(self,min): >> self.subplot = self.figure.add_subplot(111) >> self.subplot.grid(True) >> list_t1,list_peaks,t2,list_samples = >> self.computetransientall(min,min+self.maxitems) >> offset = 0 >> color = ['green','red','blue','magenta','cyan'] >> markerPeaks = ['v','<','1','3','s'] >> markerSamples = ['^','>','2','4','p'] >> self.plots=[[],[]] >> for i in range(len(self.showBands)): >> self.plots[0] += >> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i], >> linestyle='None') >> self.plots[1] += >> self.subplot.plot(t2,list_samples[i]+offset,color=color[i], >> >> marker=markerSamples[i],linestyle='None') >> offset +=1 >> >> self.subplot.set_xlim(t2[0]np.abs(t2[1]t2[0])/100,t2[1]+np.abs(t2[1]t2[0])/100) >> ymax = np.amax(list_samples) >> ymin = np.amin(list_samples) >> self.subplot.set_ylim([yminnp.abs(ymin)*0.1, ymax*1.2 + 2]) >> self.subplot.set_ylabel("abs(Sample(t)) und >> abs(Peak(t)+Offset)>",fontsize = 12) >> self.subplot.set_xlabel("Zeit in Sek. >",fontsize = 12) >> >> Any ideas how to avoid the slow down of my cpu ? >> >> regards Markus >> >> >>  >> _______________________________________________ >> Matplotlibusers mailing list >> Matplotlibusers@... >> https://lists.sourceforge.net/lists/listinfo/matplotlibusers >> >> >> >>  >> _______________________________________________ >> Matplotlibusers mailing list >> Matplotlibusers@... >> https://lists.sourceforge.net/lists/listinfo/matplotlibusers >> >> > 
From: Michael Droettboom <mdroe@st...>  20090701 20:03:03
Attachments:
Message as HTML

This should now be fixed on the maintenance branch and trunk. A Numpy array allocation was not being NULLchecked in _path.cpp:affine_transform. I know a MemoryError doesn't help the user much more than a segfault, but it always makes me feel better to get a real Python exception rather than exploding ;) Mike On 07/01/2009 03:16 PM, JaeJoon Lee wrote: > On Wed, Jul 1, 2009 at 2:34 PM, Michael Droettboom<mdroe@...> wrote: > >> I agree with JaeJoon here  try to reduce the number of points before >> passing it to matplotlib. >> >> However, I'm a little concerned about the segfault  I'd rather matplotlib >> give a MemoryError exception if that's in fact what is happening. JaeJoon >>  can you share your test that causes the segfault? >> >> The snippet below completely hogs my machine for a few minutes, but then, >> correctly, aborts with a MemoryError. >> >> This is on FC11 i586, Python 2.6, Numpy 1.3. >> >> ==== >> >> from matplotlib.pyplot import * >> import numpy as np >> >> points = np.random.random((50000000, 2)) >> plot(points) >> show() >> >> > > Yes, I also got MemoryError in this case during the plot() call. > > But I got segfault for the code below. > > x=random(50e6) > y=random(50e6) > plt.plot(x, y) > plt.show() > > In this case, plot() runs fine, but segfault during show(). > > The segfault happens in the _path_module::affine_transform method of > src/_path.cpp. > > I wonder if you can reproduce this. > > JJ > > > >> ==== >> >> Mike >> >> On 07/01/2009 01:34 PM, JaeJoon Lee wrote: >> >> A snippet of code does not help much. >> Please try to post a small concise standalone example that we can run and >> test. >> >> A general advise is to try to reduce the number of plot call, i.e., >> plot as may points as possible with a single plot call. >> >> However, 50million points seems to be awful a lot. >> 6 inch x 6 inch figure with dpi=100 has 0.36 million number of pixels. >> My guess is that it makes little sense to plot 50 million points here. >> >> Anyhow, plotting 50million points with a single plot call dies with >> some segfault error in my machine. So, I feel that matplotlib may not >> be suitable for your task. But, John or others may have some insight >> how to deal with. >> >> Regards, >> >> JJ >> >> >> >> On Tue, Jun 30, 2009 at 1:22 PM, Markus Feldmann<feldmann_markus@...> >> wrote: >> >> >> Hi All, >> >> my program lets slow down my cpu. This only appears if i plot to much >> points. I am not sure how many point i need to get this, normally i plot >> 3*14e6 + 8e3, that is round about 50million points. My system is a >> dual core 2GHz cpu with 2Gbyte Ram. >> >> Here is my method to plot, >> def drawtransientall(self,min): >> self.subplot = self.figure.add_subplot(111) >> self.subplot.grid(True) >> list_t1,list_peaks,t2,list_samples = >> self.computetransientall(min,min+self.maxitems) >> offset = 0 >> color = ['green','red','blue','magenta','cyan'] >> markerPeaks = ['v','<','1','3','s'] >> markerSamples = ['^','>','2','4','p'] >> self.plots=[[],[]] >> for i in range(len(self.showBands)): >> self.plots[0] += >> self.subplot.plot(list_t1[i],list_peaks[i],color=color[i],marker=markerPeaks[i], >> linestyle='None') >> self.plots[1] += >> self.subplot.plot(t2,list_samples[i]+offset,color=color[i], >> >> marker=markerSamples[i],linestyle='None') >> offset +=1 >> >> self.subplot.set_xlim(t2[0]np.abs(t2[1]t2[0])/100,t2[1]+np.abs(t2[1]t2[0])/100) >> ymax = np.amax(list_samples) >> ymin = np.amin(list_samples) >> self.subplot.set_ylim([yminnp.abs(ymin)*0.1, ymax*1.2 + 2]) >> self.subplot.set_ylabel("abs(Sample(t)) und >> abs(Peak(t)+Offset)>",fontsize = 12) >> self.subplot.set_xlabel("Zeit in Sek. >",fontsize = 12) >> >> Any ideas how to avoid the slow down of my cpu ? >> >> regards Markus >> >> >>  >> _______________________________________________ >> Matplotlibusers mailing list >> Matplotlibusers@... >> https://lists.sourceforge.net/lists/listinfo/matplotlibusers >> >> >> >>  >> _______________________________________________ >> Matplotlibusers mailing list >> Matplotlibusers@... >> https://lists.sourceforge.net/lists/listinfo/matplotlibusers >> >> >> 
From: Markus Feldmann <feldmann_markus@gm...>  20090703 07:29:13

Michael Droettboom schrieb: > This should now be fixed on the maintenance branch and trunk. A Numpy > array allocation was not being NULLchecked in _path.cpp:affine_transform. > > I know a MemoryError doesn't help the user much more than a segfault, > but it always makes me feel better to get a real Python exception rather > than exploding ;) > > Mike Hi All, thanks for the bugfix. I also got the segfault, but forgot to wrote this. Is there a possibility to limit the maximum points shown ? And sorry that i can not post more of my code. My program is to big. Maybe this is a feature request. :) It would be nice to set up a maximum limit and compute a average. I doesnt know which way is the fastest. I only execute the plot command three times but every plot command plots round about 15 Mill. points. regards Markus 