## Re: [Matplotlib-users] Masking data problems and fill test

 Re: [Matplotlib-users] Masking data problems and fill test From: Michaël Douchin - 2008-02-28 07:59:13 ```Thanks for you reply Eric. Since I am a beginner in matplotlib, I don't want to blame the soft first. I mean it could be a bug, but I have the feeling my code for masking the missing data is useless. My real question is : How to mask data that do not exist ??? In the masked_demo.py example, it is show how to mask data from a set like "My data are x=[1,2,3,4,5,6,7,8,9,10] y = sin(x) or anyelse continuous fonction and I want to mask 1,2,3 so I write : ym = ma.masked_where(y <3, y) But in my case, it is more like : x = [1,2,3,4,7,8,9,15,16,17] y = discrete values (no continuous fonction !) like [5,2,20,17,3,12,9,18,18,9] What I want to do with these data : 1/plot graph with no line connecting the point (x,y) = (4,17) and the point (7,3), etc. 2/fill the space under the plot but do not fill the space where there is no data (eg between the same 2 points). The 1st thing seems to be easy = just follow the example masked_demo But in my case, how can I mask data that don't exist ? I can't write ym = ma.masked_where(y <3, y) Hum, I hope I was clearer.... Michael Eric Firing a écrit : > You have run into a bug in the combination of poly_between and > fill--maybe only the former, which not taking masked arrays into > account. I have not looked at it enough to know whether it will be > easy or hard to fix, but it certainly should be fixed. I can't look > at it more right now, unfortunately. > > Eric > > Michaël Douchin wrote: >> Hi list >> >> I searched the list and google, but couldn't find a way to solve my pbm. >> >> I have data stored in a list (from an sql query) , with these "columns": >> x = time serie in hours >> y = some level value >> >> There are some missing values : eg between 08:33 and 08:40. >> Here is my code : >> sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + >> date_traite + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;" >> resa=db.query(sqla) >> data=resa.dictresult() #x = time serie >> x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" >> "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data] >> #y5 = tank level >> y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la >> colonne y1 >> figure() >> # the plot command with no mask >> >> plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False) >> >> #the mask >> #ym5 = ma.masked_where(y5 <300, y5) >> #plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False) >> #the filling under the curve >> xs, ys = poly_between(x, 0, y5) >> fill(xs,ys) >> >> >> Here is the result: >> http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png >> As you see, I commented the lines with the mask, because it did not >> change anything >> >> To see what I am looking for, here is the result under R (a >> statistical tool) >> http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg >> As you see, between 08:48 and 08:50 (for example), there is a gap, >> showing we have no data for this interval. >> >> As I want to automatically draw this graph for different set of data, >> I can't look each set in detail. >> I tried to folow the example masked_demo.py, but I could not adapt it >> to my case.... >> >> Any hint ? >> Thanks very much in advance >> >> Michael >> >> ------------------------------------------------------------------------- >> >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Matplotlib-users mailing list >> Matplotlib-users@... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > > ```

 [Matplotlib-users] Masking data problems and fill test From: Michaël Douchin - 2008-02-27 09:57:09 ```Hi list I searched the list and google, but couldn't find a way to solve my pbm. I have data stored in a list (from an sql query) , with these "columns": x = time serie in hours y = some level value There are some missing values : eg between 08:33 and 08:40. Here is my code : sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + date_traite + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;" resa=db.query(sqla) data=resa.dictresult() #x = time serie x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data] #y5 = tank level y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la colonne y1 figure() # the plot command with no mask plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False) #the mask #ym5 = ma.masked_where(y5 <300, y5) #plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False) #the filling under the curve xs, ys = poly_between(x, 0, y5) fill(xs,ys) Here is the result: http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png As you see, I commented the lines with the mask, because it did not change anything To see what I am looking for, here is the result under R (a statistical tool) http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg As you see, between 08:48 and 08:50 (for example), there is a gap, showing we have no data for this interval. As I want to automatically draw this graph for different set of data, I can't look each set in detail. I tried to folow the example masked_demo.py, but I could not adapt it to my case.... Any hint ? Thanks very much in advance Michael ```
 Re: [Matplotlib-users] Masking data problems and fill test From: Eric Firing - 2008-02-27 18:26:24 ```You have run into a bug in the combination of poly_between and fill--maybe only the former, which not taking masked arrays into account. I have not looked at it enough to know whether it will be easy or hard to fix, but it certainly should be fixed. I can't look at it more right now, unfortunately. Eric Michaël Douchin wrote: > Hi list > > I searched the list and google, but couldn't find a way to solve my pbm. > > I have data stored in a list (from an sql query) , with these "columns": > x = time serie in hours > y = some level value > > There are some missing values : eg between 08:33 and 08:40. > Here is my code : > sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + date_traite > + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;" > resa=db.query(sqla) > data=resa.dictresult() > #x = time serie > x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" > "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data] > #y5 = tank level > y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la > colonne y1 > figure() > # the plot command with no mask > > plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False) > #the mask > #ym5 = ma.masked_where(y5 <300, y5) > #plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False) > #the filling under the curve > xs, ys = poly_between(x, 0, y5) > fill(xs,ys) > > > Here is the result: > http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png > As you see, I commented the lines with the mask, because it did not > change anything > > To see what I am looking for, here is the result under R (a statistical > tool) > http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg > As you see, between 08:48 and 08:50 (for example), there is a gap, > showing we have no data for this interval. > > As I want to automatically draw this graph for different set of data, I > can't look each set in detail. > I tried to folow the example masked_demo.py, but I could not adapt it to > my case.... > > Any hint ? > Thanks very much in advance > > Michael > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users ```
 Re: [Matplotlib-users] Masking data problems and fill test From: Michaël Douchin - 2008-02-28 07:59:13 ```Thanks for you reply Eric. Since I am a beginner in matplotlib, I don't want to blame the soft first. I mean it could be a bug, but I have the feeling my code for masking the missing data is useless. My real question is : How to mask data that do not exist ??? In the masked_demo.py example, it is show how to mask data from a set like "My data are x=[1,2,3,4,5,6,7,8,9,10] y = sin(x) or anyelse continuous fonction and I want to mask 1,2,3 so I write : ym = ma.masked_where(y <3, y) But in my case, it is more like : x = [1,2,3,4,7,8,9,15,16,17] y = discrete values (no continuous fonction !) like [5,2,20,17,3,12,9,18,18,9] What I want to do with these data : 1/plot graph with no line connecting the point (x,y) = (4,17) and the point (7,3), etc. 2/fill the space under the plot but do not fill the space where there is no data (eg between the same 2 points). The 1st thing seems to be easy = just follow the example masked_demo But in my case, how can I mask data that don't exist ? I can't write ym = ma.masked_where(y <3, y) Hum, I hope I was clearer.... Michael Eric Firing a écrit : > You have run into a bug in the combination of poly_between and > fill--maybe only the former, which not taking masked arrays into > account. I have not looked at it enough to know whether it will be > easy or hard to fix, but it certainly should be fixed. I can't look > at it more right now, unfortunately. > > Eric > > Michaël Douchin wrote: >> Hi list >> >> I searched the list and google, but couldn't find a way to solve my pbm. >> >> I have data stored in a list (from an sql query) , with these "columns": >> x = time serie in hours >> y = some level value >> >> There are some missing values : eg between 08:33 and 08:40. >> Here is my code : >> sqla="SELECT * FROM import_parcelle a WHERE dat_loc='" + >> date_traite + "' AND code_uc='" + code_uc + "' ORDER BY a.heu_loc ;" >> resa=db.query(sqla) >> data=resa.dictresult() #x = time serie >> x= [ datetime.datetime(*time.strptime(a["dat_loc"]+" >> "+a["heu_loc"],'%Y-%m-%d %H:%M:%S')[0:6] ) for a in data] >> #y5 = tank level >> y5= [float(a["niv_cuv"]) for a in data] # --> extraction de la >> colonne y1 >> figure() >> # the plot command with no mask >> >> plot_date(x,y5,color='b',linestyle='None',marker='',xdate=True,ydate=False) >> >> #the mask >> #ym5 = ma.masked_where(y5 <300, y5) >> #plot_date(x,ym5,color='r',linestyle='-',xdate=True,ydate=False) >> #the filling under the curve >> xs, ys = poly_between(x, 0, y5) >> fill(xs,ys) >> >> >> Here is the result: >> http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.png >> As you see, I commented the lines with the mask, because it did not >> change anything >> >> To see what I am looking for, here is the result under R (a >> statistical tool) >> http://michaeldouchin.free.fr/17_2007-06-07_10B_vitesse.jpg >> As you see, between 08:48 and 08:50 (for example), there is a gap, >> showing we have no data for this interval. >> >> As I want to automatically draw this graph for different set of data, >> I can't look each set in detail. >> I tried to folow the example masked_demo.py, but I could not adapt it >> to my case.... >> >> Any hint ? >> Thanks very much in advance >> >> Michael >> >> ------------------------------------------------------------------------- >> >> This SF.net email is sponsored by: Microsoft >> Defy all challenges. Microsoft(R) Visual Studio 2008. >> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ >> _______________________________________________ >> Matplotlib-users mailing list >> Matplotlib-users@... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > > ```
 Re: [Matplotlib-users] Masking data problems and fill test From: Pierre GM - 2008-02-28 14:07:57 ```Michael, Give scikits.timeseries a try. It's a package designed to handle time series, with or without missing data or dates, for which matplotlib functions are also available. http://scipy.org/scipy/scikits/wiki/TimeSeries The problem you're refering to (missing dates) is trivial with this package. Let me know if you have any problem with it. Please note that plotting with hourly frequency is still experimental (and we need some feedback). HIH P. ```
 Re: [Matplotlib-users] Masking data problems and fill test From: Michaël Douchin - 2008-03-03 13:14:17 ```Thanks Pierre, I will try this package and come back here to give a feedback Michael Pierre GM a écrit : > Michael, > Give scikits.timeseries a try. It's a package designed to handle time series, > with or without missing data or dates, for which matplotlib functions are > also available. > http://scipy.org/scipy/scikits/wiki/TimeSeries > The problem you're refering to (missing dates) is trivial with this package. > Let me know if you have any problem with it. Please note that plotting with > hourly frequency is still experimental (and we need some feedback). > HIH > P. > > ------------------------------------------------------------------------- > This SF.net email is sponsored by: Microsoft > Defy all challenges. Microsoft(R) Visual Studio 2008. > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/ > _______________________________________________ > Matplotlib-users mailing list > Matplotlib-users@... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > > ```