From: Bill E. <ee...@ae...> - 2010-06-18 00:31:46
|
I'm struggling to figure out how to format my data so that I can use dates as x-data. I've tried the pylab examples and they execute beautifully on my Windoze and Linux boxes. But when I try to plot data on my own, it doesn't work. Using api example date_demo.py as a starting point. This appears to be stock price for Google from 2004 to 2008. Here are some clues: In [7]: q = np.load(datafile) In [8]: q Out[7]: rec.array([ (datetime.date(2004, 8, 19), 100.0, 104.06, 95.959999999999994, 100. 34, 22351900L, 100.34), (datetime.date(2004, 8, 20), 101.01000000000001, 109.08, 100.5, 108.31, 1 1428600L, 108.31), (datetime.date(2004, 8, 23), 110.75, 113.48, 109.05, 109.40000000000001, 9137200L, 109.40000000000001), ..., (datetime.date(2008, 10, 10), 313.16000000000003, 341.88999999999999, 310 .30000000000001, 332.0, 10597800L, 332.0), (datetime.date(2008, 10, 13), 355.79000000000002, 381.94999999999999, 345 .75, 381.01999999999998, 8905500L, 381.01999999999998), (datetime.date(2008, 10, 14), 393.52999999999997, 394.5, 357.0, 362.70999 999999998, 7784800L, 362.70999999999998)], dtype=[('date', '|O4'), ('', '|V4'), ('open', '<f8'), ('high', '<f8'), ('l ow', '<f8'), ('close', '<f8'), ('volume', '<i8'), ('adj_close', '<f8')]) In [9]: r = q.view(np.recarray) In [10]: r.date Out[10]: array([2004-08-19, 2004-08-20, 2004-08-23, ..., 2008-10-10, 2008-10-13, 2008-10-14], dtype=object) So it appears that the dtype of the date column is '|O4' I tried to import my own data. It looks like 2005-03-04,0.923115796 2005-03-05,0.915828724 2005-03-06,0.442521474 2005-03-07,0.997096213 2005-03-08,0.867752118 And to import, I use recarray myarray = np.loadtxt(fullfile, dtype=[('date', '|O4'), ('ydata', 'float')], delimiter = ',') rx = myarray.view(np.recarray) Data imports fine. But when I go to plot, I get the following error ValueError: setting an array element with a sequence. WARNING: Failure executing file: <bogus.py> This makes me think I don't quite have everything formatted correctly. Any ideas? Bill Eaton |
From: Ryan M. <rm...@gm...> - 2010-06-18 17:31:51
|
On Thu, Jun 17, 2010 at 11:39 AM, Bill Eaton <ee...@ae...> wrote: > I'm struggling to figure out how to format my data so that I can use dates > as x-data. <SNIP> > I tried to import my own data. It looks like > 2005-03-04,0.923115796 > 2005-03-05,0.915828724 > 2005-03-06,0.442521474 > 2005-03-07,0.997096213 > 2005-03-08,0.867752118 > And to import, I use recarray > myarray = np.loadtxt(fullfile, dtype=[('date', '|O4'), ('ydata', > 'float')], delimiter = ',') > rx = myarray.view(np.recarray) > > Data imports fine. But when I go to plot, I get the following error > ValueError: setting an array element with a sequence. > WARNING: Failure executing file: <bogus.py> You need to give np.loadtxt a converter so that it converts that first column of strings into datetime objects: import numpy as np import matplotlib.pyplot as plt from matplotlib import dates from datetime import datetime from StringIO import StringIO s=''' 2005-03-04,0.923115796 2005-03-05,0.915828724 2005-03-06,0.442521474 2005-03-07,0.997096213 2005-03-08,0.867752118''' dateparser = lambda s: datetime.strptime(s, '%Y-%m-%d') dt = np.dtype([('date', np.object),('ydata', np.float)]) d = np.loadtxt(StringIO(s), dtype=dt, converters={0:dateparser}, delimiter=',') plt.plot(d['date'], d['ydata']) plt.gca().xaxis.set_major_locator(dates.DayLocator()) plt.gca().xaxis.set_major_formatter(dates.DateFormatter('%Y/%m/%d')) plt.show() Personally, I find it much easier to work with python datetime objects than any other form. You can Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma |
From: Eric F. <ef...@ha...> - 2010-06-18 18:21:59
|
On 06/18/2010 07:31 AM, Ryan May wrote: > On Thu, Jun 17, 2010 at 11:39 AM, Bill Eaton<ee...@ae...> wrote: >> I'm struggling to figure out how to format my data so that I can use dates >> as x-data. > <SNIP> >> I tried to import my own data. It looks like >> 2005-03-04,0.923115796 >> 2005-03-05,0.915828724 >> 2005-03-06,0.442521474 >> 2005-03-07,0.997096213 >> 2005-03-08,0.867752118 Try matplotlib's mlab.csv2rec. It has a lot of magic built-in, including automatic date recognition. Eric >> And to import, I use recarray >> myarray = np.loadtxt(fullfile, dtype=[('date', '|O4'), ('ydata', >> 'float')], delimiter = ',') >> rx = myarray.view(np.recarray) >> >> Data imports fine. But when I go to plot, I get the following error >> ValueError: setting an array element with a sequence. >> WARNING: Failure executing file:<bogus.py> > > You need to give np.loadtxt a converter so that it converts that first > column of strings into datetime objects: > > import numpy as np > import matplotlib.pyplot as plt > from matplotlib import dates > from datetime import datetime > from StringIO import StringIO > > s=''' > 2005-03-04,0.923115796 > 2005-03-05,0.915828724 > 2005-03-06,0.442521474 > 2005-03-07,0.997096213 > 2005-03-08,0.867752118''' > > dateparser = lambda s: datetime.strptime(s, '%Y-%m-%d') > dt = np.dtype([('date', np.object),('ydata', np.float)]) > d = np.loadtxt(StringIO(s), dtype=dt, converters={0:dateparser}, delimiter=',') > plt.plot(d['date'], d['ydata']) > plt.gca().xaxis.set_major_locator(dates.DayLocator()) > plt.gca().xaxis.set_major_formatter(dates.DateFormatter('%Y/%m/%d')) > plt.show() > > Personally, I find it much easier to work with python datetime objects > than any other form. You can > > Ryan > |