From: antonv <vas...@ya...> - 2008-12-23 06:51:58
|
Hi all, I am pretty new to matplotlib and I have a big issue... I am trying to plot data from a grib file from noaa and I am not sure what's the best way to go about it. Until now, I am extracting the data i need in CSV files but I am not able to plot it. Here is a sample of the code that I am using: import matplotlib.pyplot as plt import matplotlib.mlab as mlab r = mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') plt.contour(r.longitude, r.latitude) plt.show() Which errors out and requires a 2d array. By the way, here is a data file: http://downloads.75ive.com/testdata.csv http://downloads.75ive.com/testdata.csv Any help is appreciated! -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21139727.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: Eric F. <ef...@ha...> - 2008-12-23 07:18:34
|
antonv wrote: > Hi all, > > I am pretty new to matplotlib and I have a big issue... I am trying to plot > data from a grib file from noaa and I am not sure what's the best way to go > about it. Until now, I am extracting the data i need in CSV files but I am > not able to plot it. Here is a sample of the code that I am using: > > import matplotlib.pyplot as plt > import matplotlib.mlab as mlab > > r = > mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') > plt.contour(r.longitude, r.latitude) > plt.show() > > Which errors out and requires a 2d array. > > By the way, here is a data file: http://downloads.75ive.com/testdata.csv > http://downloads.75ive.com/testdata.csv > > Any help is appreciated! I assume you meant to include your dependent variable in the call to contour, not just your longitude and latitude. In any case, you need to grid your data onto a 2D array. In the mpl examples, see griddata_demo.py. Eric |
From: Scott S. <sco...@gm...> - 2008-12-23 10:25:58
|
>2008/12/23 Eric Firing <ef...@ha...>: > antonv wrote: >> Hi all, >> >> I am trying to plot data from a grib file from noaa ... >> Here is a sample of the code that I am using: >> >> import matplotlib.pyplot as plt >> import matplotlib.mlab as mlab >> >> r = >> mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') >> plt.contour(r.longitude, r.latitude) >> plt.show() >> >> Which errors out and requires a 2d array. >> >> By the way, here is a data file: http://downloads.75ive.com/testdata.csv >> http://downloads.75ive.com/testdata.csv >> >> Any help is appreciated! > > > I assume you meant to include your dependent variable in the call to > contour, not just your longitude and latitude. In any case, you need to > grid your data onto a 2D array. In the mpl examples, see griddata_demo.py. If you just want a quick overview, you could also try >>> plt.scatter(r.longitude, r.latitude, c=r.dirpw_200812191800, s=5, edgecolors='none') >>> plt.colorbar() >>> plt.show() Cheers, Scott |
From: antonv <vas...@ya...> - 2008-12-23 16:16:01
|
Scott, the Scatter function works great but i am really looking to use the contourf one. Eric, you're right, the way I had it initially it was missing the z value. I looked at the griddata example and here is the code I came up with: from numpy.random import uniform, seed from matplotlib.mlab import griddata import matplotlib.pyplot as plt import numpy as np r = mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') x = r.longitude y = r.latitude z = r.dirpw_200812191800 xi = r.x yi = r.y # grid the data. zi = griddata(x,y,z,xi,yi) CS = plt.contourf(xi,yi,zi,15,cmap=plt.cm.jet) plt.colorbar() # draw colorbar plt.show() Which still errors out: 'output grid defined by xi,yi must be monotone increasing' The xi and yi are from the first 2 columns of the csv file and I thought they were monotonically increasing. What am I missing? antonv wrote: > > Hi all, > > I am pretty new to matplotlib and I have a big issue... I am trying to > plot data from a grib file from noaa and I am not sure what's the best way > to go about it. Until now, I am extracting the data i need in CSV files > but I am not able to plot it. Here is a sample of the code that I am > using: > > import matplotlib.pyplot as plt > import matplotlib.mlab as mlab > > r = > mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') > plt.contour(r.longitude, r.latitude) > plt.show() > > Which errors out and requires a 2d array. > > By the way, here is a data file: http://downloads.75ive.com/testdata.csv > http://downloads.75ive.com/testdata.csv > > Any help is appreciated! > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21147081.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: Jeff W. <js...@fa...> - 2008-12-23 16:58:47
|
antonv wrote: > Scott, the Scatter function works great but i am really looking to use the > contourf one. > > Eric, you're right, the way I had it initially it was missing the z value. I > looked at the griddata example and here is the code I came up with: > Since you read your data from a GRIB file, it must be on a regular grid. Therefore, you don't need to use griddata to grid your data. The best way to go would be to use PyNIO to read the grib data into a 2D array, and then plot it directly with contourf (perhaps using the Basemap toolkit if you want to display a map underneath). If you point me to your GRIB file I can show you what I mean. -Jeff > from numpy.random import uniform, seed > from matplotlib.mlab import griddata > import matplotlib.pyplot as plt > import numpy as np > > r = > mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') > > x = r.longitude > y = r.latitude > z = r.dirpw_200812191800 > xi = r.x > yi = r.y > > # grid the data. > zi = griddata(x,y,z,xi,yi) > > CS = plt.contourf(xi,yi,zi,15,cmap=plt.cm.jet) > plt.colorbar() # draw colorbar > > plt.show() > > > Which still errors out: 'output grid defined by xi,yi must be monotone > increasing' > The xi and yi are from the first 2 columns of the csv file and I thought > they were monotonically increasing. What am I missing? > > antonv wrote: > >> Hi all, >> >> I am pretty new to matplotlib and I have a big issue... I am trying to >> plot data from a grib file from noaa and I am not sure what's the best way >> to go about it. Until now, I am extracting the data i need in CSV files >> but I am not able to plot it. Here is a sample of the code that I am >> using: >> >> import matplotlib.pyplot as plt >> import matplotlib.mlab as mlab >> >> r = >> mlab.csv2rec('D:/SocalScubaInfo/Data/Outputs/CSV_files/DIRPW_12191800.csv') >> plt.contour(r.longitude, r.latitude) >> plt.show() >> >> Which errors out and requires a 2d array. >> >> By the way, here is a data file: http://downloads.75ive.com/testdata.csv >> http://downloads.75ive.com/testdata.csv >> >> Any help is appreciated! >> >> > > |
From: KURT P. <pet...@ms...> - 2008-12-23 16:30:44
|
Sorry, I meant the griddata module as part of mlab.KurtFrom: pet...@ms...To: mat...@li...Subject: RE: Plotting NOAA data...Date: Tue, 23 Dec 2008 09:28:49 -0700 I don't seem to have a mlab module as part of matplotlib. Is that something new?Kurt |
From: Ryan M. <rm...@gm...> - 2008-12-23 17:20:46
|
KURT PETERS wrote: > Sorry, I meant the griddata module as part of mlab. > Kurt > It was first released in 0.98.3. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma |
From: antonv <vas...@ya...> - 2008-12-23 17:30:56
|
Hi Jeff, here is a link to the csv file: http://downloads.75ive.com/testdata.csv Thanks again for all your help guys! Really appreciated! Ryan May-3 wrote: > > KURT PETERS wrote: >> Sorry, I meant the griddata module as part of mlab. >> Kurt >> > > It was first released in 0.98.3. > > Ryan > > -- > Ryan May > Graduate Research Assistant > School of Meteorology > University of Oklahoma > > ------------------------------------------------------------------------------ > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21148270.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: Mauro C. <mau...@gm...> - 2008-12-24 10:25:19
|
Dear Anton, 2008/12/23 antonv <vas...@ya...>: > Also, because I figured out the data I need and already have the scripts in place > to extract the CSV files I would really like to keep it that way. Would it be possible to > just show me how to get from the csv file to the plot? Here is a short recipe: import numpy as np f = open("file.csv", "r") coords = np.loadtxt(f, delimiter=",", skiprows=1) lon = coords[:,0] lat = coords[:,1] dat = coords[:,2] where "file.csv" is a regular comma-separated values file in the format: Lat,Lon,Dat -61.05,10.4,20 -79.43,9.15,50 -70.66,9.53,10 -63.11,7.91,40 ... Hope this helps! Best regards, -- Dr. Mauro J. Cavalcanti Ecoinformatics Studio P.O. Box 46521, CEP 20551-970 Rio de Janeiro, RJ, BRASIL E-mail: mau...@gm... Web: http://studio.infobio.net Linux Registered User #473524 * Ubuntu User #22717 "Life is complex. It consists of real and imaginary parts." |
From: antonv <vas...@ya...> - 2008-12-28 04:40:15
|
It seems that I just cannot grasp the way the data needs to be formatted for this to work... I've used the griddata sample that James posted but it takes about 10 minutes to prep the data for plotting so that solution seems to be out of discussion. I guess my issue is that I don't know what type of data is required by contourf function. Also as Jeff was saying earlier, the data is read from a grib file so supposedly it's already gridded. I've also looked at the basemap demo (http://matplotlib.sourceforge.net/users/screenshots.html#basemap-demo) and the data is read from 3 files, one for Lat one for Long and the Last for Z Data. Is there a way to automatically extract the data from the grib file to a format similar to the one used in the basemap example? Jeff Whitaker wrote: > > Mauro Cavalcanti wrote: >> Dear Anton, >> >> 2008/12/23 antonv <vas...@ya...>: >> >>> Also, because I figured out the data I need and already have the >>> scripts in place >>> to extract the CSV files I would really like to keep it that way. Would >>> it be possible to >>> just show me how to get from the csv file to the plot? >>> >> >> Here is a short recipe: >> >> import numpy as np >> >> f = open("file.csv", "r") >> coords = np.loadtxt(f, delimiter=",", skiprows=1) >> lon = coords[:,0] >> lat = coords[:,1] >> dat = coords[:,2] >> >> where "file.csv" is a regular comma-separated values file in the format: >> >> Lat,Lon,Dat >> -61.05,10.4,20 >> -79.43,9.15,50 >> -70.66,9.53,10 >> -63.11,7.91,40 >> ... >> >> Hope this helps! >> >> Best regards, >> >> > Since the arrays are 2D (for gridded data), a reshape is also needed, i.e. > > lon.shape = (nlats,nlons) > lat.shape = (nlats,nlons) > data.shape = (nlats,nlons) > > You'll need to know what the grid dimensons (nlats,nlons) are. > > -Jeff > > ------------------------------------------------------------------------------ > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21190078.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: antonv <vas...@ya...> - 2008-12-29 04:58:40
|
Hey Jeff, I've got it sorted out a bit now. You're right the data was an output from Degrib and I had the option to output the csv's with or without data in the land areas. As before I was using a program that was placing the pixels in an image based on the X and Y columns it didn't create an issue. That was an easy fix by switching the option in the Degrib export. Also by looking at your example I realized the way the contourf function requests the Z data and by just reshaping the array I was able to make all the stuff work properly. Numpy is amazing by the way! I had no idea how easy you can work with huge arrays! My new issue is that I need to mask the land areas in the Z array so I would have a clean plot over the basemap. Any ideas on how to achieve that? Thanks, Anton Jeff Whitaker wrote: > > antonv wrote: >> It seems that I just cannot grasp the way the data needs to be formatted >> for >> this to work... >> I've used the griddata sample that James posted but it takes about 10 >> minutes to prep the data for plotting so that solution seems to be out of >> discussion. >> >> I guess my issue is that I don't know what type of data is required by >> contourf function. Also as Jeff was saying earlier, the data is read from >> a >> grib file so supposedly it's already gridded. I've also looked at the >> basemap demo >> (http://matplotlib.sourceforge.net/users/screenshots.html#basemap-demo) >> and >> the data is read from 3 files, one for Lat one for Long and the Last for >> Z >> Data. Is there a way to automatically extract the data from the grib file >> to >> a format similar to the one used in the basemap example? >> > > Anton: I just looked at your csv file and I think I know what the > problem is. Whatever program you used to dump the grib data did not > write all the data - the missing land values were skipped. That means > you don't have the full rectangular array of data. I think you have two > choices: > > 1) insert the missing land values into the array, either in the csv file > or into the array after it is read in from the csv file. What program > did you use to dump the GRIB data to a CSV file? > > 2) use a python grib interface. If you're on Windows, PyNIO won't > work. I've written my own module (pygrib2 - > http://code.google.com/p/pygrib2) which you should be able to compile on > windows. You'll need the png and jasper (jpeg2000) libraries, however. > > I recommend (2) - in the time you've already spent messing with that csv > file, you could have already gotten a real python grib reader working! > > -Jeff >> >> >> Jeff Whitaker wrote: >> >>> Mauro Cavalcanti wrote: >>> >>>> Dear Anton, >>>> >>>> 2008/12/23 antonv <vas...@ya...>: >>>> >>>> >>>>> Also, because I figured out the data I need and already have the >>>>> scripts in place >>>>> to extract the CSV files I would really like to keep it that way. >>>>> Would >>>>> it be possible to >>>>> just show me how to get from the csv file to the plot? >>>>> >>>>> >>>> Here is a short recipe: >>>> >>>> import numpy as np >>>> >>>> f = open("file.csv", "r") >>>> coords = np.loadtxt(f, delimiter=",", skiprows=1) >>>> lon = coords[:,0] >>>> lat = coords[:,1] >>>> dat = coords[:,2] >>>> >>>> where "file.csv" is a regular comma-separated values file in the >>>> format: >>>> >>>> Lat,Lon,Dat >>>> -61.05,10.4,20 >>>> -79.43,9.15,50 >>>> -70.66,9.53,10 >>>> -63.11,7.91,40 >>>> ... >>>> >>>> Hope this helps! >>>> >>>> Best regards, >>>> >>>> >>>> >>> Since the arrays are 2D (for gridded data), a reshape is also needed, >>> i.e. >>> >>> lon.shape = (nlats,nlons) >>> lat.shape = (nlats,nlons) >>> data.shape = (nlats,nlons) >>> >>> You'll need to know what the grid dimensons (nlats,nlons) are. >>> >>> -Jeff >>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Matplotlib-users mailing list >>> Mat...@li... >>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>> >>> >>> >> >> > > > ------------------------------------------------------------------------------ > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21199108.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: Jeff W. <js...@fa...> - 2008-12-23 17:56:56
|
antonv wrote: > Hi Jeff, here is a link to the csv file: > http://downloads.75ive.com/testdata.csv > > Thanks again for all your help guys! Really appreciated! > > I meant the original GRIB file ... -Jeff > Ryan May-3 wrote: > >> KURT PETERS wrote: >> >>> Sorry, I meant the griddata module as part of mlab. >>> Kurt >>> >>> >> It was first released in 0.98.3. >> >> Ryan >> >> -- >> Ryan May >> Graduate Research Assistant >> School of Meteorology >> University of Oklahoma >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Matplotlib-users mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> >> >> > > |
From: antonv <vas...@ya...> - 2008-12-23 18:02:16
|
Here is the link to the folder that has the grib files. They are updated every 3 hours: ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/ Any of the files there would have the same info only at different times of day. -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21148738.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: antonv <vas...@ya...> - 2008-12-23 18:12:39
|
Better yet, here is the file that I am using on my testing but keep in mind that it's a 33mb file: http://downloads.75ive.com/enp.t18z.grib.grib2 antonv wrote: > > Here is the link to the folder that has the grib files. They are updated > every 3 hours: > ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/ > > Any of the files there would have the same info only at different times of > day. > > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21148866.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: Jeff W. <js...@fa...> - 2008-12-23 18:46:55
Attachments:
testgrib.png
|
antonv wrote: > Better yet, here is the file that I am using on my testing but keep in mind > that it's a 33mb file: > http://downloads.75ive.com/enp.t18z.grib.grib2 > Anton: I had already downloaded a file from the ftp site, so here's an example script: import Nio from mpl_toolkits.basemap import Basemap import matplotlib.pyplot as plt import numpy as np f = Nio.open_file('akw.t00z.grib.grib2') print f.variables.keys() lons = f.variables['lon_0'][:] # flip latitudes so data goes S-->N lats = f.variables['lat_0'][::-1] times = f.variables['forecast_time0'][:] datavar = f.variables['WWSWHGT_P0_L1_GLL0'] ntime = 10 data = datavar[ntime,::-1] print f.variables['WWSWHGT_P0_L1_GLL0'] print data.min(), data.max() m = Basemap(projection='cyl',llcrnrlat=lats[0],llcrnrlon=lons[0],\ urcrnrlat=lats[-1],urcrnrlon=lons[-1],resolution='l') x, y = m(*np.meshgrid(lons, lats)) levels = np.arange(0,9.1,0.5) m.contourf(x,y,data,levels) m.drawcoastlines() m.fillcontinents() m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0]) m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1]) m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0]) m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1]) plt.title(datavar.long_name+' %s hr fcst'%(times[ntime]),fontsize=12) plt.colorbar(orientation='horizontal',shrink=0.9,format="%g") plt.show() The resulting plot is attached. Note you will need the Basemap toolkit (available from the matplotlib download site). If you don't want to plot the coastlines, you can just use plt.contourf. You'll also need PyNIO (http://www.pyngl.ucar.edu/Nio.shtml) to read the GRIB file. The png produced by this script is attached. Let me know if you have any questions. -Jeff > > > > antonv wrote: > >> Here is the link to the folder that has the grib files. They are updated >> every 3 hours: >> ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/ >> >> Any of the files there would have the same info only at different times of >> day. >> >> >> > > |
From: Jeff W. <js...@fa...> - 2008-12-23 19:41:28
|
vas...@ya... wrote: > Hi Jeff, > > This example is just awesome! I really appreciate the time you put into this but it's way way over my head... There are a couple issues I have especially that I just started using python and matplotlib a couple days ago. First, the direct reading of the grib file is great but I am not able to download Nio. The link they sent me to confirm the registration goes to a page that is not available... Anton: I downloaded and installed it this morning and it worked fine. If you're having problems, I'd suggest emailing them - they are quite responsive. > Also, because I figured out the data I need and already have the scripts in place to extract the CSV files I would really like to keep it that way. Would it be possible to just show me how to get from the csv file to the plot? > It's a matter of reading the CSV files into 2D arrays (lons,lats,data). I don't have time right now to work that out for your data. Anyway, I think you'll be happier in the long run if you bite the bullet now and learn how to use PyNIO to read the data. -Jeff > Thanks a lot! > Anton Vasilescu > > > Jeff Whitaker wrote: > >> antonv wrote: >> >>> Better yet, here is the file that I am using on my testing but keep in >>> mind >>> that it's a 33mb file: >>> http://downloads.75ive.com/enp.t18z.grib.grib2 >>> >>> >> Anton: I had already downloaded a file from the ftp site, so here's an >> example script: >> >> import Nio >> from mpl_toolkits.basemap import Basemap >> import matplotlib.pyplot as plt >> import numpy as np >> f = Nio.open_file('akw.t00z.grib.grib2') >> print f.variables.keys() >> lons = f.variables['lon_0'][:] >> # flip latitudes so data goes S-->N >> lats = f.variables['lat_0'][::-1] >> times = f.variables['forecast_time0'][:] >> datavar = f.variables['WWSWHGT_P0_L1_GLL0'] >> ntime = 10 >> data = datavar[ntime,::-1] >> print f.variables['WWSWHGT_P0_L1_GLL0'] >> print data.min(), data.max() >> m = Basemap(projection='cyl',llcrnrlat=lats[0],llcrnrlon=lons[0],\ >> urcrnrlat=lats[-1],urcrnrlon=lons[-1],resolution='l') >> x, y = m(*np.meshgrid(lons, lats)) >> levels = np.arange(0,9.1,0.5) >> m.contourf(x,y,data,levels) >> m.drawcoastlines() >> m.fillcontinents() >> m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0]) >> m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1]) >> m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0]) >> m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1]) >> plt.title(datavar.long_name+' %s hr fcst'%(times[ntime]),fontsize=12) >> plt.colorbar(orientation='horizontal',shrink=0.9,format="%g") >> plt.show() >> >> The resulting plot is attached. >> >> Note you will need the Basemap toolkit (available from the matplotlib >> download site). If you don't want to plot the coastlines, you can just >> use plt.contourf. You'll also need PyNIO >> (http://www.pyngl.ucar.edu/Nio.shtml) to read the GRIB file. >> >> The png produced by this script is attached. >> >> Let me know if you have any questions. >> >> -Jeff >> >>> >>> antonv wrote: >>> >>> >>>> Here is the link to the folder that has the grib files. They are updated >>>> every 3 hours: >>>> ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/ >>>> >>>> Any of the files there would have the same info only at different times >>>> of >>>> day. >>>> >>>> >>>> >>>> >>> >>> >> >> ------------------------------------------------------------------------------ >> >> _______________________________________________ >> Matplotlib-users mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> >> >> > Quoted from: > http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21149409.html > > |
From: antonv <vas...@ya...> - 2008-12-23 20:14:49
|
Is there a PyNIO binary for windows as on their download page there are only Linux, Mac or Solaris versions? Jeff Whitaker wrote: > > vas...@ya... wrote: >> Hi Jeff, >> >> This example is just awesome! I really appreciate the time you put into >> this but it's way way over my head... There are a couple issues I have >> especially that I just started using python and matplotlib a couple days >> ago. First, the direct reading of the grib file is great but I am not >> able to download Nio. The link they sent me to confirm the registration >> goes to a page that is not available... > Anton: I downloaded and installed it this morning and it worked fine. > If you're having problems, I'd suggest emailing them - they are quite > responsive. >> Also, because I figured out the data I need and already have the scripts >> in place to extract the CSV files I would really like to keep it that >> way. Would it be possible to just show me how to get from the csv file to >> the plot? >> > It's a matter of reading the CSV files into 2D arrays (lons,lats,data). > I don't have time right now to work that out for your data. Anyway, I > think you'll be happier in the long run if you bite the bullet now and > learn how to use PyNIO to read the data. > > -Jeff >> Thanks a lot! >> Anton Vasilescu >> >> >> Jeff Whitaker wrote: >> >>> antonv wrote: >>> >>>> Better yet, here is the file that I am using on my testing but keep in >>>> mind >>>> that it's a 33mb file: >>>> http://downloads.75ive.com/enp.t18z.grib.grib2 >>>> >>>> >>> Anton: I had already downloaded a file from the ftp site, so here's an >>> example script: >>> >>> import Nio >>> from mpl_toolkits.basemap import Basemap >>> import matplotlib.pyplot as plt >>> import numpy as np >>> f = Nio.open_file('akw.t00z.grib.grib2') >>> print f.variables.keys() >>> lons = f.variables['lon_0'][:] >>> # flip latitudes so data goes S-->N >>> lats = f.variables['lat_0'][::-1] >>> times = f.variables['forecast_time0'][:] >>> datavar = f.variables['WWSWHGT_P0_L1_GLL0'] >>> ntime = 10 >>> data = datavar[ntime,::-1] >>> print f.variables['WWSWHGT_P0_L1_GLL0'] >>> print data.min(), data.max() >>> m = Basemap(projection='cyl',llcrnrlat=lats[0],llcrnrlon=lons[0],\ >>> urcrnrlat=lats[-1],urcrnrlon=lons[-1],resolution='l') >>> x, y = m(*np.meshgrid(lons, lats)) >>> levels = np.arange(0,9.1,0.5) >>> m.contourf(x,y,data,levels) >>> m.drawcoastlines() >>> m.fillcontinents() >>> m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0]) >>> m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1]) >>> m.drawparallels(np.arange(40,81,10),labels=[1,0,0,0]) >>> m.drawmeridians(np.arange(150,241,10),labels=[0,0,0,1]) >>> plt.title(datavar.long_name+' %s hr fcst'%(times[ntime]),fontsize=12) >>> plt.colorbar(orientation='horizontal',shrink=0.9,format="%g") >>> plt.show() >>> >>> The resulting plot is attached. >>> >>> Note you will need the Basemap toolkit (available from the matplotlib >>> download site). If you don't want to plot the coastlines, you can just >>> use plt.contourf. You'll also need PyNIO >>> (http://www.pyngl.ucar.edu/Nio.shtml) to read the GRIB file. >>> >>> The png produced by this script is attached. >>> >>> Let me know if you have any questions. >>> >>> -Jeff >>> >>>> >>>> antonv wrote: >>>> >>>> >>>>> Here is the link to the folder that has the grib files. They are >>>>> updated >>>>> every 3 hours: >>>>> ftp://ftpprd.ncep.noaa.gov/pub/data/nccf/com/wave/prod/ >>>>> >>>>> Any of the files there would have the same info only at different >>>>> times >>>>> of >>>>> day. >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> >>> ------------------------------------------------------------------------------ >>> >>> _______________________________________________ >>> Matplotlib-users mailing list >>> Mat...@li... >>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>> >>> >>> >> Quoted from: >> http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21149409.html >> >> > > > ------------------------------------------------------------------------------ > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21150608.html Sent from the matplotlib - users mailing list archive at Nabble.com. |
From: Jeff W. <js...@fa...> - 2008-12-24 14:04:14
|
Mauro Cavalcanti wrote: > Dear Anton, > > 2008/12/23 antonv <vas...@ya...>: > >> Also, because I figured out the data I need and already have the scripts in place >> to extract the CSV files I would really like to keep it that way. Would it be possible to >> just show me how to get from the csv file to the plot? >> > > Here is a short recipe: > > import numpy as np > > f = open("file.csv", "r") > coords = np.loadtxt(f, delimiter=",", skiprows=1) > lon = coords[:,0] > lat = coords[:,1] > dat = coords[:,2] > > where "file.csv" is a regular comma-separated values file in the format: > > Lat,Lon,Dat > -61.05,10.4,20 > -79.43,9.15,50 > -70.66,9.53,10 > -63.11,7.91,40 > ... > > Hope this helps! > > Best regards, > > Since the arrays are 2D (for gridded data), a reshape is also needed, i.e. lon.shape = (nlats,nlons) lat.shape = (nlats,nlons) data.shape = (nlats,nlons) You'll need to know what the grid dimensons (nlats,nlons) are. -Jeff |
From: Jeff W. <js...@fa...> - 2008-12-28 18:43:28
|
antonv wrote: > It seems that I just cannot grasp the way the data needs to be formatted for > this to work... > I've used the griddata sample that James posted but it takes about 10 > minutes to prep the data for plotting so that solution seems to be out of > discussion. > > I guess my issue is that I don't know what type of data is required by > contourf function. Also as Jeff was saying earlier, the data is read from a > grib file so supposedly it's already gridded. I've also looked at the > basemap demo > (http://matplotlib.sourceforge.net/users/screenshots.html#basemap-demo) and > the data is read from 3 files, one for Lat one for Long and the Last for Z > Data. Is there a way to automatically extract the data from the grib file to > a format similar to the one used in the basemap example? > Anton: I just looked at your csv file and I think I know what the problem is. Whatever program you used to dump the grib data did not write all the data - the missing land values were skipped. That means you don't have the full rectangular array of data. I think you have two choices: 1) insert the missing land values into the array, either in the csv file or into the array after it is read in from the csv file. What program did you use to dump the GRIB data to a CSV file? 2) use a python grib interface. If you're on Windows, PyNIO won't work. I've written my own module (pygrib2 - http://code.google.com/p/pygrib2) which you should be able to compile on windows. You'll need the png and jasper (jpeg2000) libraries, however. I recommend (2) - in the time you've already spent messing with that csv file, you could have already gotten a real python grib reader working! -Jeff > > > Jeff Whitaker wrote: > >> Mauro Cavalcanti wrote: >> >>> Dear Anton, >>> >>> 2008/12/23 antonv <vas...@ya...>: >>> >>> >>>> Also, because I figured out the data I need and already have the >>>> scripts in place >>>> to extract the CSV files I would really like to keep it that way. Would >>>> it be possible to >>>> just show me how to get from the csv file to the plot? >>>> >>>> >>> Here is a short recipe: >>> >>> import numpy as np >>> >>> f = open("file.csv", "r") >>> coords = np.loadtxt(f, delimiter=",", skiprows=1) >>> lon = coords[:,0] >>> lat = coords[:,1] >>> dat = coords[:,2] >>> >>> where "file.csv" is a regular comma-separated values file in the format: >>> >>> Lat,Lon,Dat >>> -61.05,10.4,20 >>> -79.43,9.15,50 >>> -70.66,9.53,10 >>> -63.11,7.91,40 >>> ... >>> >>> Hope this helps! >>> >>> Best regards, >>> >>> >>> >> Since the arrays are 2D (for gridded data), a reshape is also needed, i.e. >> >> lon.shape = (nlats,nlons) >> lat.shape = (nlats,nlons) >> data.shape = (nlats,nlons) >> >> You'll need to know what the grid dimensons (nlats,nlons) are. >> >> -Jeff >> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Matplotlib-users mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> >> >> > > |
From: Jeff W. <js...@fa...> - 2008-12-29 14:21:16
|
antonv wrote: > Hey Jeff, > > I've got it sorted out a bit now. You're right the data was an output from > Degrib and I had the option to output the csv's with or without data in the > land areas. As before I was using a program that was placing the pixels in > an image based on the X and Y columns it didn't create an issue. That was an > easy fix by switching the option in the Degrib export. > > Also by looking at your example I realized the way the contourf function > requests the Z data and by just reshaping the array I was able to make all > the stuff work properly. Numpy is amazing by the way! I had no idea how easy > you can work with huge arrays! > > My new issue is that I need to mask the land areas in the Z array so I would > have a clean plot over the basemap. Any ideas on how to achieve that? > Create a masked array. Say the values in the Z array set to 1.e30 over land areas in the CSV file. from numpy import ma Z = ma.masked_values(Z,1.e30) Then plot with contourf as before and the land areas will not be contoured. -Jeff > Thanks, > Anton > > > Jeff Whitaker wrote: > >> antonv wrote: >> >>> It seems that I just cannot grasp the way the data needs to be formatted >>> for >>> this to work... >>> I've used the griddata sample that James posted but it takes about 10 >>> minutes to prep the data for plotting so that solution seems to be out of >>> discussion. >>> >>> I guess my issue is that I don't know what type of data is required by >>> contourf function. Also as Jeff was saying earlier, the data is read from >>> a >>> grib file so supposedly it's already gridded. I've also looked at the >>> basemap demo >>> (http://matplotlib.sourceforge.net/users/screenshots.html#basemap-demo) >>> and >>> the data is read from 3 files, one for Lat one for Long and the Last for >>> Z >>> Data. Is there a way to automatically extract the data from the grib file >>> to >>> a format similar to the one used in the basemap example? >>> >>> >> Anton: I just looked at your csv file and I think I know what the >> problem is. Whatever program you used to dump the grib data did not >> write all the data - the missing land values were skipped. That means >> you don't have the full rectangular array of data. I think you have two >> choices: >> >> 1) insert the missing land values into the array, either in the csv file >> or into the array after it is read in from the csv file. What program >> did you use to dump the GRIB data to a CSV file? >> >> 2) use a python grib interface. If you're on Windows, PyNIO won't >> work. I've written my own module (pygrib2 - >> http://code.google.com/p/pygrib2) which you should be able to compile on >> windows. You'll need the png and jasper (jpeg2000) libraries, however. >> >> I recommend (2) - in the time you've already spent messing with that csv >> file, you could have already gotten a real python grib reader working! >> >> -Jeff >> >>> Jeff Whitaker wrote: >>> >>> >>>> Mauro Cavalcanti wrote: >>>> >>>> >>>>> Dear Anton, >>>>> >>>>> 2008/12/23 antonv <vas...@ya...>: >>>>> >>>>> >>>>> >>>>>> Also, because I figured out the data I need and already have the >>>>>> scripts in place >>>>>> to extract the CSV files I would really like to keep it that way. >>>>>> Would >>>>>> it be possible to >>>>>> just show me how to get from the csv file to the plot? >>>>>> >>>>>> >>>>>> >>>>> Here is a short recipe: >>>>> >>>>> import numpy as np >>>>> >>>>> f = open("file.csv", "r") >>>>> coords = np.loadtxt(f, delimiter=",", skiprows=1) >>>>> lon = coords[:,0] >>>>> lat = coords[:,1] >>>>> dat = coords[:,2] >>>>> >>>>> where "file.csv" is a regular comma-separated values file in the >>>>> format: >>>>> >>>>> Lat,Lon,Dat >>>>> -61.05,10.4,20 >>>>> -79.43,9.15,50 >>>>> -70.66,9.53,10 >>>>> -63.11,7.91,40 >>>>> ... >>>>> >>>>> Hope this helps! >>>>> >>>>> Best regards, >>>>> >>>>> >>>>> >>>>> >>>> Since the arrays are 2D (for gridded data), a reshape is also needed, >>>> i.e. >>>> >>>> lon.shape = (nlats,nlons) >>>> lat.shape = (nlats,nlons) >>>> data.shape = (nlats,nlons) >>>> >>>> You'll need to know what the grid dimensons (nlats,nlons) are. >>>> >>>> -Jeff >>>> >>>> ------------------------------------------------------------------------------ >>>> _______________________________________________ >>>> Matplotlib-users mailing list >>>> Mat...@li... >>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>> >>>> >>>> >>>> >>> >>> >> ------------------------------------------------------------------------------ >> _______________________________________________ >> Matplotlib-users mailing list >> Mat...@li... >> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >> >> >> > > |
From: antonv <vas...@ya...> - 2009-01-01 20:38:29
|
So my beginner saga continues with another question: I am trying to create a custom colormap using ListedColormap. The custom color map is for a range of values between 0 and 20 while the data in my file is between 0 and 8. Now my issue is that when plotting the graph the colormap is using the 0 to 8 values from the file. How can I force it to use all the 21 values in the colormap? Thanks, Anton Jeff Whitaker wrote: > > antonv wrote: >> Hey Jeff, >> >> I've got it sorted out a bit now. You're right the data was an output >> from >> Degrib and I had the option to output the csv's with or without data in >> the >> land areas. As before I was using a program that was placing the pixels >> in >> an image based on the X and Y columns it didn't create an issue. That was >> an >> easy fix by switching the option in the Degrib export. >> >> Also by looking at your example I realized the way the contourf function >> requests the Z data and by just reshaping the array I was able to make >> all >> the stuff work properly. Numpy is amazing by the way! I had no idea how >> easy >> you can work with huge arrays! >> >> My new issue is that I need to mask the land areas in the Z array so I >> would >> have a clean plot over the basemap. Any ideas on how to achieve that? >> > > Create a masked array. Say the values in the Z array set to 1.e30 over > land areas in the CSV file. > > from numpy import ma > Z = ma.masked_values(Z,1.e30) > > Then plot with contourf as before and the land areas will not be > contoured. > > -Jeff >> Thanks, >> Anton >> >> >> Jeff Whitaker wrote: >> >>> antonv wrote: >>> >>>> It seems that I just cannot grasp the way the data needs to be >>>> formatted >>>> for >>>> this to work... >>>> I've used the griddata sample that James posted but it takes about 10 >>>> minutes to prep the data for plotting so that solution seems to be out >>>> of >>>> discussion. >>>> >>>> I guess my issue is that I don't know what type of data is required by >>>> contourf function. Also as Jeff was saying earlier, the data is read >>>> from >>>> a >>>> grib file so supposedly it's already gridded. I've also looked at the >>>> basemap demo >>>> (http://matplotlib.sourceforge.net/users/screenshots.html#basemap-demo) >>>> and >>>> the data is read from 3 files, one for Lat one for Long and the Last >>>> for >>>> Z >>>> Data. Is there a way to automatically extract the data from the grib >>>> file >>>> to >>>> a format similar to the one used in the basemap example? >>>> >>>> >>> Anton: I just looked at your csv file and I think I know what the >>> problem is. Whatever program you used to dump the grib data did not >>> write all the data - the missing land values were skipped. That means >>> you don't have the full rectangular array of data. I think you have two >>> choices: >>> >>> 1) insert the missing land values into the array, either in the csv file >>> or into the array after it is read in from the csv file. What program >>> did you use to dump the GRIB data to a CSV file? >>> >>> 2) use a python grib interface. If you're on Windows, PyNIO won't >>> work. I've written my own module (pygrib2 - >>> http://code.google.com/p/pygrib2) which you should be able to compile on >>> windows. You'll need the png and jasper (jpeg2000) libraries, however. >>> >>> I recommend (2) - in the time you've already spent messing with that csv >>> file, you could have already gotten a real python grib reader working! >>> >>> -Jeff >>> >>>> Jeff Whitaker wrote: >>>> >>>> >>>>> Mauro Cavalcanti wrote: >>>>> >>>>> >>>>>> Dear Anton, >>>>>> >>>>>> 2008/12/23 antonv <vas...@ya...>: >>>>>> >>>>>> >>>>>> >>>>>>> Also, because I figured out the data I need and already have the >>>>>>> scripts in place >>>>>>> to extract the CSV files I would really like to keep it that way. >>>>>>> Would >>>>>>> it be possible to >>>>>>> just show me how to get from the csv file to the plot? >>>>>>> >>>>>>> >>>>>>> >>>>>> Here is a short recipe: >>>>>> >>>>>> import numpy as np >>>>>> >>>>>> f = open("file.csv", "r") >>>>>> coords = np.loadtxt(f, delimiter=",", skiprows=1) >>>>>> lon = coords[:,0] >>>>>> lat = coords[:,1] >>>>>> dat = coords[:,2] >>>>>> >>>>>> where "file.csv" is a regular comma-separated values file in the >>>>>> format: >>>>>> >>>>>> Lat,Lon,Dat >>>>>> -61.05,10.4,20 >>>>>> -79.43,9.15,50 >>>>>> -70.66,9.53,10 >>>>>> -63.11,7.91,40 >>>>>> ... >>>>>> >>>>>> Hope this helps! >>>>>> >>>>>> Best regards, >>>>>> >>>>>> >>>>>> >>>>>> >>>>> Since the arrays are 2D (for gridded data), a reshape is also needed, >>>>> i.e. >>>>> >>>>> lon.shape = (nlats,nlons) >>>>> lat.shape = (nlats,nlons) >>>>> data.shape = (nlats,nlons) >>>>> >>>>> You'll need to know what the grid dimensons (nlats,nlons) are. >>>>> >>>>> -Jeff >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> _______________________________________________ >>>>> Matplotlib-users mailing list >>>>> Mat...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>>>> >>>>> >>>>> >>>>> >>>> >>>> >>> ------------------------------------------------------------------------------ >>> _______________________________________________ >>> Matplotlib-users mailing list >>> Mat...@li... >>> https://lists.sourceforge.net/lists/listinfo/matplotlib-users >>> >>> >>> >> >> > > > ------------------------------------------------------------------------------ > _______________________________________________ > Matplotlib-users mailing list > Mat...@li... > https://lists.sourceforge.net/lists/listinfo/matplotlib-users > > -- View this message in context: http://www.nabble.com/Plotting-NOAA-data...-tp21139727p21244624.html Sent from the matplotlib - users mailing list archive at Nabble.com. |