NCO netCDF Operators / Discussion / Help: Zonally Defined Temperature Average

Samuel Moore - 2016-02-24

Hi Guys,

I apologise I am very new to this but I currently have a global temperature data file in .nc format that I have successfully split removing the irrelevant past data. I am looking to put the temperature data into a model that is zonally split globally into 288 boxes 15°x15° (lat/lon). However, the temperature data is currently recorded globally across 2°x2° (lat/lon). I want to average this temperature data into 15°x15° (lat/lon) globally for the input into my model. Currently I have successfully averaged the temperature across defined latitudes and longitudes using the command line:-

ncwa -a lat,lon -d lat,-20.,20. -d lon,120.,270. input.nc output.nc

I would appreciate any help in towards the correct command line that will obtain an average every 15°x15° globally. Many thanks.

Last edit: Samuel Moore 2016-02-24

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-02-24

suggest you create a 15x15 global grid (use ncks if you don't yet have one) then do a conservative remap from the source to that grid with ncremap:
ncremap -i input.nc -d 15x15.nc -o output.nc
where 15x15.nc is an existing file (that will not be overwritten) with the desired 15x15 grid.
cz

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-02-24

Thank you for you help. I apologise for the repeated questions but I have no idea how to go about creating a 15x15 global grid using ncks?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-02-24

http://nco.sf.net/nco.html#ncremap
http://nco.sf.net/nco.html#grid

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-02-24

ncks --rgr grd_ttl='Equi-Angular grid 12x24' --rgr grid=12x24_SCRIP.nc --rgr latlon=12,24 --rgr lat_typ=uni --rgr lon_typ=grn_ctr ~/foo.nc ~/foo.nc

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-02-24

Hi Charlie,

I just put together the command-line for creating the grid before I realised your reply. The only thing I do not understand is the input.nc file for the grid? What file do I use? My .nc data file or another which I need to download.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-02-24

The file foo.nc in my grid command can be any valid netCDF file. It is a dummy argument that is not used.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Thanks Charlie. Slowly understanding it. I am now using the regridding command line:
ncremap -i TempData.nc -d GeneratedGrid.nc -o Output.nc
However, I am recieving the following error message:

Started processing at Wed 24 Feb 2016 21:40:02 GMT.
NCO version is 4.5.5
Destination grid will be inferred from data-file
ncks: ERROR nco_grd_nfr() unable to identify latitude and/or longitude dimension and/or column dimension.
ncremap: ERROR Failed to generate destination grid. Debug this:
ncks -O --no_tmp_fl --gaa rgr_script=ncremap --gaa rgr_hostname=Sams-MacBook-Pro.local --gaa rgr_version=4.5.5 --hdr_pad=1000 --rgr nfr=y --rgr grid=/Users/sammoore/ncremap_tmp_grd_dst.nc /Users/sammoore/Desktop/GRIDTEST2.nc /Users/sammoore/ncremap_grd_tmp.nc

Charlie Zender - 2016-02-24

use -g (for 'gridfile') not -d (for 'datafile')

ncremap -i TempData.nc -g GeneratedGrid.nc -o Output.nc
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Apologies thats an obvious error. With the new command line I'm recieving this error:

Started processing at Wed 24 Feb 2016 22:17:27 GMT.
NCO version is 4.5.5
Destination grid supplied by user
Weight-generation type: esmf
ESMF's ESMF_RegridWeightGen will generate map-file
Algorithm used to generate weights in map-file is: bilinear
Input #00: /Users/sammoore/Desktop/Temp1995-200MASTER.nc
Grid(src): /Users/sammoore/ncremap_tmp_grd_src.nc
Grid(dst): /Users/sammoore/Desktop/GRIDTEST2.nc
/opt/local/bin/ncremap: line 826: ESMF_RegridWeightGen: command not found
ncremap: ERROR Failed to generate map-file. Debug this:
ESMF_RegridWeightGen -s /Users/sammoore/ncremap_tmp_grd_src.nc -d /Users/sammoore/Desktop/GRIDTEST2.nc -w /Users/sammoore/ncremap_tmp_map_esmf_bilinear.nc --method bilinear --ignore_unmapped > /dev/null
ncremap: HINT When ESMF fails to generate map-files, it often puts additional debugging information in the file named PET0.RegridWeightGen.Log in the invocation directory (/Users/sammoore)

Thankyou again for all your help!!

Charlie Zender - 2016-02-24

Please read the manual on ncremap. It will tell you to install ESMF_RegridWeightGen.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-02-26

Hi Charlie,

Sorry for the delayed response I have had trouble building the ESMF software in the last couple of days. However, I have managed to overcome this but am still receiving an error message.

/opt/local/bin/ncremap: line 826: ESMF_RegridWeightGen: command not found ncremap: ERROR Failed to generate map-file. Debug this: ESMF_RegridWeightGen -s /Users/sammoore/Desktop/esmf/ncremap_tmp_grd_src.nc -d /Users/sammoore/Desktop/GRIDTEST2.nc -w /Users/sammoore/Desktop/esmf/ncremap_tmp_map_esmf_bilinear.nc --method bilinear --ignore_unmapped > /dev/null

Does this mean that the ESMF directory is not stored in the correct place. I have tried adding the ESMF folder to my $PATH with the same message appearing. I am now getting .nc files in the ESMF directory called 'ncremap_tmp_grd_src.nc' and 'ncremap_grd-tmp.nc'. However, these files are not contain the plottable georeferenced variable. Any insight into where I am going wrong would be great. Thank you.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-02-26

Your $PATH does not yet lead to ESMF_RegridWeightGen. If your shell cannot find it, ncremap cannot use it. Solve that and ncremap should complete successfully.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Hi Charlie,

I am still having issues running this command, previously I didn't have the librarys stored in the correct place. I have solved this issue but I am now recieving a new error:

Sams-MacBook-Pro:~ sammoore$ ncremap -i /Users/sammoore/Desktop/Temp1995-200MASTER.nc  -g /Users/sammoore/Desktop/GRIDTEST2.nc  -o /Users/sammoore/Desktop/Test.nc
Started processing at Fri  4 Mar 2016 17:03:31 GMT.
NCO version is 4.5.5
Destination grid supplied by user
Weight-generation type: esmf
ESMF's ESMF_RegridWeightGen will generate map-file
Algorithm used to generate weights in map-file is: bilinear
Input #00: /Users/sammoore/Desktop/Temp1995-200MASTER.nc
Grid(src): /Users/sammoore/ncremap_tmp_grd_src.nc
Grid(dst): /Users/sammoore/Desktop/GRIDTEST2.nc
dyld: lazy symbol binding failed: Symbol not found: __gfortran_transfer_integer_write
  Referenced from: /usr/local/bin/ESMF_RegridWeightGen
  Expected in: /usr/local/lib/libgfortran.3.dylib

dyld: Symbol not found: __gfortran_transfer_integer_write
  Referenced from: /usr/local/bin/ESMF_RegridWeightGen
  Expected in: /usr/local/lib/libgfortran.3.dylib

/opt/local/bin/ncremap: line 826:  4998 Trace/BPT trap: 5       ESMF_RegridWeightGen -s /Users/sammoore/ncremap_tmp_grd_src.nc -d /Users/sammoore/Desktop/GRIDTEST2.nc -w /Users/sammoore/ncremap_tmp_map_esmf_bilinear.nc --method bilinear --ignore_unmapped > /dev/null
ncremap: ERROR Failed to generate map-file. Debug this:
ESMF_RegridWeightGen -s /Users/sammoore/ncremap_tmp_grd_src.nc -d /Users/sammoore/Desktop/GRIDTEST2.nc -w /Users/sammoore/ncremap_tmp_map_esmf_bilinear.nc --method bilinear --ignore_unmapped > /dev/null
ncremap: HINT When ESMF fails to generate map-files, it often puts additional debugging information in the file named PET0.RegridWeightGen.Log in the invocation directory (/Users/sammoore)

I was hoping you could help me in solving this issue?
Another option, I know it is a long shot but is a possible option for you to post a compressed file containing the working library/bin for ESMF? That would be extremely helpful.

Sam Moore.

Charlie Zender - 2016-03-04

This appears to be a problem with ESMF_RegridWeightGen being unable to find a library. On my MBP I use MacPorts and ESMF installs painlessly with one command 'port install esmf' and works fine with ncremap. Otherwise, maybe contact ESMF and they'll have an install solution for you. Sorry this has not been easy for you. Accurate regridding is a fairly involved process.
cz

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-03-04

I also have MacPorts installed and used a simular command 'sudo port install esmf' because 'port install esmf' did not work. This has entirely removed the error with the libraries. However, I am now recieving a 'debug this:' error:

ncremap: ERROR Failed to generate map-file. Debug this: ESMF_RegridWeightGen -s /Users/sammoore/ncremap_tmp_grd_src.nc -d /Users/sammoore/Desktop/GRIDTEST2.nc -w /Users/sammoore/ncremap_tmp_map_esmf_bilinear.nc --method bilinear --ignore_unmapped > /dev/null ncremap: HINT When ESMF fails to generate map-files, it often puts additional debugging information in the file named PET0.RegridWeightGen.Log in the invocation directory (/Users/sammoore)

Unlike all the other runs this is now creating the file in the specified directory and I am receiving this debugging error when opening the 'PET0.RegridWeightGen.Log' file:

20160304 192415.795 INFO PET0 Running with ESMF Version 6.3.0rp1
20160304 192415.801 ERROR PET0 ESMF_IOScrip.F90:145 ESMF_ScripInq netCDF Status Return Error
20160304 192415.801 ERROR PET0 ESMF_RegridWeightGen.F90:619 ESMF_RegridWeightGenFile Failure - Internal subroutine call returned Error

Any ideas?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-03-04

Remember, if the destination file is a grid use '-g', if it's data use '-d'. If you have done that correctly and it doesn't work then the next step is to attach the files,
/Users/sammoore/Desktop/Temp1995-200MASTER.nc -g /Users/sammoore/Desktop/GRIDTEST2.nc
cz

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-03-04

I believe I am doing this correctly the code I am using is:

ncremap -i /Users/sammoore/Desktop/Temp1995-200MASTER.nc -g /Users/sammoore/Desktop/GRIDTEST2.nc -o /Users/sammoore/Desktop/Out.nc

Is this correct?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-03-04

As an update I have tried -m, -g and -d in the place of the generated gridfile with no luck..

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-03-04

like i said, attach the files.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-03-04

Here are the attatched files. Thank you.

GRIDTEST2.nc

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Charlie Zender - 2016-03-04

The grid file in the following command is 12x24_SCRIP.nc. Use that. Not GRIDTEST2.nc, which is empty.
ncks --rgr grd_ttl='Equi-Angular grid 12x24' --rgr grid=12x24_SCRIP.nc --rgr latlon=12,24 --rgr lat_typ=uni --rgr lon_typ=grn_ctr ~/foo.nc ~/foo.nc

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Samuel Moore - 2016-03-04

It's worked!!!!!! Thank you Charlie, I know this has been a long and time consuming thread for you but I really do appreciate all your time and help, can't thank you enough you've been brilliant!!

Sam.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Zonally Defined Temperature Average

Command-line operators for netCDF and HDF files

Forums

Help

Zonally Defined Temperature Average

Zonally Defined Temperature Average

Command-line operators for netCDF and HDF files

Forums

Help

Zonally Defined Temperature Average document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Zonally Defined Temperature Average