I have a netcdf file containing monthly data, let say it has 10 years (120 months). I wanted to generated a new netcdf file containing average value of all the 120 months in such a way that the output netcdf file should contain the same no of months (120) instead of one month value by repeating this averaged calculated value (120 times) and produce same time length as the input file. In short my output file will have 120 time steps as the input but each time step will have the same constant averaged value.
Thanks,
Dogar
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One thing I want to specify is that I want time average only of the input file fields and the rest dimensions of the input variable should remain same as in the output file.
Thanks,
Mubashar
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
1. Use ncwa -x -v time -a time in.nc out.nc to get the time average
2. Add the original time coordinate to the output file with
ncks -A -v time in.nc out.nc
3. copy the averaged array by multiplying them and imposing the dimensional order
ncap2 -s 'copy=time*original_average' out.nc out2.nc
the exact meanings of all these switches are in the manual.
good luck,
cz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After applying the first command for time average as follows:
ncwa -x -v time -a time asmsw.nc out.nc !(Note asmsw.nc has 10 years i.e. 120 months which will be averaged to out.nc file)
The second command gives following error:
ncks -A -v time ncks -A -v time out.nc
ncks: ERROR attempt to write 1 dimensional input variable time to 0 dimensional space in output file.
HINT: When using -A (append) option, all appended variables must be the same rank in the input file as in the output file. The ncwa operator is useful at ridding variables of extraneous (size = 1) dimensions. See how at http://nco.sf.net/nco.html#ncwa
Any suggestion.
Thanks,
Mubashar
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
After applying the first command for time average as follows:
ncwa -x -v time -a time asmsw.nc out.nc !(Note asmsw.nc has 10 years i.e. 120 months which will be averaged to out.nc file)
The second command gives following error:
ncks -A -v time ncks -A -v time asmsw.nc out.nc
ncks: ERROR attempt to write 1 dimensional input variable time to 0 dimensional space in output file. HINT: When using -A (append) option, all appended variables must be the same rank in the input file as in the output file. The ncwa operator is useful at ridding variables of extraneous (size = 1) dimensions. See how at http://nco.sf.net/nco.html#ncwa
Any suggestion.
Thanks, Mubashar
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
out.nc appears to contain the average value of time as a scalar.
you need to remove it before step 2.
try this instead for step 1:
ncwa -C -x -v time -a time asmsw.nc out.nc
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> Do I need to specify my variables name in place of original_average?
yes. and you must repeat the command for every variable you wish to "broadcast".
asmsw_b01_copy=time*asmsw_b01;
et cetera
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks for your continued help. After applying this ncap2 command on first variable successfully I got error on second variable (error is given at the end). One more thing, as you can see from the ncdump given above I have around 18 variables in this file and in total I have 6 different files (asmsw.nc, asmlw.nc, ssasw.nc ssalw.nc etc) and each contains around 18 variables. So applying this command manually on each variable of each file will be time consuming. Is there any way to automatically mention all variables and append them. The error I got while applying ncap2 command on 2nd variable is given below:
ncap2: out2.nc exists--`e'xit, `o'verwrite (i.e., delete existing file), or `a'ppend (i.e., replace duplicate variables in and add new variables to existing file) (e/o/a)? a
ncap2: WARNING dimension "time" is already defined in out2.nc
ncap2: WARNING dimension "pfull" is already defined in out2.nc
ncap2: WARNING dimension "lat" is already defined in out2.nc
ncap2: WARNING dimension "lon" is already defined in out2.nc
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_def_var()
nco_err_exit(): ERROR Error code is -42. Translation into English with nc_strerror(-42) is "NetCDF: String match to name in use"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
Plz suggest.
Thanks,
Mubashar
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One more thing, we just need to add the time as a dimention with all the variables, but in the following command why we are multiplying time with the variable. It seems that it will change the origninally averaged value of the variable.
there is no way to automate this with ncap2. you can string all the commands together with semicolons and do it in one command if you wish. you would know this if you read the manual.
cz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thanks. I will apply your instructions. one last question:
I have 10 years monthly data in a netcdf file (asmsw.nc). I applied cdo and nco command on the same file to find time average and checked the difference of the averaged variable from both methods in GrADS. The difference is not exactly zero, although it is very very small (10^-7). Why is it so? You could test it at your level for any variable of your netcdf file to get an idea.
ncwa -x -v time -a time asmsw.nc nco_tavg.nc
cdo timmean asmsw.nc cdo_tavg.nc
OR
cdo timavg asmsw.nc cdo_tavg.nc
Thanks,
Mubashar
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
because CDO converts your "float" data to "double" before averaging/arithmetic, while NCO preserves the "float" type through all arithmetic, there will be roundoff differences.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Dear NCO experts,
I have a netcdf file containing monthly data, let say it has 10 years (120 months). I wanted to generated a new netcdf file containing average value of all the 120 months in such a way that the output netcdf file should contain the same no of months (120) instead of one month value by repeating this averaged calculated value (120 times) and produce same time length as the input file. In short my output file will have 120 time steps as the input but each time step will have the same constant averaged value.
Thanks,
Dogar
Hi NCO users,
One thing I want to specify is that I want time average only of the input file fields and the rest dimensions of the input variable should remain same as in the output file.
Thanks,
Mubashar
1. Use ncwa -x -v time -a time in.nc out.nc to get the time average
2. Add the original time coordinate to the output file with
ncks -A -v time in.nc out.nc
3. copy the averaged array by multiplying them and imposing the dimensional order
ncap2 -s 'copy=time*original_average' out.nc out2.nc
the exact meanings of all these switches are in the manual.
good luck,
cz
Dear Prof Zender,
After applying the first command for time average as follows:
ncwa -x -v time -a time asmsw.nc out.nc !(Note asmsw.nc has 10 years i.e. 120 months which will be averaged to out.nc file)
The second command gives following error:
ncks -A -v time ncks -A -v time out.nc
ncks: ERROR attempt to write 1 dimensional input variable time to 0 dimensional space in output file.
HINT: When using -A (append) option, all appended variables must be the same rank in the input file as in the output file. The ncwa operator is useful at ridding variables of extraneous (size = 1) dimensions. See how at http://nco.sf.net/nco.html#ncwa
Any suggestion.
Thanks,
Mubashar
Dear Prof Zender,
After applying the first command for time average as follows:
ncwa -x -v time -a time asmsw.nc out.nc !(Note asmsw.nc has 10 years i.e. 120 months which will be averaged to out.nc file)
The second command gives following error:
ncks -A -v time ncks -A -v time asmsw.nc out.nc
ncks: ERROR attempt to write 1 dimensional input variable time to 0 dimensional space in output file. HINT: When using -A (append) option, all appended variables must be the same rank in the input file as in the output file. The ncwa operator is useful at ridding variables of extraneous (size = 1) dimensions. See how at http://nco.sf.net/nco.html#ncwa
Any suggestion.
Thanks, Mubashar
out.nc appears to contain the average value of time as a scalar.
you need to remove it before step 2.
try this instead for step 1:
ncwa -C -x -v time -a time asmsw.nc out.nc
Dear Prof Zender,
Upto step 2 it works perfectly fine. What I am doing wrong in the 3rd step?
ncap2 -s 'copy=time*original_average' out.nc out2.nc
ncap2: WARNING prs_cls::ncap_var_init(): Unable to find variable original_average in out.nc or out2.nc
Do I need to specify my variables name in place of original_average? Following is the ncdump -h asmsw.nc file:
time = UNLIMITED ; // (120 currently)
pfull = 40 ;
lat = 90 ;
lon = 1 ;
variables:
float asmsw_b01(time, pfull, lat, lon) ;
asmsw_b01:long_name = "test data" ;
asmsw_b01:units = "unitless" ;
float asmsw_b02(time, pfull, lat, lon) ;
asmsw_b02:long_name = "test data" ;
asmsw_b02:units = "unitless" ;
float asmsw_b03(time, pfull, lat, lon) ;
asmsw_b03:long_name = "test data" ;
asmsw_b03:units = "unitless" ;
float asmsw_b04(time, pfull, lat, lon) ;
asmsw_b04:long_name = "test data" ;
asmsw_b04:units = "unitless" ;
float asmsw_b05(time, pfull, lat, lon) ;
asmsw_b05:long_name = "test data" ;
asmsw_b05:units = "unitless" ;
float asmsw_b06(time, pfull, lat, lon) ;
asmsw_b06:long_name = "test data" ;
asmsw_b06:units = "unitless" ;
float asmsw_b07(time, pfull, lat, lon) ;
asmsw_b07:long_name = "test data" ;
asmsw_b07:units = "unitless" ;
float asmsw_b08(time, pfull, lat, lon) ;
asmsw_b08:long_name = "test data" ;
asmsw_b08:units = "unitless" ;
float asmsw_b09(time, pfull, lat, lon) ;
asmsw_b09:long_name = "test data" ;
asmsw_b09:units = "unitless" ;
float asmsw_b10(time, pfull, lat, lon) ;
asmsw_b10:long_name = "test data" ;
asmsw_b10:units = "unitless" ;
float asmsw_b11(time, pfull, lat, lon) ;
asmsw_b11:long_name = "test data" ;
asmsw_b11:units = "unitless" ;
float asmsw_b12(time, pfull, lat, lon) ;
asmsw_b12:long_name = "test data" ;
asmsw_b12:units = "unitless" ;
float asmsw_b13(time, pfull, lat, lon) ;
asmsw_b13:long_name = "test data" ;
asmsw_b13:units = "unitless" ;
float asmsw_b14(time, pfull, lat, lon) ;
asmsw_b14:long_name = "test data" ;
asmsw_b14:units = "unitless" ;
float asmsw_b15(time, pfull, lat, lon) ;
asmsw_b15:long_name = "test data" ;
asmsw_b15:units = "unitless" ;
float asmsw_b16(time, pfull, lat, lon) ;
asmsw_b16:long_name = "test data" ;
asmsw_b16:units = "unitless" ;
float asmsw_b17(time, pfull, lat, lon) ;
asmsw_b17:long_name = "test data" ;
asmsw_b17:units = "unitless" ;
float asmsw_b18(time, pfull, lat, lon) ;
asmsw_b18:long_name = "test data" ;
asmsw_b18:units = "unitless" ;
float lat(lat) ;
lat:long_name = "latitude" ;
lat:units = "degrees_north" ;
float lon(lon) ;
lon:units = "degrees_east" ;
lon:long_name = "longitude" ;
float pfull(pfull) ;
pfull:positive = "up" ;
pfull:long_name = "pressure" ;
pfull:units = "hPa" ;
float time(time) ;
time:units = "days since 1800-01-01 00:00:00" ;
time:calendar = "julian" ;
// global attributes:
:history = "Sun Feb 10 16:55:50 2013: ncks -d time,1680,1799 ../asmsw.nc asmsw_1990-1999.nc" ;
:NCO = "4.0.5" ;
}
Thanks,
Mubashar
Thanks,
Mubashar
> Do I need to specify my variables name in place of original_average?
yes. and you must repeat the command for every variable you wish to "broadcast".
asmsw_b01_copy=time*asmsw_b01;
et cetera
Dear Prof Zender,
Thanks for your continued help. After applying this ncap2 command on first variable successfully I got error on second variable (error is given at the end). One more thing, as you can see from the ncdump given above I have around 18 variables in this file and in total I have 6 different files (asmsw.nc, asmlw.nc, ssasw.nc ssalw.nc etc) and each contains around 18 variables. So applying this command manually on each variable of each file will be time consuming. Is there any way to automatically mention all variables and append them. The error I got while applying ncap2 command on 2nd variable is given below:
ncap2 -s 'asmsw_b02=time*asmsw_b02' out.nc out2.nc
ncap2: out2.nc exists--`e'xit, `o'verwrite (i.e., delete existing file), or `a'ppend (i.e., replace duplicate variables in and add new variables to existing file) (e/o/a)? a
ncap2: WARNING dimension "time" is already defined in out2.nc
ncap2: WARNING dimension "pfull" is already defined in out2.nc
ncap2: WARNING dimension "lat" is already defined in out2.nc
ncap2: WARNING dimension "lon" is already defined in out2.nc
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_def_var()
nco_err_exit(): ERROR Error code is -42. Translation into English with nc_strerror(-42) is "NetCDF: String match to name in use"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)
Plz suggest.
Thanks,
Mubashar
Hi Prof,
One more thing, we just need to add the time as a dimention with all the variables, but in the following command why we are multiplying time with the variable. It seems that it will change the origninally averaged value of the variable.
ncap2 -s 'asmsw_b02=time*asmsw_b02' out.nc out2.nc
Thanks,
Mubashar
1. you are defining a new variable so you have to change the name
ncap2 -s 'asmsw_b02_new=time*asmsw_b02' out.nc out2.nc
2. you want to multiply by a variable whose value is one but whose dimension is time so do
ncap2 -s 'one=time*0.0+1;asmsw_b02_new=one*asmsw_b02' out.nc out2.nc
there is no way to automate this with ncap2. you can string all the commands together with semicolons and do it in one command if you wish. you would know this if you read the manual.
cz
Dear Prof,
Thanks. I will apply your instructions. one last question:
I have 10 years monthly data in a netcdf file (asmsw.nc). I applied cdo and nco command on the same file to find time average and checked the difference of the averaged variable from both methods in GrADS. The difference is not exactly zero, although it is very very small (10^-7). Why is it so? You could test it at your level for any variable of your netcdf file to get an idea.
ncwa -x -v time -a time asmsw.nc nco_tavg.nc
cdo timmean asmsw.nc cdo_tavg.nc
OR
cdo timavg asmsw.nc cdo_tavg.nc
Thanks,
Mubashar
because CDO converts your "float" data to "double" before averaging/arithmetic, while NCO preserves the "float" type through all arithmetic, there will be roundoff differences.