Charlie Zender posted a comment on discussion Open Discussion

In this first step we use ncwa to average-out the time dimension from the target variable (emi_co). We also extract it's spatial coordinates (which will not change) into a new file, out.nc. The -C prevent associated coordinate variables (like time) from being pulled into the output file. ncwa -a time -C -v emi_co,lat,lon in.nc out.nc In this second step we append the three time-containing coordinate variables from the original file to the output of the previous command. ncks -A -v date,datesec,time...

bartok posted a comment on discussion Open Discussion

Can you explain the steps above? Thanks. Blanka

bartok posted a comment on discussion Open Discussion

Hi, Have you managed to remove the time dimension? I have the same problem, just iIwould like to remove the second dimesion from lat, lon variables. float lat(lat, lon) ; lat:standard_name = "lat" ; lat:long_name = "lat" ; lat:units = "degrees_north" ; float lon(lat, lon) ; lon:standard_name = "lon" ; lon:long_name = "lon" ; lon:units = "degrees_east" ; Can you explain the steps above? Thanks. Blanka

Noah modified a comment on discussion Open Discussion

Thank you so much Charlie! After regridding all the files with ncremap -d smallest_grid file_x_name file_x_name_re, making all variable data types uniform with ncap2 -O -s 'var_name=double(var_name);' file_name file_name and removing the time_bnds variable, everything worked perfectly!

Noah modified a comment on discussion Open Discussion

Thank you so much Charlie! After regridding all the files with ncremap -d smallest_grid file_x_name file_x_name, making all variable data types uniform with ncap2 -O -s 'var_name=double(var_name);' file_name file_name and removing the time_bnds variable, everything worked perfectly!

Noah posted a comment on discussion Open Discussion

Thank you so much Charlie! After regridding all the files with ncap2 -O -s 'var_name=double(var_name);' file_name file_name and removing the time_bnds variable, everything worked perfectly!

NCO netCDF Operators released /nco-5.2.7.tar.gz

henry Butowsky posted a comment on ticket #30

Either the dim ens_member is already defined as 0 (UNLIMITED) in the input file or your shell script is setting it to 0.

Husain Najafi modified a comment on ticket #30

Thanks Henry, Now ens_member = UNLIMITED ; // (0 currently)) is there any way to define this dimension as fixes size? I am getting ERROR nco_cnk_sz_set_trv() reports variable pre_new has dim_sz == 0L for non-record dimension ens_member. This should not occur and it will cause chunking to fail...

Husain Najafi posted a comment on ticket #30

Thanks Henry, Now ens_member = UNLIMITED ; // (0 currently)) is there any way to define this dimension as fixes size?

henry Butowsky posted a comment on ticket #30

its a bit tricky because you are redefining the variable "pre" that is already in input. try something like. defdim("ens",40 ); pre_new[$time,$latitude,$longitude,$ens]=pre; ...Henry

Husain Najafi modified a comment on ticket #30

Dear Henry, Thanks for the tip. I used the follwoing script cat << EOF > test.nco defdim("ens_member", $ens); pre[\$time, \$latitude, \$longitude, \$ens_member] = pre; EOF ($ens is expanded from bash as a variable as it changes from 00 to 20) . With this, ncap2 passed without any error from the bash script. But then when I look into the output file, no displayable variables found. I also tried defdim("ens",40 ), but it also did not work out.

Husain Najafi posted a comment on ticket #30

Dear Henry, Thanks for the tip. I used the follwoing script cat << EOF > test.nco defdim("ens_member", $ens); pre[\$time, \$latitude, \$longitude, \$ens_member] = pre; EOF ($ens is expanded from bash as a variable as it changes from 00 to 20) . With this, ncap2 passed without any error from the bash script. But then when I look into the output file, no displayable variables found. I also tried defdim("ens",40 ), but it also does did not work out.

henry Butowsky modified a comment on ticket #30

I suggest you get the command working in a script first then once its working move it to the command line. eg test.nco defdim("ens",40 ); pre[$time,$latitude,$longitude,$ens]=pre; then run the above script with the command ncap2 -S test.nco in.nc out.nc ...Henry

henry Butowsky posted a comment on ticket #30

I suggest you get the command working in a script first then once its working move it to the command line. eg test.nco defdim("ens",40 ); pre[$time,$latitude,$longitude,$ens]=pre then run the above script with the command ncap2 -S test.nco in.nc out.nc ...Henry

Husain Najafi created ticket #30

ncap2 on shell gives error

Charlie Zender posted a comment on discussion Open Discussion

Every file needs to be the same shape, and have the same variables with your command. However MIROC-ES2L.nc, contains time_bnds but CanESM5-1.nc does not. Furthermore, CNRM-ESM2-1.nc is four times the resolution of CanESM5-1.nc. These files need to be regridded and regularized before they can be averaged.

Noah posted a comment on discussion Open Discussion

I am using: ncra MIROC-ES2L.nc GISS-E2-1-G.nc CNRM-ESM2-1.nc EC-Earth3-AerChem.nc IPSL-CM6A-LR-INCA.nc CanESM5-1.nc CanESM5-CanOE.nc GISS-E2-1-H.nc GISS-E2-2-G.nc MIROC6.nc MRI-ESM2-0.nc CanESM5.nc output.nc -O To average the attached files on a Linux machine, however when I try this I get the following error: ERROR: nco_inq_varid() reports requested variable "time_bnds" is not defined in file nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco_inq_varid()...

NCO netCDF Operators released /nco-5.2.6.tar.gz

NCO netCDF Operators released /nco-5.2.5.tar.gz

Koichi Sakaguchi posted a comment on discussion Help

Hi Charlie, thank you very much for the reply, and for quickly fixing the problem! I tried the version under ...bin_perlmutter/ncclimo and the--exclude option worked for me. Being able to use this exclude option helps me debug another (original) problem I am having while creating a timeseries of global averages. I may report in another post if I can't resolve it myself. Best regards, Koichi

Charlie Zender posted a comment on discussion Help

Hello Again, I noticed you are at PNNL and work with E3SM colleagues so I bumped-up the priority of this feature. As of today and NCO 5.2.5-alpha03, ncclimo supports --exclude in timeseries mode. You can test the new executable in /global/homes/z/zender/bin_perlmutter/ncclimo. Please LMK how it works for you. Charlie

Charlie Zender posted a comment on discussion Help

Hello Koichi, Thank you for brining this to my attention. Unfortunately this is an undocumented limitation of the --excludeoption for ncclimo, and there is no simple workaround. I just updated the manual to mention it. I have added removing this limitation to the general feature request list. However, it will require deep dive on my part. If your project is DOE or NASA-funded and this would be useful, then send me the details and I can prioritize it on the feature list. Charlie

Koichi Sakaguchi posted a comment on discussion Help

Dear NCO developers and community, I use ncclimo binary on Perlumtter avaiable on /global/homes/z/zender/bin_perlmutter. I am trying to create global average timeseries from monthly mean cam output using ncclimo. When I try the option to change the variable list into exclusion, I get the following error(s): $ ncclimo --exclude -p serial -v ABSORB --split -m cam --rgn_avg -s ${styr} -e ${edyr} -c ${icase} -i ${indir} -o ${outdir} ERROR: Unrecognized option --exclude_variables ERROR: Unrecognized option...

Gus Correa posted a comment on discussion Help

Oops, it should be export PATH=/path/to/your/conda_installation/bin:$PATH not set PATH=/path/to/your/conda_installation/bin:$PATH On Tue, Jun 4, 2024 at 3:16 PM Gus Correa gus@ldeo.columbia.edu wrote: Hi Sam May I ask why you want to install NCO from source? In my experience, installing software from source on Ginsburg hits missing libraries very often, and it is a hassle to build them all in your home directory. (It would be much easier for the superuser to install them from packages, but that is...

Gus Correa posted a comment on discussion Help

Hi Sam May I ask why you want to install NCO from source? In my experience, installing software from source on Ginsburg hits missing libraries very often, and it is a hassle to build them all in your home directory. (It would be much easier for the superuser to install them from packages, but that is another story.) A reasonable alternative, which will get you the latest (or close to the latest) NCO is to use the Anaconda conda installer. There are instructions in the link below, on https://nco.sourceforge.net/#RTFM...

Sam Chester posted a comment on discussion Help

I am trying to install NCO on a HPC. I configured it successfully but make fails. make all-recursive make[1]: Entering directory '/burg/home/sc5122/nco' Making all in data make[2]: Entering directory '/burg/home/sc5122/nco/data' /cm/shared/apps/netcdf/gcc/64/4.7.3/bin/ncgen -k netCDF-4 -lb -o "file://in_zarr4#mode=nczarr,file" in_zarr4.cdl ncgen: Permission denied (ncgen:63) make[2]: [Makefile:657: in_zarr4] Error 1 (ignored) ln -s -f in_zarr4 h0001 ln -s -f in_zarr4 h0002 ln -s -f in_zarr4 h0003...

Charlie Zender posted a comment on discussion Help

Building NCO from source is non-trivial. It looks to me like you are close. You will need to install or point to the missing libraries libexpat.a and libcurl.a to succeed.

Sam Chester posted a comment on discussion Help

I am trying to install NCO on a HPC. I configured it successfully but make fails. make all-recursive make[1]: Entering directory '/burg/home/sc5122/nco' Making all in data make[2]: Entering directory '/burg/home/sc5122/nco/data' /cm/shared/apps/netcdf/gcc/64/4.7.3/bin/ncgen -k netCDF-4 -lb -o "file://in_zarr4#mode=nczarr,file" in_zarr4.cdl ncgen: Permission denied (ncgen:63) make[2]: [Makefile:657: in_zarr4] Error 1 (ignored) ln -s -f in_zarr4 h0001 ln -s -f in_zarr4 h0002 ln -s -f in_zarr4 h0003...

Xinyuan Lv posted a comment on discussion Help

Dear Charlie, I am a student of the First Institute of Oceanography, Ministry of Natural Resources, China. As far as I know, my advisor is not funded by DOE or NASA. Sorry for the trouble.

Charlie Zender posted a comment on discussion Help

What University are you a student at and does any of your or your advisor's funding come from DOE or NASA? If so, what projects? Knowing this will help me determine how much time I can put into addressing the issues you have encountered.

Xinyuan Lv posted a comment on discussion Help

Dear Charlie, Thanks for the response! About Q1 There's really no other data in the folder. If I use -i, I will be prompted that I cannot find the corresponding file, such asncclimo: ERROR Unable to find required input file /media/lxyccyethz/EXTERNAL_USB/daytest/mpaso.hist.am.timeSeriesStatsMonthly.0001-01-01.nc, Because my file name is like mine-800-GMPAS-JRA1p5_TL319_oARRM60to10-intel2023.mpaso.hist.am.DailyTSP.0001-01-01.nc. Here are the run commands and errors using -c name and -i (you can see...

Angela Bernaldez posted a comment on discussion Open Discussion

Thank you very much, Charlie. I have just tried running that and can confirm it does work for me! Unstructured file has successfully been remapped into regular grid. Thank you for your help and time.

Charlie Zender posted a comment on discussion Open Discussion

Try this: ncremap --pdq=Time,nVertLevels,nIsoLevelsT,nIsoLevelsZ,nCells -m map_mpas.nc diag.2023-10-26_13.00.00.nc data_rgr.nc LMK how it works. Charlie

Charlie Zender posted a comment on discussion Help

Please let me know where you are employed so I can determine whether putting more time into this is justified. In the meantime, the -iswitch causes ncclimo to try to ingest every file in the input directory. Perhaps you have non-input files in that directory? If so, I suggest you pipe only the desired files as input to ncclimo. For dimension permutation, it seems like your file has dimensions (Depth?) that ncclimo does not expect. You can use ncpdq or the --pdq_opt option to ncremap to rectify t...

Xinyuan Lv modified a comment on discussion Help

Dear Charlie, I have just discovered a new problem. I am trying to use nco-5.2.4 to horizontally interpolate the data after MPAS_analysis vertical interpolation, but the following error is reported: ➜ ncremap -6 -P mpaso -m /D/Data/lxyccyethz/E3SM_Diags/map720and1440.nc /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4.0002-01-01.nc /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4_to_720and1440.0002-01-01.nc Input #00: /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4.0002-01-01.nc...

Xinyuan Lv modified a comment on discussion Help

Xinyuan Lv posted a comment on discussion Help

Dear Charlie, I have just discovered a new problem. I am trying to use nco-5.2.4 to horizontally interpolate the data after MPAS_analysis vertical interpolation, but the following error is reported: ➜ ncremap -6 -P mpaso -m /D/Data/lxyccyethz/E3SM_Diags/map720and1440.nc /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4.0002-01-01.nc /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4_to_720and1440.0002-01-01.nc Input #00: /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4.0002-01-01.nc...

Xinyuan Lv posted a comment on discussion Help

Dear Charlie, I have just discovered a new problem. I am trying to use nco-5.2.4 to horizontally interpolate the data after MPAS_analysis vertical interpolation, but the following error is reported: ➜ ncremap -6 -P mpaso -m /D/Data/lxyccyethz/E3SM_Diags/map720and1440.nc /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4.0002-01-01.nc /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4_to_720and1440.0002-01-01.nc Input #00: /D/Data/lxyccyethz/mpas_analysis/huaweibook/mpaso.T_S_to_EN4.0002-01-01.nc...

Angela Bernaldez posted a comment on discussion Open Discussion

Hello Charlie, Thank you very much for your help. I have tried the command line you provided and it does work for the case of MSLP, but when I try with the big file containing several variables, some of them being multi-level ones, I keep having the issue described at the beginning. I have uploaded the whole diagnostic file (860.2 MB) in the following link: https://zenodo.org/records/11401287?token=eyJhbGciOiJIUzUxMiJ9.eyJpZCI6ImZjY2I3ZmZlLWE5NzAtNGFlYy1hYWIxLWVmMjA2Y2IzNmVlYiIsImRhdGEiOnt9LCJyYW5kb20iOiJmN2MwYWNjYWUyYTE5Y2RmNGVjZDc3N2Q5YmMxZmU3ZSJ9.Ual6eKaHX74ZMVfjxv9z79HKVYvA6WfkMGrxQS-B8Oh-JSVzItVfrM0PXdrIsO6NV8qSidkC91mHa_ewvUxlng...

Xinyuan Lv posted a comment on discussion Help

Dear Charlie, Regarding the first question, since I have more files, it's a bit hard to list all the files, I tried to use -c name *~/.nc, but the error message suggests that my files are not integer multiples of 12, but my folder does have a total of 216 files in 12 months per year for the years of 0002-0019, so why is it like that? Here is the command and the error message: ➜ ncclimo -6 -P mpaso --split -s 2 -e 19 -c name /D/Data/input/*.nc -o ${dir_out} -O ${dir_rgr} -r ~/map720and1440.nc ncclimo:...

Charlie Zender posted a comment on discussion Open Discussion

The files you uploaded regrid without issue with this command and the latest NCO (5.2.4): ncremap -m map_mpas.nc diag_mslp_2023-10-26_13.00.00.nc data_rgr.nc This tells me that the mapfile is correct and the input data file is valid. The -P mpas option should, apparently not be used (because it causes a hang) to regrid MPAS-A files. However, without a multi-level variable to regrid, I cannot determine how to solve the other issue you encountered, the one that asked you to permute the input dimension...

Charlie Zender posted a comment on discussion Help

To give ncclimo a specified set of input files, just put the filenames on the command line, e.g., ncclimo -6 -P mpaso --split -s 2 -e 19 -c name in1.nc in2.nc ... inN.nc If the time resolution of the file(s) to be split is less than monthly, then add the --hfs option. Yes, ncremap can interpolate to a specified depth, as the user guide says. It should give similar results to MPAS-analysis.

Angela Bernaldez posted a comment on discussion Open Discussion

Hello Charlie, Thank you for your response. I have uploaded both files to Zenodo, after considerably reducing the size of the diagnostic file by just selecting one variable, MSLP in this case. Here is the link to access the unstructured grid file and the corresponding mapping one: https://zenodo.org/records/11396016?token=eyJhbGciOiJIUzUxMiJ9.eyJpZCI6ImZjY2I3ZmZlLWE5NzAtNGFlYy1hYWIxLWVmMjA2Y2IzNmVlYiIsImRhdGEiOnt9LCJyYW5kb20iOiJmN2MwYWNjYWUyYTE5Y2RmNGVjZDc3N2Q5YmMxZmU3ZSJ9.Ual6eKaHX74ZMVfjxv9z79HKVYvA6WfkMGrxQS-B8Oh-JSVzItVfrM0PXdrIsO6NV8qSidkC91mHa_ewvUxlng...

Xinyuan Lv posted a comment on discussion Help

Hi, Recently I am working on batch interpolation of MPAS_O output results and have two questions for horizontal and vertical interpolation respectively. Question 1: I am trying to use the 0002-0019 monthly customized output file DailySurface (file name like mine-800-final-GMPAS-JRA1p5_TL319_oARRM60to10-intel2023.mpaso.hist.am.DailySurface.0018- 04-01.nc) as the input file for MPAS_O using ncclimo. Using the same command, it will report an error in nco-5.2.4 (not recognizing the nc file in the input...

Charlie Zender posted a comment on discussion Open Discussion

Hi Angela, Please make diag.2023-10-26_13.00.00.nc and map_mpas.nc (or, equivalently, both the source and destination gridfiles) available on a publicly available site (e.g., derecho, perlmutter) or by attachment to this chain. Charlie

Xinyuan Lv posted a comment on discussion Help

Thank you very much, both methods you mentioned will solve my problem. Thank you for your prompt and detailed response!

Charlie Zender posted a comment on discussion Help

The problem appears to be that Matlab does not recognize the 64BIT_DATA file format. You can convert this type to 64BIT_OFFSET with ncks -6 in.nc out.nc or to NETCDF4_CLASSIC with ncks -7 in.nc out.nc. Try doing that to the input files and then I expect Matlab will recognize the output. Actually it's probably simpler to just produce one of those formats in the output file of ncremap by using ncremap -6 ... or ncremap -7 ...

Xinyuan Lv posted a comment on discussion Help

Hi, After I installed the latest version of nco using 'conda install -c conda-forge nco==5.2.4', I was able to convert the MPAS_O output file to a standard grid point file using a command that I could run successfully before. Run the commands in order: ncremap -g grd720and1440.nc -G latlon=720,1440 ncremap -s ocean.ARRM60to10.scrip.200413.nc -g grd720and1440.nc -m map720and1440.nc ncremap -P mpaso -m map720and1440.nc mpaso.hist.am.DailySurface.0019-12-01.nc output_lonlat720and1440.nc But the output...

Angela Bernaldez modified a comment on discussion Open Discussion

I am encountering difficulties in attempting to remap a diagnostic output file from MPAS Atmospheric component (unstructured grid) into a regular lat-lon one. For that purpose, I created the SCRIP file for both the source MPAS-A grid and a desired regular lat-lon grid. With those two SCRIP files, I was able to generate the map file which contains the weights and is used in the remapping process. But the issue comes when trying to use ncremap functionality for a MPAS model and executing: ncremap -P...

Angela Bernaldez posted a comment on discussion Open Discussion

I am encountering difficulties in attempting to remap an output file from MPAS Atmospheric component (unstructured grid) into a regular lat-lon one. For that purpose, I created the SCRIP file for both the source MPAS-A grid and a desired regular lat-lon grid. With those two SCRIP files, I was able to generate the map file which contains the weights and is used in the remapping process. But the issue comes when trying to use ncremap functionality for a MPAS model and executing: ncremap -P mpas -m...

Charlie Zender posted a comment on discussion Open Discussion

Unfortunately not.

Burcu Boza posted a comment on discussion Open Discussion

I want to regrid irregularly spaced vertical dimension of the ocean data to a uniformly spaced one. I can do it linearly with ncremap --vrt. However, I need to take account the layer thicknesses and the contribution of the data at surface which have depths smaller than the first layer of the destination grid. So, is there a way to make a weighted vertical interpolation?

NCO netCDF Operators released /nco-5.2.4.tar.gz

henry Butowsky posted a comment on discussion Help

to do it all on the command line - something like ncap2 -O -h -s "t2@units=\"hours since 2013-09-30 14:00:00\";" -S "$nco_script" "$input_file" "$output_file"

Paul MAUGER modified a comment on discussion Help

Thanks Charlie for taking time to answer. Thank you Henry for finding a way ; the script below works fine : #!/bin/bash # Input and output file names input_file="RAINNC.nc" output_file="processed_RAINNC.nc" # Start time counter start_time=$(date +%s) # NCO script content nco_script=$(cat << 'END_SCRIPT' t2=array(0.0,1.0,$Time); t2@units="hours since 2013-09-30 14:00:00"; sz=$Time.size; for(idx=0;idx<sz;idx++) Times(idx,:)=sprint(strftime(t2(idx),"%Y-%m-%d_%H:%M:%S")); END_SCRIPT ) # Execute NCO script...

Paul MAUGER posted a comment on discussion Help

Thanks Charlie for taking time to answer. Thank you Henry for finding a way ; the script below works fine : #!/bin/bash # Input and output file names input_file="RAINNC.nc" output_file="processed_RAINNC.nc" # Start time counter start_time=$(date +%s) # NCO script content nco_script=$(cat << 'END_SCRIPT' t2=array(0.0,1.0,$Time); t2@units="hours since 2013-09-30 14:00:00"; sz=$Time.size; for(idx=0;idx<sz;idx++) Times(idx,:)=sprint(strftime(t2(idx),"%Y-%m-%d %H:%M:%S")); END_SCRIPT ) # Execute NCO script...

henry Butowsky modified a comment on discussion Help

HI Paul, its bit fiddly, but you can do by first creating a regular time t2 variable. then using that and strftime() to create the timestamps t2=array(0.0,1.0,$Time); t2@units="hours since 2013-10-01 10:00:00"; sz=$Time.size; for(idx=0;idx<sz;idx++) Times(idx,:)=sprint(strftime(t2(idx),"%Y-%m-%d %H:%M:%S")); regards Henry

henry Butowsky posted a comment on discussion Help

HI Paul, its bit fiddly, but you can do by first creating a regular time t2 variable. then using that and strftime() to create the timestamps t2=array(0.0,1.0,$Time); t2@units="hours since 2013-10-01 10:00:00"; sz=$time.size; for(idx=0;idx<sz;idx++) Times(idx,:)=sprint(strftime(t2(idx),"%Y-%m-%d %H:%M:%S")); regards Henry

Charlie Zender posted a comment on discussion Help

Hi Paul, Great question. Unfortunately WRF uses a time convention that is as far from CF-compliant as I can imagine, and there is no straightforward way to change it to a different calendar or timezone. It has to be done with brute force recomputation of the strings. The functions that accomplish this in ncap2 are documented in the manual: strtime, strftime, regular. I'm unfamiliar with them and will have to leave it at that, though maybe someone who knows more will chime in.

Paul MAUGER posted a comment on discussion Help

Hello, I've been looking at shifting time in WRF outputs files. For example, I would like to shift date-time from UTC to UTC-10. The time format in those files is a string of characters as seen below : nctime wrfout_d02_2013-10-01_00\:00\:00 |tms |head -n 5 "2013-10-01_00:00:00", "2013-10-01_01:00:00", "2013-10-01_02:00:00", "2013-10-01_03:00:00", "2013-10-01_04:00:00", Also see below the header information for relevant dimension and variable names : nch wrfout_d02_2013-10-01_00\:00\:00 |head -n...

NCO netCDF Operators released /nco-5.2.3.tar.gz

Charlie Zender posted a comment on discussion Open Discussion

Maybe you will have better results with the bilinear algorithm. Try ncremap -a bilin ...

Ankur Gupta posted a comment on discussion Open Discussion

I faced the same issue. It is due to the fact that the destination grid has longitudes which are not present in source grid. So, we to tell NCO to extrapolate the data or interpolate (between two ends of longitudes). For example, if the 1x1 source grid is defined on lon=(0 to 359) and 0.5x0.5 is the destination grid with data desired at lon-359.5. In such case NCO need to either extrapolate data beyond 359 or interpolate between lon=359 and 0. Unfortunately, I do not know how to make NCO do this...

NCO netCDF Operators released /nco-5.2.2.tar.gz

Paul MAUGER modified a comment on discussion Help

Thanks so much, Henry! I wouldn't have found this in a million years. The second solution works well : # Loop over time steps for time_idx in $(seq 0 $((timesteps - 1))); do # Loop over latitude indices for lat_idx in $(seq $start_lat_idx $end_lat_idx); do ncap2 -h -O -s 'global@tmp=array(SST('${time_idx}','${lat_idx}',31),(SST('${time_idx}','${lat_idx}',80)-SST('${time_idx}','${lat_idx}',31))/50,$west_east); SST('${time_idx}','${lat_idx}',31:80)=global@tmp(0:49);' $output_file $output_file done...

Paul MAUGER posted a comment on discussion Help

Thanks so much, Henry! I wouldn't have found this in a million years. The second solution works well : # Loop over time steps for time_idx in $(seq 0 $((timesteps - 1))); do # Loop over latitude indices for lat_idx in $(seq $start_lat_idx $end_lat_idx); do ncap2 -O -s 'global@tmp=array(SST('${time_idx}','${lat_idx}',31),(SST('${time_idx}','${lat_idx}',80)-SST('${time_idx}','${lat_idx}',31))/50,$west_east); SST('${time_idx}','${lat_idx}',31:80)=global@tmp(0:49);' $output_file $output_file done do...

henry Butowsky modified a comment on discussion Help

HI Paul, The point is that the "array" function has to return a proper variable that means it must use valid dimensions. You can of create your own dimension of the correct size. e.g defdim("my_west_east",61) ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$my_west_east);' test.nc output_test.nc The alternative is to use attributes e.g global@tmp=array(290.0f,0.1,$west_east); SST(1,60,30:90)=global@tmp(0:60); ...Henry

henry Butowsky modified a comment on discussion Help

HI Paul, The point is that the "array" function has to return a proper variable that means it must use valid dimensions. You can of create your own dimension of the correct size. e.g defdim("my_west_east",61) ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$my_west_east);' test.nc output_test.nc The alternative is to use attributes e.g gloabl@tmp=array(290.0f,0.1,$west_east); SST(1,60,30:90)=global@tmp(0:60); ...Henry

henry Butowsky posted a comment on discussion Help

HI Paul, The point is that the "array" function has to return a proper variable that means it must use valid dimensions. You can of create your own dimension of the correct size. e.g defdim("my_west_east",61) ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$my_west_east);' test.nc output_test.nc The alternative is to use attributes e.g gloabl@tmp=array(290.0f,0.1,$west_east); SST(1,60,30:90)=global@tmp(0:30); ...Henry

Paul MAUGER modified a comment on discussion Help

Thank you for your time, Henry Unfortunately, I must say I had great trouble understanding your point. I did read thoroughly the User Guide - multiple times for the concerned section - and couldn't find relevant information for my problem. Nor did I found relevant thread on this forum, although I got some ideas to try. Please be assured I did spend time on that matter, wracking my brain... For now, I'm still asking myself : Can I use ncap2 and array() function on a limited range of longitudes ? And...

Paul MAUGER posted a comment on discussion Help

Thank you for your time, Henry Unfortunately, I must say I had great trouble understanding your point. I did read thoroughly the User Guide - multiple times for the concerned section - and couldn't find relevant information for my problem. Nor did I found relevant thread on this forum, although I got some ideas to try. Please be assured I did spend time on that matter, wracking my brain... For now, I'm still asking myself : Can I use ncap2 and array() function on a limited range of longitudes ? And...

henry Butowsky posted a comment on discussion Help

Hi Paul, Sorry for you confusion . The final argument of the array() function is overloaded usage: var_out=array(start_exp,inc_exp,$dim|dim_list|var) So the final argument can be a single dimension .eg ncap2 -v -O -s 'a1=array(0,1, $time );' in.nc out.nc Or a list of dimensions e.g ncap2 -v -O -s 'a1=array(0,1, /$time, $lat, $lon/ );' in.nc out.nc ( note the / .. / this indicates a list of dimensions Or a variable from which the array takes its shape e.g So if we have say variable float three_dmn_var(lat,...

Paul MAUGER modified a comment on discussion Help

Tries : ncap2 -O -s 'SST(1,60,:) = array(290.0f,0.1,$west_east)' test.nc output_test.nc This above is changing all the longitudes values from lat_idx = 60 with an array rightfully incremented. ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$west_east)' test.nc output_test.nc ncap2: ERROR assign(): Hyperslab for SST - number of elements on LHS(61) doesn't equal number of elements on RHS(120) Erreur de segmentation (core dumped) Of course this above does not work because the longitude range LHS is...

Paul MAUGER modified a comment on discussion Help

Tries : ncap2 -O -s 'SST(1,60,:) = array(290.0f,0.1,$west_east)' test.nc output_test.nc This above is changing all the longitudes values from lat_idx = 60 with an array rightfully incremented. ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$west_east)' test.nc output_test.nc ncap2: ERROR assign(): Hyperslab for SST - number of elements on LHS(61) doesn't equal number of elements on RHS(120) Erreur de segmentation (core dumped) Of course this above does not work because the longitude range LHS is...

Paul MAUGER modified a comment on discussion Help

Tries : ncap2 -O -s 'SST(1,60,:) = array(290.0f,0.1,$west_east)' test.nc output_test.nc This above is changing all the longitudes values from lat_idx = 60 with an array rightfully incremented. ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$west_east)' test.nc output_test.nc ncap2: ERROR assign(): Hyperslab for SST - number of elements on LHS(61) doesn't equal number of elements on RHS(120) Erreur de segmentation (core dumped) Of course this above does not work because the longitude range LHS is...

Paul MAUGER posted a comment on discussion Help

Tries : ncap2 -O -s 'SST(1,60,:) = array(290.0f,0.1,$west_east)' test.nc output_test.nc This above is changing all the longitudes values from lat_idx = 60 with an array rightfully incremented. ncap2 -O -s 'SST(1,60,30:90) = array(290.0f,0.1,$west_east)' test.nc output_test.nc ncap2: ERROR assign(): Hyperslab for SST - number of elements on LHS(61) doesn't equal number of elements on RHS(120) Erreur de segmentation (core dumped) Of course this above does not work because the longitude range LHS is...

Paul MAUGER modified a comment on discussion Help

On track : close to a working solution but not there yet. Command below is able to limit process to desired longitudes : ncap2 -O -s "SST(1,60,30:90) = array(SST(1,60,30), ((SST(1,60,90) - SST(1,60,30)) / (90 - 30 + 1)), 30); " test.nc output_test.nc It seems the starting longitude is enough while replacing "$west_east" (dim). For my case, I will use a script like (I changed the boundaries on lat/lon) : #!/bin/bash # Define input and output filenames input_file="test.nc" output_file="output_test.nc"...

Paul MAUGER modified a comment on discussion Help

On track : close to a working solution but not there yet. Command below is able to limit process to desired longitudes : ncap2 -O -s "SST(1,60,30:90) = array(SST(1,60,30), ((SST(1,60,90) - SST(1,60,30)) / (90 - 30 + 1)), 30); " test.nc output_test.nc It seems the starting longitude is enough while replacing "$west_east" (dim). For my case, I will use a script like (I changed the boundaries on lat/lon) : #!/bin/bash # Define input and output filenames input_file="test.nc" output_file="output_test.nc"...

Paul MAUGER modified a comment on discussion Help

Working solution : ncap2 -O -s "SST(1,60,30:90) = array(SST(1,60,30), ((SST(1,60,90) - SST(1,60,30)) / (90 - 30 + 1)), 30); " test.nc output_test.nc It seems the starting longitude is enough while replacing "$west_east" (dim). For my case, I will use a script like (I changed the boundaries on lat/lon) : #!/bin/bash # Define input and output filenames input_file="test.nc" output_file="output_test.nc" # Define start and end indices for latitude and longitude start_lat_idx=45 end_lat_idx=75 start_lon_idx=30...

Paul MAUGER modified a comment on discussion Help

Hello, I must correct some SST values across a certain range of latitudes and longitudes. As trial, I used ncap2 and array function to replace original SST values by computed ones for all longitudes : ncap2 -O -s 'SST(1,60,:)=array(SST(1,60,0), (SST(1,60,119)-SST(1,60,0))/120, $west_east);' test.nc output_test.nc It works fine, but I would like to apply this procedure to only a subset of SST field : I would like to process only data from latitude_idx [30 to 90] (dim=south_north) and longitude_idx...

Paul MAUGER posted a comment on discussion Help

Working solution : ncap2 -O -s "SST(1,60,30:90) = array(SST(1,60,30), ((SST(1,60,90) - SST(1,60,30)) / (90 - 30 + 1)), 30); " test.nc output_test.nc It seems the starting longitude is enough while replacing "$west_east" (dim). For my case, I will use a script like (I changed the boundaries on lat/lon) : #!/bin/bash # Define input and output filenames input_file="test.nc" output_file="output_test.nc" # Define start and end indices for latitude and longitude start_lat_idx=45 end_lat_idx=75 start_lon_idx=30...

Paul MAUGER posted a comment on discussion Help

Hello, I must correct some SST values across a certain range of latitudes and longitudes. As trial, I used ncap2 and array function to replace original SST values by computed ones for all longitudes : ncap2 -O -s 'SST(1,60,:)=array(SST(1,60,0), (SST(1,60,119)-SST(1,60,0))/120, $west_east);' test.nc output_test.nc It works fine, but I would like to apply this procedure to only a subset of SST field : I would like to process only data from latitude_idx [30 to 90] (dim=south_north) and longitude_idx...

Charlie Zender posted a comment on discussion Open Discussion

You're exactly right, Keith. The mask/weight method hadn't occurred to me, but it could be done using a multi-step approach. And then ncwa could apply that mask to all variables in a dataset using the -y ttl option, and the result would be what you want (I think). The mask would be a 3D variable that would never change for a given resolution/grid. Once created "offline" it could simply be appended to any dataset you wished to apply this method to. Creating the mask is the crux of the problem. That...

Keith Lindsay posted a comment on discussion Open Discussion

Hi Charlie, Thanks for the quick response. I suspected that might be the case, but wanted to see if I was overlooking an applicable/slick NCO feature. A multistep NCO based workflow could be to create a 3d mask or weight variable that is 1 in the bottom cells and 0 elsewhere, and apply ncwa with that mask or weight. There are details like adding the mask/weight variable to the input file, or to a subsetted copy of the input file, but I think this could be made to work. That said, the folks I'm working...

Charlie Zender posted a comment on discussion Open Discussion

Hi Keith, Storing indices in a variable (KMT) and then using that variable as dimension bounds for another variable is a level of indirection that existing NCO hyperslabbing routines are not well suited to handle. The only NCO method I can think of to grab the field[:,:,KMT] is by implementing in ncap2 a brute-force loop over the KMT variable and use the current KMT value to subscript and fill-in your field variable, element-by-element. I realize that KMT-type descriptions are common in ocean models,...

Keith Lindsay posted a comment on discussion Open Discussion

Hi, In CESM POP output there is a variable KMT that specifies the (1-based) k/z_t index of the deepest ocean cell. That is k<=KMT cells are ocean cells and k>KMT cells are land. KMT is a 2d variable, it depends on lateral spatial dimensions and not time. So given a 3d variable name field, field[:,:,KMT] would be the values of field in the deepest model cell, at each point (assuming 1-based indexing). I attempted to extract this with variants of ncwa -a z_t -B "KMT==k+1" ... but didn't come up with...

Leandra Loyola posted a comment on discussion Open Discussion

Thank you. It did work with certain datasets, but i found that i have other datasets that have an unstructed grid, and it doesn't work with those ones.

Charlie Zender posted a comment on discussion Help

Glad to hear it!

Charlie Zender posted a comment on discussion Open Discussion

Try ncremap --map=map.nc in.nc out.nc which is documented here (and in the NCO manual): https://acme-climate.atlassian.net/wiki/spaces/DOC/pages/754286611/Regridding+E3SM+Data+with+ncremap

Leandra Loyola posted a comment on discussion Help

It's been a long time, but thank you very much!!! I was able to do it with the ncea command form nco and it works perfectly.

Leandra Loyola posted a comment on discussion Open Discussion

Hello!, I am working with the CESM2 Large Ensemble output varibales and i have problems regridding 4D variables. I was able to regrid ocean ant atmospheric variables (sea surface temperature, winds, sea surface height) but for the ocean oxycen concentrations wichs is an 4D ocea variable with dimensions (time, level, latitude, longitude) i have not been able to do it. Please if anybody has ideas, they are welcome. Thank you!

Michelle Irizarry Ortiz posted a comment on discussion Help

Gotcha! I will try that. Have also been looking into NCL reshape, but not sure I am getting anywhere. LOL! Thanks!

Charlie Zender posted a comment on discussion Help

You could do it with NCO (specifically ncap2 and maybe ncrcat). I'm unsure that it would be faster or more efficient than R because the ncap2 script would need to use loops and array sub-setting. Unfortunately I do not have the time to develop and demonstrate an approach. Basically I would create the new dimension, then loop over it and fill each timestep with the the appropriate data. Good luck!

Michelle Irizarry Ortiz posted a comment on discussion Help

Hi! I have a netCDF file for ERA5 from NCAR (https://rda.ucar.edu/datasets/ds633.0/dataaccess/) that does not have a time dimension (see attached), but instead has two separate dimensions: forecast_initial_time and a forecast_hour. I would like to combine those dimensions into just time which is forecast_initial_time + (forecast_hour-1) in the appropriate units. Then I'd have to change the dimensions of all the variables to remove forecast_initial_time and forecast_hour and replace with just time....

Charlie Zender posted a comment on discussion Help

Hi Matt, Good question. I would grep for that element in the traditional printed output from ncks, e.g., ncks --trd -v U fvcore_internal_rst.20240118_09z.nc4 | grep 16330475

Matthew Thompson posted a comment on discussion Help

Recently, I had occasion to use the --chk_nan functionality of ncks and: $ ncks --chk_nan fvcore_internal_rst.20240118_09z.nc4 ncks: WARNING nco_chk_nan() reports variable /U has first NaN at hyperslab element 16330475 ncks: INFO nco_chk_nan() reports total number of floating-point variables with NaN elements is 1 So, it found one, yay. But, is there a way to have ncks output that in a more "user friendly" format? Such as "we found NaN at lat/lon/level"? I know we could calculate it since we know...

NCO netCDF Operators released /nco-5.2.1.tar.gz

NCO netCDF Operators released /nco-5.2.0.tar.gz