My intention is to use files in sequential order from 1895-01 through 2015-07 (the files tmean_189501.nc through tmean_201507.nc). I want the loop to run 1447 times, using the final six digits of the file name as the numeric part to use for counting, increment by one at each iteration, and after 12 steps we flip back over to 1 (i.e. these are monthly files and we only want to use files named 01.nc through 12.nc).
I must be doing something wrong in my specification of the various loop indices, as I'm seeing the following messages when I launch the command:
ncrcat: INFO Input file 0 is tmean_189501.nc
ncrcat: INFO Extended filetype of tmean_189501.nc is NC_FORMAT_NC3, mode = 0
ncrcat: INFO Input file 1 is tmean_000001.nc
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is 2. Translation into English with nc_strerror(2) is "No such file or directory"
So it looks like it knows to use tmean_189501.nc as the initial file, but the loop specification is incorrect and hence it thinks that tmean_000001.nc is the next file, instead of tmean_189502.nc.
Can anyone suggest the way I can fix the loop specification to achieve the results I'm after? BTW I'm running this from within a Python script on a Windows machine, so I unfortunately can't use a pipe for this and must rely upon the -n <loop> approach.
Thanks in advance for any help.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have some bad news for you, James. The -n switch cannot currently do what you want. What you want to do is clear, but -n wasn't implemented for it. I will try to hack it soon to do this. The reason it gives a bogus second filename is that it's confused with why the first six digits of the first filename exceed your wrap counter, 12. It thinks the whole six digits should wrap, not just the last two. On windows one way to accomlish your goal , currently, is to pre-generate the filenames and the entire command with perl/python/whatever and execute that.
cz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
OK this is good to know, Charlie, and thanks so much for the immediate feedback. I'm sure I can work my way around this, I've become lazy enough already from leaning so much on you guys' good work. I've been naming the files according to the date but I'll modify my Python to start the file name numbering at 000000 (or should I start with 000001?) and your suggested approach should work fine then. I appreciate your help!
--James
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The lack of this particular feature has been a low-level irritant for some time. I just committed a patch to resolve it by allowing an optional, as yet undocumented, sixth argument to -n. Specify that as 'yyyymm'. When present, that tells NCO to increment the left-most N-2 digits by one everytime the rightmost 2 digits (the month) exceeds the specified maximum month number. To accomplish your goal you would use
ncrcat -n 1447,6,1,12,1,yyyymm tmean_189501.nc out.nc
feedback welcome.
cz
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That sounds great, Charlie. I have limited confidence in my ability to build software like NCO these days, it might take me a month to finally get everything right, so I'll use what I've got for now and wait on the next release that has this included.
Again thanks a lot for your help.
--James
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'm running a NCO command to concatenate around 1500 monthly files into a single file containing the full period of record.
The NCO command I'm using is
ncrcat -O -h -D 2 -n 1447,6,1,12 tmean_189501.nc out.nc
My intention is to use files in sequential order from 1895-01 through 2015-07 (the files tmean_189501.nc through tmean_201507.nc). I want the loop to run 1447 times, using the final six digits of the file name as the numeric part to use for counting, increment by one at each iteration, and after 12 steps we flip back over to 1 (i.e. these are monthly files and we only want to use files named 01.nc through 12.nc).
I must be doing something wrong in my specification of the various loop indices, as I'm seeing the following messages when I launch the command:
ncrcat: INFO Input file 0 is tmean_189501.nc
ncrcat: INFO Extended filetype of tmean_189501.nc is NC_FORMAT_NC3, mode = 0
ncrcat: INFO Input file 1 is tmean_000001.nc
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is 2. Translation into English with nc_strerror(2) is "No such file or directory"
So it looks like it knows to use tmean_189501.nc as the initial file, but the loop specification is incorrect and hence it thinks that tmean_000001.nc is the next file, instead of tmean_189502.nc.
Can anyone suggest the way I can fix the loop specification to achieve the results I'm after? BTW I'm running this from within a Python script on a Windows machine, so I unfortunately can't use a pipe for this and must rely upon the -n <loop> approach.
Thanks in advance for any help.
I have some bad news for you, James. The -n switch cannot currently do what you want. What you want to do is clear, but -n wasn't implemented for it. I will try to hack it soon to do this. The reason it gives a bogus second filename is that it's confused with why the first six digits of the first filename exceed your wrap counter, 12. It thinks the whole six digits should wrap, not just the last two. On windows one way to accomlish your goal , currently, is to pre-generate the filenames and the entire command with perl/python/whatever and execute that.
cz
OK this is good to know, Charlie, and thanks so much for the immediate feedback. I'm sure I can work my way around this, I've become lazy enough already from leaning so much on you guys' good work. I've been naming the files according to the date but I'll modify my Python to start the file name numbering at 000000 (or should I start with 000001?) and your suggested approach should work fine then. I appreciate your help!
--James
The lack of this particular feature has been a low-level irritant for some time. I just committed a patch to resolve it by allowing an optional, as yet undocumented, sixth argument to -n. Specify that as 'yyyymm'. When present, that tells NCO to increment the left-most N-2 digits by one everytime the rightmost 2 digits (the month) exceeds the specified maximum month number. To accomplish your goal you would use
ncrcat -n 1447,6,1,12,1,yyyymm tmean_189501.nc out.nc
feedback welcome.
cz
That sounds great, Charlie. I have limited confidence in my ability to build software like NCO these days, it might take me a month to finally get everything right, so I'll use what I've got for now and wait on the next release that has this included.
Again thanks a lot for your help.
--James