ncrcat: time rebase [version "4.3.9"]

Help
2014-02-28
2014-10-14
  • Dennis Shea

    Dennis Shea - 2014-02-28

    It is possible that I misunderstand time rebase with ncrcat.

    I have downloaded 3-hrly TRMM netCDF files from NASA's mirador server.
    A sample file dump for 3B42.20090731.12.7A.nc looks like:

    %> ncdump -v time 3B42.20090731.12.7A.nc

    netcdf \3B42.20090731.12.7A {
    dimensions:
    time = UNLIMITED ; // (1 currently)
    longitude = 1440 ;
    latitude = 400 ;
    variables:
    double time(time) ;
    time:units = "hours since 2009-7-31 12" ;

    [SNIP a bunch of variables]

    // global attributes:
    :Conventions = "COARDS" ;
    :calendar = "standard" ;
    :comments = "file created by grads using lats4d
    available from http://dao.gsfc.nasa.gov/software/grads/lats4d/" ;
    :model = "geos/das" ;
    :center = "gsfc" ;
    data:

    time = 0 ;
    }
    =============
    Each file has its own unique time base units attribute which matches
    the date in the file name.
    =============
    Next, I concatenate all 8*31 times (files) into one file.

    %> ncrcat -h -O 3B42.200907*nc TEST.nc

    %> ncdump -v time TEST.nc

    netcdf TEST {
    dimensions:
    time = UNLIMITED ; // (248 currently)
    longitude = 1440 ;
    latitude = 400 ;
    variables:
    double time(time) ;
    time:units = "hours since 2009-7-1 0" ;
    [SNIP]
    // global attributes:
    :Conventions = "COARDS" ;
    :calendar = "standard" ;
    :comments = "file created by grads using lats4d
    available from http://dao.gsfc.nasa.gov/software/grads/lats4d/" ;
    :model = "geos/das" ;
    :center = "gsfc" ;
    data:

    time = 0, 0, 0, 0, 0, 0, 0, 0, 24, 24, 24, 24, 24, 24, 24, 24, 48, 48,
    48,
    48, 48, 48, 48, 48, 72, 72, 72, 72, 72, 72, 72, 72, 96, 96, 96, 96,
    96,
    96, 96, 96, 120, 120, 120, 120, 120, 120, 120, 120, 144, 144, 144,
    144,
    144, 144, 144, 144, 168, 168, 168, 168, 168, 168, 168, 168, 192, 192,
    192, 192, 192, 192, 192, 192, 216, 216, 216, 216, 216, 216, 216, 216,
    240, 240, 240, 240, 240, 240, 240, 240, 264, 264, 264, 264, 264, 264,
    264, 264, 288, 288, 288, 288, 288, 288, 288, 288, 312, 312, 312, 312,
    312, 312, 312, 312, 336, 336, 336, 336, 336, 336, 336, 336, 360, 360,
    360, 360, 360, 360, 360, 360, 384, 384, 384, 384, 384, 384, 384, 384,
    408, 408, 408, 408, 408, 408, 408, 408, 432, 432, 432, 432, 432, 432,
    432, 432, 456, 456, 456, 456, 456, 456, 456, 456, 480, 480, 480, 480,
    480, 480, 480, 480, 504, 504, 504, 504, 504, 504, 504, 504, 528, 528,
    528, 528, 528, 528, 528, 528, 552, 552, 552, 552, 552, 552, 552, 552,
    576, 576, 576, 576, 576, 576, 576, 576, 600, 600, 600, 600, 600, 600,
    600, 600, 624, 624, 624, 624, 624, 624, 624, 624, 648, 648, 648, 648,
    648, 648, 648, 648, 672, 672, 672, 672, 672, 672, 672, 672, 696, 696,
    696, 696, 696, 696, 696, 696, 720, 720, 720, 720, 720, 720, 720, 720 ;

    In my view, the time should be 0,3,6,9,12,15,18,21,24,27,30,....

    ====
    %> ncrcat --version
    NCO netCDF Operators version "4.3.9" last modified 2013/12/02 built Dec
    30 2013 on cottonwood.cgd.ucar.edu by root
    ncrcat version 4.3.9

    %> uname -a
    Linux harmon.cgd.ucar.edu 3.13.1-1.el6.elrepo.x86_64 #1 SMP Wed Jan 29
    12:55:36 EST 2014 x86_64 x86_64 x86_64 GNU/Linux

    THX
    D

     
  • Charlie Zender

    Charlie Zender - 2014-02-28

    Hi Dennis,

    It took me a while to figure this out.
    The input files use a time units value not fully
    recognized by UDUnits:

    "hours since 2013-11-30 6"

    should be

    "hours since 2013-11-30 6:00:00"

    or something similar. A naked integer for the HH hour
    is not correctly recognized. You can use the udunits
    command to verify this. Or you can use the undocumented
    ncks --tst_udunits function to show that the difference
    between times on consecutive days with the incorrect formats
    is always 24 hours. The suggested workaround is to append ":00:00" to
    all time units attributes using

    ncatted -a units,,a,c,":00:00" inout.nc

    I am uncertain whether this is a problem with UDUnits
    not supporting a format it says it does, or whether TRMM
    data uses a units format that UDUnits does not explicitly support.
    I suspect the latter. If not, please forward this problem to
    Unidata for fixing.

    Best,
    c

    zender@neige:/data/zender/hdf$ ncks --tst_udunits='hours since 2013-11-30 6','hours since 2013-12-01 21',standard ~/nco/data/in.nc
    Units in=hours since 2013-11-30 6, units out=hours since 2013-12-01 21, difference (date) or conversion (non-date) = -24.000000

    zender@neige:/data/zender/hdf$ ncks --tst_udunits='hours since 2013-11-30 06:00:00','hours since 2013-12-01 21:00:00',standard ~/nco/data/in.nc
    Units in=hours since 2013-11-30 06:00:00, units out=hours since 2013-12-01 21:00:00, difference (date) or conversion (non-date) = -39.000000

     
  • Charlie Zender

    Charlie Zender - 2014-03-01

    oops, should be

    ncatted -a units,time,a,c,":00:00" inout.nc

    or you'll change all units
    cz

     
  • romzi04

    romzi04 - 2014-10-13

    Hi all -

    I'm replying to this thread cause my issue is quite the same, but with nco 4.4.4. However I looked at the units and it seems to be all right.

    Here is my point:
    I do have two monthly files with a time attribute referring to the first time step of the file:
    $>ncdump -h ARPERAREC-201201.nc
    netcdf ARPERAREC-201201 {
    dimensions:
    time = UNLIMITED ; // (124 currently)
    [...]
    int time(time) ;
    time:calendar = "gregorian" ;
    time:long_name = "time" ;
    time:units = "hours since 2012-01-01 00:00:00" ;

    and
    $>ncdump -h ARPERAREC-201202.nc
    netcdf ARPERAREC-201202 {
    dimensions:
    time = UNLIMITED ; // (116 currently)
    [...]
    int time(time) ;
    time:calendar = "gregorian" ;
    time:long_name = "time" ;
    time:units = "hours since 2012-02-01 00:00:00" ;

    Each one has records indexed by 6,12,18,...
    But when I am using ncrcat the second file is not rebased to the first one and I have the time variable going to 6,12,18,...,6,12,18,... :

    ncrcat ARPERAREC-201201.nc ARPERAREC-201202.nc test.nc
    [...]
    data:

    time = 6, 12, 18, 24, 30, 36, 42, 48, 54, 60, 66, 72, 78, 84, 90, 96, 102,
    108, 114, 120, 126, 132, 138, 144, 150, 156, 162, 168, 174, 180, 186,
    192, 198, 204, 210, 216, 222, 228, 234, 240, 246, 252, 258, 264, 270,
    276, 282, 288, 294, 300, 306, 312, 318, 324, 330, 336, 342, 348, 354,
    360, 366, 372, 378, 384, 390, 396, 402, 408, 414, 420, 426, 432, 438,
    444, 450, 456, 462, 468, 474, 480, 486, 492, 498, 504, 510, 516, 522,
    528, 534, 540, 546, 552, 558, 564, 570, 576, 582, 588, 594, 600, 606,
    612, 618, 624, 630, 636, 642, 648, 654, 660, 666, 672, 678, 684, 690,
    696, 702, 708, 714, 720, 726, 732, 738, 744, 6, 12, 18, 24, 30, 36, 42,
    48, 54, 60, 66, 72, 78, 84, 90, 96, 102, 108, 114, 120, 126, 132, 138,
    144, 150, 156, 162, 168, 174, 180, 186, 192, 198, 204, 210, 216, 222,
    228, 234, 240, 246, 252, 258, 264, 270, 276, 282, 288, 294, 300, 306,
    312, 318, 324, 330, 336, 342, 348, 354, 360, 366, 372, 378, 384, 390,
    396, 402, 408, 414, 420, 426, 432, 438, 444, 450, 456, 462, 468, 474,
    480, 486, 492, 498, 504, 510, 516, 522, 528, 534, 540, 546, 552, 558,
    564, 570, 576, 582, 588, 594, 600, 606, 612, 618, 624, 630, 636, 642,
    648, 654, 660, 666, 672, 678, 684, 690, 696 ;
    }

    Any suggestion ? I used to deal with those files and it worked with a previous version of nco... Unfortunately I do not retrieve the one installed on my previous computer, and I assume is better to fix the issue with the current version than downgrading each time I want to use ncrcat.

    Thank you !
    R.

     
    Last edit: romzi04 2014-10-13
  • Charlie Zender

    Charlie Zender - 2014-10-13

    Hello R.,

    Re-basing has not changed in quite some time. Version 4.4.4 should work fine.
    The current version works fine for me with your files. Was the NCO you are using built with UDUnits support? If not, that would explain the problem you see. UDUnits is required for rebasing. Post results of ncrcat -r if unsure.

    cz

    zender@roulee:~$ ncrcat -O ARPERAREC-201201.nc ARPERAREC-201202.nc foo.nc
    zender@roulee:~$ ncks --cdl -C -v time foo.nc
    netcdf foo {
    dimensions:
    time = UNLIMITED ; // (240 currently)

    variables:
    int time(time) ;
    time:calendar = "gregorian" ;
    time:long_name = "time" ;
    time:units = "hours since 2012-01-01 00:00:00" ;

    data:
    time = 6, 12, 18, 24, 30, 36, 42, 48, 54, 60, 66, 72, 78, 84, 90, 96, 102, 108, 114, 120, 126, 132, 138, 144, 150, 156, 162, 168, 174, 180, 186, 192, 198, 204, 210, 216, 222, 228, 234, 240, 246, 252, 258, 264, 270, 276, 282, 288, 294, 300, 306, 312, 318, 324, 330, 336, 342, 348, 354, 360, 366, 372, 378, 384, 390, 396, 402, 408, 414, 420, 426, 432, 438, 444, 450, 456, 462, 468, 474, 480, 486, 492, 498, 504, 510, 516, 522, 528, 534, 540, 546, 552, 558, 564, 570, 576, 582, 588, 594, 600, 606, 612, 618, 624, 630, 636, 642, 648, 654, 660, 666, 672, 678, 684, 690, 696, 702, 708, 714, 720, 726, 732, 738, 744, 750, 756, 762, 768, 774, 780, 786, 792, 798, 804, 810, 816, 822, 828, 834, 840, 846, 852, 858, 864, 870, 876, 882, 888, 894, 900, 906, 912, 918, 924, 930, 936, 942, 948, 954, 960, 966, 972, 978, 984, 990, 996, 1002, 1008, 1014, 1020, 1026, 1032, 1038, 1044, 1050, 1056, 1062, 1068, 1074, 1080, 1086, 1092, 1098, 1104, 1110, 1116, 1122, 1128, 1134, 1140, 1146, 1152, 1158, 1164, 1170, 1176, 1182, 1188, 1194, 1200, 1206, 1212, 1218, 1224, 1230, 1236, 1242, 1248, 1254, 1260, 1266, 1272, 1278, 1284, 1290, 1296, 1302, 1308, 1314, 1320, 1326, 1332, 1338, 1344, 1350, 1356, 1362, 1368, 1374, 1380, 1386, 1392, 1398, 1404, 1410, 1416, 1422, 1428, 1434, 1440 ;

    } // group /

     
    • romzi04

      romzi04 - 2014-10-14

      Hello CZ,

      Fantastic, there was the 'trick'.
      I did not notice I had an issue in my NCO installation coming from the udunits2 lib. I still don't know why but I clean everything and re-install them, paying attention to the UDUNITS2_PATH env variable... And it works !

      Thanks for the support, you save me quite a lot of time !!

      R.

       

Log in to post a comment.