#111 ncap2 cannot append to compressed file

[raglan_road]

I am trying to add two variables in a file and write it to a third variable in the same file with ncap2. If the file is uncompressed, this works. E.g.:

ncap2 -L 6 -4 -A -h -s 'co2_flux_land=bio_flux_opt+fire_flux_imp' guineapig_uncomp.nc

However, if I compress the variables in the file first with 'ncks',

ncks -O -4 -L 6 -o guineapig_comp_level6.nc guineapig_uncomp.nc

then ncap2 can no longer do this:

$ ncap2 -L 6 -4 -A -h -s 'co2_flux_land=bio_flux_opt+fire_flux_imp' guineapig_comp_level6.nc 
ERROR: nco__open() unable to open file "guineapig_comp_level6.nc"
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -101. Translation into English with nc_strerror(-101) is "NetCDF: HDF error"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

even if the compression is level 0,

$ ncks -O -4 -L 0 -o guineapig_comp_level0.nc guineapig_uncomp.nc 
$ ncap2 -L 6 -4 -A -h -s 'co2_flux_land=bio_flux_opt+fire_flux_imp' guineapig_comp_level0.nc 
ERROR: nco__open() unable to open file "guineapig_comp_level0.nc"
nco_err_exit(): ERROR Short NCO-generated message (usually name of function that triggered error): nco__open()
nco_err_exit(): ERROR Error code is -101. Translation into English with nc_strerror(-101) is "NetCDF: HDF error"
nco_err_exit(): ERROR NCO will now exit with system call exit(EXIT_FAILURE)

However, if instead of appending I write the output to a new file, then ncap2 can handle compressed files:

$ ncap2 -L 6 -4 -h -o guineapig_comp_level6_added.nc -s 'co2_flux_land=bio_flux_opt+fire_flux_imp' guineapig_comp_level6.nc

and the new variable is also compressed,

$ ncdump -h -s guineapig_comp_level6_added.nc 
netcdf guineapig_comp_level6_added {
dimensions:
        date = UNLIMITED ; // (8 currently)
        lat = 180 ;
        lon = 360 ;
        date_components = 6 ;
variables:
        double co2_flux_land(date, lat, lon) ;
                co2_flux_land:_FillValue = -1.e+34 ;
                co2_flux_land:units = "mol m-2 s-1" ;
                co2_flux_land:_Storage = "chunked" ;
                co2_flux_land:_ChunkSizes = 8, 180, 360 ;
                co2_flux_land:_DeflateLevel = 6 ;
                co2_flux_land:_Shuffle = "true" ;
                co2_flux_land:_Endianness = "little" ;

This looks like a bug, unless there's a good reason why ncap2 can append to uncompressed files, and write to a new compressed file, but not append to a compressed file.

[henry Butowsky]

HI Raglan.
I know this is abit odd but repeat the append file name twice

**ncap2 -L 6 -4 -A -h -s 'co2_flux_land=bio_flux_opt+fire_flux_imp' guineapig_comp_level6.nc guineapig_comp_level6.nc **

This works for me on the latest code in repo

....Henry