Deflation (lossless compression)

[Charlie Zender]

FYI netCDF4 Lempel-Ziv deflation is working in the CVS snapshot, e.g.,

ncks -O -4 --dfl_lvl=9 ~/nco/data/in.nc ~/foo.nc

My random tests show climate data compresses up to 30-40%.

The switch is described in the manual

http://nco.sf.net/nco.html#dfl_lvl

Charlie

[Charlie Zender]

Hi Henry,

I've just finished implementing the --deflate switch in all NCOs
except ncap and ncap2. Would you please finish the implementation
in ncap and ncap2? I've put the switch processing in the main()
routines and the nco_def_var_deflate() calls in the low level routines.
I think it would be better if you decided how to propagate
the single variable (dfl_lvl) necessary through the parse tree
to the nco_def_var_deflate() calls.

For ncap, all that remains is to propagate dfl_lvl into the
var_write() routine and remove the local copy. (TODO nco822)

For ncap2, all that remains is to propagate dfl_lvl into
ncap_var_write() and remove the local copy. (TODO nco823).

You don't need to test the routines (requires netCDF4).
Just make sure the variable gets passed and I'll test your mods.

Let me know if you have any questions.

Thanks!
Charlie

[henry Butowsky]

Hi Charlie,
all done.
By the way it looks like Unidata have done a decent implementation of compression - So once the var has been defined
as compressed ---- the compression routine is transparent to the regular netcdf lib functions ?? O

Regards Henry

[Charlie Zender]

Thanks, Henry.

> the compression routine is transparent to the regular netcdf lib functions ?

Yes, and it can sit on top of packing for added compression.

Charlie