hi gary,
if the behavior has changed, it was inadvertent and i'll restore it. please send me the location of the file so i can verify whether current/future NCO works as you wish.
c
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
hi gary,
if the behavior has changed, it was inadvertent and i'll restore it.
please send me the location of the file so i can verify whether
current/future NCO works as you wish.
c
and you don't need glasses, except perhaps as style accessories. i never added a user-visible switch to toggle shuffling, i'll put that on the TODO list (and turn it on by default unless someone objects).
c
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Just committed the changes to turn on shuffle by default when manually deflating files. Also tweaked default chunking behavior to be more efficient and faster. Both changes will be in 4.4.9 hopefully by next week. Please try if possible executables in ~zender/bin/LINUXAMD64 to see if they work well for you.
Just committed the changes to turn on shuffle by default when manually
deflating files. Also tweaked default chunking behavior to be more
efficient and faster. Both changes will be in 4.4.9 hopefully by next week.
Please try if possible executables in ~zender/bin/LINUXAMD64 to see if they
work well for you.
I've noticed an interesting change in behavior between NCO 4.[1-3].N and NCO 4.4.N, in that applying
'ncks -4 -L 1 infile.nc outfile.nc'
no longer has shuffling, which results in compressed files that are 8%-10% larger, for example:
uncompressed : 228391916 b
compressed_4.1.0 : 131404552 b
compressed_4.2.0 : 131404552
compressed_4.2.0 : 131404565
compressed_4.2.3 : 131404565
compressed_4.3.2 : 131404871
compressed_4.3.4 : 131404871
compressed_4.3.7 : 131404871
compressed_4.4.2 : 174752905
compressed_4.4.4 : 174752620
I didn't see an option to restore the shuffling in the documentation. Do I need reading glasses?
hi gary,
if the behavior has changed, it was inadvertent and i'll restore it. please send me the location of the file so i can verify whether current/future NCO works as you wish.
c
The data are on
yellowstone:/glade/scratch/strandwg/testing_compression
Thanks, Charlie!
On Thu, May 14, 2015 at 9:44 AM, Charlie Zender zender@users.sf.net wrote:
--
Gary Strandwww.cgd.ucar.edu/ccr/strandwg strandwg@ucar.edu
and you don't need glasses, except perhaps as style accessories. i never added a user-visible switch to toggle shuffling, i'll put that on the TODO list (and turn it on by default unless someone objects).
c
Just committed the changes to turn on shuffle by default when manually deflating files. Also tweaked default chunking behavior to be more efficient and faster. Both changes will be in 4.4.9 hopefully by next week. Please try if possible executables in ~zender/bin/LINUXAMD64 to see if they work well for you.
zender@yslogin5:~/nco$ ncks -O -4 -L 1 ~/uncompressed.nc ~/foo.nc
zender@yslogin5:~/nco$ ls -l ~/foo.nc
-rw-r--r-- 1 zender ncar 130653330 May 14 21:34 /glade/u/home/zender/foo.nc
I did not realize how important shuffle was until you wrote this request.
So thanks again.
c
Much much thanksly, Charlie!
On Thu, May 14, 2015 at 10:37 PM, Charlie Zender zender@users.sf.net
wrote:
--
Gary Strandwww.cgd.ucar.edu/ccr/strandwg strandwg@ucar.edu