Hi, I read in the changelog that clonezilla switched to pbzip2 because of a bug in lbzip2, now I know this was a long time ago (+3 years) and those issues seem to be solved for a long time.
Could Clonezilla switch back to lbzip2, which is faster and seems to be more feature-rich?
Thx in advance, Steven
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
You might consider using one of the LZMA2 compression types. bzip2 is slow to compress and slow decompress. LZMA2 used by clonezilla appears to have adequately small window sizes to do its job effectively. And compression / decompression speeds are quite good.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Sure. We should switch back to lbzip2. It will be done in the next release.
However, as szfong mentioned, pixz (option "-z5p" in Clonezilla) might be a better alternative option.
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I just tested the new lbzip2 debian test-build, with the old pbzip2 build I got a partclone imaging speed of 625 MB/min. When I used the new 20140120-trusty testing build using lbzip2, the rate kept rising and even topped at 1 GB/min when halfway the image! The avg. was 955MB/min. Impressive, that's an improvement of almost 60%.
Update: I tested pixz too and I got 500 MB/min. Yeah I know it's not the fastest computer. The computer has an AMD dual-core processor. The backup was around 8% larger. However you save some time with the check if backup is restorable, (4.5GHz/min vs 1.5GHz/min) but not enough to justify the difference in compression speed.
My conclusion is that with lbzip2, parallel bzip2 now offers a good middle ground between parallel xz and parallel gzip if decompression speed isn't important.
So thank you for the implementation, Steven! I love your work.
Regards Steven Goris
Last edit: Steven Goris 2014-01-21
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I'm the author of pixz. I just wanted to point out that the default compression level of xz/pixz is -6, which is pretty strong. If you will be compressing huge amounts of data, you might consider using levels -1 through -4. The compression ratio will still be very good, probably better than bzip2; but the speed will be much more reasonable.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
@Steven Goris,
Thanks. Yes, it depends on what your purpose is. Xz format is good when uncompressing, about the same speed of gunzip. However, not as fast as that of gzip when compressing. Therefore if you want to have minimum size and good speed when uncompressing, xz is a very good choice.
@Dave,
Thanks for your comments. Yes, in Clonezilla we use "-3" level.
BTW, thanks for developing pixz. Great program!
Steven.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi, I read in the changelog that clonezilla switched to pbzip2 because of a bug in lbzip2, now I know this was a long time ago (+3 years) and those issues seem to be solved for a long time.
Could Clonezilla switch back to lbzip2, which is faster and seems to be more feature-rich?
Thx in advance, Steven
You might consider using one of the LZMA2 compression types. bzip2 is slow to compress and slow decompress. LZMA2 used by clonezilla appears to have adequately small window sizes to do its job effectively. And compression / decompression speeds are quite good.
Sure. We should switch back to lbzip2. It will be done in the next release.
However, as szfong mentioned, pixz (option "-z5p" in Clonezilla) might be a better alternative option.
Steven.
Program "lbzip2" is the default program if bzip2 option is chosen in Clonezilla lives 2.2.2-1 and 20140120-*.
Thanks for reminding this.
Steven.
Awesome! About pixz, should this be faster to compress too? It always felt that bzip2 still compressed a little bit faster.
I just tested the new lbzip2 debian test-build, with the old pbzip2 build I got a partclone imaging speed of 625 MB/min. When I used the new 20140120-trusty testing build using lbzip2, the rate kept rising and even topped at 1 GB/min when halfway the image! The avg. was 955MB/min. Impressive, that's an improvement of almost 60%.
Update: I tested pixz too and I got 500 MB/min. Yeah I know it's not the fastest computer. The computer has an AMD dual-core processor. The backup was around 8% larger. However you save some time with the check if backup is restorable, (4.5GHz/min vs 1.5GHz/min) but not enough to justify the difference in compression speed.
My conclusion is that with lbzip2, parallel bzip2 now offers a good middle ground between parallel xz and parallel gzip if decompression speed isn't important.
So thank you for the implementation, Steven! I love your work.
Regards Steven Goris
Last edit: Steven Goris 2014-01-21
Hi, I'm the author of pixz. I just wanted to point out that the default compression level of xz/pixz is -6, which is pretty strong. If you will be compressing huge amounts of data, you might consider using levels -1 through -4. The compression ratio will still be very good, probably better than bzip2; but the speed will be much more reasonable.
@Steven Goris,
Thanks. Yes, it depends on what your purpose is. Xz format is good when uncompressing, about the same speed of gunzip. However, not as fast as that of gzip when compressing. Therefore if you want to have minimum size and good speed when uncompressing, xz is a very good choice.
@Dave,
Thanks for your comments. Yes, in Clonezilla we use "-3" level.
BTW, thanks for developing pixz. Great program!
Steven.