How about using LZMA compression for SquashFS v3?
LZMA is the compression used in 7-zip (http://www.7-zip.org) and compresses really great (even better than RAR and ACE) while being LGPL and having very fast decompression. Not quite as fast as gzip but still very fast compared to bzip2, RAR, ACE and most others.. Compression is slow, but that doesn't matter on a read-only FS..
The LZMA SDK also contains a small pure ANSI-C implementation of the decompression algorithm.
The LZMA SDK can be downloaded at http://www.7-zip.org/sdk.html
Here are some tests I did (decompression was done from a ramdisk):
test.tar = 198830080 bytes
# gzip -9c test.tar >test.tar.gz
test.tar.gz = 44020506 bytes
real 0m32.161s
user 0m31.151s
sys 0m0.301s
# bzip2 -9c test.tar >test.tar.bz2
test.tar.bz2 = 35164492 bytes
real 0m46.667s
user 0m45.823s
sys 0m0.290s
# lzma e test.tar test.tar.lzma
test.tar.lzma = 29236224 bytes
real 8m57.717s
user 8m48.395s
sys 0m1.102s
# mksquashfs test.tar test.tar.sqsh
test.tar.sqsh = 46735360 bytes
real 0m29.736s
user 0m28.984s
sys 0m0.454s
# gzcat test.tar.gz >/dev/null
real 0m1.797s
user 0m1.726s
sys 0m0.034s
# bzcat test.tar.bz2 >/dev/null
real 0m11.445s
user 0m11.247s
sys 0m0.070s
# lzma d test.tar.lzma -so >/dev/null
real 0m4.897s
user 0m4.781s
sys 0m0.051s
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
kernel guys say that they dont wanna inlude lzma decompressor in kernel, so even if v3 will be w/ lzma support, it won't be included into _official_ kernel :-(
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
patches to use squashfs with lzma are readily available. Basically the patches add the lzma decompression code, and change some squashfs initialisation code and the call to uncompress a block. This means only one type of compress method is supported and has to be selected at compile time.
It would be much nicer if the compression type of a squashfs filesystem could be detected and hence the proper uncompres code can be used. If it is using zlib compression use the kernels internal unomcpress routines, and when it is lzma used the uncompress code inside the squashfs module (as long as lzma is not natively supported by the kernel.)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
How about using LZMA compression for SquashFS v3?
LZMA is the compression used in 7-zip (http://www.7-zip.org) and compresses really great (even better than RAR and ACE) while being LGPL and having very fast decompression. Not quite as fast as gzip but still very fast compared to bzip2, RAR, ACE and most others.. Compression is slow, but that doesn't matter on a read-only FS..
The LZMA SDK also contains a small pure ANSI-C implementation of the decompression algorithm.
The LZMA SDK can be downloaded at http://www.7-zip.org/sdk.html
Here are some tests I did (decompression was done from a ramdisk):
test.tar = 198830080 bytes
# gzip -9c test.tar >test.tar.gz
test.tar.gz = 44020506 bytes
real 0m32.161s
user 0m31.151s
sys 0m0.301s
# bzip2 -9c test.tar >test.tar.bz2
test.tar.bz2 = 35164492 bytes
real 0m46.667s
user 0m45.823s
sys 0m0.290s
# lzma e test.tar test.tar.lzma
test.tar.lzma = 29236224 bytes
real 8m57.717s
user 8m48.395s
sys 0m1.102s
# mksquashfs test.tar test.tar.sqsh
test.tar.sqsh = 46735360 bytes
real 0m29.736s
user 0m28.984s
sys 0m0.454s
# gzcat test.tar.gz >/dev/null
real 0m1.797s
user 0m1.726s
sys 0m0.034s
# bzcat test.tar.bz2 >/dev/null
real 0m11.445s
user 0m11.247s
sys 0m0.070s
# lzma d test.tar.lzma -so >/dev/null
real 0m4.897s
user 0m4.781s
sys 0m0.051s
kernel guys say that they dont wanna inlude lzma decompressor in kernel, so even if v3 will be w/ lzma support, it won't be included into _official_ kernel :-(
Why not? Do you have more information?
dam I need to learn how to write kernel modules I have all these wonderful ideas rolling around in my head.
patches to use squashfs with lzma are readily available. Basically the patches add the lzma decompression code, and change some squashfs initialisation code and the call to uncompress a block. This means only one type of compress method is supported and has to be selected at compile time.
It would be much nicer if the compression type of a squashfs filesystem could be detected and hence the proper uncompres code can be used. If it is using zlib compression use the kernels internal unomcpress routines, and when it is lzma used the uncompress code inside the squashfs module (as long as lzma is not natively supported by the kernel.)