use 7-zip as archiver
Brought to you by:
herrekberg
Menu
▾
▴
#84 use 7-zip as archiver
Hello!
I'm Alukard, and I will post a new idea about archiver formats.
The p7zip software can handle with a couple of various formats, like zip, rar, tar, bzip, gzip, etc.
It can be used as an alternative, and to padronize the interface of Comix and the archive.
As I have said, it is only an alternative - in case of p7zip isn´t installed, it will use other alternatives (unrar, zip, tar, etc).
note that 7z's rar support is usually limited to rar version 2.0 archives, because it uses the Free Software rar code. There is a plugin to add non-free rar support to 7z (p7zip-rar in Ubuntu), but it's of course non-free software. So you definitely want to default to using rar/unrar for rar archives.
7z's standard rar support works for uncompressed (m0g) files in rar archives made by current versions of rar, but you get "unsupported method" if you try to extract anything compressed. Note that "unrar l" lists the rar version requirement at the end of the line, and it's 2.0 for uncompressed, and 2.9 for compressed. Free (libre) rar software only handles rar version 2.0-compatible stuff.
Before adding 7z support, you should probably re-work comix to extract all the files from an archive with one exec() of 7z, instead of the current one fork()/exec() per archive member, called by the thumbnailer loop. This would be deadly slow on a solid archive, where extracting one file requires decrypting from the beginning of the solid block (typically the beginning of the archive). 7z defaults to creating solid archives, so comix is more likely to be used on solid archives with 7z than rar.
rar and 7z both let you ask for multiple specific files to extract from the archive in one go, so probably a good idea to extract the whole archive. Maybe with heuristics to do it in batches for archives > 100MB. But if you're going to extract the whole archive to /tmp eventually, then you should definitely do it in one go, in case of solid archives.
But if the thumbnailer isn't active (e.g. full-screen), then you don't need all the files, so it's a waste to extract everything if the archive isn't solid. This could make going to the last file in the archive faster (except that currently every file is extracted when you hit END, although I don't know why. I've been looking at the code trying to see how to make comix faster at going to the end of an archive and then the next archive (after looking at only a couple files near the beginning.))
I should really have started a new feature-request for this. sorry.
I think some of what I said in my last post was wrong, because I misunderstood the code. It seems comix always starts an extractor thread to extract all the files in the order it wants them. So going to the last file will always have to wait for that thread to finish. However, when I run comix -f (with comix 4.0.3 on my Ubuntu Intrepid GNU/Linux system), the thread doesn't proceed to the end right away. It gets a few files done every time you go to the next image. Sounds like a locking problem with the _condition semaphore. I don't know python well at all, and I haven't figured out how to get pdb to do anything to debug threads.
I guess it makes sense to extract first the image you're going to display first, and you can't guarantee that if you give unrar the set of files to extract in one go. If the files aren't stored in order comix sorts them to, then you have to wait before the first image displays.
Not to mention that you'd have to poll the filesystem to know when any given file was finished extracting, or use inotify (non-portable).
So maybe running unrar/7z to extract in batches of 4 or 5 images would be a good trade-off, for non-solid archives only, of course.
It would be nice to support going to the end of an archive quickly, without waiting for all the files to extract. (esp. without waiting for as many forks/execs as there are images in the archive! It's a significant delay even on a core2duo.)
Hi,
thanks for the enlightening comments. I didn't know that 7-zip used "solid" archives by default, that explains why it was so deadly slow when I tried it earlier.
I am a bit surprised, though, that you get noticeable delays because of the multiple forks. On my rather slow Athlon machine, the time difference in extracting a "normal" .cbr all in one go or using one call per file is less than half a second. Sure, it is not negligible, but seeing as Comix will start displaying as soon as the first file is extracted it shouldn't "appear" slower (it should even seem faster in many cases). Maybe there is some other factor which makes forks/execs slower or faster on different systems.
The heuristic that Comix uses in deciding the order in which to extract files is simply to first extract the images immediately around the first viewed page (normally the first page, but not necessarily so if you e.g. use a bookmark or go the previous archive), and then the rest of the images from first to last. There might be a better heuristic of course, but none will be best in all cases. An alternative would be to extract files only on demand, i.e. when they are going to be displayed. This has the added benefit of using less space in the temporary directory, but could instead make things seem sluggish when changing pages.
About the strange behaviour of not extracting all files directly when the thumbnail sidebar isn't shown. I had never noticed this, probably because I ran all the tests of the extracting mechanisms with the thumbnails turned on. I doesn't seem to be a problem with the condition though, but something even more subtle. It appears that when the main thread is idle, the extracting thread, and even the spawned unrar process (!), are somehow paused. If you give the main thread something to do again (it is enough to move the mouse cursor over the window to generate some X window events) the extracting will resume! I have yet to come up with an explanation, and indeed it might be some bug with the condition, but it doesn't seem so to me.