PKWARE's APPNOTE.TXT Version: 6.3.0 with LZMA and PPMd support
Hello,
on the 29 of September PKWARE published a new revision of its definition of .ZIP File Format Specification (http://www.pkware.com/business_and_developers/developer/popups/appnote.txt) which now includes LZMA and PPMd compression methods. Since both compressions are already present in 7-Zip and its ZIP support was undergoing a some modifications (http://sourceforge.net/forum/forum.php?thread_id=1570341&forum_id=45797 & http://sourceforge.net/forum/forum.php?thread_id=1572625&forum_id=45797\), I was wondering if
the new version of 7-Zip may include these compression schemes? Including such support will make it the first software to do so.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
RE: The most popular File archiver...
2006-09-29 22:41
PKWARE's APPNOTE.TXT (Version: 6.3.0, September 29, 2006) now describes LZMA and PPMd.
So probably I also will support LZMA and PPMd in ZIP. But now I don't see any program from pkware that supports LZMA in .zip. I want to try executable from them to test compatibility.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I don't see the point. The only reason people use ZIP is for compatibility or speed. Even Deflate64, or whatever it was called, is hardly used, and for a good reason.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Deflate64 was not properly/fully documended in the appnote, which is why it was never included in Info-Zip. With PPMd and LZMA offically specified, they can now be used. Of course modern formats are more efficient, but on the other hand gzip, tar and Z are still supported and are present in 7-Zip.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It's not a question of documentation. Even if open/free apps supported these new formats, it still doesn't mean it's compatible.
A lot of people will still use outdated versions because they're unaware of the new versions, or they don't know how to update, or they don't have enough rights, or no one compiled it for their OS, or something else.
The only thing going for ZIP is total compatibility and speed. Newer ZIP formats will break compatibility (and won't be faster).
If you're going to distribute something in "new ZIP" and tell people they have to upgrade, you might just as well tell them to use 7z.
sheh.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is like saying that Microsoft should never improve their MS Word or MS Excel formats so that users from 1988 MS Word 1.0 can still use their antiquated software to read the latest documents.
If compatibility is an issue, then by all means produce a PKZip 2.0 file, most commercial archivers allow you to do this.
The day will come when 7Zip will improve it's own archive format which will break compatibility with the existing format of today. That the way of the world.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>...like saying that Microsoft should never improve their MS Word or MS Excel...
These serve a need completely different from an archiver. And either way, I'm not saying software shouldn't evolve, only that I think it doesn't make much sense for ZIP, which I believe, as stated, is popular for its compatibility and speed. (And perhaps, more recently, WinXP integration).
If someone's interest is not universal compatibility, they might just as well be already using existing, much more tested, lightweight, software: 7zip or WinRAR.
>The day will come when 7Zip will improve it's own archive format which will break compatibility...
7z users are power users, and are looking for improved compression and newer versions. They will know and want to upgrade. ZIP targets computer illiterates or arcane platforms.
---
>Al he sees is the dramatic size gain, and he ruches to tell/send it to his friends/colleagues...
If that were the case, why didn't this type of viral popularization make WinRAR mainstream already? (7-zip may be at a disadvantage because of the GUI, but not WinRAR.)
How about this scenario (I wonder... :): Incompatible ZIP archives start appearing on the internet, people get frustrated, ZIP loses favor, on the lookout for newer software people widen their horizons and find out about RAR (or 7z, given better GUI). :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> >Al he sees is the dramatic size gain, and he ruches to tell/send it to his friends/colleagues...
> If that were the case, why didn't this type of viral popularization make WinRAR mainstream already? (7-zip may be at a disadvantage because of the GUI, but not WinRAR.)
I agree that it is stretching (quite) a bit :-) , but still, WinZip has an advantage: it's name. You would agree that when one speak of ZIP, this names comes in mind, before WinRar and even PKZIP (sigh). I also believe WinZip has a larger install base than any other archiver (sigh 2).
> How about this scenario (I wonder... :): Incompatible ZIP archives start appearing on the internet, people get frustrated, ZIP loses favor,
That's a gamble, indeed.
> on the lookout for newer software people widen their horizons and find out about RAR (or 7z, given better GUI). :)
Lambda user will not look for newer software, he will take whatever will be pushed into his throat. :-) So it all will be up to the marketing departments and acceptance by some (THE, sigh 3) major OS(es).
An interesting “test case” is developing right now with OSX dropping their standard SIT(X) in favour of ZIP.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> like saying that Microsoft should never improve their MS Word or MS Excel formats ...
MS Word and MS Excel format is the IP of Microsoft, they have the right to change whatever they want.
NOTE: ZIP format does not belong to WinZIP. PKWare ZIP format belong to PKWARE, so why is WinZIP making improper changes to the format then expect PKWARE to add it in to it's specification.
Take for example: .TXT, .CSV file is so simple and the whole world expect any text editor can open/read it. If you want to change something to it, you are FREE to do it on your own format and extension. Why choose a universal accepted format and extension such as .ZIP?
Can't WinZIP create their own format/extension?
I suggest WinZIP, use this extension .WZP
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> why is WinZIP making improper changes to the format then expect PKWARE to add it in to it's specification.
To please their customers and to try to prove that they an compression experts, not GUI wrapper builders :-)
> Can't WinZIP create their own format/extension?
> I suggest WinZIP, use this extension .WZP
Well, they already do: in their setup the archive is called setup.WZ
But they will never do that extrenally, because WinZip's the major selling point is its (stolen) association with the ZIP format on windows platform (the other being nice interface and wizards).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It is true that ZIP strongest point is its compatibility. But I don't agree that it can't evolve. IMHO the reasons for non adapting previous enhancements to ZIP format were either their closeness (Deflate64/ZIP64) and/or inability to provide much improvement, for example in term of compression (bzip2). With well documented LZMA and PPMd, it is not the case and LZMA is fast. Also, PPMd implementation is actually WinZip's extension. And one have to give them credit for being VERY good salesmen. WinZip primary market is lambda users, be it private or companies, and a lambda user mostly have text files, like e-mails, documents, spreadsheets, etc. And PPMd is particularly good for compressing that, so they expect the users will notice this difference, and be impressed. At this point, the “ubiquitousness” comes into play. To a lambda use a zip is a zip, whatever compression was, as he won't bother reading the explanations and fine print. Al he sees is the dramatic size gain, and he ruches to tell/send it to his friends/colleagues. And when they fail to open his files he tells them to upgrade to the “latest and shiniest” version of WinZip, just like his. And this is what WinZip marketing want.
One of the strongest boost's for adaptation of the new compression schemes would come if Micro$oft decides to to add it to their OS, be it as a patch or as a Plu$! update. And one of their reasons can be that they sell software to produce these text documents.
An other player in the field if the Info-Zip. There have been little development in the zip/unzip code base because the ZIP format have fossilized. It remains to be seen if they will include these new options, but it is far from sure, not because they don't like them, but because Info-Zip uses zlib for compression and there is no (yet) lzmalib and ppmdlib.
7-Zip already have these libraries, so it might be the first software to implement it. Except that Igor rightly prefers to wait for an implementation from PKZIP to ensure compatibility. Let hope that they will release such a version soon, perhaps as part of their SDK.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
LZP method might be included into ZIP format as well if WinZIP or PKWARE implement them in their next version. Since it perform as well as Deflate but faster.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> To please their customers and to try to prove that they an compression experts, not GUI wrapper builders :-)
Oh please. Prove what? Prove PPMII develop from WinZIP? Just don't make the whole world laugh. PPMII is from Dmitry Shkarin http://www.compression.ru/ds/
WinZIP is still a GUI wrapper builders expert and nothing else.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> Oh please. Prove what? Prove PPMII develop from WinZIP? Just don't make the whole world laugh. PPMII is from Dmitry Shkarin http://www.compression.ru/ds/
Well let not be too hard on them, they do give proper credits on their page describing this extension to the ZIP format. One have to credit them with the idea of “gluing” a new compression technique to the old format. :-) (Even though ZIP was thought extendable in this direction from the start).
That's the way it is: WinZip is stuck with the ZIP format, for the good and bad of it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> gluing a new compression technique to the old format..
Do you want to see more and more this kind of gluing? Then there is alot to glue here. Glue LZP, LZS, LZSS, LZX, PPMZ, ROLZ, CM, etc.
I wouldn't want such gluing. ZIP is unable to create Solid Archive, so no matter what good is the 'glue', their don't 'glue' (compress) as well as solid archive.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
>> WinZIP is still a GUI wrapper builders expert and nothing else. <<
Yes, and the GUI is very importand because it must contain all the nessesary functions. Compressing a file is just a small part of a good GUI and that's why Wizip and Winrar are so big.
A packer will stand or fall by it's GUI.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
> Then, give it a GUI similar to Winrar and Winzip and notice the difference......
> The GUI IS very importand.
I don't see WinRK, WinAce and Squeez having much of the market share. And their GUIs are not too bad.
uha is great, and have nice GUI (see other thread) but need at least some website to show that there is development behind, a road map, something to show it is alive. Even simple periodic bug fixes would do.
(And opensourcing it will be the pinnacle :-) )
So neither good compression, nor sleek GUI is enough to pierce nowadays. You need something extra, like 7-Zip has OSS/LGPL.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
2.Even better, add your own compression algorithm and make it fast, reliable and stable. It must be very fast on the fast modes and quite powerful when you select better compression, the most importand thing is a descent speed with a quite good compression. Repair archives, update solid archives, merge archives, split archives, encrypt archives, convert archives, make sfx archives, multimedia filters, exclude filetypes you don't want to be compressed, Link archives. Would be a good start.
(link archives means that you can link any archive/archives together no matter where it is on your harddrive or ftp. When you open an archive it will also show the content in the "linked archive/archives" and unpack it if you want, almost like volumes but more handy because you can link any excisting archives together, anytime. You can off course "unlink" any archives, anytime. If your computer can't find the linked archive/archives on your harddrive or/and ftp it will automatically search for those linked archives, if your computer don't find it/them you can still without problems unpack the content in the one you're working with. You could go as far as every archive has a unique number, in that way we could link archives over the net too)
2.A good looking GUI with lots of handy features that makes everything easy and comfortable for everyone who uses the packer. Convert archives, direct ftp upload, scheduled backup jobs, a fileexplorer like norton commander would be just the beginning. Every handy feature you can think of to make it easier for people to work with your packer. Leave the competition standing by inventing smart and practical ideas. The look of the GUI is gonna be very importand because people look at the GUI, if it looks real good, it's easier to get people to download your packer.
3.Open source would be great. Assign lots of people to work with the new archiver, let all people do what they are doing best, let some designwizard create the GUI. Tell people to pack lots of files with the new format and practically fill up torrent sites and places like that with your format so people are forced to download something to unpack them with. Mass convert zip and rar to the new format. Open a site to show people how good your packer is, put in some flash videos to show them how to work with it and why it is so much better then other packers.
4.Make the Zip part in your packer very powerful and a good reason for anybody to choose your archiver, when they use your packer they are going to use the new format as well.
5.A lot of job, but if many people work together.........
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
PKWARE's APPNOTE.TXT Version: 6.3.0 with LZMA and PPMd support
Hello,
on the 29 of September PKWARE published a new revision of its definition of .ZIP File Format Specification (http://www.pkware.com/business_and_developers/developer/popups/appnote.txt) which now includes LZMA and PPMd compression methods. Since both compressions are already present in 7-Zip and its ZIP support was undergoing a some modifications (http://sourceforge.net/forum/forum.php?thread_id=1570341&forum_id=45797 & http://sourceforge.net/forum/forum.php?thread_id=1572625&forum_id=45797\), I was wondering if
the new version of 7-Zip may include these compression schemes? Including such support will make it the first software to do so.
Look what Igor said here:
http://sourceforge.net/forum/forum.php?thread_id=1580258&forum_id=45797
RE: The most popular File archiver...
2006-09-29 22:41
PKWARE's APPNOTE.TXT (Version: 6.3.0, September 29, 2006) now describes LZMA and PPMd.
So probably I also will support LZMA and PPMd in ZIP. But now I don't see any program from pkware that supports LZMA in .zip. I want to try executable from them to test compatibility.
Tnx, I missed this one (the thread was quite long :-) )
so we'll have zip in 7zip and lzma in zip
nice :)
freeko
I don't see the point. The only reason people use ZIP is for compatibility or speed. Even Deflate64, or whatever it was called, is hardly used, and for a good reason.
Deflate64 was not properly/fully documended in the appnote, which is why it was never included in Info-Zip. With PPMd and LZMA offically specified, they can now be used. Of course modern formats are more efficient, but on the other hand gzip, tar and Z are still supported and are present in 7-Zip.
It's not a question of documentation. Even if open/free apps supported these new formats, it still doesn't mean it's compatible.
A lot of people will still use outdated versions because they're unaware of the new versions, or they don't know how to update, or they don't have enough rights, or no one compiled it for their OS, or something else.
The only thing going for ZIP is total compatibility and speed. Newer ZIP formats will break compatibility (and won't be faster).
If you're going to distribute something in "new ZIP" and tell people they have to upgrade, you might just as well tell them to use 7z.
sheh.
What a load of crap!!
This is like saying that Microsoft should never improve their MS Word or MS Excel formats so that users from 1988 MS Word 1.0 can still use their antiquated software to read the latest documents.
If compatibility is an issue, then by all means produce a PKZip 2.0 file, most commercial archivers allow you to do this.
The day will come when 7Zip will improve it's own archive format which will break compatibility with the existing format of today. That the way of the world.
>What a load of crap!!
How eloquent.
>...like saying that Microsoft should never improve their MS Word or MS Excel...
These serve a need completely different from an archiver. And either way, I'm not saying software shouldn't evolve, only that I think it doesn't make much sense for ZIP, which I believe, as stated, is popular for its compatibility and speed. (And perhaps, more recently, WinXP integration).
If someone's interest is not universal compatibility, they might just as well be already using existing, much more tested, lightweight, software: 7zip or WinRAR.
>The day will come when 7Zip will improve it's own archive format which will break compatibility...
7z users are power users, and are looking for improved compression and newer versions. They will know and want to upgrade. ZIP targets computer illiterates or arcane platforms.
---
>Al he sees is the dramatic size gain, and he ruches to tell/send it to his friends/colleagues...
If that were the case, why didn't this type of viral popularization make WinRAR mainstream already? (7-zip may be at a disadvantage because of the GUI, but not WinRAR.)
How about this scenario (I wonder... :): Incompatible ZIP archives start appearing on the internet, people get frustrated, ZIP loses favor, on the lookout for newer software people widen their horizons and find out about RAR (or 7z, given better GUI). :)
> >Al he sees is the dramatic size gain, and he ruches to tell/send it to his friends/colleagues...
> If that were the case, why didn't this type of viral popularization make WinRAR mainstream already? (7-zip may be at a disadvantage because of the GUI, but not WinRAR.)
I agree that it is stretching (quite) a bit :-) , but still, WinZip has an advantage: it's name. You would agree that when one speak of ZIP, this names comes in mind, before WinRar and even PKZIP (sigh). I also believe WinZip has a larger install base than any other archiver (sigh 2).
> How about this scenario (I wonder... :): Incompatible ZIP archives start appearing on the internet, people get frustrated, ZIP loses favor,
That's a gamble, indeed.
> on the lookout for newer software people widen their horizons and find out about RAR (or 7z, given better GUI). :)
Lambda user will not look for newer software, he will take whatever will be pushed into his throat. :-) So it all will be up to the marketing departments and acceptance by some (THE, sigh 3) major OS(es).
An interesting “test case” is developing right now with OSX dropping their standard SIT(X) in favour of ZIP.
> like saying that Microsoft should never improve their MS Word or MS Excel formats ...
MS Word and MS Excel format is the IP of Microsoft, they have the right to change whatever they want.
NOTE: ZIP format does not belong to WinZIP. PKWare ZIP format belong to PKWARE, so why is WinZIP making improper changes to the format then expect PKWARE to add it in to it's specification.
Take for example: .TXT, .CSV file is so simple and the whole world expect any text editor can open/read it. If you want to change something to it, you are FREE to do it on your own format and extension. Why choose a universal accepted format and extension such as .ZIP?
Can't WinZIP create their own format/extension?
I suggest WinZIP, use this extension .WZP
> why is WinZIP making improper changes to the format then expect PKWARE to add it in to it's specification.
To please their customers and to try to prove that they an compression experts, not GUI wrapper builders :-)
> Can't WinZIP create their own format/extension?
> I suggest WinZIP, use this extension .WZP
Well, they already do: in their setup the archive is called setup.WZ
But they will never do that extrenally, because WinZip's the major selling point is its (stolen) association with the ZIP format on windows platform (the other being nice interface and wizards).
It is true that ZIP strongest point is its compatibility. But I don't agree that it can't evolve. IMHO the reasons for non adapting previous enhancements to ZIP format were either their closeness (Deflate64/ZIP64) and/or inability to provide much improvement, for example in term of compression (bzip2). With well documented LZMA and PPMd, it is not the case and LZMA is fast. Also, PPMd implementation is actually WinZip's extension. And one have to give them credit for being VERY good salesmen. WinZip primary market is lambda users, be it private or companies, and a lambda user mostly have text files, like e-mails, documents, spreadsheets, etc. And PPMd is particularly good for compressing that, so they expect the users will notice this difference, and be impressed. At this point, the “ubiquitousness” comes into play. To a lambda use a zip is a zip, whatever compression was, as he won't bother reading the explanations and fine print. Al he sees is the dramatic size gain, and he ruches to tell/send it to his friends/colleagues. And when they fail to open his files he tells them to upgrade to the “latest and shiniest” version of WinZip, just like his. And this is what WinZip marketing want.
One of the strongest boost's for adaptation of the new compression schemes would come if Micro$oft decides to to add it to their OS, be it as a patch or as a Plu$! update. And one of their reasons can be that they sell software to produce these text documents.
An other player in the field if the Info-Zip. There have been little development in the zip/unzip code base because the ZIP format have fossilized. It remains to be seen if they will include these new options, but it is far from sure, not because they don't like them, but because Info-Zip uses zlib for compression and there is no (yet) lzmalib and ppmdlib.
7-Zip already have these libraries, so it might be the first software to implement it. Except that Igor rightly prefers to wait for an implementation from PKZIP to ensure compatibility. Let hope that they will release such a version soon, perhaps as part of their SDK.
LZP method might be included into ZIP format as well if WinZIP or PKWARE implement them in their next version. Since it perform as well as Deflate but faster.
> To please their customers and to try to prove that they an compression experts, not GUI wrapper builders :-)
Oh please. Prove what? Prove PPMII develop from WinZIP? Just don't make the whole world laugh. PPMII is from Dmitry Shkarin http://www.compression.ru/ds/
WinZIP is still a GUI wrapper builders expert and nothing else.
> Oh please. Prove what? Prove PPMII develop from WinZIP? Just don't make the whole world laugh. PPMII is from Dmitry Shkarin http://www.compression.ru/ds/
Well let not be too hard on them, they do give proper credits on their page describing this extension to the ZIP format. One have to credit them with the idea of “gluing” a new compression technique to the old format. :-) (Even though ZIP was thought extendable in this direction from the start).
That's the way it is: WinZip is stuck with the ZIP format, for the good and bad of it.
> gluing a new compression technique to the old format..
Do you want to see more and more this kind of gluing? Then there is alot to glue here. Glue LZP, LZS, LZSS, LZX, PPMZ, ROLZ, CM, etc.
I wouldn't want such gluing. ZIP is unable to create Solid Archive, so no matter what good is the 'glue', their don't 'glue' (compress) as well as solid archive.
>> WinZIP is still a GUI wrapper builders expert and nothing else. <<
Yes, and the GUI is very importand because it must contain all the nessesary functions. Compressing a file is just a small part of a good GUI and that's why Wizip and Winrar are so big.
A packer will stand or fall by it's GUI.
> A packer will stand or fall by it's GUI.
No. By its ubiquitousness! :-)
>> WinZIP is still a GUI wrapper builders expert and nothing else. <<
That's the difference, that's why they are so big.
Release a new great command line packer and try to make it big because it's great.
Good Luck!
Then, give it a GUI similar to Winrar and Winzip and notice the difference......
The GUI IS very importand.
> Then, give it a GUI similar to Winrar and Winzip and notice the difference......
> The GUI IS very importand.
I don't see WinRK, WinAce and Squeez having much of the market share. And their GUIs are not too bad.
uha is great, and have nice GUI (see other thread) but need at least some website to show that there is development behind, a road map, something to show it is alive. Even simple periodic bug fixes would do.
(And opensourcing it will be the pinnacle :-) )
So neither good compression, nor sleek GUI is enough to pierce nowadays. You need something extra, like 7-Zip has OSS/LGPL.
I think the concept for a great packer is.
1.A compression engine, could be zip for exemple.
2.Even better, add your own compression algorithm and make it fast, reliable and stable. It must be very fast on the fast modes and quite powerful when you select better compression, the most importand thing is a descent speed with a quite good compression. Repair archives, update solid archives, merge archives, split archives, encrypt archives, convert archives, make sfx archives, multimedia filters, exclude filetypes you don't want to be compressed, Link archives. Would be a good start.
(link archives means that you can link any archive/archives together no matter where it is on your harddrive or ftp. When you open an archive it will also show the content in the "linked archive/archives" and unpack it if you want, almost like volumes but more handy because you can link any excisting archives together, anytime. You can off course "unlink" any archives, anytime. If your computer can't find the linked archive/archives on your harddrive or/and ftp it will automatically search for those linked archives, if your computer don't find it/them you can still without problems unpack the content in the one you're working with. You could go as far as every archive has a unique number, in that way we could link archives over the net too)
2.A good looking GUI with lots of handy features that makes everything easy and comfortable for everyone who uses the packer. Convert archives, direct ftp upload, scheduled backup jobs, a fileexplorer like norton commander would be just the beginning. Every handy feature you can think of to make it easier for people to work with your packer. Leave the competition standing by inventing smart and practical ideas. The look of the GUI is gonna be very importand because people look at the GUI, if it looks real good, it's easier to get people to download your packer.
3.Open source would be great. Assign lots of people to work with the new archiver, let all people do what they are doing best, let some designwizard create the GUI. Tell people to pack lots of files with the new format and practically fill up torrent sites and places like that with your format so people are forced to download something to unpack them with. Mass convert zip and rar to the new format. Open a site to show people how good your packer is, put in some flash videos to show them how to work with it and why it is so much better then other packers.
4.Make the Zip part in your packer very powerful and a good reason for anybody to choose your archiver, when they use your packer they are going to use the new format as well.
5.A lot of job, but if many people work together.........