there are data after end of archive

[sopcore]

please help!
when i extract tar file using 7-zip, occur a error massage dialog: there are data after end of archive;
but if the tar  only be combined with txt files will no this error.

plus: the tar file created in unix env,

[Igor Pavlov]

Please describe full information to reproduce such "bad" tar archive:
1) System version
2) TAR software version
3) what files do you put to TAR archive?
Try to create simple and small "bad" TAR file.
Then check "Physical Size" property of that bad tar archive in 7-zip.
And look your TAR file in hex editior.
What do you see after "Physical Size" offset in that TAR file?

[Alexander Lehmann]

I have noticed the same problem with some tar files,all of the files are from the same open source project, so this may be an issue related to the way they build their files.

When I look at the end of the file after the physical size displayed by 7zip, I see the last two tar entries.

I cannot reproduce the issue with smaller files (since I didn't create the problematic files), but you could download a file for testing, since these are from an open source project, they are public anyways.

http://www.apache.org/dyn/closer.cgi/pig/pig-0.8.0/

(also broken are two other files from the project I tried)

http://www.apache.org/dyn/closer.cgi/hadoop/pig/pig-0.5.0/
http://www.apache.org/dyn/closer.cgi/hadoop/pig/pig-0.7.0/

[Igor Pavlov]

alexlehm:
So you see that there is problem with that TAR.
7-Zip notifies about that problem.

You can ask developers of these bad tars about tar software that was used to create these files.

[Alexander Lehmann]

The files work correctly with Linux tar and Cygwin tar, also with other windows archiver programs there is no problem (e.g. ultimatezip).

The extracted files are all correct, so there is not actually a problem with file processing of the files but with checking the archive structure.

I have also noticed that the problem only happens with current versions of 7zip (9.15, 9,20), but not with an older version that I have installed on another machine (version 4.65)

[redstun]

Same error with latest 7.22 beta, when opening files downloaded from apache.org, such as:
http://archive.apache.org/dist/ant/binaries/apache-ant-1.7.1-bin.tar.bz2
http://labs.renren.com/apache-mirror//maven/binaries/apache-maven-3.0.3-bin.tar.gz

But there's no error for this:
http://archive.apache.org/dist/ant/binaries/apache-ant-1.8.2-bin.tar.bz2

Never seen this error with 7-zip 4.xx

[Igor Pavlov]

Old 7-zip ignored some errors.
New 7-zip notifies user about problems TAR archives.

So if there is error in old and, and there is no error in new and, probably now they use "good" TAR software to create tars.

I suppose it's good that 7-Zip notifies about problems in other popular TAR software.
Note that these TAR problems (like data after end of archive) can be sucurity problems also.

[Nostromov]

Old 7-zip ignored some errors.
New 7-zip notifies user about problems TAR archives.

& and thank goodness for that!

[Alexander Lehmann]

maven 3.0.3 is pretty new, however there may be an issue with the tar package they use to package the distribution.

[Igor Pavlov]

Can you ask them what TAR software they use?

[Alexander Lehmann]

I was willing to bet that maven uses their own tools to create archives and it turned out that this is correct and the problem can be analyzed with maven projects and by comparing it to the ant implementation.

The Ant and Maven implementation are essentially the same, including the same error, however the error was fixed in Ant about 2 years ago (ant 1.8.0) while the Maven version is still using the code that puts junk at the end of the file.

First of all, let me explain what the problem is. Tar files were originally intended to be writte to tape (Tape ARchive), where the data is written in 512 byte blocks including the file or directory enties and always 20 blocks are groups into a record so that each record is 10MB long, the record size for the tapes. if an archive is shorter than the record size, the last record is written fully nevertheless so that all tar files are a multiple size of 10MB (there may be tar implementations that do not do that, which is accepted by most tar readers). To signify the end of the archive, 2 512 byte records with 0 bytes are written, so that the tar reader can determine when the data is actually ending, regardless of what is written to fill the last block, the padding blocks are supposed to be 0 filled as well but this is not necessarily checked since the eof records stop processing beforehand.

Now, the old ant and the maven implementation is a bit lazy about generating the padding records and simply uses the buffer from the previous block so that the padding data is really an segment from before in the file that is repeated.

The issue that describes the fix for ant is this one: https://issues.apache.org/bugzilla/show_bug.cgi?id=47421 and the commit to fix this is the following: https://fisheye6.atlassian.com/browse/ant/core/trunk/src/main/org/apache/tools/tar/TarBuffer.java?r2=789556&r1=597087

I will try to propose the same fix for maven, however it would be nice if 7zip could mention that this problem can probably be ignored for archives created by older ant or maven versions.

[Alexander Lehmann]

opened an issue at codehaus

http://jira.codehaus.org/browse/PLXCOMP-176

[Alexander Lehmann]

I wonder if it would be possible to detect the structure of the junk padding data and give a message accordingly, e.g. this file likely created with ant or maven before xxx.

Assuming that the buffer is overwritten at the beginning, the record sequence has to be like this (if the blocksize were 6 for example)

0
1
2
3
4
5

6
null
null
3
4
5

i.e. any record x after the null records has to be identical to the record at x-(blocksize)

[Roger Pack]

FWIW we overcame this problem locally by starting to tar our files using java library commons-io 1.9 instead of 1.0 (so I'd guess this was a bug in older versions of it). (apparently it was benign?)

[Shaun Forsyth]

Sorry to tag on to an old thread, however I am seeing this with 16.04, and a tgz I have (which I can share) which works fine with 17.04. This feels like a bug.

[Igor Pavlov]

there is no 7-zip 17.04 still.
Please check 7-zip version again.
Check also:

7z x a.tar.gz -o111 -t#
7z x a.tar.gz
7z x a.tar -o222 -t#