Re: [Module::Build] Archive::Tar file format

[Glenn L. <pe...@ne...>]

On approximately 1/20/2004 12:34 AM, came the following characters from
the keyboard of Jos I. Boumans:
> 
> On Jan 13, 2004, at 9:59 AM, Glenn Linderman wrote:
> 
>> On approximately 1/13/2004 12:22 AM, came the following characters from
>> the keyboard of Jos I. Boumans:
>>
>>> On 12-jan-04, at 19:38, Glenn Linderman wrote:
>>> what needs to happen to make this work with M::B is:
>>> A) patch M::B to use the $Archive::Tar::DO_NOT_USE_PREFIX flag
>>> B) create an archive with M::B from a dir structure with at least one 
>>>  file of > 100
>>>     chars, like in:
>>>     http://search.cpan.org/CPAN/authors/id/K/KA/KAKE/Bot-BasicBot- 
>>> Pluggable-Module-SimpleBlog-0.02.tar.gz
>>> C) attempt to extract this file with your tar extractor of choice
>>
>>
>> Why should I make my module have a file of > 100 chars?  Such a name 
>> is much too long, in my opinion, for this module of interest.
> 
> *sigh*
> 
> illustrated above is the way to guarantee that M::B will work properly 
> with every archive A::T will create. The problem that came to light by 
> your test case was an incompatibillity issue with older tar clients. 
> This particular problem has been fixed as described in a previous mail, 
> and i'm waiting for results from the testers (ie you and randy).
> 
>> I believe that you have my test case, and can see that there are no 
>> files with names > 100 chars.
> 
> your test case is no longer relevant here, as stated above, since it 
> merely pointed at the incompatibillity, rather than using it in the 
> 'worst case possible'.

So let me see if I understand: you have fixed an incompatibility with 
older tar clients, and are waiting for Randy and I to test it.  Except 
that my test case is no longer relevant, so why wait for me to test it?

>> If the problems with PowerArchiver (and other utilities) seeing 
>> pathnames are related to pathnames that are longer than 100 
>> characters, why are those other utilities having problems displaying 
>> the directory names from the output of A::T when the pathnames are 
>> less than 100 characters?
>>
>> I think we are still missing something here.
> 
> no, still same problem -- A::T uses the prefix header (even sometimes 
> for paths < 100 characters) and older clients just don't recognise it...

And here, as far as I can tell, you state that there will still be an 
incompatibility with older tar clients, so as far as I can tell, there 
isn't a lot of point in testing something that has no chance of helping 
the incompatibility.

So you may understand perfectly well what is going on, but from what I 
can tell, even if I should come up with a test case to test your fixes, 
I still won't have a solution to the incompatibility I am actually 
experiencing with older tar clients.

The way I understand this issue, which may be totally wrong, is as follows:

1) older tars didn't have prefix headers, and were limited to pathnames 
<= 100 bytes in total length.

2) Some tar or another implemented a solution that allowed > 100 byte 
pathnames, but had its own limit of <= 256 bytes.

3) GNU tar implemented one or more extensions that allowed pathnames > 
100 bytes.

4) Some tar standards committee implemented some extensions that allowed 
pathnames greater than 100 bytes.  The file format implemented by this 
solution is incompatible with all older tar clients.

5) implementations 2, 3, and 4 are incompatible, when path names are 
greater than 100 bytes.

So then, it now appears that although my test case is demonstrating a 
problem with using the new file format for archives containing only 
pathnames <= 100 bytes, the problem you are fixing has to do with the 
incompatibilities among the various solutions for archives containing 
pathnames > 100 bytes?

On my very first job back in 1979, I was introduced to the concept of 
"backward compatibility"... the goal being to keep the data 
transportable between older and newer versions of software, and the 
programming APIs from older versions to still function the same in newer 
versions.  The purpose behind that goal was to maximize the ability of 
the customer to get work done.

It appears that since the new tar file format in point (4) above causes 
incompatibility problems with older software programs, that it should 
only be used when it is needed, and that the older file format (from 
point (1) above) should be used when the new format is not needed.  This 
would maximize the ability of users to interoperate with their data 
using various old and new software programs.  Are there any other 
solutions that achieve this maximal data compatibility?

Clearly if a user of a new program, needing the new feature of > 100 
byte pathnames, creates an archive, it will not be compatible with old 
programs that can only deal with <= 100 byte pathnames.  At the time of 
creation of such an archive, the user should be warned that it uses 
features from whatever (point 2, 3, and/or 4) above that permit the 
pathnames to be >= 100 bytes, and that the resultant archive may not be 
compatible with tar programs that do not have that feature.

> On Jan 20, 2004, at 4:40 AM, Glenn Linderman wrote:
> 
>> On approximately 1/13/2004 12:59 AM, came the following characters from
>> the keyboard of Glenn Linderman:
>>
>>> Especially, because I still don't know why people are fixing and 
>>> discussing > 100 character pathnames, when my test case that fails 
>>> doesn't have any!
>>
>> Any progress on this point yet?  I've been away for a few days, but 
>> didn't see any new traffic on the topic.
> 
> point A) from the above list needs to first be adressed and tested

So I think that is what Randy did in the message below.  I think 
basically that he verified your statement that my test case (which is 
also his test case for my problem) is not relevant to what you fixed.

So we, as testers (at least I, as tester), am waiting for a fix for the 
problem I actually have, rather than a fix for a problem that I don't 
have (but which sounds like it needs to be fixed, nonetheless).

OK, so how far off am I in my understanding of what is going on?


On 1/13/2004 3:35 PM, Randy W. Sims wrote:
> On 1/12/2004 6:15 AM, Jos I. Boumans wrote:
> 
>>> YES! Looks good. I will test more extensively when a snapshot is
>>> available. Thanks.
>>
>> You can find a snapshot attached. To use the new feature, follow this bit of documentation:
> 
> I've tried both versions with no success after modifying Module::Build with the patch below. I get the same result whether DO_NOT_USE_PREFIX is  true or false. WinZip and PowerArchiver can not read the path info, while WinRAR and at least one port of GNU tar can read it.
> 
> Randy.
> 
> $ diff -u Base.pm.orig Base.pm
> --- Base.pm.orig        2004-01-10 23:19:59.000000000 -0500
> +++ Base.pm     2004-01-13 18:01:36.000000000 -0500
> @@ -1741,6 +1741,9 @@
>      $self->do_system($self->{args}{gzip}, "$dir.tar") if $self->{args}{gzip};
>    } else {
>      require Archive::Tar;
> +    # Archive::Tar versions >= 1.09 use the following to enable a compatibility
> +    # hack so that the resulting archive is compatible with older clients.
> +    $Archive::Tar::DO_NOT_USE_PREFIX = 1;
>      my $files = $self->rscan_dir($dir);
>      Archive::Tar->create_archive("$dir.tar.gz", 1, @$files);
>    }

-- 
Glenn -- http://nevcal.com/
===========================
The best part about procrastination is that you are never bored,
because you have all kinds of things that you should be doing.