Thread: [Netatalk-devel] cannot copy long filename including #hex

Hi, 

The following filenames can be copied from Mac OS X to netatalk.

aaa#11.txt
bbb#12.txt
ccc#13.txt

But, the following filenames might not be able to be copied.

aaaaaaaaaaaaaaaaaaaaaa#11.txt
bbbbbbbbbbbbbbbbbbbbbb#12.txt
cccccccccccccccccccccc#13.txt

This problem happens even if using either Finder or cp command
in Terminal.app.

Finder's Error Message:

>
>The Finder cannot complete the operation because some data
>in "aaaaaaaaaaaaaaaaaaaaaa#11.txt" could not be read or
>written.
>(Error code -36)
>

cp command's Error Message:

>
># cp aaaaaaaaaaaaaaaaaaaaaa#11.txt /Volumes/neta-vol/dir/
>cp: /Volumes/neta-vol/dir/aaaaaaaaaaaaaaaaaaaaaa#11.txt: Is a directory
>

If filename is long and include #hex, Mac OS X request FPGetFileDirParms
and FPResolveID command.

When there is a directory of the same CNID as #hex on netatalk's volume,
netatalk return kFPObjectTypeErr to FPResolveID, and the copy fails.
(See osx105-neta21dev.dump)

Mac OS X return kFPIDNotFound to FPResolveID, and the copy succeeds.
(See osx105-osx104.dump)

#0 - #10 are no problem because these CNID is reserved.

Netatalk's result code is more right than Mac OS X.
However, netatalk should behave as well as Mac OS X.
I wrote a one-line patch.
(netatalk-2.1dev-resolveid_dir.patch)

Still another problem exists.
When a file or directory of the same CNID as #hex is in the distination
directory, the following message is displayed.

>
>A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
>this location. Do you want to replace it with the older one you
>are moving?
>                                             [Stop]   [Replace]
>

If "Replace" is clicked, this file or directory is overwrited,
though it is another filenmame.
This problem occur on both netatalk and Mac OS X.
Mac OS X is defective.
I think there is no countermeasure.

-- 
HAT

Le vendredi 19 décembre 2008 à 02:08 +0900, HAT a écrit :
> Hi, 
> 
> The following filenames can be copied from Mac OS X to netatalk.
> 
> aaa#11.txt
> bbb#12.txt
> ccc#13.txt
> 
> But, the following filenames might not be able to be copied.
> 
> aaaaaaaaaaaaaaaaaaaaaa#11.txt
> bbbbbbbbbbbbbbbbbbbbbb#12.txt
> cccccccccccccccccccccc#13.txt
> 
> This problem happens even if using either Finder or cp command
> in Terminal.app.
> 
> Finder's Error Message:
> 
> >
> >The Finder cannot complete the operation because some data
> >in "aaaaaaaaaaaaaaaaaaaaaa#11.txt" could not be read or
> >written.
> >(Error code -36)
> >
> 
> cp command's Error Message:
> 
> >
> ># cp aaaaaaaaaaaaaaaaaaaaaa#11.txt /Volumes/neta-vol/dir/
> >cp: /Volumes/neta-vol/dir/aaaaaaaaaaaaaaaaaaaaaa#11.txt: Is a directory
Yep, in the spec 11 hexa is the old OS9 'Network Trash Folder'.
 
> >
> 
> If filename is long and include #hex, Mac OS X request FPGetFileDirParms
> and FPResolveID command.
> 
> When there is a directory of the same CNID as #hex on netatalk's volume,
> netatalk return kFPObjectTypeErr to FPResolveID, and the copy fails.
> (See osx105-neta21dev.dump)
> 
> Mac OS X return kFPIDNotFound to FPResolveID, and the copy succeeds.
> (See osx105-osx104.dump)
> 
> #0 - #10 are no problem because these CNID is reserved.
> 
> Netatalk's result code is more right than Mac OS X.
> However, netatalk should behave as well as Mac OS X.
> I wrote a one-line patch.
> (netatalk-2.1dev-resolveid_dir.patch)
Thanks, I'll try to do more tests because in the test suite we
specifically test for both values, meaning that at least some Apple
server versions (pre OSX?) were returning kFPObjectTypeErr and it's the
kind of stuff where returning the wrong value can badly confuse a client
in weird places.

After these tests I think I will put something like:
 if (afp_version < 3) return  kFPObjectTypeErr else kFPIDNotFound.

> Still another problem exists.
> When a file or directory of the same CNID as #hex is in the distination
> directory, the following message is displayed.
> 
> >
> >A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
> >this location. Do you want to replace it with the older one you
> >are moving?
> >                                             [Stop]   [Replace]
> >
> 
> If "Replace" is clicked, this file or directory is overwrited,
> though it is another filenmame.
> This problem occur on both netatalk and Mac OS X.
> Mac OS X is defective.
> I think there is no countermeasure.
I agree.

This problem is worse with netatalk because volumes export low cnid
numbers. On a Mac system these low numbers are more likely to be used by
unshared files (system files, whatever).

For a user 'foo item #1000.txt' can look like a sensible name,
'foo item #4BDF.txt'  a little less :)   
   
Didier

Hi,
Le samedi 20 décembre 2008 à 07:01 +0100, didier a écrit :

> > Netatalk's result code is more right than Mac OS X.
> > However, netatalk should behave as well as Mac OS X.
> > I wrote a one-line patch.
> > (netatalk-2.1dev-resolveid_dir.patch)
> Thanks, I'll try to do more tests because in the test suite we
> specifically test for both values, meaning that at least some Apple
> server versions (pre OSX?) were returning kFPObjectTypeErr and it's the
> kind of stuff where returning the wrong value can badly confuse a client
> in weird places.
> 
> After these tests I think I will put something like:
>  if (afp_version < 3) return  kFPObjectTypeErr else kFPIDNotFound.
> 
Commit in both branches, thanks

Didier
 

Hi,

On *20 Dec 2008 07:01:33 +0100*
didier <dga...@ma...> wrote:

> Le vendredi 19 décembre 2008 à 02:08 +0900, HAT a écrit :
> > Still another problem exists.
> > When a file or directory of the same CNID as #hex is in the distination
> > directory, the following message is displayed.
> > 
> > >
> > >A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
> > >this location. Do you want to replace it with the older one you
> > >are moving?
> > >                                             [Stop]   [Replace]
> > >
> > 
> > If "Replace" is clicked, this file or directory is overwrited,
> > though it is another filenmame.
> > This problem occur on both netatalk and Mac OS X.
> > Mac OS X is defective.
> > I think there is no countermeasure.
> I agree.
> 
> This problem is worse with netatalk because volumes export low cnid
> numbers. On a Mac system these low numbers are more likely to be used by
> unshared files (system files, whatever).
> 
> For a user 'foo item #1000.txt' can look like a sensible name,
> 'foo item #4BDF.txt'  a little less :)   

I wrote a patch.

-#define CNID_START               17
+#define CNID_START_SPEC          17
+#define CNID_START               0x10000

Please see a attached file.

In case of existing volume, (new CNID) is (last CNID)+1.
In case of new volume, CNID begins at 0x10001.

This is not a perfect method.
However, I think it is safe enough.

-- 
HAT

Hi,

2009/6/30 HAT <ha...@fa...>:
> Hi,
>
> On *20 Dec 2008 07:01:33 +0100*
> didier <dga...@ma...> wrote:
>
>> Le vendredi 19 décembre 2008 à 02:08 +0900, HAT a écrit :
>> > Still another problem exists.
>> > When a file or directory of the same CNID as #hex is in the distination
>> > directory, the following message is displayed.
>> >
>> > >
>> > >A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
>> > >this location. Do you want to replace it with the older one you
>> > >are moving?
>> > >                                             [Stop]   [Replace]
>> > >
>> >
>> > If "Replace" is clicked, this file or directory is overwrited,
>> > though it is another filenmame.
>> > This problem occur on both netatalk and Mac OS X.
>> > Mac OS X is defective.
>> > I think there is no countermeasure.
>> I agree.
>>
>> This problem is worse with netatalk because volumes export low cnid
>> numbers. On a Mac system these low numbers are more likely to be used by
>> unshared files (system files, whatever).

Afaik that doesn't prevent a collision on OS X if the file "matched"
by the #<id> is not in the share. I.e. netatalk and OS X suffer the
same.

>> For a user 'foo item #1000.txt' can look like a sensible name,
>> 'foo item #4BDF.txt'  a little less :)
>
> I wrote a patch.
>
> -#define CNID_START               17
> +#define CNID_START_SPEC          17
> +#define CNID_START               0x10000
>
> Please see a attached file.
>
> In case of existing volume, (new CNID) is (last CNID)+1.
> In case of new volume, CNID begins at 0x10001.
>
> This is not a perfect method.
> However, I think it is safe enough.

I'm not sure if I like the idea of introducing a hack for a bug, but
in the end it might be the cleanest thing to do.

If you think it's beneficial I can also add the functionality to dbd
to adjust any cnid between 17 and 0xffff ?

-Frank

Hi,

> I'm not sure if I like the idea of introducing a hack for a bug, but
> in the end it might be the cleanest thing to do.

iTunes user found this problem first.
Title of suite have #opus.
The symphony of 10000 opus doesn't exist.
Therefore, a lot of iTunes users will be happy.

> If you think it's beneficial I can also add the functionality to dbd
> to adjust any cnid between 17 and 0xffff ?

I cannot judge whether it is really necessary. 

# rm -rf .AppleDB/

-- 
HAT

2009/6/30 HAT <ha...@fa...>:
> Hi,
>
>> I'm not sure if I like the idea of introducing a hack for a bug, but
>> in the end it might be the cleanest thing to do.
>
> iTunes user found this problem first.
> Title of suite have #opus.
> The symphony of 10000 opus doesn't exist.
> Therefore, a lot of iTunes users will be happy.

;o)

>> If you think it's beneficial I can also add the functionality to dbd
>> to adjust any cnid between 17 and 0xffff ?
>
> I cannot judge whether it is really necessary.

Well, your patch doesn't do anything for existing iTunes users and
they're existing volumes.

> # rm -rf .AppleDB/

Yeah. Even better:
# rm rf <volume>
Problem solved.

-Frank

Le mardi 30 juin 2009 à 19:14 +0200, Frank Lahm a écrit :
> Hi,
> 
> 2009/6/30 HAT <ha...@fa...>:
> > Hi,
> >
> > On *20 Dec 2008 07:01:33 +0100*
> > didier <dga...@ma...> wrote:
> >
> >> Le vendredi 19 décembre 2008 à 02:08 +0900, HAT a écrit :
> >> > Still another problem exists.
> >> > When a file or directory of the same CNID as #hex is in the distination
> >> > directory, the following message is displayed.
> >> >
> >> > >
> >> > >A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
> >> > >this location. Do you want to replace it with the older one you
> >> > >are moving?
> >> > >                                             [Stop]   [Replace]
> >> > >
> >> >
> >> > If "Replace" is clicked, this file or directory is overwrited,
> >> > though it is another filenmame.
> >> > This problem occur on both netatalk and Mac OS X.
> >> > Mac OS X is defective.
> >> > I think there is no countermeasure.
> >> I agree.
> >>
> >> This problem is worse with netatalk because volumes export low cnid
> >> numbers. On a Mac system these low numbers are more likely to be used by
> >> unshared files (system files, whatever).
> 
> Afaik that doesn't prevent a collision on OS X if the file "matched"
> by the #<id> is not in the share. I.e. netatalk and OS X suffer the
> same.
> 
> >> For a user 'foo item #1000.txt' can look like a sensible name,
> >> 'foo item #4BDF.txt'  a little less :)
> >
> > I wrote a patch.
> >
> > -#define CNID_START               17
> > +#define CNID_START_SPEC          17
> > +#define CNID_START               0x10000
> >
> > Please see a attached file.
> >
> > In case of existing volume, (new CNID) is (last CNID)+1.
> > In case of new volume, CNID begins at 0x10001.
> >
> > This is not a perfect method.
> > However, I think it is safe enough.
> 
> I'm not sure if I like the idea of introducing a hack for a bug, but
> in the end it might be the cleanest thing to do.
Does it still use 17 for "Network Trash Folder"? We need it for
compatibility with OS9.
> 
> If you think it's beneficial I can also add the functionality to dbd
> to adjust any cnid between 17 and 0xffff ?
I don't think we need it, it may create more problems than it solves
some applications do stupid things with cnid. Affected user on old
volumes can rm the DB.

Didier

Hi!

2009/7/16 didier <dga...@ma...>:
> Le mardi 30 juin 2009 à 19:14 +0200, Frank Lahm a écrit :
>> Hi,
>>
>> 2009/6/30 HAT <ha...@fa...>:
>> > Hi,
>> >
>> > On *20 Dec 2008 07:01:33 +0100*
>> > didier <dga...@ma...> wrote:
>> >
>> >> Le vendredi 19 décembre 2008 à 02:08 +0900, HAT a écrit :
>> >> > Still another problem exists.
>> >> > When a file or directory of the same CNID as #hex is in the distination
>> >> > directory, the following message is displayed.
>> >> >
>> >> > >
>> >> > >A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
>> >> > >this location. Do you want to replace it with the older one you
>> >> > >are moving?
>> >> > >                                             [Stop]   [Replace]
>> >> > >
>> >> >
>> >> > If "Replace" is clicked, this file or directory is overwrited,
>> >> > though it is another filenmame.
>> >> > This problem occur on both netatalk and Mac OS X.
>> >> > Mac OS X is defective.
>> >> > I think there is no countermeasure.
>> >> I agree.
>> >>
>> >> This problem is worse with netatalk because volumes export low cnid
>> >> numbers. On a Mac system these low numbers are more likely to be used by
>> >> unshared files (system files, whatever).
>>
>> Afaik that doesn't prevent a collision on OS X if the file "matched"
>> by the #<id> is not in the share. I.e. netatalk and OS X suffer the
>> same.
>>
>> >> For a user 'foo item #1000.txt' can look like a sensible name,
>> >> 'foo item #4BDF.txt'  a little less :)
>> >
>> > I wrote a patch.
>> >
>> > -#define CNID_START               17
>> > +#define CNID_START_SPEC          17
>> > +#define CNID_START               0x10000
>> >
>> > Please see a attached file.
>> >
>> > In case of existing volume, (new CNID) is (last CNID)+1.
>> > In case of new volume, CNID begins at 0x10001.
>> >
>> > This is not a perfect method.
>> > However, I think it is safe enough.
>>
>> I'm not sure if I like the idea of introducing a hack for a bug, but
>> in the end it might be the cleanest thing to do.
>
> Does it still use 17 for "Network Trash Folder"? We need it for
> compatibility with OS9.

I guess, I have to test it though.

-Frank

2009/7/16 Frank Lahm <fra...@go...>:
> 2009/7/16 didier <dga...@ma...>:
>> Le mardi 30 juin 2009 à 19:14 +0200, Frank Lahm a écrit :
>>> 2009/6/30 HAT <ha...@fa...>:
>>> > On *20 Dec 2008 07:01:33 +0100*
>>> > didier <dga...@ma...> wrote:
>>> >
>>> >> Le vendredi 19 décembre 2008 à 02:08 +0900, HAT a écrit :
>>> >> > Still another problem exists.
>>> >> > When a file or directory of the same CNID as #hex is in the distination
>>> >> > directory, the following message is displayed.
>>> >> >
>>> >> > >
>>> >> > >A newer item named "xxxxxxxxxxxxxxxxxxxxxxxx" already exists in
>>> >> > >this location. Do you want to replace it with the older one you
>>> >> > >are moving?
>>> >> > >                                             [Stop]   [Replace]
>>> >> > >
>>> >> >
>>> >> > If "Replace" is clicked, this file or directory is overwrited,
>>> >> > though it is another filenmame.
>>> >> > This problem occur on both netatalk and Mac OS X.
>>> >> > Mac OS X is defective.
>>> >> > I think there is no countermeasure.
>>> >> I agree.
>>> >>
>>> >> This problem is worse with netatalk because volumes export low cnid
>>> >> numbers. On a Mac system these low numbers are more likely to be used by
>>> >> unshared files (system files, whatever).
>>>
>>> Afaik that doesn't prevent a collision on OS X if the file "matched"
>>> by the #<id> is not in the share. I.e. netatalk and OS X suffer the
>>> same.
>>>
>>> >> For a user 'foo item #1000.txt' can look like a sensible name,
>>> >> 'foo item #4BDF.txt'  a little less :)
>>> >
>>> > I wrote a patch.
>>> >
>>> > -#define CNID_START               17
>>> > +#define CNID_START_SPEC          17
>>> > +#define CNID_START               0x10000
>>> >
>>> > Please see a attached file.
>>> >
>>> > In case of existing volume, (new CNID) is (last CNID)+1.
>>> > In case of new volume, CNID begins at 0x10001.
>>> >
>>> > This is not a perfect method.
>>> > However, I think it is safe enough.
>>>
>>> I'm not sure if I like the idea of introducing a hack for a bug, but
>>> in the end it might be the cleanest thing to do.
>>
>> Does it still use 17 for "Network Trash Folder"? We need it for
>> compatibility with OS9.
>
> I guess, I have to test it though.

As it seems only raising CNID_START is not enough because as suggested
by Didier "Network Trash Folder" wont be assigned CNID 17:

$ s dbd -d /Volumes/CNID/
CNID database dump:
dbd stamp: 0x4a645a11 (Jul 20 2009 13:50:41), next free CNID: 65542
id:      65537, did:          2, type: f, dev: 0x41, ino:
0x000000000fcec120, name: .DS_Store
id:      65538, did:          2, type: d, dev: 0x41, ino:
0x000000000fcec084, name: Network Trash Folder
id:      65539, did:          2, type: d, dev: 0x41, ino:
0x000000000fcec08a, name: Temporary Items
id:      65540, did:          2, type: d, dev: 0x41, ino:
0x000000000fe5801a, name: dir
id:      65541, did:          2, type: d, dev: 0x41, ino:
0x000000000ed24002, name: netatalk-docs
5 CNIDs in database. Max CNID: 65541.

-Frank