afpfs-ng-devel

[Afpfs-ng-devel] length of volumename is wrong

[HAT <ha...@fa...>]

Alex,

Have you already noticed a problem of volumename length?

If volumename is non-ASCII, UTF8 string is very long.
Actually, I cannot mount two of three volumes on Mac OS X 10.5.4.
This problem is fatal.

In include/afp_protocol.h,

#define AFP_VOLUME_NAME_LEN 33

For AFP2.x, this command returns volume names in ANSI format with a
maximum length of 27 bytes.


#define AFP_VOLUME_NAME_UTF8_LEN 33

For AFP3.x, this command returns volume names in UTF-8 format with a one
byte length byte specifying any length up to 255. Note that the Finder
limits setting volume names to no more than 27 characters.

#define AFP_MACHINETYPE_LEN 33

A string in Pascal format of up to 16 characters that describes the file
servers hardware and software but has no significance to AFP.


If you do not know it, I report after examining it a little more.

-- 
HAT

Re: [Afpfs-ng-devel] length of volumename is wrong

[Alex d. <ale...@gm...>]

HAT,

Yes, that doesn't seem right and I cannot remember where I got that  
data from.  I probably wrote that when I started this two years, and  
in looking through the 3.1 docs it looks incorrect.  I may have taken  
this from netatalk.

Is the solution just to increase the length of  
AFP_VOLUME_NAME_UTF8_LEN to 255?

It looks like AFP_VOLUME_NAME_LEN should be 27.

Are there similiar problems with server name lengths?

I actually use the machine type to detect the server type, which is  
useful for:
- compensating for broken servers, particularly netatalk but also the  
Apple extreme
- displaying logos in GUIs when browsing for servers

There is a larger problem with filenames, I have a maximum filename  
length of 255 bytes, but in reality the length is up to 16 bits.  In  
many places I preallocate the space for the filename, which is fine  
for a small number, but there are many places I don't want to allocate  
64k.  Changing this requires a lot of rework of anything that  
allocates space that references a filename.   This is left for a later  
release.  I expect you'll hit this too.



- Alex



On 16-Jul-08, at 9:28 AM, HAT wrote:

> Alex,
>
> Have you already noticed a problem of volumename length?
>
> If volumename is non-ASCII, UTF8 string is very long.
> Actually, I cannot mount two of three volumes on Mac OS X 10.5.4.
> This problem is fatal.
>
> In include/afp_protocol.h,
>
> #define AFP_VOLUME_NAME_LEN 33
>
> For AFP2.x, this command returns volume names in ANSI format with a
> maximum length of 27 bytes.
>
>
> #define AFP_VOLUME_NAME_UTF8_LEN 33
>
> For AFP3.x, this command returns volume names in UTF-8 format with a  
> one
> byte length byte specifying any length up to 255. Note that the Finder
> limits setting volume names to no more than 27 characters.
>
> #define AFP_MACHINETYPE_LEN 33
>
> A string in Pascal format of up to 16 characters that describes the  
> file
> servers hardware and software but has no significance to AFP.
>
>
> If you do not know it, I report after examining it a little more.
>
> -- 
> HAT
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Afpfs-ng-devel mailing list
> Afp...@li...
> https://lists.sourceforge.net/lists/listinfo/afpfs-ng-devel

Re: [Afpfs-ng-devel] length of volumename is wrong

[HAT <ha...@fa...>]

Alex,

>Yes, that doesn't seem right and I cannot remember where I got that  
>data from.  I probably wrote that when I started this two years, and  
>in looking through the 3.1 docs it looks incorrect.  I may have taken  
>this from netatalk.

netatalk's volumename length is buggy.
http://www003.upp.so-net.ne.jp/hat/netatalk/cjk.html

>Is the solution just to increase the length of  
>AFP_VOLUME_NAME_UTF8_LEN to 255?
>
>It looks like AFP_VOLUME_NAME_LEN should be 27.
>
>Are there similiar problems with server name lengths?

Your sources are not clear which bare string or Pascal string.
Pascal string include leading one byte.

-------------------------------------------------------
Apple Filing Protocol Programming Guide
Page 16

>AFPNames are encoded in conformance to the Unicode standard (UTF-8),
>which uses 16-bits to encode more than 65,000 characters.
>To keep character coding simple and efficient, the Unicode Standard
>assigns each character a unique numeric value and name.
>To help when converting from UTF-8 to other script systems,
>AFPNames begin with a four-byte text encoding hint that specifies
>the script that was originally used to compose the name.
>The text encoding hint is followed by a two-byte length field
>specifying the length of the UTF-8 encoded name that follows.

-------------------------------------------------------
#define AFP_SERVER_NAME_LEN

Apple Filing Protocol Programming Guide
Page 11

> A string in Pascal format of up to 32 characters.

A leading-octet of Pascal string is length.

#define AFP_SERVER_NAME_LEN 31

-------------------------------------------------------
#define AFP_SERVER_NAME_UTF8_LEN

Apple Filing Protocol Reference
Page 97

>UTF-8ServerName (AFPName)
>AFPName containing the UTF-8-encoded name of the server.

#define AFP_SERVER_NAME_UTF8_LEN 65535

This is not realistic.
If UTF-8ServerName is 255 over, this volume should be disregarded.

#define AFP_SERVER_NAME_UTF8_LEN 255

-------------------------------------------------------
#define AFP_VOLUME_NAME_LEN

Apple Filing Protocol Reference
Page 101

>For AFP 2.x, this command returns volume names in ANSI format
>with a maximum length of 27 bytes.

#define AFP_VOLUME_NAME_LEN 27

Maybe, the classic client has length of 31 bytes for volumename.
4 bytes have been reserved for suffix so that the same volumename may exist.

31 - 4 = 27

-------------------------------------------------------
#define AFP_VOLUME_NAME_UTF8_LEN

Apple Filing Protocol Reference
Page 101

>For AFP 3.x, this command returns volume names in UTF-8 format
>with a one byte length byte  specifying any length up to 255.
>Note that the Finder limits setting volume names to no more than 
>27 characters. 

#define AFP_VOLUME_NAME_UTF8_LEN 255

Do not confuse "character" with "byte".
Generally,
One Latin character is 2 bytes as UTF8.
One CJK character is 3 bytes as UTF8.

27 CJK characters are 81 bytes.

If character is decomposed, it is more complex.

-------------------------------------------------------
#define AFP_SIGNATURE_LEN 16

ok.

-------------------------------------------------------
#define AFP_MACHINETYPE_LEN

Apple Filing Protocol Programming Guide
Page 11

>A string in Pascal format of up to 16 characters that describes
>the file server's hardware and software but has no significance to AFP.

#define AFP_MACHINETYPE_LEN 15

-------------------------------------------------------
#define AFP_LOGINMESG_LEN

Apple Filing Protocol Reference
Page 100

>The usual size of any of these messages is 200 bytes including
>the length byte (a Str199). 
>AFP 3.x clients can request that the server send longer attention
>messages by setting the attention quantum size in the Option field
>of the DSOpenSession command (described in the document Apple Filing 
>Protocol Client).

#define AFP_LOGINMESG_LEN 199 or 200 ???

If UTF8 string is 200 over, this should be disregarded.

or

#define AFP_LOGINMESG_UTF8_LEN 65535

-------------------------------------------------------
#define AFP_VOLPASS_LEN 8

maybe, ok.

If DHX, 255.

-------------------------------------------------------
#define AFP_HOSTNAME_LEN 255

maybe, ok.

-------------------------------------------------------
#define AFP_ZONE_LEN 255

I don't know it.

-------------------------------------------------------
#define AFP_SERVER_ICON_LEN 256

ok.

Apple Filing Protocol Programming Guide
Page 11

>A optional value of 256 bytes that is used to customize
>the appearance of server volumes on the Mac OS Desktop.
>It consists of a 32-by-32 bit (128 bytes) icon bitmap followed
>by a 32-by-32 bit (128 bytes) icon mask.
>The mask usually consists of the icon's outline filled with black
>(bits that are set). For more information about icons,
>refer to Inside Mac OS X.

mask is 128.

-------------------------------------------------------
#define AFP_MAX_USERNAME_LEN

Apple Filing Protocol Reference
Page 116 
>The name can be a string of up to 255 Macintosh Roman characters
>or an AFPName of up to 255 characters.

#define AFP_MAX_USERNAME_LEN 255

-------------------------------------------------------
#define AFP_MAX_PASSWORD_LEN

maybe, 255

If cleartext or random number exchange, it is 8.

-------------------------------------------------------

>There is a larger problem with filenames, I have a maximum filename  
>length of 255 bytes, but in reality the length is up to 16 bits.  In  
>many places I preallocate the space for the filename, which is fine  
>for a small number, but there are many places I don't want to allocate  
>64k.  Changing this requires a lot of rework of anything that  
>allocates space that references a filename.   This is left for a later  
>release.  I expect you'll hit this too.

I will explain it after tomorrow.

-- 
HAT

Re: [Afpfs-ng-devel] length of volumename is wrong

[Alex d. <ale...@gm...>]

HAT,

Thanks for going through this.  I'll make the appropriate changes in  
CVS soon.  Most of them are pretty obvious.

Some comments on some of the broken items:

On 16-Jul-08, at 1:16 PM, HAT wrote:
>
> Your sources are not clear which bare string or Pascal string.
> Pascal string include leading one byte.

All these string lengths are for non-pascal strings.


> -------------------------------------------------------
> #define AFP_LOGINMESG_LEN
>
> Apple Filing Protocol Reference
> Page 100
>
>> The usual size of any of these messages is 200 bytes including
>> the length byte (a Str199).
>> AFP 3.x clients can request that the server send longer attention
>> messages by setting the attention quantum size in the Option field
>> of the DSOpenSession command (described in the document Apple Filing
>> Protocol Client).
>
> #define AFP_LOGINMESG_LEN 199 or 200 ???
>
> If UTF8 string is 200 over, this should be disregarded.
>
> or
>
> #define AFP_LOGINMESG_UTF8_LEN 65535

Is there such a thing as a UTF8 login message?

In reality, I've never seen a DSIAttention message that had anything  
in it (except for the login message).

I think I'll leave this at 199.


> -------------------------------------------------------
> #define AFP_VOLPASS_LEN 8
>
> maybe, ok.
>
> If DHX, 255.

Well, with DHX the volume password isn't really a volume password; it  
is done by authenticating with any username and a password.  So I  
think this definition is correct
.
> -------------------------------------------------------
> #define AFP_MAX_PASSWORD_LEN
>
> maybe, 255
>
> If cleartext or random number exchange, it is 8.

Okay, that's a UAM-specific change.  I'll delve into that code to see  
if that's respected.


>
> -------------------------------------------------------
>
>> There is a larger problem with filenames, I have a maximum filename
>> length of 255 bytes, but in reality the length is up to 16 bits.  In
>> many places I preallocate the space for the filename, which is fine
>> for a small number, but there are many places I don't want to  
>> allocate
>> 64k.  Changing this requires a lot of rework of anything that
>> allocates space that references a filename.   This is left for a  
>> later
>> release.  I expect you'll hit this too.
>
> I will explain it after tomorrow.

Yeah, this is a lot of work to redo this, and not the highest item on  
my list.  It also needs to be done in an AFP version specific way.


- Alex

Re: [Afpfs-ng-devel] length of volumename is wrong

[HAT <ha...@fa...>]

Sorry.
I experimented on the connection from Mac OS 9.2.2 to Mac OS X 10.5.4.

>-------------------------------------------------------
>#define AFP_SERVER_NAME_LEN
>
>Apple Filing Protocol Programming Guide
>Page 11
>
>> A string in Pascal format of up to 32 characters.
>
>A leading-octet of Pascal string is length.
>
>#define AFP_SERVER_NAME_LEN 31

about servername:

abcdefghijklmnopqrstuvwxyzABCDE
possible to access

abcdefghijklmnopqrstuvwxyzABCDEF
possible to access

abcdefghijklmnopqrstuvwxyzABCDEFG
impossible to access

#define AFP_SERVER_NAME_LEN 32

>-------------------------------------------------------
>#define AFP_MACHINETYPE_LEN
>
>Apple Filing Protocol Programming Guide
>Page 11
>
>>A string in Pascal format of up to 16 characters that describes
>>the file server's hardware and software but has no significance to AFP.
>
>#define AFP_MACHINETYPE_LEN 15

maybe,
#define AFP_MACHINETYPE_LEN 16

-- 
HAT

Re: [Afpfs-ng-devel] length of volumename is wrong

[Alex d. <ale...@gm...>]

Okay, I have committed the suggested changes to CVS.
  Thanks!
- A



On 16-Jul-08, at 1:16 PM, HAT wrote:

> Alex,
>
>> Yes, that doesn't seem right and I cannot remember where I got that
>> data from.  I probably wrote that when I started this two years, and
>> in looking through the 3.1 docs it looks incorrect.  I may have taken
>> this from netatalk.
>
> netatalk's volumename length is buggy.
> http://www003.upp.so-net.ne.jp/hat/netatalk/cjk.html
>
>> Is the solution just to increase the length of
>> AFP_VOLUME_NAME_UTF8_LEN to 255?
>>
>> It looks like AFP_VOLUME_NAME_LEN should be 27.
>>
>> Are there similiar problems with server name lengths?
>
> Your sources are not clear which bare string or Pascal string.
> Pascal string include leading one byte.
>
> -------------------------------------------------------
> Apple Filing Protocol Programming Guide
> Page 16
>
>> AFPNames are encoded in conformance to the Unicode standard (UTF-8),
>> which uses 16-bits to encode more than 65,000 characters.
>> To keep character coding simple and efficient, the Unicode Standard
>> assigns each character a unique numeric value and name.
>> To help when converting from UTF-8 to other script systems,
>> AFPNames begin with a four-byte text encoding hint that specifies
>> the script that was originally used to compose the name.
>> The text encoding hint is followed by a two-byte length field
>> specifying the length of the UTF-8 encoded name that follows.
>
> -------------------------------------------------------
> #define AFP_SERVER_NAME_LEN
>
> Apple Filing Protocol Programming Guide
> Page 11
>
>> A string in Pascal format of up to 32 characters.
>
> A leading-octet of Pascal string is length.
>
> #define AFP_SERVER_NAME_LEN 31
>
> -------------------------------------------------------
> #define AFP_SERVER_NAME_UTF8_LEN
>
> Apple Filing Protocol Reference
> Page 97
>
>> UTF-8ServerName (AFPName)
>> AFPName containing the UTF-8-encoded name of the server.
>
> #define AFP_SERVER_NAME_UTF8_LEN 65535
>
> This is not realistic.
> If UTF-8ServerName is 255 over, this volume should be disregarded.
>
> #define AFP_SERVER_NAME_UTF8_LEN 255
>
> -------------------------------------------------------
> #define AFP_VOLUME_NAME_LEN
>
> Apple Filing Protocol Reference
> Page 101
>
>> For AFP 2.x, this command returns volume names in ANSI format
>> with a maximum length of 27 bytes.
>
> #define AFP_VOLUME_NAME_LEN 27
>
> Maybe, the classic client has length of 31 bytes for volumename.
> 4 bytes have been reserved for suffix so that the same volumename  
> may exist.
>
> 31 - 4 = 27
>
> -------------------------------------------------------
> #define AFP_VOLUME_NAME_UTF8_LEN
>
> Apple Filing Protocol Reference
> Page 101
>
>> For AFP 3.x, this command returns volume names in UTF-8 format
>> with a one byte length byte  specifying any length up to 255.
>> Note that the Finder limits setting volume names to no more than
>> 27 characters.
>
> #define AFP_VOLUME_NAME_UTF8_LEN 255
>
> Do not confuse "character" with "byte".
> Generally,
> One Latin character is 2 bytes as UTF8.
> One CJK character is 3 bytes as UTF8.
>
> 27 CJK characters are 81 bytes.
>
> If character is decomposed, it is more complex.
>
> -------------------------------------------------------
> #define AFP_SIGNATURE_LEN 16
>
> ok.
>
> -------------------------------------------------------
> #define AFP_MACHINETYPE_LEN
>
> Apple Filing Protocol Programming Guide
> Page 11
>
>> A string in Pascal format of up to 16 characters that describes
>> the file server's hardware and software but has no significance to  
>> AFP.
>
> #define AFP_MACHINETYPE_LEN 15
>
> -------------------------------------------------------
> #define AFP_LOGINMESG_LEN
>
> Apple Filing Protocol Reference
> Page 100
>
>> The usual size of any of these messages is 200 bytes including
>> the length byte (a Str199).
>> AFP 3.x clients can request that the server send longer attention
>> messages by setting the attention quantum size in the Option field
>> of the DSOpenSession command (described in the document Apple Filing
>> Protocol Client).
>
> #define AFP_LOGINMESG_LEN 199 or 200 ???
>
> If UTF8 string is 200 over, this should be disregarded.
>
> or
>
> #define AFP_LOGINMESG_UTF8_LEN 65535
>
> -------------------------------------------------------
> #define AFP_VOLPASS_LEN 8
>
> maybe, ok.
>
> If DHX, 255.
>
> -------------------------------------------------------
> #define AFP_HOSTNAME_LEN 255
>
> maybe, ok.
>
> -------------------------------------------------------
> #define AFP_ZONE_LEN 255
>
> I don't know it.
>
> -------------------------------------------------------
> #define AFP_SERVER_ICON_LEN 256
>
> ok.
>
> Apple Filing Protocol Programming Guide
> Page 11
>
>> A optional value of 256 bytes that is used to customize
>> the appearance of server volumes on the Mac OS Desktop.
>> It consists of a 32-by-32 bit (128 bytes) icon bitmap followed
>> by a 32-by-32 bit (128 bytes) icon mask.
>> The mask usually consists of the icon's outline filled with black
>> (bits that are set). For more information about icons,
>> refer to Inside Mac OS X.
>
> mask is 128.
>
> -------------------------------------------------------
> #define AFP_MAX_USERNAME_LEN
>
> Apple Filing Protocol Reference
> Page 116
>> The name can be a string of up to 255 Macintosh Roman characters
>> or an AFPName of up to 255 characters.
>
> #define AFP_MAX_USERNAME_LEN 255
>
> -------------------------------------------------------
> #define AFP_MAX_PASSWORD_LEN
>
> maybe, 255
>
> If cleartext or random number exchange, it is 8.
>
> -------------------------------------------------------
>
>> There is a larger problem with filenames, I have a maximum filename
>> length of 255 bytes, but in reality the length is up to 16 bits.  In
>> many places I preallocate the space for the filename, which is fine
>> for a small number, but there are many places I don't want to  
>> allocate
>> 64k.  Changing this requires a lot of rework of anything that
>> allocates space that references a filename.   This is left for a  
>> later
>> release.  I expect you'll hit this too.
>
> I will explain it after tomorrow.
>
> -- 
> HAT
>
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's  
> challenge
> Build the coolest Linux based applications with Moblin SDK & win  
> great prizes
> Grand prize is a trip for two to an Open Source event anywhere in  
> the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> Afpfs-ng-devel mailing list
> Afp...@li...
> https://lists.sourceforge.net/lists/listinfo/afpfs-ng-devel

[Afpfs-ng-devel] volumename length patch (Re: length of volumename is wrong)

[HAT <ha...@fa...>]

I wrote a patch for current CVS HEAD.

I replaced a lot of AFP_VOLUME_NAME_LEN with AFP_VOLUME_NAME_UTF8_LEN
because neither classic-volumename nor UTF-volumename are distinguished
mostly in sources.

I tested Mac OS X 10.5.4 - Fedora 9.
This run well.

It is also needed to examin AFP_SERVER_NAME_LEN and AFP_SERVER_NAME_UTF8_LEN.

-- 
HAT

Re: [Afpfs-ng-devel] volumename length patch (Re: length of volumename is wrong)

[Alex d. <ale...@gm...>]

HAT,

That all seems pretty reasonable.  I'll apply it tonight.  Have you  
tried this code with long volume names?

I'll verify it against a Mac OS 9 system, but I can't see how that  
would be broken.

And congratulations for hunting through my code, I hope that wasn't  
too unpleasant.


- A


On 18-Jul-08, at 11:33 AM, HAT wrote:

> I wrote a patch for current CVS HEAD.
>
> I replaced a lot of AFP_VOLUME_NAME_LEN with AFP_VOLUME_NAME_UTF8_LEN
> because neither classic-volumename nor UTF-volumename are  
> distinguished
> mostly in sources.
>
> I tested Mac OS X 10.5.4 - Fedora 9.
> This run well.
>
> It is also needed to examin AFP_SERVER_NAME_LEN and  
> AFP_SERVER_NAME_UTF8_LEN.
>
> -- 
> HAT
> <cvs20080718-volname-length.patch.gz>

Re: [Afpfs-ng-devel] volumename length patch

[HAT <ha...@fa...>]

Alex,

> That all seems pretty reasonable.  I'll apply it tonight.  Have you  
> tried this code with long volume names?

The following volumename is ok.


"ÀÀÀÀÀÀÀÀÀÀ"
	30Bytes as decomposed-UTF8
"HAT のパブリックフォルダ"
	34Bytes as precomposed-UTF8
"がぎぐげごがぎぐげご"
	60Bytes as decomposed-UTF8
"각각각각각각각각각각"
	90Bytes as decomposed-UTF8


"HAT のパブリックフォルダ" means "HAT's Public Folder".
Public Folder's volumename is precomposed-UTF8.
This is Mac OS X's bug.
Other volumename is decomposed-UTF8.

When too long foldername is made a shared folder, Finder becomes  
unstable. This is Mac OS X's bug.

The folder that is normally shared can be normally mounted by afpfs-ng.


> I'll verify it against a Mac OS 9 system, but I can't see how that  
> would be broken.

I do not test Mac OS 9 at all because MacJapanese is not supported.

-- 
HAT

Re: [Afpfs-ng-devel] volumename length patch

[Alex d. <ale...@gm...>]

HAT,

Okay, I've committed about half of these; the others overlap with some  
other local changes, and I'll wait with committing those until I have  
some other things done.

By the way, you have some changes to files in the 'fuse' directory,  
which has now disappeared.  You may want to prune your cvs checkout.

Thanks for your help!

- A

On 18-Jul-08, at 2:48 PM, HAT wrote:

> Alex,
>
>> That all seems pretty reasonable.  I'll apply it tonight.  Have you  
>> tried this code with long volume names?
>
> The following volumename is ok.
>
>
> "ÀÀÀÀÀÀÀÀÀÀ"
> 	30Bytes as decomposed-UTF8
> "HAT のパブリックフォルダ"
> 	34Bytes as precomposed-UTF8
> "がぎぐげごがぎぐげご"
> 	60Bytes as decomposed-UTF8
> "각각각각각각각각각각"
> 	90Bytes as decomposed-UTF8
>
>
> "HAT のパブリックフォルダ" means "HAT's Public Folder".
> Public Folder's volumename is precomposed-UTF8.
> This is Mac OS X's bug.
> Other volumename is decomposed-UTF8.
>
> When too long foldername is made a shared folder, Finder becomes  
> unstable. This is Mac OS X's bug.
>
> The folder that is normally shared can be normally mounted by afpfs- 
> ng.
>
>
>> I'll verify it against a Mac OS 9 system, but I can't see how that  
>> would be broken.
>
> I do not test Mac OS 9 at all because MacJapanese is not supported.
>
> -- 
> HAT
>