[Netatalk-admins] Re: AppleDouble -> Mac OS X resource fork?

[Thomas K. <Tho...@ph...>]

Craig Ringer wrote:

> I don't understand why we're having this argument.

Because there seem to be some system administrators that refuse to do their
homework and better think about patching a software down to a level where it
seems to meet *their* criteria. When this software is broken enough, they
call it 'cross platform'.

> Among other things, there's an existing system (OS/X server) that can do
> what you want, and if you feel that strongly about it, perhaps it's worth
> looking at it.

Sure? The last time I tried it, it didn't work. Files saved by MacOS X via
SMB lead to two files on the server's HFS(+) file system. Local access on
the server as well as AFP access leads to loss of metadata and resource
forks.

Okay, maybe some people are dumb enough to share UFS volumes with MacOS X
server. Then this might work probably but the drawbacks (speed and CNID
compatibility issues) are enormous.

If you use both SMB and AFP with a MacOS X machine on the same server you
always will run into problems, no matter which file server is in use. This
applies to Windows servers, EtherShare/PCShare, MacOS X itself, whatever.

> I can think of two less harmful possibilities that may be worthy of
> consideration, though I personally just don't care:
> (a) Writing a Samba VFS layer to map ._ files to .AppleDouble/ files,
> so MacOS/X can see and modify the subset of the information in the
> AppleDouble file that it understands when talking over SMB without
> destroying the other information NetATalk needs. I don't know if this is
> even possible, but it may be worth looking at.

There are already early patches available for both a netatalk and a samba
VFS in CVS. But the aim is not following this totally brain-dead "._"
attempt but instead creating a more *cross platform* solution by mapping Mac
specific metadata and resource forks as multiple streams on the samba side
(providing 'cross platform' compatibility in a way a Windows client will
automagically move/rename/delete our AppleDouble stuff when it handles files
and folders)

When people are starting their 'cross platform' bushwa as an excuse for the
lame "._" implementation then they actually speak about nothing more or less
than 'mixed protocol compatibility' (it's all Mac but a crude mixture of
protocols involved). They never really spoke about cross *platform* since
they never thought about the fact how to deal with all the metadata and
resource fork stuff when other *platforms* (like Windows for example) access
the same data on the server.

> Problems:
> - Locking, locking, locking. Preventing NetATalk and Samba stepping on each
> other is probably not trivial.

This is a more general problem since it is not working even with the pure
data fork as well. When locking is working in the future it would be
possible to extend locking to all files involved with multiple streams
(Didiers Patches in CVS do not only care about Mac specific stuff
:Afp_AfpInfo and :Afp_Resource but will provide Samba 3 with a more generic
support for ADS <http://www.alcpress.com/articles/ads.html>)

> - I'm not sure if it would even be possible to modify just the parts of the
> AppleDouble file that OSX understands and still have a usable AppleDouble
> file. It's well and truly outside my knowledge.

Okay. Back to the basics.

Netatalk uses AppleDouble v2 as MacOS X does.

Netatalk *fully* supports AppleDouble v2. MacOS X doesn't (they write just 2
IDs and they read just 2 IDs in a fixed order which is actually a mess since
it just met the criteria of the Finder folks but not them of more advanced
solutions)

You can simply take a ._ file from Mac OS X, rename and move it and afpd
will be happy with it (even with adouble:v2 set in AppleVolumes.default)

    ._file (crippled ADv2)
    file

    file
    .AppleDouble/file (still crippled ADv2, full support when first accessed
                       by an afpd client)

Due to Netatalk implementing Apple's specs correctly we *can* read that ADv2
file MacOS X created and live with it (since we can add our own set of other
IDs and metadata). Just the other way around isn't working since Apple
refuses to fully support their own specifications.

> (b) A utility to convert .AppleDouble based resource fork files to OS/X
> style ._ resource fork files and vice versa.

See Philon's posting in this thread. The whole problem wouldn't be existent
if Apple would fully implement their own specs. And Phil is *totally* wrong
when he repeats again and again that Apple now uses a different AppleDouble
scheme. It is still AppleDouble v2. The problem is the broken implementation
(fixed ID order assumed, not all IDs supported) and afpd simply cannot use
such a crippled metadata storage attempt without forfeiting functionality.
For the Finder folks it might work but an AFP server has to take care about
other things as well as just the simply resource fork and the 'Standard
Macintosh Finder Information'!

Everyone who is concerned by this should write bug reports to Apple.

If Apple would support their own specs correctly you would've simply renamed
the ADv2 files and you're done in such situations like recovering from a
backup.

> At least this would help people who are concerned about reliance on the
> NetATalk server, and I can imagine situations where such a tool could be
> handy. For example, restoring a backup tape of a NetATalk volume onto an
> OS/X machine.

Please keep in mind, that with this full load of volume options a backup/
restore solution has always to take care of

    $volroot/.AppleDesktop/.volinfo

where vital information about encodings, naming options and AppleDouble
schemes are stored. An example:

    MAC_CHARSET:MAC_ROMAN
    VOL_CHARSET:UTF8
    ADOUBLE_VER:v2
    CNIDBACKEND:cdb
    CNIDDBDHOST:localhost
    CNIDDBDPORT:4700
    CNID_DBPATH:/var/volumes/transfer
    VOLUME_OPTS:USEDOTS
    VOLCASEFOLD:

> Ideally the tool would be available for OS/X as well. For all I know
> such a tool may already exist or even be in NetATalk - I haven't really
> cared to look.

    http://netatalk.sourceforge.net/2.0/htmldocs/megatron.1.html

This utility can be used for such a purpose for example. Adding native
HFS(+) support can be taken from utilities like RSyncX for example and
providing a GUI shouldn't be that hard as well.

But someone has to start writing such a tool...

But before I would have a look to rsync. If I understand it correctly, the
rsync folks thought about adding generic support for HFS(+) metadata /
resource forks and different ways to implement the storage and transport of
these on flat filesystems. I've seen a discussion on the rsync list where
Wes Craig (one of the original Netatalk authors) has been involved...

Regards,

Thomas