Re: [linuxdc-dev] VMU filesystem question

[M. R. B. <mr...@0x...>]

* David Willmore <dav...@ia...> on Mon, Dec 09, 2002:

>=20
> I have read that in the past and I just reread it as you requested.
>=20
[...]
>=20
> *what* extra info?
>=20

Read it again.  Every VMU file has an extra file "header" that is not
considered part of the file data, but is used when
displaying/parsing/validating the file's contents.  There should be two
fields that immediately stick out in that header: CRC and "Number of bytes
of actual file data following header". =20

On the proceeding page, there is a field in the root directory structure
labeled "offset of header (in blocks) from file start".  Although this could
be assumed to always be 0 for data files, it isn't a constant so there is a
possibility it could change.

Outside of vmufs, there is absolutely no way to update that data using
standard Unix VFS semantics.  On vmufs of course, the driver can keep it as
part of it's private inode data and update it dynamically whenever the file
is written to.

I'm surprised you asked that question since those fields are the reason Adr=
ian
asked his question in the first place.

>=20
> Yes.  Take a look at the SCSI subsystem.  I've been a linux developer
> since version .12 in fall of '92.  I'm not here to start some kind os
> pissing match and I'll ask you to respect that.
>=20

It was not my intention to start a pissing match - I posed that question as
a valid one since you seem to be missing the big picture, esp. on VFS
issues.  I apologize if my statements were peceived as such.

>=20
> What I am suggesting is to have the vmufs format the data in such a was
> as it will appear as normal files to the user and be editable as such.
> I don't see anything in the description of the directory nor the files
> that makes this unimplementable.

Check LinuxDC CVS, vmufs already exists and does what you mention.  The
problem is not how to expose the content of the files, it's what to do with
file header data that can't be transferred outside of vmufs.

>=20
> > The only disadvantage is a non-intuitive way to access file data, but at
> > least it's compatible with every other vmufs implementation (including =
the
> > DC browser itself) outside of Linux.
>=20
> Pardon?  I'm not suggesting changing *one byte* on the card.  We're talki=
ng
> about the vfs, right?  That's the interface in the kernel between high-ish
> level calls (open, write, read, close, dlookup, etc.) and the code which
> determines what is allowed and where it goes/comes from on the backing me=
dia.
>=20

Sorry if I misunderstood you, but in your preceeding post you proposed
changing the first block of the VMU file data into a non-standard format.

> The kernel does not care what the fs driver does with the data it's given
> nor how it comes up with the data it presents.  It just wants to give
> it paths and files and perform some basic fops on them or get errors tryi=
ng.
>=20

You forget that the kernel does care how the file data are represented.
Please see struct inode in include/linux/fs.h.

>=20
> I'll go into detail since this appears not to be to your liking:
>=20
[...]
> It looks like all of the data in the header will work well if exposed as
> seperate files in the .header directory associated with the file in quest=
ion.
>=20
> Shall I go into more detail?
>=20

Please look at the big picture.  Your above scheme is beautiful for
everything that just does vmufs <-> vmufs.  But what happens when I want to
create a brand new file on the mounted vmu, and I do a "touch myfile"?
Where does the header come from?  In order to remain compatible it must
either come from the user or have a default supplied by the vmufs driver.
Both scenarios aren't pretty but the latter makes it easier for the end
user to use the VMU as a generic storage device.  Note that without the
file header it's possible that the VMU will break for anything outside of
Linux (that hasn't been confirmed yet, but all DC software that uses the
vmu accesses the header).

The case that Adrian mentioned is "cp oldvmu/file newvmu/file".  Under your
proposed scheme this quickly become unmanageable as you now have:

$ cp oldvmu/file newvmu/file
$ cp oldvmu/.header/file newvmu/.header/file

and so on for each file that you want to copy.  Can you see the problem with
that?  Everytime you add a new virtual file that has to be tagged along
when the real file leaves vmufs, you decrease your flexibility.  Even
copying the entire directory tree(s) becomes problematic because as soon as
you copy, say the global VMU color you've destroyed what was on the other
VMU.  And unless you have everything under directories (since vmufs can't
physically support subdirectories) any special file in the root directory
pollutes your namespace.

To me, your scheme sounds more like you would have both a vmufs
driver that exposed the normal Unix files, and a /proc-like implementation
that exposed global VMU metadata.

Unfortunately, there is no precedence for exposing vmufs data in this way
other than MacOS-specific filesystems, which have always needed hacks to
play nice with PC-based OSes (data/rsrc crap comes to mind).  I can't think
of any other filesystem that doesn't do a simple map of file, directory, and
attributes on top of the VFS.

>=20
> That's the same kind of thing I'm proposing.  If you chose to rm -rf the
> .header data in the vmufs, you're going to mess things up.  So, uhh, don't
> do that if you don't want to mess up the VMU.
>=20

I wasn't on HFS/AppleShare when I did the rm.  vmufs can enforce the policy
that you can't remove .header/, but what other filesystem will?

> The .afpvol trick was just a way for the Mac to translate the way it saw
> things into the way the unix box saw things.  If the rules on how this
> 'hidden' or 'extra' information is handled is built into the filesystem
> so that you can't mess it up, you have a very useful tool.  If you just
> want to make it look like some simple files, you're going to allow people
> more chance to rm -rf your .afpvol so-to-speak.

The point is to make it look like simple files (the vmufs files
themselves).  A frequently requested feature of vmufs is (was, before it was
written) to a allow the VMU to function as a generic storage device.
Enforcing such policies within the filesystem driver makes this impossible.

Study the source in the fs/ directory, and see how other filesystems do it.
There is only a limited set of information that is transferable between
filesystems, and no .header/ scheme will ever overcome that.

>=20
> Yes, touching raw blocks when you don't now what you're doing is a bad id=
ea.
> With the 'make it look like a bunch of files' method, the user never gets
> a chance to do that.
>=20

Again, I think your scheme works well when using the VMU as strictly a
VMU-to-VMU transfer device.  I don't think it can hold up to other uses
without the vmufs driver making sweeping decisions over what data the VMU
can hold.  This is what Adrian is attempting to avoid in his design of
vmufs.

(Sorry if it looks like I'm speaking for Adrian in these posts, but this is
just rehashed stuff we've discussed on and off the list in the last few
weeks.)

M. R.