Re: [Relfs-devel] About relfs-devel

[Vincenzo a. N. N. <vin...@ya...>]

On Thursday 29 July 2004 16:13, Fabio Tranchitella wrote:

> I didn't try any database-based filesystem, but AFAIK Apple is
> developing a similar solution. I'm sure we need next generation
> filesystems, with automatic handles of file types, automatic indexing
> and a powerful search engine.
>

As anybody willing to write something "different", I know I will have to 
face sooner or later the challenge of prooving that I am not copying 
apple :) Well, the author of storage already had to do the same thing, 
see

http://www.gnome.org/~seth/blog/document-indexing

basically apple's new jewel is an indexing system and not a complete 
filesystem.

> You answered me that mssql server is not lightweight. Yes, I agree,
> but PostgreSQL too. In general, a database-based filesystem will be
> slower (and slower) than a traditional tree-based filesystem.

Postgres is not going to eat that much memory, but in general what we 
should try to achieve is asynchronous indexing, in order to keep 
storage speed comparable to the native one.

> IMHO this type of filesystem can't replace traditional filesystems.
> I'd prefer to use a traditional fs for my debian system, and I think
> this is the best choice. A relational filesystem can be useful only
> for document and personal users folders. Do you agree?

Well, I think that there are many files in an unix filesystem which 
would live better if stored and retrieved through a database (this is 
not in contrast with the principle of keeping data on disk, but maybe 
that principle is too restrictive anyway). For example, linux 
distributions have huge package databases, which are often queried with 
grep or awk, and would fit perfectly into a database model (they _are_ 
databases in fact). Also, the ability to synthesize a file on the fly 
would allow for automatic maintenance of things like /etc/modutils/ 
-> /etc/modules, and so on. There are many kinds of files which would 
benefit from a db indexing - if you have ever typed "find /usr/lib 
-exec nm "{}" ";" |grep something" you know what I mean. 

> > Regarding to storage, I have written to seth but received no reply.
> > Either he is busy, or he is not trusting an open development
> > process. Or he just missed my message, I will try to contact him
> > again and on some mailing list, too.
>
> I think we have to avoid code duplication. I've not yet tried
> storage.. Did you do this? Is that usable? The best thing would be
> extend storage adding some features actually are missing... Maybe
> relfs could be integrated in storage, as you suggested.
>

I still can't compile storage, but I read on seth's page that the 
version in gnome-cvs is not recent code. So our first task seems to be 
"look better at storage". I'll have a second look, if you can look at 
it yourself. In the meanwhile I have written a second e-mail to Seth 
asking about recent code.

> > (e.g. I would tend to keep
> > real data on the hard drive, to ensure data visibility in case of
> > the hard drive being mounted from a PC without postgresql - Maybe
> > psql can do this by itself, I don't know yet)

> And in the case you used to mount a relfs filesystem "without
> postgresql" and removed, added or modified some documents, how the
> metadatas could be updated? How do you imagine this case?
>

I am still unsure about how files should be _stored_, but regarding to 
indexing, if plugins have the proof obligation that their effects on 
the db are the same if data stored in a file is the same, we can 
reindex modified files (sigh, looking at the modification time or upon 
user request...). It's not an ideal solution but I can't think of 
anything better right now, without totally replacing the filesystem 
with the database (but this way destroying any hope of native-like 
speed, I suppose, not to mention disk usage). Metadata can be stored, 
as a backup, in XFS/patched ext2/ext3 extended attributes (but will not 
be updated on external modifications). The main reason to store data on 
the disk, besides speed, is that users need to trust the system. Nobody 
would use a system which makes user files "disappear" from the disk.

Bye

Vincenzo