Re: [ivdfs-devel] Greetings

[Nigel K. <nk...@lb...>]

>
> There appear to be no mail archives yet, has conversation yet begun?

Just begun now, I'm still getting everybody to join the list though.

> =20
>  Incredibly Versatile?
> =20
>  Perhaps we should nail down what "Versatile" features we are looking f=
or.
Agreed.

> =20
>  CornFS was an attempt at making a simple copy-on-write mirroring=20
>  filesystem that could be easily recovered. The primary goal was archiv=
al=20
>  use, with an eye toward rapid searchability, layering on top of any=20
>  resilient networked filesystem.
> =20
>  Is this targeted to be a POSIX filesystem (complete with permissions=20
>  and/or ACLs) or just a heirarchical archival filesystem?
Well, all the features of underlying filesystem will be available
through FUSE... think passthrough. I want this to work as simply as
possible, so one can even stop FUSE and still access the files.

File operations like rename, unlink, create and on-close-writes are then
journalled and sync'd between nodes.

> =20
>  While talking with others about their pet projects, the primary=20
>  attributes of focus seem to be:
> =20
>  Distributed metadata.
>  - Distributing the filesystem namespace across N number of networked n=
odes.
>  Distributed data.
Agreed

>  - Distributing the file data across N number of networked nodes (eithe=
r=20
>  full file copies or block striping).
Or even other forms of RAID-like behaviour? I think FiST (RAIF) has some
other interesting RAID-like options, can always peek at the code and
dump chunks into our own implementation.

>  Distributed locking.
>  - Changes to the metadata and/or data need to be synchronized to keep =

>  the filesystem coherent.
Agreed

>  Redundancy.
>  - Having N copies available on the network, so that content is always =

>  available.
Yea ... I was thinking about a GFS-like approach where all nodes can
ping eachother and have a vote when one of them cannot be reached to
outcast it from the group. Apon being outcast, the outcasted node could
switch to read-only mode or just return IO errors, or even carry on and
re-sync later.... *shrug*, all are pretty easy to implement.


>  Single instance store.
>  - Use an MD5/SHA1 hash to uniquely identify and universally store a fi=
le=20
>  (or blocks of a file) for later recall.
The less we touch the files on the underlying filesystem the better,
hashes can be stored in the FUSE's own .whatever directory if needed.

I think our first goal should be plain and simple replication ...
nothing special, if a file changes. copy the entire thing. If created,
create and copy data. If deleted delete it. No locking. Lets get a very
simple working FS together, then implement the features above with the
most important first.


>  Others have mentioned P2P networking instead of a private cluster mode=
l.
> =20
>  While I'm interested in the above, I'd really like to see P2P get=20
>  factored into this.
> =20
>  With a little public key cryptography, it would be possible to join=20
>  something theoretically like Freenet with something more like bittorre=
nt:
>  - Have "confederations" of metadata namespaces that link to one anothe=
r=20
>  using UUID or unique cryptographic hashes.
>  - Have a single instance store of files or blocks using MD5/SHA1 hashe=
s.
>  - Have "webs of trust" for both metadata and file data so that only yo=
ur=20
>  friends can access your filesystems or file data, yet still allow=20
>  objects to be replicated without knowing what is inside a file (ala=20
>  Freenet). With webs of trust, one unified network would be possible,=20
>  with storage availablity spanning _all_ nodes. "Dark" nets would not b=
e=20
>  necessary, as it would be part of the model.
>  - Swarm file downloads from both "seed" and other "downloading" nodes =

>  (ala Bittorrent).
I'm thinking p2p is more for another dedicated project, or maybe a
branch of this one?

My major goal is to provide 100% redundant file storage across
datacenters and provide a true globally distributed filesystem.

>  I'm envisioning something that allows content to become a "permanent=20
>  network copy", that will only go away if nobody is interested in it.
> =20
>  Also, while FUSE is neat, FiST holds much promise. Something like RAIF=
=20
>  is one of the alternatives to CornFS, and I've thought about rewriting=
=20
>  CornFS to be a layed FiST filesystem.
> =20
>    http://www.filesystems.org
> =20
>  If the goal is to go toward P2P, implementing a loopback WebDAV=20
>  interface might be the most universal way to present a filesystem=20
>  (though it is non-POSIX compliant).
Not sure if FiST is included in the vanilla kernel? the less patching
needed the better, one tends to loose ALOT of interest in a project the
second you need to patch something. Especially if the server being
patched is under a maintenance contract with a vendor, patching the
kernel in this scenario is almost always guaranteed to void your contract=
=2E


>  I look forward to your thoughts.
The main requirement I have for this project is to replicate virtual
hosting data between datacenters, clients websites and email (maildir).
I need to switch between centers should a natural disaster or major
outage occur.

-Nigel