Re: [ivdfs-devel] Greetings

[Ian C. B. <ia...@bl...>]

There appear to be no mail archives yet, has conversation yet begun?

Incredibly Versatile?

Perhaps we should nail down what "Versatile" features we are looking for.

CornFS was an attempt at making a simple copy-on-write mirroring 
filesystem that could be easily recovered. The primary goal was archival 
use, with an eye toward rapid searchability, layering on top of any 
resilient networked filesystem.

Is this targeted to be a POSIX filesystem (complete with permissions 
and/or ACLs) or just a heirarchical archival filesystem?

While talking with others about their pet projects, the primary 
attributes of focus seem to be:

Distributed metadata.
- Distributing the filesystem namespace across N number of networked nodes.
Distributed data.
- Distributing the file data across N number of networked nodes (either 
full file copies or block striping).
Distributed locking.
- Changes to the metadata and/or data need to be synchronized to keep 
the filesystem coherent.
Redundancy.
- Having N copies available on the network, so that content is always 
available.
Single instance store.
- Use an MD5/SHA1 hash to uniquely identify and universally store a file 
(or blocks of a file) for later recall.

Others have mentioned P2P networking instead of a private cluster model.

While I'm interested in the above, I'd really like to see P2P get 
factored into this.

With a little public key cryptography, it would be possible to join 
something theoretically like Freenet with something more like bittorrent:
- Have "confederations" of metadata namespaces that link to one another 
using UUID or unique cryptographic hashes.
- Have a single instance store of files or blocks using MD5/SHA1 hashes.
- Have "webs of trust" for both metadata and file data so that only your 
friends can access your filesystems or file data, yet still allow 
objects to be replicated without knowing what is inside a file (ala 
Freenet). With webs of trust, one unified network would be possible, 
with storage availablity spanning _all_ nodes. "Dark" nets would not be 
necessary, as it would be part of the model.
- Swarm file downloads from both "seed" and other "downloading" nodes 
(ala Bittorrent).

If anonymity is a goal, possibly work something functionally equivalent 
to The Onion Router into the mix.

I'm envisioning something that allows content to become a "permanent 
network copy", that will only go away if nobody is interested in it.

Also, while FUSE is neat, FiST holds much promise. Something like RAIF 
is one of the alternatives to CornFS, and I've thought about rewriting 
CornFS to be a layed FiST filesystem.

  http://www.filesystems.org

If the goal is to go toward P2P, implementing a loopback WebDAV 
interface might be the most universal way to present a filesystem 
(though it is non-POSIX compliant).

I look forward to your thoughts.

- Ian C. Blenke <ia...@bl...> http://ian.blenke.com/projects/cornfs/