ivdfs-devel Mailing List for Incredibly Versatile Distributed FS
Status: Pre-Alpha
Brought to you by:
nkukard
You can subscribe to this list here.
2006 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(6) |
Oct
|
Nov
(1) |
Dec
|
---|---|---|---|---|---|---|---|---|---|---|---|---|
2007 |
Jan
|
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
(1) |
Oct
|
Nov
|
Dec
|
2008 |
Jan
|
Feb
|
Mar
(1) |
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
2011 |
Jan
(1) |
Feb
|
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
From: Karthik R. <kr...@gm...> - 2011-01-01 22:47:35
|
Hi, I would like to join the project for developing the versatile distributed file system. This is the first time I'm joining a project on SourceForge. It would be of great help if you tell how we get started on this project. -- Thanks Karthik |
From: <sv...@li...> - 2008-03-24 18:30:51
|
Author: nkukard Date: 2008-03-24 18:07:26 +0000 (Mon, 24 Mar 2008) New Revision: 6 Added: trunk/TODO Log: * Added some thoughts Added: trunk/TODO =================================================================== --- trunk/TODO (rev 0) +++ trunk/TODO 2008-03-24 18:07:26 UTC (rev 6) @@ -0,0 +1,31 @@ +* Tracking of file versions +Store create create time for each file and version number + +file => createtime, version + + + + +* Check file is available +- Broadcast IS_FILE_AVAIL to all nodes + + +* Read a block +- Broadcase IS_BLOCK_AVAIL to all nodes + + + +FILE OPERATIONS: + + +When file is closed... +- If file was written to + - Increase version number + - Journal change diff + offset, len, data_block + +When file is unlinked... +- If file was unlinked + - Delete file + - Journal change + |
From: <sv...@li...> - 2007-09-29 08:35:41
|
Author: nkukard Date: 2007-09-29 08:05:40 +0000 (Sat, 29 Sep 2007) New Revision: 5 Modified: trunk/src/Makefile.am trunk/src/Makefile.in trunk/src/common.h trunk/src/fsop_open.c trunk/src/fsop_write.c trunk/src/listener.c Log: * More preliminary work Modified: trunk/src/Makefile.am =================================================================== --- trunk/src/Makefile.am 2007-09-08 16:07:41 UTC (rev 4) +++ trunk/src/Makefile.am 2007-09-29 08:05:40 UTC (rev 5) @@ -58,5 +58,5 @@ ivdfs_SOURCES += fsop_removexattr.c ivdfs_CFLAGS = @FUSE_CFLAGS@ $(AM_CFLAGS) -ivdfs_LDFLAGS = @FUSE_LIBS@ +ivdfs_LDFLAGS = @FUSE_LIBS@ -levent Modified: trunk/src/Makefile.in =================================================================== --- trunk/src/Makefile.in 2007-09-08 16:07:41 UTC (rev 4) +++ trunk/src/Makefile.in 2007-09-29 08:05:40 UTC (rev 5) @@ -193,7 +193,7 @@ fsop_release.c fsop_flush.c fsop_fsync.c fsop_setxattr.c \ fsop_getxattr.c fsop_listxattr.c fsop_removexattr.c ivdfs_CFLAGS = @FUSE_CFLAGS@ $(AM_CFLAGS) -ivdfs_LDFLAGS = @FUSE_LIBS@ +ivdfs_LDFLAGS = @FUSE_LIBS@ -levent all: all-am .SUFFIXES: Modified: trunk/src/common.h =================================================================== --- trunk/src/common.h 2007-09-08 16:07:41 UTC (rev 4) +++ trunk/src/common.h 2007-09-29 08:05:40 UTC (rev 5) @@ -24,10 +24,31 @@ #include <fuse.h> +// Journal stuff +struct journal_entry_t { + uint64_t timestamp; // Timestamp the operation ahppened on + char *fpath; // Path of tile + uint32_t type; // Type of entry this is? write/truncate/create/unlink/rename + off_t pos_start; // Offset this starts at + off_t pos_end; // Offset it ends at + char *data; // The actual data +}; + + + +// Macro's to get the FD temporarily #define GET_FD(aaa) aaa->fh #define SET_FD(aaa,bbb) aaa->fh = bbb +// Debug macro +#define DEBUG(aaa,bbb...) fprintf(stderr,bbb) + +// Bitops +#define BIT_ISSET(aaa,bbb) (aaa & bbb) + + + // fsop functions void *fsop_init(void); void fsop_destroy(void *); @@ -71,3 +92,5 @@ void *listener(void *); + +// vim: ts=4 Modified: trunk/src/fsop_open.c =================================================================== --- trunk/src/fsop_open.c 2007-09-08 16:07:41 UTC (rev 4) +++ trunk/src/fsop_open.c 2007-09-29 08:05:40 UTC (rev 5) @@ -29,18 +29,88 @@ int fsop_open(const char *fpath, struct fuse_file_info *fi) { int fd; + int flags = fi->flags; char path[PATH_MAX]; setpath(fpath,path); - - fd = open(path, fi->flags); + + + + if (flags & O_RDONLY) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_RDONLY\n",path); + } + + if (flags & O_WRONLY) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_WRONLY\n",path); + } + + if (flags & O_RDWR) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_RDWR\n",path); + } + + if (flags & O_CREAT) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_CREAT\n",path); + } + + if (flags & O_EXCL) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_EXCL\n",path); + } + + if (flags & O_TRUNC) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_TRUNC\n",path); + } + + if (flags & O_APPEND) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_APPEND\n",path); + } + + if (flags & O_NONBLOCK) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_NONBLOCK\n",path); + } + + if (flags & O_SYNC) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_SYNC\n",path); + } + + if (flags & O_ASYNC) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_ASYNC\n",path); + } + + + + if (fi->flags & O_DIRECT) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_DIRECT\n",path); + } + + if (fi->flags & O_DIRECTORY) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_DIRECTORY\n",path); + } + + if (fi->flags & O_NOFOLLOW) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_NOFOLLOW\n",path); + } + + if (fi->flags & O_NOATIME) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_NOATIME\n",path); + } + + + + if (fi->flags & O_LARGEFILE) { + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_LARGEFILE\n",path); + } + + + DEBUG(LOG_DEBUG,"IVDFS - File '%s' => %i/%i/%i\n",path,flags,O_RDONLY,O_LARGEFILE); + + + fd = open(path, flags); if (fd == -1) return -errno; SET_FD(fi,fd); + DEBUG(LOG_DEBUG,"IVDFS - File '%s' open, fd %i\n",path,fd); - fprintf(stderr,"IVDFS - File open as %i\n",fd); - return 0; } Modified: trunk/src/fsop_write.c =================================================================== --- trunk/src/fsop_write.c 2007-09-08 16:07:41 UTC (rev 4) +++ trunk/src/fsop_write.c 2007-09-29 08:05:40 UTC (rev 5) @@ -39,7 +39,7 @@ if (res == -1) return -errno; - fprintf(stderr,"ONCLOSE OPERATION: update contents => %s\n",fpath); + DEBUG(LOG_DEBUG,"IVDFS - Write '%s' %u@%u res %i\n",path,size,offset,res); return res; } Modified: trunk/src/listener.c =================================================================== --- trunk/src/listener.c 2007-09-08 16:07:41 UTC (rev 4) +++ trunk/src/listener.c 2007-09-29 08:05:40 UTC (rev 5) @@ -20,19 +20,184 @@ #include "common.h" +#include <sys/types.h> +#include <sys/socket.h> +#include <netinet/in.h> +#include <errno.h> #include <stdio.h> +#include <string.h> +#include <netdb.h> +#include <event.h> -// Listener thread +/* Event handle */ +struct event_base *event_base; + + +/* Read callback for udp */ +void callback_uread(int fd, short event, void *arg) { + size_t nbytes; + char msgbuf[1024]; + struct sockaddr addr; + int addrlen; + + + addrlen = sizeof(addr); + + fprintf(stderr,"CALLBACK_UREAD\n"); + + if ((nbytes = recvfrom(fd,&msgbuf,1024,0,(struct sockaddr *) &addr, &addrlen)) < 0) { + fprintf(stderr,"Error recvfrom(): %s\n",strerror(errno)); + } + fprintf(stderr,"Message: %s\n",msgbuf); + + event_add(arg,NULL); +} +/* Write callback for udp */ +void callback_uwrite(int fd, short event, void *arg) { + fprintf(stderr,"CALLBACK_UWRITE\n"); +} + + + +/* Read callback for tcp */ +void callback_tread(int fd, short event, void *arg) { + size_t nbytes; + char msgbuf[1024]; + struct sockaddr addr; + int addrlen; + + + addrlen = sizeof(addr); + + fprintf(stderr,"CALLBACK_TREAD\n"); + + /* Accept */ + int conn = accept(fd,&addr,&addrlen); + shutdown(conn,SHUT_RDWR); + close(conn); +/* + if ((nbytes = recvfrom(fd,&msgbuf,1024,0,(struct sockaddr *) &addr, &addrlen)) < 0) { + fprintf(stderr,"Error recvfrom(): %s\n",strerror(errno)); + } + fprintf(stderr,"Message: %s\n",msgbuf); +*/ + event_add(arg,NULL); +} + +/* Write callback for tcp */ +void callback_twrite(int fd, short event, void *arg) { + fprintf(stderr,"CALLBACK_TWRITE\n"); +} + + + +/* Listener thread */ void *listener(void *data) { + /* Event handler */ + struct event event_uread; + struct event event_tread; + + + /* IPv6 */ + struct sockaddr_in6 from; + struct sockaddr_in6 sin; + struct servent *sp; + + int usock, tsock; + int reuse = 1; + + + memset((char *)&sin, 0, sizeof(sin)); + + sin.sin6_port = htons(6066); + + sin.sin6_addr = in6addr_any; + + if ((usock = socket(AF_INET6, SOCK_DGRAM, 0)) == -1) { + fprintf(stderr,"Failed to listen ip6 for dgram: %s\n",strerror(errno)); + return NULL; + } + + if (setsockopt(usock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)) < 0) { + fprintf(stderr,"Failed to setsockopt: %s\n",strerror(errno)); + return NULL; + } + + if (bind(usock, (struct sockaddr *) &sin, sizeof(sin)) < 0) { + fprintf(stderr,"Failed to bind: %s\n",strerror(errno)); + return NULL; + } + + + /* And IPv6/TCP */ + memset((char *)&sin, 0, sizeof(sin)); + + sin.sin6_port = htons(6066); + + sin.sin6_addr = in6addr_any; + + if ((tsock = socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP)) == -1) { + fprintf(stderr,"Failed to listen ip6 for stream: %s\n",strerror(errno)); + return NULL; + } + + if (setsockopt(tsock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)) == -1) { + fprintf(stderr,"Failed to setsockopt: %s\n",strerror(errno)); + return NULL; + } + + if (bind(tsock, (struct sockaddr *) &sin, sizeof(sin)) == -1) { + fprintf(stderr,"Failed to bind: %s\n",strerror(errno)); + return NULL; + } + + if (listen(tsock, 16) == -1) { + fprintf(stderr,"Failed to listen: %s\n",strerror(errno)); + return NULL; + } + + + /* Initialize even library */ + event_base = event_init(); + + event_set(&event_uread,usock,EV_READ,callback_uread,&event_uread); + event_base_set(event_base,&event_uread); + event_add(&event_uread,NULL); + + event_set(&event_tread,tsock,EV_READ,callback_tread,&event_tread); + event_base_set(event_base,&event_tread); + event_add(&event_tread,NULL); + + event_dispatch(); + + fprintf(stderr,"We exited from event_dispatch()\n"); + +/* + while (1) { + struct sockaddr_in addr; + int addrlen; + size_t nbytes; + char msgbuf[1024]; + + addrlen = sizeof(addr); + + if ((nbytes = recvfrom(sock,&msgbuf,1024,0,(struct sockaddr *) &addr, &addrlen)) < 0) { + fprintf(stderr,"Error recvfrom(): %s\n",strerror(errno)); + } + fprintf(stderr,"Message: %s\n",msgbuf); + } + while (1) { sleep(30); printf("THREAD TRIGGER\n"); } +*/ return NULL; } +// vim: ts=4 |
From: <sv...@li...> - 2006-11-15 10:47:40
|
Author: nkukard Date: 2006-11-15 10:30:01 +0000 (Wed, 15 Nov 2006) New Revision: 3 Modified: trunk/src/chmod.c trunk/src/chown.c trunk/src/fsync.c trunk/src/ftruncate.c trunk/src/link.c trunk/src/mknod.c trunk/src/removexattr.c trunk/src/rename.c trunk/src/rmdir.c trunk/src/setxattr.c trunk/src/symlink.c trunk/src/truncate.c trunk/src/unlink.c trunk/src/utime.c Log: * Added some debugging info Modified: trunk/src/chmod.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/chmod.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/chmod.c 2006-11-15 10:30:01 UTC (rev 3) @@ -36,6 +36,8 @@ res =3D chmod(path, mode); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: chmod =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/chown.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/chown.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/chown.c 2006-11-15 10:30:01 UTC (rev 3) @@ -36,6 +36,8 @@ res =3D lchown(path, uid, gid); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: chown =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/fsync.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/fsync.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/fsync.c 2006-11-15 10:30:01 UTC (rev 3) @@ -34,9 +34,15 @@ setpath(fpath,path); =20 if (isdatasync) + { + fprintf(stderr,"OPERATION: sync data =3D> %s\n",fpath); res =3D fdatasync(fi->fh); + }=09 else + { + fprintf(stderr,"OPERATION: sync all =3D> %s\n",fpath); res =3D fsync(fi->fh); + } =20 if (res =3D=3D -1) return -errno; Modified: trunk/src/ftruncate.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/ftruncate.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/ftruncate.c 2006-11-15 10:30:01 UTC (rev 3) @@ -37,6 +37,8 @@ res =3D ftruncate(fi->fh, size); if (res =3D=3D -1) return -errno; + =09 + fprintf(stderr,"ONCLOSE OPERATION: update =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/link.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/link.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/link.c 2006-11-15 10:30:01 UTC (rev 3) @@ -38,6 +38,8 @@ res =3D link(from, to); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: link =3D> %s\n",ffrom); =20 return 0; } Modified: trunk/src/mknod.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/mknod.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/mknod.c 2006-11-15 10:30:01 UTC (rev 3) @@ -34,9 +34,16 @@ setpath(fpath,path); =20 if (S_ISFIFO(mode)) + { + fprintf(stderr,"OPERATION: mkfifo =3D> %s\n",fpath); res =3D mkfifo(path, mode); + } else + { + fprintf(stderr,"OPERATION: mknod =3D> %s\n",fpath); res =3D mknod(path, mode, rdev); + } + if (res =3D=3D -1) return -errno; =20 Modified: trunk/src/removexattr.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/removexattr.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/removexattr.c 2006-11-15 10:30:01 UTC (rev 3) @@ -36,6 +36,8 @@ res =3D lremovexattr(path, name); if (res =3D=3D -1) return -errno; + =09 + fprintf(stderr,"OPERATION: remove xattr =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/rename.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/rename.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/rename.c 2006-11-15 10:30:01 UTC (rev 3) @@ -38,6 +38,8 @@ res =3D rename(from, to); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: rename =3D> %s\n",ffrom); =20 return 0; } Modified: trunk/src/rmdir.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/rmdir.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/rmdir.c 2006-11-15 10:30:01 UTC (rev 3) @@ -36,6 +36,8 @@ res =3D rmdir(path); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: rmdir =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/setxattr.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/setxattr.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/setxattr.c 2006-11-15 10:30:01 UTC (rev 3) @@ -36,6 +36,8 @@ res =3D lsetxattr(path, name, value, size, flags); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: set xattr =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/symlink.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/symlink.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/symlink.c 2006-11-15 10:30:01 UTC (rev 3) @@ -37,6 +37,8 @@ if (res =3D=3D -1) return -errno; =20 + fprintf(stderr,"OPERATION: symlink =3D> %s\n",from); + return 0; } =20 Modified: trunk/src/truncate.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/truncate.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/truncate.c 2006-11-15 10:30:01 UTC (rev 3) @@ -36,6 +36,8 @@ res =3D truncate(path, size); if (res =3D=3D -1) return -errno; +=09 + fprintf(stderr,"OPERATION: truncate =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/unlink.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/unlink.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/unlink.c 2006-11-15 10:30:01 UTC (rev 3) @@ -37,7 +37,7 @@ if (res =3D=3D -1) return -errno; =20 - fprintf(stderr,"OPERATION: remove =3D> %s\n",fpath); + fprintf(stderr,"OPERATION: unlink =3D> %s\n",fpath); =20 return 0; } Modified: trunk/src/utime.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- trunk/src/utime.c 2006-09-29 07:11:29 UTC (rev 2) +++ trunk/src/utime.c 2006-11-15 10:30:01 UTC (rev 3) @@ -37,6 +37,8 @@ if (res =3D=3D -1) return -errno; =20 + fprintf(stderr,"OPERATION: utime =3D> %s\n",fpath); + return 0; } =20 |
From: Ian C. B. <ia...@bl...> - 2006-09-29 15:36:59
|
Has anyone played with cleversafe's dsgrid stuff yet? http://www.cleversafe.org This is the kind of thing I think we're looking for. - Ian C. Blenke <ia...@bl...> http://ian.blenke.com/ |
From: Sean E. <ed...@no...> - 2006-09-20 21:28:32
|
Ok, that sounds good. I must have misunderstood. On Wednesday 20 September 2006 16:49, Nigel Kukard wrote: > Yo, > > Stupid subject line. > > > Just a thought regarding the client/server as opposed to the P2P > > architecture, the way I see it is that doing a P2P system would be > > beneficial in the sense of collaborative editing. On the other hand, you > > could end up with a client/server architecture as well, through the use > > of dedicated servers, which just mirror the network filesystem as a > > whole. It would allow, say, 2 workstations and a backup server to be > > constantly synced together, > > I think the subject is getting a bit confused. > > P2P is peer-to-peer. Defined as each peer being equal in the duties it > performs. This is what we want. It eliminates the single point of > failure with a master-slave architecture. > > > whereas with a client-server structure you wouldn't have that > > advantage. Really the only difference would be that a workstation > > running IVDFS would be able to accept updates and update its own local > > copy, as well as sending updates to the "server(s)", and the "server(s)" > > would be able to function as > > workstations as well, and write to its own local system and update the > > other machines on the grid. And, more importantly, the filesystem /could/ > > still function as the proposed client/server layout. > > This is the behavior were after. > > The point I tried to make is the goal for ivdfs is not anonymous file > sharing, for instance bittorrent, gnutella ... etc. Or storing of > anonymous data in using hashes ... etc. It could be designed into ivdfs, > I have nothing against that, its just not one of the primary goals for > the project. > > The primary goal is replication and raid-like behavior for high > availability file storage across different networks, servers or whatever > in the simplest manner possible, and to preserve the underlying > filesystem in such a way that using plain replications its possible to > stop ivdfs and still access the files on the local filesystem. > > So to address your comment above, one could mount a server fs on his/her > workstation and have it setup in such a way that only files in ones own > directory be replicated and the rest pulled directly off the server, or > any other way in which you see fit :o) > > > -Nigel |
From: Nigel K. <nk...@lb...> - 2006-09-20 20:52:02
|
Yo, Stupid subject line. > Just a thought regarding the client/server as opposed to the P2P > architecture, the way I see it is that doing a P2P system would be > beneficial in the sense of collaborative editing. On the other hand, y= ou > could end up with a client/server architecture as well, through the us= e of > dedicated servers, which just mirror the network filesystem as a whole= =2E It > would allow, say, 2 workstations and a backup server to be constantly = synced > together, I think the subject is getting a bit confused. P2P is peer-to-peer. Defined as each peer being equal in the duties it performs. This is what we want. It eliminates the single point of failure with a master-slave architecture. > whereas with a client-server structure you wouldn't have that > advantage. Really the only difference would be that a workstation runn= ing > IVDFS would be able to accept updates and update its own local copy, a= s well > as sending updates to the "server(s)", and the "server(s)" would be ab= le to > function as > workstations as well, and write to its own local system and update the = other > machines on the grid. And, more importantly, the filesystem /could/ sti= ll > function as the proposed client/server layout. This is the behavior were after. The point I tried to make is the goal for ivdfs is not anonymous file sharing, for instance bittorrent, gnutella ... etc. Or storing of anonymous data in using hashes ... etc. It could be designed into ivdfs, I have nothing against that, its just not one of the primary goals for the project. The primary goal is replication and raid-like behavior for high availability file storage across different networks, servers or whatever in the simplest manner possible, and to preserve the underlying filesystem in such a way that using plain replications its possible to stop ivdfs and still access the files on the local filesystem. So to address your comment above, one could mount a server fs on his/her workstation and have it setup in such a way that only files in ones own directory be replicated and the rest pulled directly off the server, or any other way in which you see fit :o) -Nigel |
From: Sean E. <ed...@no...> - 2006-09-20 20:36:08
|
Just a thought regarding the client/server as opposed to the P2P architecture, the way I see it is that doing a P2P system would be beneficial in the sense of collaborative editing. On the other hand, you could end up with a client/server architecture as well, through the use of dedicated servers, which just mirror the network filesystem as a whole. It would allow, say, 2 workstations and a backup server to be constantly synced together, whereas with a client-server structure you wouldn't have that advantage. Really the only difference would be that a workstation running IVDFS would be able to accept updates and update its own local copy, as well as sending updates to the "server(s)", and the "server(s)" would be able to function as workstations as well, and write to its own local system and update the other machines on the grid. And, more importantly, the filesystem /could/ still function as the proposed client/server layout. ~Sean |
From: Nigel K. <nk...@lb...> - 2006-09-20 09:42:41
|
> > There appear to be no mail archives yet, has conversation yet begun? Just begun now, I'm still getting everybody to join the list though. > =20 > Incredibly Versatile? > =20 > Perhaps we should nail down what "Versatile" features we are looking f= or. Agreed. > =20 > CornFS was an attempt at making a simple copy-on-write mirroring=20 > filesystem that could be easily recovered. The primary goal was archiv= al=20 > use, with an eye toward rapid searchability, layering on top of any=20 > resilient networked filesystem. > =20 > Is this targeted to be a POSIX filesystem (complete with permissions=20 > and/or ACLs) or just a heirarchical archival filesystem? Well, all the features of underlying filesystem will be available through FUSE... think passthrough. I want this to work as simply as possible, so one can even stop FUSE and still access the files. File operations like rename, unlink, create and on-close-writes are then journalled and sync'd between nodes. > =20 > While talking with others about their pet projects, the primary=20 > attributes of focus seem to be: > =20 > Distributed metadata. > - Distributing the filesystem namespace across N number of networked n= odes. > Distributed data. Agreed > - Distributing the file data across N number of networked nodes (eithe= r=20 > full file copies or block striping). Or even other forms of RAID-like behaviour? I think FiST (RAIF) has some other interesting RAID-like options, can always peek at the code and dump chunks into our own implementation. > Distributed locking. > - Changes to the metadata and/or data need to be synchronized to keep = > the filesystem coherent. Agreed > Redundancy. > - Having N copies available on the network, so that content is always = > available. Yea ... I was thinking about a GFS-like approach where all nodes can ping eachother and have a vote when one of them cannot be reached to outcast it from the group. Apon being outcast, the outcasted node could switch to read-only mode or just return IO errors, or even carry on and re-sync later.... *shrug*, all are pretty easy to implement. > Single instance store. > - Use an MD5/SHA1 hash to uniquely identify and universally store a fi= le=20 > (or blocks of a file) for later recall. The less we touch the files on the underlying filesystem the better, hashes can be stored in the FUSE's own .whatever directory if needed. I think our first goal should be plain and simple replication ... nothing special, if a file changes. copy the entire thing. If created, create and copy data. If deleted delete it. No locking. Lets get a very simple working FS together, then implement the features above with the most important first. > Others have mentioned P2P networking instead of a private cluster mode= l. > =20 > While I'm interested in the above, I'd really like to see P2P get=20 > factored into this. > =20 > With a little public key cryptography, it would be possible to join=20 > something theoretically like Freenet with something more like bittorre= nt: > - Have "confederations" of metadata namespaces that link to one anothe= r=20 > using UUID or unique cryptographic hashes. > - Have a single instance store of files or blocks using MD5/SHA1 hashe= s. > - Have "webs of trust" for both metadata and file data so that only yo= ur=20 > friends can access your filesystems or file data, yet still allow=20 > objects to be replicated without knowing what is inside a file (ala=20 > Freenet). With webs of trust, one unified network would be possible,=20 > with storage availablity spanning _all_ nodes. "Dark" nets would not b= e=20 > necessary, as it would be part of the model. > - Swarm file downloads from both "seed" and other "downloading" nodes = > (ala Bittorrent). I'm thinking p2p is more for another dedicated project, or maybe a branch of this one? My major goal is to provide 100% redundant file storage across datacenters and provide a true globally distributed filesystem. > I'm envisioning something that allows content to become a "permanent=20 > network copy", that will only go away if nobody is interested in it. > =20 > Also, while FUSE is neat, FiST holds much promise. Something like RAIF= =20 > is one of the alternatives to CornFS, and I've thought about rewriting= =20 > CornFS to be a layed FiST filesystem. > =20 > http://www.filesystems.org > =20 > If the goal is to go toward P2P, implementing a loopback WebDAV=20 > interface might be the most universal way to present a filesystem=20 > (though it is non-POSIX compliant). Not sure if FiST is included in the vanilla kernel? the less patching needed the better, one tends to loose ALOT of interest in a project the second you need to patch something. Especially if the server being patched is under a maintenance contract with a vendor, patching the kernel in this scenario is almost always guaranteed to void your contract= =2E > I look forward to your thoughts. The main requirement I have for this project is to replicate virtual hosting data between datacenters, clients websites and email (maildir). I need to switch between centers should a natural disaster or major outage occur. -Nigel |
From: Ian C. B. <ia...@bl...> - 2006-09-19 20:11:58
|
There appear to be no mail archives yet, has conversation yet begun? Incredibly Versatile? Perhaps we should nail down what "Versatile" features we are looking for. CornFS was an attempt at making a simple copy-on-write mirroring filesystem that could be easily recovered. The primary goal was archival use, with an eye toward rapid searchability, layering on top of any resilient networked filesystem. Is this targeted to be a POSIX filesystem (complete with permissions and/or ACLs) or just a heirarchical archival filesystem? While talking with others about their pet projects, the primary attributes of focus seem to be: Distributed metadata. - Distributing the filesystem namespace across N number of networked nodes. Distributed data. - Distributing the file data across N number of networked nodes (either full file copies or block striping). Distributed locking. - Changes to the metadata and/or data need to be synchronized to keep the filesystem coherent. Redundancy. - Having N copies available on the network, so that content is always available. Single instance store. - Use an MD5/SHA1 hash to uniquely identify and universally store a file (or blocks of a file) for later recall. Others have mentioned P2P networking instead of a private cluster model. While I'm interested in the above, I'd really like to see P2P get factored into this. With a little public key cryptography, it would be possible to join something theoretically like Freenet with something more like bittorrent: - Have "confederations" of metadata namespaces that link to one another using UUID or unique cryptographic hashes. - Have a single instance store of files or blocks using MD5/SHA1 hashes. - Have "webs of trust" for both metadata and file data so that only your friends can access your filesystems or file data, yet still allow objects to be replicated without knowing what is inside a file (ala Freenet). With webs of trust, one unified network would be possible, with storage availablity spanning _all_ nodes. "Dark" nets would not be necessary, as it would be part of the model. - Swarm file downloads from both "seed" and other "downloading" nodes (ala Bittorrent). If anonymity is a goal, possibly work something functionally equivalent to The Onion Router into the mix. I'm envisioning something that allows content to become a "permanent network copy", that will only go away if nobody is interested in it. Also, while FUSE is neat, FiST holds much promise. Something like RAIF is one of the alternatives to CornFS, and I've thought about rewriting CornFS to be a layed FiST filesystem. http://www.filesystems.org If the goal is to go toward P2P, implementing a loopback WebDAV interface might be the most universal way to present a filesystem (though it is non-POSIX compliant). I look forward to your thoughts. - Ian C. Blenke <ia...@bl...> http://ian.blenke.com/projects/cornfs/ |