ivdfs-devel — IVDFS Development Mailing List

[ivdfs-devel] Participation in the project

[Karthik R. <kr...@gm...>]

Hi,

I would like to join the project for developing the versatile
distributed file system. This is the first time I'm joining a project
on SourceForge. It would be of great help if you tell how we get
started on this project.

-- 
Thanks
Karthik

[ivdfs-devel] COMMIT - r6 - trunk

[<sv...@li...>]

Author: nkukard
Date: 2008-03-24 18:07:26 +0000 (Mon, 24 Mar 2008)
New Revision: 6

Added:
   trunk/TODO
Log:
* Added some thoughts


Added: trunk/TODO
===================================================================
--- trunk/TODO	                        (rev 0)
+++ trunk/TODO	2008-03-24 18:07:26 UTC (rev 6)
@@ -0,0 +1,31 @@
+* Tracking of file versions
+Store create create time for each file and version number
+
+file => createtime, version
+
+
+
+
+* Check file is available
+- Broadcast   IS_FILE_AVAIL  to all nodes
+
+
+* Read a block
+- Broadcase   IS_BLOCK_AVAIL  to all nodes
+
+
+
+FILE OPERATIONS:
+
+
+When file is closed...
+- If file was written to
+	- Increase version number
+	- Journal change diff
+		offset, len, data_block
+
+When file is unlinked...
+- If file was unlinked
+	- Delete file
+	- Journal change
+

[ivdfs-devel] COMMIT - r5 - trunk/src

[<sv...@li...>]

Author: nkukard
Date: 2007-09-29 08:05:40 +0000 (Sat, 29 Sep 2007)
New Revision: 5

Modified:
   trunk/src/Makefile.am
   trunk/src/Makefile.in
   trunk/src/common.h
   trunk/src/fsop_open.c
   trunk/src/fsop_write.c
   trunk/src/listener.c
Log:
* More preliminary work


Modified: trunk/src/Makefile.am
===================================================================
--- trunk/src/Makefile.am	2007-09-08 16:07:41 UTC (rev 4)
+++ trunk/src/Makefile.am	2007-09-29 08:05:40 UTC (rev 5)
@@ -58,5 +58,5 @@
 ivdfs_SOURCES += fsop_removexattr.c
 
 ivdfs_CFLAGS = @FUSE_CFLAGS@ $(AM_CFLAGS) 
-ivdfs_LDFLAGS = @FUSE_LIBS@
+ivdfs_LDFLAGS = @FUSE_LIBS@ -levent
 

Modified: trunk/src/Makefile.in
===================================================================
--- trunk/src/Makefile.in	2007-09-08 16:07:41 UTC (rev 4)
+++ trunk/src/Makefile.in	2007-09-29 08:05:40 UTC (rev 5)
@@ -193,7 +193,7 @@
 	fsop_release.c fsop_flush.c fsop_fsync.c fsop_setxattr.c \
 	fsop_getxattr.c fsop_listxattr.c fsop_removexattr.c
 ivdfs_CFLAGS = @FUSE_CFLAGS@ $(AM_CFLAGS) 
-ivdfs_LDFLAGS = @FUSE_LIBS@
+ivdfs_LDFLAGS = @FUSE_LIBS@ -levent
 all: all-am
 
 .SUFFIXES:

Modified: trunk/src/common.h
===================================================================
--- trunk/src/common.h	2007-09-08 16:07:41 UTC (rev 4)
+++ trunk/src/common.h	2007-09-29 08:05:40 UTC (rev 5)
@@ -24,10 +24,31 @@
 #include <fuse.h>
 
 
+// Journal stuff
+struct journal_entry_t {
+		uint64_t timestamp;  // Timestamp the operation ahppened on
+		char *fpath;  // Path of tile
+		uint32_t type;  // Type of entry this is?   write/truncate/create/unlink/rename
+		off_t pos_start;  // Offset this starts at
+		off_t pos_end;  // Offset it ends at
+		char *data;  // The actual data
+};
+
+
+
+// Macro's to get the FD temporarily
 #define GET_FD(aaa) aaa->fh
 #define SET_FD(aaa,bbb) aaa->fh = bbb
 
+// Debug macro
+#define DEBUG(aaa,bbb...) fprintf(stderr,bbb)
 
+
+// Bitops
+#define BIT_ISSET(aaa,bbb) (aaa & bbb)
+
+
+
 // fsop functions
 void *fsop_init(void);
 void fsop_destroy(void *);
@@ -71,3 +92,5 @@
 void *listener(void *);
 
 
+
+// vim: ts=4

Modified: trunk/src/fsop_open.c
===================================================================
--- trunk/src/fsop_open.c	2007-09-08 16:07:41 UTC (rev 4)
+++ trunk/src/fsop_open.c	2007-09-29 08:05:40 UTC (rev 5)
@@ -29,18 +29,88 @@
 int fsop_open(const char *fpath, struct fuse_file_info *fi)
 {
 	int fd;
+	int flags = fi->flags;
 
 	char path[PATH_MAX];
 	setpath(fpath,path);
-    
-	fd = open(path, fi->flags);
+
+
+
+	if (flags & O_RDONLY) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_RDONLY\n",path);
+	}
+
+	if (flags & O_WRONLY) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_WRONLY\n",path);
+	}
+
+	if (flags & O_RDWR) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_RDWR\n",path);
+	}
+
+	if (flags & O_CREAT) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_CREAT\n",path);
+	}
+
+	if (flags & O_EXCL) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_EXCL\n",path);
+	}
+
+	if (flags & O_TRUNC) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_TRUNC\n",path);
+	}
+
+	if (flags & O_APPEND) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_APPEND\n",path);
+	}
+
+	if (flags & O_NONBLOCK) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_NONBLOCK\n",path);
+	}
+
+	if (flags & O_SYNC) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_SYNC\n",path);
+	}
+
+	if (flags & O_ASYNC) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_ASYNC\n",path);
+	}
+
+
+
+	if (fi->flags & O_DIRECT) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_DIRECT\n",path);
+	}
+
+	if (fi->flags & O_DIRECTORY) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_DIRECTORY\n",path);
+	}
+
+	if (fi->flags & O_NOFOLLOW) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_NOFOLLOW\n",path);
+	}
+
+	if (fi->flags & O_NOATIME) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_NOATIME\n",path);
+	}
+
+
+
+	if (fi->flags & O_LARGEFILE) {
+		DEBUG(LOG_DEBUG,"IVDFS - File '%s' open with O_LARGEFILE\n",path);
+	}
+
+
+	DEBUG(LOG_DEBUG,"IVDFS - File '%s' => %i/%i/%i\n",path,flags,O_RDONLY,O_LARGEFILE);
+
+
+	fd = open(path, flags);
 	if (fd == -1)
 		return -errno;
 
 	SET_FD(fi,fd);
+	DEBUG(LOG_DEBUG,"IVDFS - File '%s' open, fd %i\n",path,fd);
 
-	fprintf(stderr,"IVDFS - File open as %i\n",fd);
-
 	return 0;
 }
 

Modified: trunk/src/fsop_write.c
===================================================================
--- trunk/src/fsop_write.c	2007-09-08 16:07:41 UTC (rev 4)
+++ trunk/src/fsop_write.c	2007-09-29 08:05:40 UTC (rev 5)
@@ -39,7 +39,7 @@
 	if (res == -1)
 		return -errno;
 
-	fprintf(stderr,"ONCLOSE OPERATION: update contents => %s\n",fpath);
+	DEBUG(LOG_DEBUG,"IVDFS - Write '%s' %u@%u res %i\n",path,size,offset,res);
 
 	return res;
 }

Modified: trunk/src/listener.c
===================================================================
--- trunk/src/listener.c	2007-09-08 16:07:41 UTC (rev 4)
+++ trunk/src/listener.c	2007-09-29 08:05:40 UTC (rev 5)
@@ -20,19 +20,184 @@
 
 #include "common.h"
 
+#include <sys/types.h>
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <errno.h>
 #include <stdio.h>
+#include <string.h>
+#include <netdb.h>
+#include <event.h>
 
 
-// Listener thread
+/* Event handle */
+struct event_base *event_base;
+
+
+/* Read callback for udp */
+void callback_uread(int fd, short event, void *arg) {
+	size_t nbytes;
+	char msgbuf[1024];
+	struct sockaddr addr;
+	int addrlen;
+
+
+	addrlen = sizeof(addr);
+
+	fprintf(stderr,"CALLBACK_UREAD\n");
+
+	if ((nbytes = recvfrom(fd,&msgbuf,1024,0,(struct sockaddr *) &addr, &addrlen)) < 0) {
+		fprintf(stderr,"Error recvfrom(): %s\n",strerror(errno));
+	}
+	fprintf(stderr,"Message: %s\n",msgbuf);
+
+	event_add(arg,NULL);
+}
+/* Write callback for udp */
+void callback_uwrite(int fd, short event, void *arg) {
+	fprintf(stderr,"CALLBACK_UWRITE\n");
+}
+
+
+
+/* Read callback for tcp */
+void callback_tread(int fd, short event, void *arg) {
+	size_t nbytes;
+	char msgbuf[1024];
+	struct sockaddr addr;
+	int addrlen;
+
+
+	addrlen = sizeof(addr);
+
+	fprintf(stderr,"CALLBACK_TREAD\n");
+
+	/* Accept */
+	int conn = accept(fd,&addr,&addrlen);
+	shutdown(conn,SHUT_RDWR);
+	close(conn);
+/*
+	if ((nbytes = recvfrom(fd,&msgbuf,1024,0,(struct sockaddr *) &addr, &addrlen)) < 0) {
+		fprintf(stderr,"Error recvfrom(): %s\n",strerror(errno));
+	}
+	fprintf(stderr,"Message: %s\n",msgbuf);
+*/
+	event_add(arg,NULL);
+}
+
+/* Write callback for tcp */
+void callback_twrite(int fd, short event, void *arg) {
+	fprintf(stderr,"CALLBACK_TWRITE\n");
+}
+
+
+
+/* Listener thread */
 void *listener(void *data)
 {
+	/* Event handler */
+	struct event event_uread;
+	struct event event_tread;
+
+
+	/* IPv6 */
+	struct sockaddr_in6 from;
+	struct sockaddr_in6 sin;
+	struct servent *sp;
+
+	int usock, tsock;
+	int reuse = 1;
+
+
+	memset((char *)&sin, 0, sizeof(sin));
+
+	sin.sin6_port = htons(6066);
+
+	sin.sin6_addr = in6addr_any;
+
+	if ((usock = socket(AF_INET6, SOCK_DGRAM, 0)) == -1) {
+		fprintf(stderr,"Failed to listen ip6 for dgram: %s\n",strerror(errno));
+		return NULL;
+	}
+
+	if (setsockopt(usock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)) < 0) {
+		fprintf(stderr,"Failed to setsockopt: %s\n",strerror(errno));
+		return NULL;
+	}
+
+	if (bind(usock, (struct sockaddr *) &sin, sizeof(sin)) < 0) {
+		fprintf(stderr,"Failed to bind: %s\n",strerror(errno));
+		return NULL;
+	}
+
+
+	/* And IPv6/TCP */
+	memset((char *)&sin, 0, sizeof(sin));
+
+	sin.sin6_port = htons(6066);
+
+	sin.sin6_addr = in6addr_any;
+
+	if ((tsock = socket(AF_INET6, SOCK_STREAM, IPPROTO_TCP)) == -1) {
+		fprintf(stderr,"Failed to listen ip6 for stream: %s\n",strerror(errno));
+		return NULL;
+	}
+
+	if (setsockopt(tsock, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof(reuse)) == -1) {
+		fprintf(stderr,"Failed to setsockopt: %s\n",strerror(errno));
+		return NULL;
+	}
+
+	if (bind(tsock, (struct sockaddr *) &sin, sizeof(sin)) == -1) {
+		fprintf(stderr,"Failed to bind: %s\n",strerror(errno));
+		return NULL;
+	}
+
+	if (listen(tsock, 16) == -1) {
+		fprintf(stderr,"Failed to listen: %s\n",strerror(errno));
+		return NULL;
+	}
+
+
+	/* Initialize even library */
+	event_base = event_init();
+
+	event_set(&event_uread,usock,EV_READ,callback_uread,&event_uread);
+	event_base_set(event_base,&event_uread);
+	event_add(&event_uread,NULL);
+
+	event_set(&event_tread,tsock,EV_READ,callback_tread,&event_tread);
+	event_base_set(event_base,&event_tread);
+	event_add(&event_tread,NULL);
+
+	event_dispatch();
+
+	fprintf(stderr,"We exited from event_dispatch()\n");
+
+/*	
+	while (1) {
+		struct sockaddr_in addr;
+		int addrlen;
+		size_t nbytes;
+		char msgbuf[1024];
+
+		addrlen = sizeof(addr);
+
+		if ((nbytes = recvfrom(sock,&msgbuf,1024,0,(struct sockaddr *) &addr, &addrlen)) < 0) {
+			fprintf(stderr,"Error recvfrom(): %s\n",strerror(errno));
+		}
+		fprintf(stderr,"Message: %s\n",msgbuf);
+	}
+
 	while (1)
 	{
 		sleep(30);
 		printf("THREAD TRIGGER\n");
 	}
+*/
 
 	return NULL;
 }
 
 
+// vim: ts=4

[ivdfs-devel] COMMIT - r3 - trunk/src

[<sv...@li...>]

Author: nkukard
Date: 2006-11-15 10:30:01 +0000 (Wed, 15 Nov 2006)
New Revision: 3

Modified:
   trunk/src/chmod.c
   trunk/src/chown.c
   trunk/src/fsync.c
   trunk/src/ftruncate.c
   trunk/src/link.c
   trunk/src/mknod.c
   trunk/src/removexattr.c
   trunk/src/rename.c
   trunk/src/rmdir.c
   trunk/src/setxattr.c
   trunk/src/symlink.c
   trunk/src/truncate.c
   trunk/src/unlink.c
   trunk/src/utime.c
Log:
* Added some debugging info


Modified: trunk/src/chmod.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/chmod.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/chmod.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -36,6 +36,8 @@
 	res =3D chmod(path, mode);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: chmod =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/chown.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/chown.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/chown.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -36,6 +36,8 @@
 	res =3D lchown(path, uid, gid);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: chown =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/fsync.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/fsync.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/fsync.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -34,9 +34,15 @@
 	setpath(fpath,path);
=20
 	if (isdatasync)
+	{
+		fprintf(stderr,"OPERATION: sync data =3D> %s\n",fpath);
 		res =3D fdatasync(fi->fh);
+	}=09
 	else
+	{
+		fprintf(stderr,"OPERATION: sync all =3D> %s\n",fpath);
 		res =3D fsync(fi->fh);
+	}
=20
 	if (res =3D=3D -1)
 		return -errno;

Modified: trunk/src/ftruncate.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/ftruncate.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/ftruncate.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -37,6 +37,8 @@
 	res =3D ftruncate(fi->fh, size);
 	if (res =3D=3D -1)
 		return -errno;
+	=09
+	fprintf(stderr,"ONCLOSE OPERATION: update =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/link.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/link.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/link.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -38,6 +38,8 @@
 	res =3D link(from, to);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: link =3D> %s\n",ffrom);
=20
 	return 0;
 }

Modified: trunk/src/mknod.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/mknod.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/mknod.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -34,9 +34,16 @@
 	setpath(fpath,path);
=20
 	if (S_ISFIFO(mode))
+	{
+		fprintf(stderr,"OPERATION: mkfifo =3D> %s\n",fpath);
 		res =3D mkfifo(path, mode);
+	}
 	else
+	{
+		fprintf(stderr,"OPERATION: mknod =3D> %s\n",fpath);
 		res =3D mknod(path, mode, rdev);
+	}
+
 	if (res =3D=3D -1)
 		return -errno;
=20

Modified: trunk/src/removexattr.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/removexattr.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/removexattr.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -36,6 +36,8 @@
 	res =3D lremovexattr(path, name);
 	if (res =3D=3D -1)
 		return -errno;
+	=09
+	fprintf(stderr,"OPERATION: remove xattr =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/rename.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/rename.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/rename.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -38,6 +38,8 @@
 	res =3D rename(from, to);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: rename =3D> %s\n",ffrom);
=20
 	return 0;
 }

Modified: trunk/src/rmdir.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/rmdir.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/rmdir.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -36,6 +36,8 @@
 	res =3D rmdir(path);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: rmdir =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/setxattr.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/setxattr.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/setxattr.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -36,6 +36,8 @@
 	res =3D lsetxattr(path, name, value, size, flags);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: set xattr =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/symlink.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/symlink.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/symlink.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -37,6 +37,8 @@
 	if (res =3D=3D -1)
 		return -errno;
=20
+	fprintf(stderr,"OPERATION: symlink =3D> %s\n",from);
+
 	return 0;
 }
=20

Modified: trunk/src/truncate.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/truncate.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/truncate.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -36,6 +36,8 @@
 	res =3D truncate(path, size);
 	if (res =3D=3D -1)
 		return -errno;
+=09
+	fprintf(stderr,"OPERATION: truncate =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/unlink.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/unlink.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/unlink.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -37,7 +37,7 @@
 	if (res =3D=3D -1)
 		return -errno;
=20
-	fprintf(stderr,"OPERATION: remove =3D> %s\n",fpath);
+	fprintf(stderr,"OPERATION: unlink =3D> %s\n",fpath);
=20
 	return 0;
 }

Modified: trunk/src/utime.c
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
--- trunk/src/utime.c	2006-09-29 07:11:29 UTC (rev 2)
+++ trunk/src/utime.c	2006-11-15 10:30:01 UTC (rev 3)
@@ -37,6 +37,8 @@
 	if (res =3D=3D -1)
 		return -errno;
=20
+	fprintf(stderr,"OPERATION: utime =3D> %s\n",fpath);
+
 	return 0;
 }
=20

[ivdfs-devel] cleversafe's dsgrid

[Ian C. B. <ia...@bl...>]

Has anyone played with cleversafe's dsgrid stuff yet?

http://www.cleversafe.org

This is the kind of thing I think we're looking for.

- Ian C. Blenke <ia...@bl...> http://ian.blenke.com/

[ivdfs-devel] Re: Greetings

[Sean E. <ed...@no...>]

Ok, that sounds good. I must have misunderstood.

On Wednesday 20 September 2006 16:49, Nigel Kukard wrote:
> Yo,
>
> Stupid subject line.
>
> > Just a thought regarding the client/server as opposed to the P2P
> >  architecture, the way I see it is that doing a P2P system would be
> >  beneficial in the sense of collaborative editing. On the other hand, you
> >  could end up with a client/server architecture as well, through the use
> > of dedicated servers, which just mirror the network filesystem as a
> > whole. It would allow, say, 2 workstations and a backup server to be
> > constantly synced together,
>
> I think the subject is getting a bit confused.
>
> P2P is peer-to-peer. Defined as each peer being equal in the duties it
> performs. This is what we want. It eliminates the single point of
> failure with a master-slave architecture.
>
> > whereas with a client-server structure you wouldn't have that
> >  advantage. Really the only difference would be that a workstation
> > running IVDFS would be able to accept updates and update its own local
> > copy, as well as sending updates to the "server(s)", and the "server(s)"
> > would be able to function as
> > workstations as well, and write to its own local system and update the
> > other machines on the grid. And, more importantly, the filesystem /could/
> > still function as the proposed client/server layout.
>
> This is the behavior were after.
>
> The point I tried to make is the goal for ivdfs is not anonymous file
> sharing, for instance bittorrent, gnutella ... etc. Or storing of
> anonymous data in using hashes ... etc. It could be designed into ivdfs,
> I have nothing against that, its just not one of the primary goals for
> the project.
>
> The primary goal is replication and raid-like behavior for high
> availability file storage across different networks, servers or whatever
> in the simplest manner possible, and to preserve the underlying
> filesystem in such a way that using plain replications its possible to
> stop ivdfs and still access the files on the local filesystem.
>
> So to address your comment above, one could mount a server fs on his/her
> workstation and have it setup in such a way that only files in ones own
> directory be replicated and the rest pulled directly off the server, or
> any other way in which you see fit  :o)
>
>
> -Nigel

[ivdfs-devel] Re: Greetings

[Nigel K. <nk...@lb...>]

Yo,

Stupid subject line.

> Just a thought regarding the client/server as opposed to the P2P
>  architecture, the way I see it is that doing a P2P system would be
>  beneficial in the sense of collaborative editing. On the other hand, y=
ou
>  could end up with a client/server architecture as well, through the us=
e of
>  dedicated servers, which just mirror the network filesystem as a whole=
=2E It
>  would allow, say, 2 workstations and a backup server to be constantly =
synced
>  together,

I think the subject is getting a bit confused.

P2P is peer-to-peer. Defined as each peer being equal in the duties it
performs. This is what we want. It eliminates the single point of
failure with a master-slave architecture.


> whereas with a client-server structure you wouldn't have that
>  advantage. Really the only difference would be that a workstation runn=
ing
>  IVDFS would be able to accept updates and update its own local copy, a=
s well
>  as sending updates to the "server(s)", and the "server(s)" would be ab=
le to
>  function as
> workstations as well, and write to its own local system and update the =
other
> machines on the grid. And, more importantly, the filesystem /could/ sti=
ll
> function as the proposed client/server layout.

This is the behavior were after.

The point I tried to make is the goal for ivdfs is not anonymous file
sharing, for instance bittorrent, gnutella ... etc. Or storing of
anonymous data in using hashes ... etc. It could be designed into ivdfs,
I have nothing against that, its just not one of the primary goals for
the project.

The primary goal is replication and raid-like behavior for high
availability file storage across different networks, servers or whatever
in the simplest manner possible, and to preserve the underlying
filesystem in such a way that using plain replications its possible to
stop ivdfs and still access the files on the local filesystem.

So to address your comment above, one could mount a server fs on his/her
workstation and have it setup in such a way that only files in ones own
directory be replicated and the rest pulled directly off the server, or
any other way in which you see fit  :o)


-Nigel

Re: [ivdfs-devel] Greetings

[Sean E. <ed...@no...>]

Just a thought regarding the client/server as opposed to the P2P
 architecture, the way I see it is that doing a P2P system would be
 beneficial in the sense of collaborative editing. On the other hand, you
 could end up with a client/server architecture as well, through the use of
 dedicated servers, which just mirror the network filesystem as a whole. It
 would allow, say, 2 workstations and a backup server to be constantly synced
 together, whereas with a client-server structure you wouldn't have that
 advantage. Really the only difference would be that a workstation running
 IVDFS would be able to accept updates and update its own local copy, as well
 as sending updates to the "server(s)", and the "server(s)" would be able to
 function as
workstations as well, and write to its own local system and update the other
machines on the grid. And, more importantly, the filesystem /could/ still
function as the proposed client/server layout.

~Sean

Re: [ivdfs-devel] Greetings

[Nigel K. <nk...@lb...>]

>
> There appear to be no mail archives yet, has conversation yet begun?

Just begun now, I'm still getting everybody to join the list though.

> =20
>  Incredibly Versatile?
> =20
>  Perhaps we should nail down what "Versatile" features we are looking f=
or.
Agreed.

> =20
>  CornFS was an attempt at making a simple copy-on-write mirroring=20
>  filesystem that could be easily recovered. The primary goal was archiv=
al=20
>  use, with an eye toward rapid searchability, layering on top of any=20
>  resilient networked filesystem.
> =20
>  Is this targeted to be a POSIX filesystem (complete with permissions=20
>  and/or ACLs) or just a heirarchical archival filesystem?
Well, all the features of underlying filesystem will be available
through FUSE... think passthrough. I want this to work as simply as
possible, so one can even stop FUSE and still access the files.

File operations like rename, unlink, create and on-close-writes are then
journalled and sync'd between nodes.

> =20
>  While talking with others about their pet projects, the primary=20
>  attributes of focus seem to be:
> =20
>  Distributed metadata.
>  - Distributing the filesystem namespace across N number of networked n=
odes.
>  Distributed data.
Agreed

>  - Distributing the file data across N number of networked nodes (eithe=
r=20
>  full file copies or block striping).
Or even other forms of RAID-like behaviour? I think FiST (RAIF) has some
other interesting RAID-like options, can always peek at the code and
dump chunks into our own implementation.

>  Distributed locking.
>  - Changes to the metadata and/or data need to be synchronized to keep =

>  the filesystem coherent.
Agreed

>  Redundancy.
>  - Having N copies available on the network, so that content is always =

>  available.
Yea ... I was thinking about a GFS-like approach where all nodes can
ping eachother and have a vote when one of them cannot be reached to
outcast it from the group. Apon being outcast, the outcasted node could
switch to read-only mode or just return IO errors, or even carry on and
re-sync later.... *shrug*, all are pretty easy to implement.


>  Single instance store.
>  - Use an MD5/SHA1 hash to uniquely identify and universally store a fi=
le=20
>  (or blocks of a file) for later recall.
The less we touch the files on the underlying filesystem the better,
hashes can be stored in the FUSE's own .whatever directory if needed.

I think our first goal should be plain and simple replication ...
nothing special, if a file changes. copy the entire thing. If created,
create and copy data. If deleted delete it. No locking. Lets get a very
simple working FS together, then implement the features above with the
most important first.


>  Others have mentioned P2P networking instead of a private cluster mode=
l.
> =20
>  While I'm interested in the above, I'd really like to see P2P get=20
>  factored into this.
> =20
>  With a little public key cryptography, it would be possible to join=20
>  something theoretically like Freenet with something more like bittorre=
nt:
>  - Have "confederations" of metadata namespaces that link to one anothe=
r=20
>  using UUID or unique cryptographic hashes.
>  - Have a single instance store of files or blocks using MD5/SHA1 hashe=
s.
>  - Have "webs of trust" for both metadata and file data so that only yo=
ur=20
>  friends can access your filesystems or file data, yet still allow=20
>  objects to be replicated without knowing what is inside a file (ala=20
>  Freenet). With webs of trust, one unified network would be possible,=20
>  with storage availablity spanning _all_ nodes. "Dark" nets would not b=
e=20
>  necessary, as it would be part of the model.
>  - Swarm file downloads from both "seed" and other "downloading" nodes =

>  (ala Bittorrent).
I'm thinking p2p is more for another dedicated project, or maybe a
branch of this one?

My major goal is to provide 100% redundant file storage across
datacenters and provide a true globally distributed filesystem.

>  I'm envisioning something that allows content to become a "permanent=20
>  network copy", that will only go away if nobody is interested in it.
> =20
>  Also, while FUSE is neat, FiST holds much promise. Something like RAIF=
=20
>  is one of the alternatives to CornFS, and I've thought about rewriting=
=20
>  CornFS to be a layed FiST filesystem.
> =20
>    http://www.filesystems.org
> =20
>  If the goal is to go toward P2P, implementing a loopback WebDAV=20
>  interface might be the most universal way to present a filesystem=20
>  (though it is non-POSIX compliant).
Not sure if FiST is included in the vanilla kernel? the less patching
needed the better, one tends to loose ALOT of interest in a project the
second you need to patch something. Especially if the server being
patched is under a maintenance contract with a vendor, patching the
kernel in this scenario is almost always guaranteed to void your contract=
=2E


>  I look forward to your thoughts.
The main requirement I have for this project is to replicate virtual
hosting data between datacenters, clients websites and email (maildir).
I need to switch between centers should a natural disaster or major
outage occur.

-Nigel

Re: [ivdfs-devel] Greetings

[Ian C. B. <ia...@bl...>]

There appear to be no mail archives yet, has conversation yet begun?

Incredibly Versatile?

Perhaps we should nail down what "Versatile" features we are looking for.

CornFS was an attempt at making a simple copy-on-write mirroring 
filesystem that could be easily recovered. The primary goal was archival 
use, with an eye toward rapid searchability, layering on top of any 
resilient networked filesystem.

Is this targeted to be a POSIX filesystem (complete with permissions 
and/or ACLs) or just a heirarchical archival filesystem?

While talking with others about their pet projects, the primary 
attributes of focus seem to be:

Distributed metadata.
- Distributing the filesystem namespace across N number of networked nodes.
Distributed data.
- Distributing the file data across N number of networked nodes (either 
full file copies or block striping).
Distributed locking.
- Changes to the metadata and/or data need to be synchronized to keep 
the filesystem coherent.
Redundancy.
- Having N copies available on the network, so that content is always 
available.
Single instance store.
- Use an MD5/SHA1 hash to uniquely identify and universally store a file 
(or blocks of a file) for later recall.

Others have mentioned P2P networking instead of a private cluster model.

While I'm interested in the above, I'd really like to see P2P get 
factored into this.

With a little public key cryptography, it would be possible to join 
something theoretically like Freenet with something more like bittorrent:
- Have "confederations" of metadata namespaces that link to one another 
using UUID or unique cryptographic hashes.
- Have a single instance store of files or blocks using MD5/SHA1 hashes.
- Have "webs of trust" for both metadata and file data so that only your 
friends can access your filesystems or file data, yet still allow 
objects to be replicated without knowing what is inside a file (ala 
Freenet). With webs of trust, one unified network would be possible, 
with storage availablity spanning _all_ nodes. "Dark" nets would not be 
necessary, as it would be part of the model.
- Swarm file downloads from both "seed" and other "downloading" nodes 
(ala Bittorrent).

If anonymity is a goal, possibly work something functionally equivalent 
to The Onion Router into the mix.

I'm envisioning something that allows content to become a "permanent 
network copy", that will only go away if nobody is interested in it.

Also, while FUSE is neat, FiST holds much promise. Something like RAIF 
is one of the alternatives to CornFS, and I've thought about rewriting 
CornFS to be a layed FiST filesystem.

  http://www.filesystems.org

If the goal is to go toward P2P, implementing a loopback WebDAV 
interface might be the most universal way to present a filesystem 
(though it is non-POSIX compliant).

I look forward to your thoughts.

- Ian C. Blenke <ia...@bl...> http://ian.blenke.com/projects/cornfs/