Hi!
I need a really big NAS for some of my applications and so tried Kfs. Some things work so far, some don't.

On my 32 bit build system I have the following packages( I removed the ones I thought are irrelevant):
Ubuntu Linux with
autotools-dev     20070725.1
base-files     4.0.1ubuntu5
binutils     2.18.1~cvs20080103-0ubuntu1
cmake     2.4.7-1build1
coreutils     6.10-3ubuntu2
cpio     2.9-6ubuntu1
cpp     4:4.2.3-1ubuntu3
cpp-3.3     1:3.3.6-15ubuntu4
cpp-4.2     4.2.3-2ubuntu7
g++     4:4.2.3-1ubuntu3
g++-3.3     1:3.3.6-15ubuntu4
g++-4.2     4.2.3-2ubuntu7
g++-4.2-multilib     4.2.3-2ubuntu7
gcc     4:4.2.3-1ubuntu3
gcc-3.3     1:3.3.6-15ubuntu4
gcc-3.3-base     1:3.3.6-15ubuntu4
gcc-4.2     4.2.3-2ubuntu7
gcc-4.2-base     4.2.3-2ubuntu7
gcc-4.2-multilib     4.2.3-2ubuntu7
gcj-4.2-base     4.2.3-2ubuntu6
gdb     6.8-1ubuntu1
lib64gcc1     1:4.2.3-2ubuntu7
lib64gomp1     4.2.3-2ubuntu7
lib64stdc++6     4.2.3-2ubuntu7
libboost-date-time-dev     1.34.1-4ubuntu3
libboost-date-time1.34.1     1.34.1-4ubuntu3
libboost-dev     1.34.1-4ubuntu3
libboost-doc     1.34.1-4ubuntu3
libboost-filesystem-dev     1.34.1-4ubuntu3
libboost-filesystem1.34.1     1.34.1-4ubuntu3
libboost-graph-dev     1.34.1-4ubuntu3
libboost-iostreams-dev     1.34.1-4ubuntu3
libboost-iostreams1.34.1     1.34.1-4ubuntu3
libboost-program-options-dev     1.34.1-4ubuntu3
libboost-program-options1.34.1     1.34.1-4ubuntu3
libboost-python-dev     1.34.1-4ubuntu3
libboost-python1.34.1     1.34.1-4ubuntu3
libboost-regex-dev     1.34.1-4ubuntu3
libboost-regex1.34.1     1.34.1-4ubuntu3
libboost-serialization-dev     1.34.1-4ubuntu3
libboost-serialization1.34.1     1.34.1-4ubuntu3
libboost-signals-dev     1.34.1-4ubuntu3
libboost-signals1.34.1     1.34.1-4ubuntu3
libboost-test-dev     1.34.1-4ubuntu3
libboost-test1.34.1     1.34.1-4ubuntu3
libboost-thread-dev     1.34.1-4ubuntu3
libboost-thread1.34.1     1.34.1-4ubuntu3
libboost-wave-dev     1.34.1-4ubuntu3
libboost-wave1.34.1     1.34.1-4ubuntu3
libc6     2.7-10ubuntu3
libc6-amd64     2.7-10ubuntu3
libc6-dev     2.7-10ubuntu3
libc6-dev-amd64     2.7-10ubuntu3
libc6-i686     2.7-10ubuntu3
libcap1     1:1.10-14build1
libgc1c2     1:6.8-1.1
libgcc1     1:4.2.3-2ubuntu7
libgcj-bc     4.2.3-1ubuntu3
libgcj-common     1:4.2.3-1ubuntu3
libgcj8-1     4.2.3-2ubuntu6
libgcrypt11     1.2.4-2ubuntu7
libgnutls13     2.0.4-1ubuntu2
libgomp1     4.2.3-2ubuntu7
libkeyutils1     1.2-4
libklibc     1.5.7-4ubuntu3
libncurses5     5.6+20071124-1ubuntu2
libncursesw5     5.6+20071124-1ubuntu2
libsigc++-2.0-0c2a     2.0.17-2ubuntu3
libstdc++5     1:3.3.6-15ubuntu4
libstdc++5-3.3-dev     1:3.3.6-15ubuntu4
libstdc++6     4.2.3-2ubuntu7
libstdc++6-4.2-dev     4.2.3-2ubuntu7
make     3.81-3build1
makedev     2.3.1-84ubuntu1
sun-java5-bin     1.5.0-15-0ubuntu1
sun-java5-demo     1.5.0-15-0ubuntu1
sun-java5-jdk     1.5.0-15-0ubuntu1
sun-java5-jre     1.5.0-15-0ubuntu1
zlib1g     1:1.2.3.3.dfsg-7ubuntu1
---------------------------------------------------

calling g++ --version leads to version 4.2.3

cmake and make are working fine.

During compiling Kfs 0.1.2 I get the following error:

/home/code/kfs/src/cc/chunk/ChunkManager.cc: In member function ‚int KFS::ChunkManager::WriteChunk(KFS::WriteOp*)‚:
/home/code/kfs/src/cc/chunk/ChunkManager.cc:470: error: no matching function for call to ‚max(size_t, long unsigned int)‚
/home/code/kfs/src/cc/chunk/ChunkManager.cc:476: warning: comparison between signed and unsigned integer expressions
make[2]: *** [src/cc/chunk/CMakeFiles/chunk.dir/ChunkManager.o] Error 1
make[1]: *** [src/cc/chunk/CMakeFiles/chunk.dir/all] Error 2
make: *** [all] Error 2

I "fixed" it by casting to (size_t) in ChunkManager.cc (line 469):

#else
    size_t addedBytes = max((size_t) 0,
            op->offset + op->numBytesIO - cih->chunkInfo.chunkSize);

#else
    size_t addedBytes = max((size_t) 0, (size_t)
            op->offset + op->numBytesIO - cih->chunkInfo.chunkSize);


Afterwards I have the following warnings:
[ 37%] Building CXX object src/cc/chunk/CMakeFiles/chunk.dir/ChunkManager.o
/home/code/kfs/src/cc/chunk/ChunkManager.cc: In member function ‚int KFS::ChunkManager::WriteChunk(KFS::WriteOp*)‚:
/home/code/kfs/src/cc/chunk/ChunkManager.cc:476: warning: comparison between signed and unsigned integer expressions

[ 49%] Building CXX object src/cc/libkfsClient/CMakeFiles/kfsClient.dir/KfsClient.o
/home/code/kfs/src/cc/libkfsClient/KfsClient.cc: In function ‚std::string KFS::ErrorCodeToStr(int)‚:
/home/code/kfs/src/cc/libkfsClient/KfsClient.cc:1730: warning: deprecated conversion from string constant to ‚char*‚

[ 52%] Building CXX object src/cc/libkfsClient/CMakeFiles/kfsClient.dir/KfsWrite.o
/home/code/kfs/src/cc/libkfsClient/KfsWrite.cc: In member function ‚ssize_t KFS::KfsClient::DoLargeWriteToServer(int, off_t, const char*, size_t)‚:
/home/code/kfs/src/cc/libkfsClient/KfsWrite.cc:453: warning: comparison between signed and unsigned integer expressions

[ 53%] Building CXX object src/cc/libkfsClient/CMakeFiles/kfsClient.dir/KfsRead.o
/home/code/kfs/src/cc/libkfsClient/KfsRead.cc: In member function ‚ssize_t KFS::KfsClient::DoLargeReadFromServer(int, char*, size_t)‚:
/home/code/kfs/src/cc/libkfsClient/KfsRead.cc:383: warning: comparison between signed and unsigned integer expressions

[ 68%] Building CXX object src/cc/libkfsClient/CMakeFiles/kfsClient-shared.dir/KfsClient.o
/home/code/kfs/src/cc/libkfsClient/KfsClient.cc: In function ‚std::string KFS::ErrorCodeToStr(int)‚:
/home/code/kfs/src/cc/libkfsClient/KfsClient.cc:1730: warning: deprecated conversion from string constant to ‚char*‚

[ 71%] Building CXX object src/cc/libkfsClient/CMakeFiles/kfsClient-shared.dir/KfsWrite.o
/home/code/kfs/src/cc/libkfsClient/KfsWrite.cc: In member function ‚ssize_t KFS::KfsClient::DoLargeWriteToServer(int, off_t, const char*, size_t)‚:
/home/code/kfs/src/cc/libkfsClient/KfsWrite.cc:453: warning: comparison between signed and unsigned integer expressions

[ 72%] Building CXX object src/cc/libkfsClient/CMakeFiles/kfsClient-shared.dir/KfsRead.o
/home/code/kfs/src/cc/libkfsClient/KfsRead.cc: In member function ‚ssize_t KFS::KfsClient::DoLargeReadFromServer(int, char*, size_t)‚:
/home/code/kfs/src/cc/libkfsClient/KfsRead.cc:383: warning: comparison between signed and unsigned integer expressions

[ 98%] Building CXX object src/cc/tests/CMakeFiles/KfsWriter.dir/KfsWriter_main.o
/home/code/kfs/src/cc/tests/KfsWriter_main.cc: In function ‚int main(int, char**)‚:
/home/code/kfs/src/cc/tests/KfsWriter_main.cc:69: warning: deprecated conversion from string constant to ‚char*‚


These warnings appear in both debug and release build mode. How worrying are they?

Anyway, I managed to set up a local system with 4 chunkservers, each gets 3 GB.
The scripts are all working fine except

python kfslaunch.py -f machines.cfg --stop

because there are two issues in the stop()-function in kfsrun.sh  :

if [[ $PROC_COUNT -gt  0 ]];

should be

if [ $PROC_COUNT -gt  0 ];

Otherwise the .pid file are deleted but the processes not stopped, a bit annoying.

The second issue is pure cosmetics, it must be $PROG in

echo -n $"Stopping $prog ( $PROCID )"


When running KfsPing I get
s=frankb, p=55003, total=-1(GB), used=0(GB), util=0%
s=frankb, p=55001, total=-1(GB), used=0(GB), util=0%
s=frankb, p=55002, total=-1(GB), used=0(GB), util=0%
s=frankb, p=55004, total=-1(GB), used=0(GB), util=0%

so the total is a bit wrong. Also, what is util % ?

When copying some few boring binaries (150 MB in total) to the KFS via tools/cpFs2Kfs and get them back via cpKfs2Fs this only works when the tools are build in release. When debug binaries, cpKfs2Fs fails with "Segmentation fault".
When doing the cpFs2Kfs with more interesting data, meaning many files with spaces in names etc, cpFs2Kfs fails always on the same file with
terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

When I restore the successfully written files with cpKfs2Fs, it also sometimes breaks with the same error. I cannot see any errors in the meta or chunk server logs.

What is the error?

---------------------------------

I checked out the latest version via svn.

This time compiling breaks with

[ 39%] Building CXX object src/cc/chunk/CMakeFiles/chunk.dir/ChunkManager.o
In file included from /home/svncode/kosmosfs/trunk/src/cc/chunk/ChunkManager.h:40,
                 from /home/svncode/kosmosfs/trunk/src/cc/chunk/ChunkManager.cc:39:
/home/svncode/kosmosfs/trunk/src/cc/common/cxxutil.h:36: error: redefinition of ‚struct std::tr1::hash<long long int>‚
/usr/include/c++/4.2/tr1/functional_hash.h:68: error: previous definition of ‚struct std::tr1::hash<long long int>‚
/home/svncode/kosmosfs/trunk/src/cc/chunk/ChunkManager.cc: In member function ‚int KFS::ChunkManager::WriteChunk(KFS::WriteOp*)‚:
/home/svncode/kosmosfs/trunk/src/cc/chunk/ChunkManager.cc:478: warning: comparison between signed and unsigned integer expressions
/home/svncode/kosmosfs/trunk/src/cc/chunk/ChunkManager.cc: In member function ‚void KFS::ChunkManager::ReplayWriteDone(KFS::kfsChunkId_t, size_t, off_t, std::vector<unsigned int, std::allocator<unsigned int> >)‚:
/home/svncode/kosmosfs/trunk/src/cc/chunk/ChunkManager.cc:1060: warning: comparison between signed and unsigned integer expressions
make[2]: *** [src/cc/chunk/CMakeFiles/chunk.dir/ChunkManager.o] Error 1
make[1]: *** [src/cc/chunk/CMakeFiles/chunk.dir/all] Error 2
make: *** [all] Error 2

I guess the __i386__ switch is set on my machine, since it is 32 bits. Maybe the functional_hash.h should be included because it was nt expected to collide, since long long is something 64 bit like?
However, I replaced
#if defined (__APPLE__) || defined (__i386__)
by
#if defined (__APPLE__)
and it compiled with similar warnings as above. The results are the same on my machine, strange output for kfsping and cptokfs breaks with  'std::bad_alloc'.

--------------------------------------

I am a bit new to linux development, I hope someone can give me some advice. I suspect the unsigned/signed issues are mainly because I'm compiling on 32 bit, so far I do not have a 64 dev box, but if Kfs works at the required scale, i will move to 64 anyway.

As far as I understood the current trunk does some clustering stuff and data-pushing from chunkserver to chunkserver instead of a push to every single chunkserver to maximize write performance ( I saw the cluster key string required in machines.cfg). How stable are the current developments? When will be the next release?

All hints are welcome!!!

Cheers,
Frank