moosefs-users — Mailing list for MooseFS Users

[Moosefs-users] Mac OS X Snow Leopard mount issues

[Marshall A. <man...@ic...>]

I have successfully built 1.6.13 on Mac OS X Snow Leopard and have started a mfsmaster and mfschunkserver; however, when I try to mount it, I receive an error from mfsmount:

mb:test1 tty1$ ./mfsmaster -c mfsmaster.cfg -du
mb:test1 tty2$ ./mfschunkserver -c mfschunkserver.cfg -d
mb:test1 tty3$ mfsmount testMountPoint/ -o mfsmaster=172.23.10.252,mfsdebug,mfssubfolder=/
mfsmaster register error: Permission denied


For reference, here are the configuration files. I am running as my own user for testing in this test1 directory.

mfsmaster.cfg
===================
EXPORTS_FILENAME = ./mfsexports.cfg
DATA_PATH = ./masterData/

mfschunkserver.cfg:
===================
DATA_PATH = ./chunk1/
HDD_CONF_FILENAME = /Users/manschutz/LocalProjects/moosefs/test1/chunk1mfshdd.cfg
BIND_HOST = 192.168.110.1
MASTER_HOST = 192.168.110.1
MASTER_PORT = 9420
CSSERV_LISTEN_PORT = 9422

mfsexports.cfg
==================
# allow everything but "meta"
*			/	rw,alldirs,maproot=0
# allow "meta"
*			.	rw


chunk1mfshdd.cfg
==================
/Users/manschutz/LocalProjects/moosefs/test1/chunk1Data1/
/Users/manschutz/LocalProjects/moosefs/test1/chunk1Data2/


Thanks for the assistance!



Marshall Anschutz
Ice House Productions
Senior Software Engineer

[Moosefs-users] Mac OS X Snow Leopard mount issues

[Marshall A. <man...@ic...>]

I have successfully built 1.6.13 on Mac OS X Snow Leopard and have started a mfsmaster and mfschunkserver; however, when I try to mount it, I receive an error from mfsmount:

mb:test1 tty1$ ./mfsmaster -c mfsmaster.cfg -du
mb:test1 tty2$ ./mfschunkserver -c mfschunkserver.cfg -d
mb:test1 tty3$ mfsmount testMountPoint/ -o mfsmaster=172.23.10.252,mfsdebug,mfssubfolder=/
mfsmaster register error: Permission denied


For reference, here are the configuration files. I am running as my own user for testing in this test1 directory.

mfsmaster.cfg
===================
EXPORTS_FILENAME = ./mfsexports.cfg
DATA_PATH = ./masterData/

mfschunkserver.cfg:
===================
DATA_PATH = ./chunk1/
HDD_CONF_FILENAME = /Users/manschutz/LocalProjects/moosefs/test1/chunk1mfshdd.cfg
BIND_HOST = 192.168.110.1
MASTER_HOST = 192.168.110.1
MASTER_PORT = 9420
CSSERV_LISTEN_PORT = 9422

mfsexports.cfg
==================
# allow everything but "meta"
*			/	rw,alldirs,maproot=0
# allow "meta"
*			.	rw


chunk1mfshdd.cfg
==================
/Users/manschutz/LocalProjects/moosefs/test1/chunk1Data1/
/Users/manschutz/LocalProjects/moosefs/test1/chunk1Data2/


Thanks for the assistance!



Marshall Anschutz
Ice House Productions
Senior Software Engineer

Re: [Moosefs-users] Question about chart

[Michał B. <mic...@ge...>]

Which chart are you talking about? Do you mean the CGI monitor (which
screenshots you can see at sourceforge.net?)

 

 

Kind regards

Michał Borychowski 

 

From: pa pig [mailto:pig...@gm...] 
Sent: Wednesday, March 10, 2010 7:59 AM
To: moo...@li...
Subject: [Moosefs-users] Question about chart

 

Hi,

 

I am a open source fan, and now focus on the MFS and reading code. The code
is perfect but I am a new, So have a question about chart and chartdata, I
have no idea what are their role in the system, Could you help to give me a
answer or some docs.

 

Thanks a lot,

Stone Qiao

[Moosefs-users] Question about chart

[pa p. <pig...@gm...>]

Hi,

I am a open source fan, and now focus on the MFS and reading code. The code
is perfect but I am a new, So have a question about chart and chartdata, I
have no idea what are their role in the system, Could you help to give me a
answer or some docs.

Thanks a lot,
Stone Qiao

Re: [Moosefs-users] mfssetgoal 1 to trashfiles

[Michał B. <mic...@ge...>]

Now please try doing something like this:
cd /media/mfsmeta/trash
find . -name '[0-9A-F]*' | sed "s/^/'/" | sed "s/$/'/" | xargs -n 1 mfssetgoal 1

In the future we plan to put trash into a normal file structure. Than it would be enough to do something like: 
mfssetgoal -r /mnt/mfs/.trash

But at the moment you have to do it as shown in the first example.


Kind regards
Michal Borychowski   



-----Original Message-----
From: jose maria [mailto:aso...@zo...] 
Sent: Wednesday, February 24, 2010 4:02 PM
To: moo...@li...
Subject: [Moosefs-users] mfssetgoal 1 to trashfiles

* I am trying to apply mssetgoal 1 to the files in the trash bin, I get a list 
of 2 in two ways, in both, many files is an incorrect path to implementing the 
order.

* Example 1:
ls -1Q -I undel /media/mfsmeta/trash/ > trashfiles.txt

* Result, ignore the line break

"00001723|ftp03|CBR101419|eb|escritorio 22.02.2010 12.35|
escritorio_20100210_1200.zip.z012.1.pgp"

* and apply the order:
awk -F'|' '{ print "/media/mfs/" $2 "/" $3 "/" $4 "/" $5 "/"
    substr($6,1,length($6)-1)}' trashfiles.txt | while read i; do mfssetgoal 1
    "$i"; done;

* ignore the line break, apply mfssetgoal to:

/media/mfs/ftp03/CBR101419/eb/escritorio 22.02.2010 
12.35/escritorio_20100210_1200.zip.z012.1.pgp

* many files is an incorrect path


* Example 2, obtain list.
ls -1 -I undel /media/mfsmeta/trash/ > trashfiles.txt

* Result, ignore de line break

00001723|ftp03|CBR101419|eb|escritorio 22.02.2010 12.35|
escritorio_20100210_1200.zip.z012.1.pgp

* and apply de order:

awk -F'|' '{ print "/media/mfs/" $2 "/" $3 "/" $4 "/" $5 "/" $6}'
   trashfiles.txt | while read i; do mfssetgoal 1 "$i"; done;

* many files is an incorrect path.

* obtaining the listing is erroneous? ¿What am I doing wrong?
I need this feature, usually I have 700,000 files in the trash.

*I have seen in http://www.moosefs.org/user-contributed-files.html
¿have any practical relation to the question?

if so, as you get the files
{META-DUMP-FILE} {FILE-INODE}

* cheers, jose maria.

------------------------------------------------------------------------------
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] time populate new chunkserver

[Michał B. <mic...@ge...>]

Hi Jose!

Yes, this is normal. Speed of "rebalancing" replication is quite slow - the
overall performance of the system is more important than even usage of hard
drives. Quick replication would make the whole system considerably slow
through several days.

At our environment it is normal to take 3-4 weeks for a new chunkserver to
get a standard hdd usage.


Kind regards
Michal Borychowski 

-----Original Message-----
From: jose maria [mailto:aso...@zo...] 
Sent: Wednesday, February 24, 2010 6:24 PM
To: moo...@li...
Subject: [Moosefs-users] time populate new chunkserver

* I added a new chunkserver 48+ hours, the rest of chunkservers continues to

increase its occupancy   +90%, is this normal?


http://control.seycob.es:9425/mfs.cgi?sections=HD

* cheers, jose maria.


----------------------------------------------------------------------------
--
Download Intel® Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

[Moosefs-users] moosefs build problem on Solaris 10 (sparc)

[Sowa, B. (N. - PL/Wroclaw) <bar...@ns...>]

Hello,
 
I have tried to build moosefs master 1.6.13 on Solaris 10 (sparc)
machine. I have hit some problems I cannot resolve, maybe you can help
me a little...
The installed SW stack is as follows:
standard Solaris 10 box with CSW environment:
CSWbinutils               2.19.1,REV=2009.02.21
CSWcommon                 1.4.6,REV=2008.04.28
CSWgcc3                   3.4.5
CSWgcc3core               3.4.5
CSWgcc3corert             3.4.5,REV=2009.11.23
CSWgcc3g++                3.4.5
CSWgcc3g++rt              3.4.5,REV=2009.11.23
CSWgcc3rt                 3.4.5
CSWgcc4                   4.3.4,REV=2009.09.17
CSWgcc4corert             4.3.4,REV=2009.09.17
CSWgcc4g++rt              4.3.4,REV=2009.11.23
CSWgmake                  3.81,REV=2010.02.08
CSWlibpthreadstubs        0.1,REV=2008.11.16
I tried to compile it under gcc3
(PATH=/opt/csw/gcc3/bin:/opt/csw/gnu/bin:/opt/csw/bin:/usr/atria/bin:/op
t/csw/bin:/usr/sbin:/usr/bin:/usr/sfw/bin)
# which make
/opt/csw/bin/make
# which gcc
/opt/csw/gcc3/bin/gcc
[# which ar
/opt/csw/gnu/bin/ar

so the toolchain seems to be complete.
The problem is following:
 
make[2]: Entering directory `/root/moose/mfs-1.6.13/mfsmaster'
gcc -DHAVE_CONFIG_H -I. -I..  -std=c99 -I../mfscommon
-DAPPNAME=mfsmaster -D_GNU_SOURCE  -g -O2 -W -Wall -Wshadow -MT main.o
-MD -MP -MF .deps/main.Tpo -c -o main.o `test -f '../mfscommon/main.c'
|| echo './'`../mfscommon/main.c
../mfscommon/main.c: In function `set_signal_handlers':
../mfscommon/main.c:377: error: storage size of 'sa' isn't known
../mfscommon/main.c:380: error: `SA_RESTART' undeclared (first use in
this function)
../mfscommon/main.c:380: error: (Each undeclared identifier is reported
only once
../mfscommon/main.c:380: error: for each function it appears in.)
../mfscommon/main.c:381: warning: implicit declaration of function
`sigemptyset'
../mfscommon/main.c:385: warning: implicit declaration of function
`sigaction'
../mfscommon/main.c:377: warning: unused variable `sa'
../mfscommon/main.c: In function `errno_to_str':
../mfscommon/main.c:403: warning: implicit declaration of function
`strerror_r'
../mfscommon/main.c: In function `wdlock':
../mfscommon/main.c:525: warning: implicit declaration of function
`kill'
../mfscommon/main.c: In function `makedaemon':
../mfscommon/main.c:739: warning: implicit declaration of function
`fdopen'
../mfscommon/main.c:739: warning: assignment makes pointer from integer
without a cast
../mfscommon/main.c: In function `main':
../mfscommon/main.c:831: warning: implicit declaration of function
`strdup'
../mfscommon/main.c:831: warning: assignment makes pointer from integer
without a cast
../mfscommon/main.c:838: warning: implicit declaration of function
`getopt'
../mfscommon/main.c:853: error: `optarg' undeclared (first use in this
function)
../mfscommon/main.c:857: warning: assignment makes pointer from integer
without a cast
../mfscommon/main.c:867: error: `optind' undeclared (first use in this
function)
../mfscommon/main.c:870: warning: implicit declaration of function
`strcasecmp'
../mfscommon/main.c:926: error: `RLIMIT_MEMLOCK' undeclared (first use
in this function)
../mfscommon/main.c:950: warning: assignment makes pointer from integer
without a cast
make[2]: *** [main.o] Error 1
make[2]: Leaving directory `/root/moose/mfs-1.6.13/mfsmaster'
make[1]: *** [all-recursive] Error 1

When trying to build with gcc4 I get an error, that pthreads library
cannot be found on configure level.
I have googled a bit and checked sources about this and it seems to me
that it can have something with signal.h to do.
I found another signal.h in /usr/ucbinclude/signal.h and
/usr/ucbinclude/sys/signal.h. CFLAGS="-I ..." does not help much either.
 
My question is - is it possible to build moosefs 1.6.13 on this solaris
box? If yes, then what is the toolchain recommendation for it?
 
Ps. you guys did a great job - congratulations! I was looking a long
time until I found this filesystem and it is even more than I actually
needed :)
 
Best Regards,
Bartlomiej Sowa

[Moosefs-users] time populate new chunkserver

[jose m. <aso...@zo...>]

* I added a new chunkserver 48+ hours, the rest of chunkservers continues to 
increase its occupancy   +90%, is this normal?


http://control.seycob.es:9425/mfs.cgi?sections=HD

* cheers, jose maria.

[Moosefs-users] mfssetgoal 1 to trashfiles

[jose m. <aso...@zo...>]

* I am trying to apply mssetgoal 1 to the files in the trash bin, I get a list 
of 2 in two ways, in both, many files is an incorrect path to implementing the 
order.

* Example 1:
ls -1Q -I undel /media/mfsmeta/trash/ > trashfiles.txt

* Result, ignore the line break

"00001723|ftp03|CBR101419|eb|escritorio 22.02.2010 12.35|
escritorio_20100210_1200.zip.z012.1.pgp"

* and apply the order:
awk -F'|' '{ print "/media/mfs/" $2 "/" $3 "/" $4 "/" $5 "/"
    substr($6,1,length($6)-1)}' trashfiles.txt | while read i; do mfssetgoal 1
    "$i"; done;

* ignore the line break, apply mfssetgoal to:

/media/mfs/ftp03/CBR101419/eb/escritorio 22.02.2010 
12.35/escritorio_20100210_1200.zip.z012.1.pgp

* many files is an incorrect path


* Example 2, obtain list.
ls -1 -I undel /media/mfsmeta/trash/ > trashfiles.txt

* Result, ignore de line break

00001723|ftp03|CBR101419|eb|escritorio 22.02.2010 12.35|
escritorio_20100210_1200.zip.z012.1.pgp

* and apply de order:

awk -F'|' '{ print "/media/mfs/" $2 "/" $3 "/" $4 "/" $5 "/" $6}'
   trashfiles.txt | while read i; do mfssetgoal 1 "$i"; done;

* many files is an incorrect path.

* obtaining the listing is erroneous? ¿What am I doing wrong?
I need this feature, usually I have 700,000 files in the trash.

*I have seen in http://www.moosefs.org/user-contributed-files.html
¿have any practical relation to the question?

if so, as you get the files
{META-DUMP-FILE} {FILE-INODE}

* cheers, jose maria.

Re: [Moosefs-users] problem connect mfs-ftp

[Michał B. <mic...@ge...>]

Hi Jose!

We suggest that you run in one console window MooseFS with debugging enabled (-d -o mfsdebug) and on the second you run the test with ftp. Please check what debug log shows.


Kind regards
Michał Borychowski 


-----Original Message-----
From: jose maria [mailto:aso...@zo...] 
Sent: Friday, February 19, 2010 9:51 PM
To: moo...@li...
Subject: [Moosefs-users] problem connect mfs-ftp

* another serious problem, I tested ftp servers over moosefs with sslftp client , http://ayuda.seycob.es/sslftp for download.
user = test pass = test on ftp server.

./sslftp -d te...@ft...

* with delay +-35s for pass , all operations ok mkdir testing rmdir testing put, get, cd ls -la 

All OK there is no delay  , see the attached log.


* Any other ftp client, including backup application is able to connect, filezilla, gftp, curl, fireftp ........

* explicit ssl mandatory on server

* In some cases the list command times out.
* In other receiving the certificate ssl.

* ftp server setup is correct, if I change the ftp server directory to a local directory, the problem disappears.

* fuse, mfs, vsftpd problem? , any idea to debug the problem.

* thank you very much for your patience

[Moosefs-users] problem connect mfs-ftp

[jose m. <aso...@zo...>]

* another serious problem, I tested ftp servers over moosefs  
with sslftp client , http://ayuda.seycob.es/sslftp for download.
user = test pass = test on ftp server.

./sslftp -d te...@ft...

* with delay +-35s for pass , all operations ok
mkdir testing
rmdir testing
put, get, cd
ls -la 

All OK there is no delay  , see the attached log.


* Any other ftp client, including backup application is able to connect, 
filezilla, gftp, curl, fireftp ........

* explicit ssl mandatory on server

* In some cases the list command times out.
* In other receiving the certificate ssl.

* ftp server setup is correct, if I change the ftp server directory to a local 
directory, the problem disappears.

* fuse, mfs, vsftpd problem? , any idea to debug the problem.

* thank you very much for your patience

Re: [Moosefs-users] error mount mfsmeta (solved)

[jose m. <aso...@zo...>]

El Lunes 15 Febrero 2010, jose maria escribió:
> * thanks for the help received
> 

dot on mfsexports.cfg ---->        .    argggg ......

* cheers

[Moosefs-users] ftp virtual servers on MFS (solved)

[jose m. <aso...@zo...>]

* ftp server deployment with virtual users and virtual servers on cluster MFS 
solved.

* MFS compiled - with-default-user = virtual and group virtual , uid 20000 gid 
20000 on all machines, mfsclients, mfsmaster and chunkservers

* other combinations or users are possible.

*  on mfsclients , /media/mfs mount point for the cluster.

* on mfsclients, homedir for user  "virtual" /media/mfs  and real shell 
/bin/bash for tests

* /etc/passwd ----> virtual:x:20000:20000::/media/mfs:/bin/bash

* Create directories for base of virtual servers

* /media/mfs/ftp00 /media/mfs/ftp01 /media/mfsftp02  ......

 ------------------
* /etc/vsftpd-ftp00.conf

# virtual users
guest_enable=YES
guest_username=virtual  # user system that manages the virtual users
virtual_use_local_privs=YES
local_enable=YES
user_sub_token=$USER
local_root=/media/mfs/ftp00/$USER    #create directories for virtualusers
chroot_local_user=YES

--------------------
* /etc/vsftpd-ftp01.conf

# virtual users
guest_enable=YES
guest_username=virtual  # user system that manages the virtual users
virtual_use_local_privs=YES
local_enable=YES
user_sub_token=$USER
local_root=/media/mfs/ftp01/$USER
chroot_local_user=YES

---------------------------------------
* others ..........

* a small delay occurs in the FTP connection, but it works fine.

*  I can not find solution to mount the system mfsmeta (trash) , HELP, HELP 
................................

* jose maria.

Re: [Moosefs-users] error mount mfsmeta

[Michał B. <mic...@ge...>]

Hi Jose!

You have to use "." as a path in mfsexports.cfg

Tab "exports" should look similarly to ours:
http://80.48.16.122/mfs.cgi?sections=EX

PS. There would be no more "meta" resource in 1.7 branch, trash would be
part of a base tree.


If you need any further assistance please let us know.

Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01


-----Original Message-----
From: jose maria [mailto:aso...@zo...] 
Sent: Monday, February 15, 2010 4:12 PM
To: moo...@li...
Subject: [Moosefs-users] error mount mfsmeta

* thanks for the help received

* I get an error when trying to mount filesystem mfsmeta (trash)
I created the mount point in various parts of the tree /mnt/mfsmeta 
/media/mfsmeta, to test.

* in view of the errors that occur, I created the MFS directory mfsmeta at 
different points /usr/local/mfs/var/mfs/mfsmeta , /usr/local/mfs/var/mfsmeta
, 
to test.

* chown mountpoint an created MFS mfsmeta to apropiate user and group 

* mfsexports.cg 

172.26.0.0/24           /       rw,alldirs,maproot=20000

* /usr/local/mfs/bin/mfsmount /media/mfs -H 172.26.0.50 -m /media/mfsmeta
fuse: invalid argument `/media/mfsmeta/'

* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -m
mfsmaster register error: Permission denied


* /usr/local/mfs/bin/mfsmount /media/mfs -H 172.26.0.50
mfsmaster accepted connection with parameters: read-write,restricted_ip ;
root 
mapped to virtual:virtual

* mount

 172.26.0.50:9421 on /media/mfs type fuse.mfs 
(rw,nosuid,nodev,allow_other,default_permissions)

* /usr/local/mfs/bin/mfsgettrashtime -n /media/mfs/*
/media/mfs/ftp00: 259200
/media/mfs/ftp01: 259200
/media/mfs/ftp02: 259200
/media/mfs/ftp03: 259200
/media/mfs/ftp04: 259200
/media/mfs/walqa: 259200

* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -m
mfsmaster register error: Permission denied

* /usr/local/mfs/bin/mfsmount -H 172.26.0.50 -m /media/mfs
mfsmaster register error: Permission denied


* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -o mfsmeta
mfsmaster register error: Permission denied

* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -o mfsmeta -w
mfsmaster register error: Permission denied

* /usr/local/mfs/bin/mfsmount -H 172.26.0.50 -o mfsmeta -w /media/mfsmeta
mfsmaster register error: Permission denied

* I changed mfsexports.cfg to rw, and mfsmount -d option does not provide
more 
information.

* I can not find the solution. Here you can view the status of the cluster
in 
tests  http://control.seycob.es:9425
 

----------------------------------------------------------------------------
--
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

[Moosefs-users] error mount mfsmeta

[jose m. <aso...@zo...>]

* thanks for the help received

* I get an error when trying to mount filesystem mfsmeta (trash)
I created the mount point in various parts of the tree /mnt/mfsmeta 
/media/mfsmeta, to test.

* in view of the errors that occur, I created the MFS directory mfsmeta at 
different points /usr/local/mfs/var/mfs/mfsmeta , /usr/local/mfs/var/mfsmeta , 
to test.

* chown mountpoint an created MFS mfsmeta to apropiate user and group 

* mfsexports.cg 

172.26.0.0/24           /       rw,alldirs,maproot=20000

* /usr/local/mfs/bin/mfsmount /media/mfs -H 172.26.0.50 -m /media/mfsmeta
fuse: invalid argument `/media/mfsmeta/'

* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -m
mfsmaster register error: Permission denied


* /usr/local/mfs/bin/mfsmount /media/mfs -H 172.26.0.50
mfsmaster accepted connection with parameters: read-write,restricted_ip ; root 
mapped to virtual:virtual

* mount

 172.26.0.50:9421 on /media/mfs type fuse.mfs 
(rw,nosuid,nodev,allow_other,default_permissions)

* /usr/local/mfs/bin/mfsgettrashtime -n /media/mfs/*
/media/mfs/ftp00: 259200
/media/mfs/ftp01: 259200
/media/mfs/ftp02: 259200
/media/mfs/ftp03: 259200
/media/mfs/ftp04: 259200
/media/mfs/walqa: 259200

* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -m
mfsmaster register error: Permission denied

* /usr/local/mfs/bin/mfsmount -H 172.26.0.50 -m /media/mfs
mfsmaster register error: Permission denied


* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -o mfsmeta
mfsmaster register error: Permission denied

* /usr/local/mfs/bin/mfsmount /media/mfsmeta -H 172.26.0.50 -o mfsmeta -w
mfsmaster register error: Permission denied

* /usr/local/mfs/bin/mfsmount -H 172.26.0.50 -o mfsmeta -w /media/mfsmeta
mfsmaster register error: Permission denied

* I changed mfsexports.cfg to rw, and mfsmount -d option does not provide more 
information.

* I can not find the solution. Here you can view the status of the cluster in 
tests  http://control.seycob.es:9425
 

Re: [Moosefs-users] questions mfs 1.6.x

[Michał B. <mic...@ge...>]

Hi Jose!

> * This open source project is essential for mass storage work, its characteristics are necessary.
What exact characteristics would you like to know? If you could ask more specifically.


> * GlusterFS has different ones.
What do you mean here?


> * The answer to question number 6 will make you unique, along with a backup procedure in real time machine to another metaserver in passive mode, ready for immediate replacement. 

It is possible to implement the feature of storing one copy in another group of chunkservers.



Regarding backup procedure - it can be done by configuring metalogger which can start working as the master server in case of emergency. Please find the attachment on how to configure it.



> * Low cost and fundamental characteristics plus a tar.gz of 15Kb, can you ask for more?
> * Start the deployment to replace the secondary and tertiary servers replicated.
> * Much obliged and made available to project developers, both privileged access to the cluster, like any other network or storage resources of our institution according to their convenience and free of course.
Unfortunately we don't understand all these points :(




> > > 2 .- Very important, vsftpd can write directly on the MFS mount point,
> > > or
> > > need to mount --bind. A better idea? commercial code, other
> > > implementation
> > > of mfsmount, native module?
> >
> > We are not sure if we fully understand the question, but we'll try to
> > answer.
> >
> > There is no ready ftp server which would communicate directly with the
> > master server and chunkservers so the solution is to "go through" the
> > normal mfsmount. It gives better results than creating a special code for
> > communication - kernel really supports I/O operations very efficiently
> > (eg. by using read-ahead, buffering, writing in background, etc.). That's
> > why we also use classic file systems for storing data in chunkservers
> rather than using the disk directly as a block-device.
> * clarifying example
> 
> * On the client, mfs mounted in /media/mfs.
> configured vsftpd root directory at:
> local_root = /media/mfs/virtualusers/$USER
> user_sub_token = $USER
> 
> * Virtualusers directory physically reside in the cluster fysesystem, or
> virtualusers2 3 4 5.
> * Vsftpd I can not write to that directory as it really is a socket.
Write operations in MooseFS are made in the same way as on any other standard file system.

> * I would need to mount --bind /media/mfs /media/local and change
> local_root = /media/local/virtualusers/$USER in sshfs using FUSE is necessary.
> * Was it necessary in mfs?
> * on vsftpd.conf , use_sendfile = NO ¿Would the solution? proves it.
We can try to install vsftpd and check if there is any problem in running it with MooseFS.



> > 4.- stored millions of small files, backup software cript. compress and
> > > split
> > > files 5/30 Mb, any advice on file systems to use ext3-ext4 or block
> > > size. kernel
> > > elevator, another tuning?
> >
> > (Again we are not fully sure about the question)
> >
> > At our company we do not use any tuning, just normal ext3.
> >
> > How many milions of files do you have? Generally speaking 1 million files
> > takes approximately 300 MiB of RAM. Installation of 25 million files
> > requires about 8GiB of RAM and 25GiB space on HDD.
> >
> 
> * command executed in all primary servers
> find /media/volumen0/virtualusers -type f | wc -l
> 
> Current all total 369,026,500
If you have this number of files it may be reasonable to tar the files before storing them on MooseFS.



> > > 8.- What would be the best option on the clients automounter or
> > > desconecction, autofs, daemon guard, cron script?
> >
> > We have not used mfsmount with automounter nor autofs, we think you have to
> > just monitor its activity. If the master server or chunkservers are
> > termporarily not available, mfsmount tries to reconnect automatically.
> >
> 
> * I understand that with the reconnect feature of mfsmount or FUSE
> in /etc/fstab

We have just these entries in /etc/fstab (_netdev option is specific do debian only):
mfsmount	/mnt/mfs	fuse	rw,nosuid,nodev,_netdev	0	0

We do not use any automounts, simply mfsmounts are constantly mounted and they take care of connections to the master and chunk servers.



Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01 

Re: [Moosefs-users] cache in moosefs

[Michał B. <mic...@ge...>]

> -O mfscachefiles
>  preserve file data in cache (default is not to use data cache)

Now this option should be used:
-o mfscachemode=X

where X can be:
YES - all files are stored in a local cache
NO - no files are stored in a local cache
AUTO - storing in a local cache depends on 'allowdatacache' flag which can
be set individually for every file

MooseFS is a network file system and generally speaking files should not be
kept in cache because from the moment of putting a file in cache another
computer may change it. So cache should be used only there where you are
sure about this.


 
> -O mfsattrcacheto=SEC
> set attributes cache timeout in seconds (default: 1.0)
This is time of keeping file attributes in cache which are used by kernel by
simple operations like "rm", "mv", etc. The system tests many times
attributes of the same i-nodes and that's why we should allow for using
cache here. The time should be relevantly short but not shorter than time
needed for an individual operation like "rm" or "mv". We suggest leaving the
default value.

 
 
> -O mfsentrycacheto=SEC
> set file entry cache timeout in seconds (default: 0.0, I.e. no
> cache)
  
> -O mfsdirentrycacheto=SEC
> set directory entry cache timeout in seconds (default: 1.0)
 
Both options affect keeping entry in a folder ("lookup" operation - mapping
file or folder name to its i-node). This situation is similar to the above.
Every operation in which we refer to the full path causes that the system
every time scans file system in order to locate the given object. Most of
these operations concerns folders and folder's attributes do not change
often. So we decided to store in cache subfolders entries. That's why there
are two options: "mfsdirentrycacheto" for subfolders and "mfsentrycacheto"
for other objects. We also suggest leaving the default values here.

 

> -O mfswritecachesize=N
> specify write cache size in MiB (in range: 16..2048 - default:
> 250)
In order to increase performance writing in MooseFS is doen by "mfsmount" in
background. That's why some part of RAM has to be used as a write buffer.
This option allows to change this buffer. Lower value means lower memory
usage but also potentially slower writing speed. Greater value means higher
memory usage and in some cases can increase write operations. If there is no
need to keep this value smaller, we recommend not to change it. In pratice
increasing this buffer value does not influence a lot.


So the most important is possibility to control cache of file content (which
is important for the real system performance) with these options:

mfsseteattr -f allowdatacache FILE_1 [ FILE_2 ... ]
mfsdeleattr -f allowdatacache FILE_1 [ FILE_2 ... ]



Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01



> -----Original Message-----
> From: Steve [mailto:st...@bo...]
> Sent: Monday, January 25, 2010 3:17 PM
> To: Michał Borychowski
> Subject: RE: [Moosefs-users] mfs mount generates high trafic from
> chunks
> 
> I understand the outline of what a cache does but under what situation
> would
> it be useful ? What is the downside ?
> 
> Any guidance on choosing parameters or is a case of trying values and
> measuring results ?
> 
> 
> 
> 
> 
> 
> 
> -------Original Message-------
> 
> 
> 
> From: Michał Borychowski
> 
> Date: 25/01/2010 10:48:36
> 
> To: 'Steve'
> 
> Subject: RE: [Moosefs-users] mfs mount generates high traffic from
> chunks
> 
> 
> 
> Steve, let me know more precisely what did you mean by your question
> below??
> 
> 
> 
> 
> 
> 
> Regards
> 
> Michał
> 
> 
> 
> -----Original Message-----
> 
> From: Steve [mailto:st...@bo...]
> 
> Sent: Thursday, January 21, 2010 3:27 PM
> 
> To: moo...@li...
> 
> Subject: Re: [Moosefs-users] mfs mount generates high traffic from
> chunks
> 
> 
> 
> Hi,
> 
> 
> 
> 
> 
> 
> 
> Is there any extra docs on the cache pro's and con's, guidelines for
> usage ?
> 
> 
> more explicit than the man page.

Re: [Moosefs-users] questions mfs 1.6.x

[jose m. <aso...@zo...>]

El Martes 26 Enero 2010, Michał Borychowski escribió:
> Hola Jose!
> 
> Estoy muy contento que tenemos usuarios en Espana :)
> 
* This open source project is essential for mass storage work, its 
characteristics are necessary.

* GlusterFS has different ones.

* The answer to question number 6 will make you unique, along with a backup 
procedure in real time machine to another metaserver in passive mode, ready 
for immediate replacement. 

* Low cost and fundamental characteristics plus a tar.gz of 15Kb, can you ask 
for more?

* Start the deployment to replace the secondary and tertiary servers 
replicated.

* Much obliged and made available to project developers, both privileged 
access to the cluster, like any other network or storage resources of our 
institution according to their convenience and free of course.

> 
> > 2 .- Very important, vsftpd can write directly on the MFS mount point,
> > or
> > need to mount --bind. A better idea? commercial code, other
> > implementation
> > of mfsmount, native module?
> 
> We are not sure if we fully understand the question, but we'll try to
>  answer.
> 
> There is no ready ftp server which would communicate directly with the
>  master server and chunkservers so the solution is to "go through" the
>  normal mfsmount. It gives better results than creating a special code for
>  communication - kernel really supports I/O operations very efficiently
>  (eg. by using read-ahead, buffering, writing in background, etc.). That's
>  why we also use classic file systems for storing data in chunkservers
>  rather than using the disk directly as a block-device.
> 

* clarifying example

* On the client, mfs mounted in /media/mfs.
configured vsftpd root directory at:
local_root = /media/mfs/virtualusers/$USER
user_sub_token = $USER

* Virtualusers directory physically reside in the cluster fysesystem, or 
virtualusers2 3 4 5.
* Vsftpd I can not write to that directory as it really is a socket.
* I would need to mount --bind /media/mfs /media/local and change
local_root = /media/local/virtualusers/$USER in sshfs using FUSE is necessary.

* Was it necessary in mfs?

* on vsftpd.conf , use_sendfile = NO   ¿Would the solution? proves it.

> 
> > 4.- stored millions of small files, backup software cript. compress and
> > split
> > files 5/30 Mb, any advice on file systems to use ext3-ext4 or block
> > size. kernel
> > elevator, another tuning?
> 
> (Again we are not fully sure about the question)
> 
> At our company we do not use any tuning, just normal ext3.
> 
> How many milions of files do you have? Generally speaking 1 million files
>  takes approximately 300 MiB of RAM. Installation of 25 million files
>  requires about 8GiB of RAM and 25GiB space on HDD.
> 

* command executed in all primary servers 
find /media/volumen0/virtualusers -type f | wc -l    

Current all total  369,026,500 

* 80%+  write process 20%- read process (recovery backups)

* On mfs X 3 copies

* Ram required metaserver no problem.

* In chunservers clients and whether

* It intends to use existing or adquire conventional hardware and he added 
that support.

* Currently, both the primary servers, secondary and tertiary are HP ML110
2GB ECC RAM, six 1TB SATA disks each, 2 netlinks 2Gb/s per server, and 
ridiculous price for wholesale

* The current uptime restarts excluding kernel changes and changes of hard 
drives failing is 2 years. 24x7

*  openSUSE distribution, and are totally loose in performance and ram.


> > 5.- ¿Any suggestions for the network, MTU, bandwitch Agrégation,
> > windows
> > size, etc ... on sysctl.conf?
> 
> We strongly recommend using jumbo-frames (MTU=9000). With the greater
>  amount of chunkservers switches have to be connected through optical fiber
>  or links aggregated.
> 
* OK

> > 6.- I want to store 3 copies of the files, is there any possibility to
> > specify that one of them is stored in another group of chunkservers
> > remote
> > but connected by optical fiber on the same subnet?
> > ¿planned in de future?
> 
> There is no such a possibility now. But maybe it will be introduced in the
>  future.
> 

* Excellent.

> > 7.- I maintain high retention time of deleted files. What is the
> > technique that
> > uses MFS?. ¿It retains the three copies scheduling their final
> > deletion?
> > ¿Delete two copies and keep one?. if so, ¿how that machine is stored?
> 
> All the copies in the trash are retained. And still for the files in the
>  trash you can set the goal (ie. number of copies), so you can easily write
>  a simple script which would for example change goal for all files in the
>  trash from 3 to 2 (and later to 1).
> 

* goodly

> > 8.- What would be the best option on the clients automounter or
> > desconecction, autofs, daemon guard, cron script?
> 
> We have not used mfsmount with automounter nor autofs, we think you have to
>  just monitor its activity. If the master server or chunkservers are
>  termporarily not available, mfsmount tries to reconnect automatically.
> 

* I understand that with the reconnect feature of mfsmount or FUSE 
in /etc/fstab

* Jose Maria.

Re: [Moosefs-users] [SPAM] questions mfs 1.6.x

[Michał B. <mic...@ge...>]

Hola Jose!

Estoy muy contento que tenemos usuarios en Espana :) 


> -----Original Message-----
> From: logs [mailto:lo...@se...]
> Sent: Sunday, January 24, 2010 8:36 AM
> To: moo...@li...
> Subject: [Moosefs-users] [SPAM] questions mfs 1.6.x
 

[...]

> Questions:
> 
> 1 .- In the chunservers is best to mount a raid5 with one big chunk or
> 6 individually mounted hard drives?
It should work better with 6 individually mounted hard drives.

 
> 2 .- Very important, vsftpd can write directly on the MFS mount point,
> or
> need to mount --bind. A better idea? commercial code, other
> implementation
> of mfsmount, native module?
We are not sure if we fully understand the question, but we'll try to answer.

There is no ready ftp server which would communicate directly with the master server and chunkservers so the solution is to "go through" the normal mfsmount. It gives better results than creating a special code for communication - kernel really supports I/O operations very efficiently (eg. by using read-ahead, buffering, writing in background, etc.). That's why we also use classic file systems for storing data in chunkservers rather than using the disk directly as a block-device. 


> 
> 3.- Reduce to 1 or 2 client machines to mount the MFS cluster would
> increase performance?
Mount devices does not influence performance. It is rather the contrary - for the overall performance the more clients the better.

 
> 4.- stored millions of small files, backup software cript. compress and
> split
> files 5/30 Mb, any advice on file systems to use ext3-ext4 or block
> size. kernel
> elevator, another tuning?
(Again we are not fully sure about the question)

At our company we do not use any tuning, just normal ext3.

How many milions of files do you have? Generally speaking 1 million files takes approximately 300 MiB of RAM. Installation of 25 million files requires about 8GiB of RAM and 25GiB space on HDD.


> 5.- ¿Any suggestions for the network, MTU, bandwitch Agrégation,
> windows
> size, etc ... on sysctl.conf?
We strongly recommend using jumbo-frames (MTU=9000). With the greater amount of chunkservers switches have to be connected through optical fiber or links aggregated. 

 
> 6.- I want to store 3 copies of the files, is there any possibility to
> specify that one of them is stored in another group of chunkservers
> remote
> but connected by optical fiber on the same subnet?
> ¿planned in de future?
There is no such a possibility now. But maybe it will be introduced in the future. 


 
> 7.- I maintain high retention time of deleted files. What is the
> technique that
> uses MFS?. ¿It retains the three copies scheduling their final
> deletion?
> ¿Delete two copies and keep one?. if so, ¿how that machine is stored?
All the copies in the trash are retained. And still for the files in the trash you can set the goal (ie. number of copies), so you can easily write a simple script which would for example change goal for all files in the trash from 3 to 2 (and later to 1).



> 8.- What would be the best option on the clients automounter or
> desconecction, autofs, daemon guard, cron script?
We have not used mfsmount with automounter nor autofs, we think you have to just monitor its activity. If the master server or chunkservers are termporarily not available, mfsmount tries to reconnect automatically.


> 
> 9 .- Do you know of any recent future risk of abandoning the project?
No, we do not plan to abandon the project :) The system is supported by a big Polish research company with several branches abroad. We have about 1 petabyte of data kept on MooseFS. And it is an open source project.



Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01


> 
> * Any suggestions on hardware, configurations, usability ...... will be
> greatly
> appreciated.
> 
> Thank you very much everyone and especially to the developers of MFS.
> 
> jose maria
> 
> 
> -----------------------------------------------------------------------
> -------
> Throughout its 18-year history, RSA Conference consistently attracts
> the
> world's best and brightest in the field, creating opportunities for
> Conference
> attendees to learn about information security's most important issues
> through
> interactions with peers, luminaries and emerging and established
> companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] blank server charts

[Steve <st...@bo...>]

Michal,



Yes, that does indeed fix.



My thanks to you and the devs



Steve

 

 



 

 

-------Original Message------- 

 

From: Michał Borychowski 

Date: 25/01/2010 10:47:41 

To: 'Steve'; moo...@li... 

Subject: RE: [Moosefs-users] blank server charts 

 

Hi Steve! 

 

The probable cause of your blank screens is lack of "zlib-dev" (which is 

called "zlib1g-dev") package in the chunkservers. You have to install this 

and recompile chunkservers (remember to rerun configure script: 

./configure). 

 

Let us know if that helps. 

 

 

Kind regards 

Michał Borychowski 

MooseFS Support Manager 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

Gemius S.A. 

ul. Wołoska 7, 02-672 Warszawa 

Budynek MARS, klatka D 

Tel.: +4822 874-41-00; 

Fax : +4822 874-41-01 

 

 

 

-----Original Message----- 

From: Steve [mailto:st...@bo...] 

Sent: Thursday, January 21, 2010 3:25 PM 

To: moo...@li... 

Subject: [Moosefs-users] blank server charts 

 

Hi, 

 

 

 

3 out of my 4 chunkserver graphs are all blank. 

 

 

 

Steve. 

 

----------------------------------------------------------------------------


-- 

Throughout its 18-year history, RSA Conference consistently attracts the 

world's best and brightest in the field, creating opportunities for 

Conference 

attendees to learn about information security's most important issues 

through 

interactions with peers, luminaries and emerging and established companies. 

http://p.sf.net/sfu/rsaconf-dev2dev 

_______________________________________________ 

moosefs-users mailing list 

moo...@li... 

https://lists.sourceforge.net/lists/listinfo/moosefs-users 

 

 

Re: [Moosefs-users] mfs mount generates high trafic from chunks

[Michał B. <mic...@ge...>]

Hi Eduard!

So... From the number of 'getattr', 'open' i 'read' operations we conclude
that there are about 4500 files read in one minute. Given three chunkservers
it takes 1500 reads per minute per one server, which is 25 reads per second
on one server. The system shows there are 24MB read per second which would
mean an average file size of about 950kB and transfer about 190Mb/s.

On the other hand there are about 1 160 000 files which with an average file
size of 950kB would occupy about 1 TB. From the Info tab we see the system
has 14 TB of space with 13 TB free - everything sums up properly.

So the question would rather be - why the server requests so many reads
though it does not send that amount of data? Please check your webserver and
its configuration. What webserver do you use? Thereotically none webserver
should make unnecessary reads. You can try connecting to the webserver by a
tool like "strace" and see what operations are made.


Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01


-----Original Message-----
From: Eduard Iacoboaia [mailto:edu...@di...] 
Sent: Friday, January 22, 2010 3:26 PM
To: moo...@li...
Subject: Re: [Moosefs-users] mfs mount generates high trafic from chunks

There are 2 chunks, goal 2 on all files, one active mount (the
webserver): 3 virtual machines, each hosted on separate servers, nothing
else on those servers. The master, metaloger and mfscgi are 3 virtual
machines hosted on a 4th server. I attached some screenshots. I noticed the
getattr chart resembles a lot those from traffic in chunk servers.

--
Eduard

Michał Borychowski wrote:
> Don't you have any additional replications between chunkservers? Does 
> the webserver and one of chunkservers use the same machine?
>
>
> Kind regards
> Michał Borychowski
> MooseFS Support Manager
>
>
>
> -----Original Message-----
> From: Eduard Iacoboaia [mailto:edu...@di...]
> Sent: Friday, January 22, 2010 12:31 PM
> To: moo...@li...
> Subject: Re: [Moosefs-users] mfs mount generates high trafic from 
> chunks
>
> Hi.
>
> This is the traffic incoming on port 9422 in the mount server:
>  --------------------------------------------------------
>  |  Rate  |              Average for                    |
>  --------------------------------------------------------
>  |########|    3 sec     |    15 sec     |    30 sec    |
>  --------------------------------------------------------
>  |  bits  |    154.2M    |     142.4M    |     145.7M   |
>  |  pkts  |     13.1K    |      12.1K    |      12.3K   |
>  --------------------------------------------------------
>
>
> This is the outgoing port 80 from the same server:
>  --------------------------------------------------------
>  |  Rate  |              Average for                    |
>  --------------------------------------------------------
>  |########|    3 sec     |    15 sec     |    30 sec    |
>  --------------------------------------------------------
>  |  bits  |      1.6M    |       1.2M    |       1.2M   |
>  |  pkts  |      171     |       129     |       127    |
>  --------------------------------------------------------
>
>
> I also attached a screenshot. If it's not what you needed, please let 
> me know.
>
> Thanks,
> --
> Eduard
>
> Michał Borychowski wrote:
>   
>> And what do you have on graphs from chunkservers? Could you send us 
>> the screenshots from the cgi monitor?
>>
>> Is there really 35-70Mb transfer between mfsmount and chunkservers?
>>
>>
>> Kind regards
>> Michal
>>
>> -----Original Message-----
>> From: Eduard Iacoboaia [mailto:edu...@di...]
>> Sent: Thursday, January 21, 2010 11:29 AM
>> To: moo...@li...
>> Subject: [Moosefs-users] mfs mount generates high trafic from chunks
>>
>> Hi.
>>
>> I have a webserver that has the files (~1.1mil files and 280gb) 
>> mounted over mfs 1.6.11. Although the server generates about 2-3mbit 
>> of http trafic, it retrieves from chunk servers between 35 and 70mbit 
>> (5sec average). The mount has:
>>
>> mfscachefiles,mfsattrcacheto=10,mfsentrycacheto=10,mfsdirentrycacheto
>> =
>> 20
>>
>> Is there anything I can do to lower the amount of traffic from chunks?
>>
>> Thanks,
>> --
>> Eduard
>>
>>
>> ---------------------------------------------------------------------
>> -
>> ------
>> --
>> Throughout its 18-year history, RSA Conference consistently attracts 
>> the world's best and brightest in the field, creating opportunities 
>> for Conference attendees to learn about information security's most 
>> important issues through interactions with peers, luminaries and 
>> emerging and established companies.
>> http://p.sf.net/sfu/rsaconf-dev2dev
>> _______________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>>
>>   
>>     
>
>   

Re: [Moosefs-users] blank server charts

[Michał B. <mic...@ge...>]

Hi Steve!

The probable cause of your blank screens is lack of "zlib-dev" (which is
called "zlib1g-dev") package in the chunkservers. You have to install this
and recompile chunkservers (remember to rerun configure script:
./configure).

Let us know if that helps.


Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00;
Fax : +4822 874-41-01



-----Original Message-----
From: Steve [mailto:st...@bo...] 
Sent: Thursday, January 21, 2010 3:25 PM
To: moo...@li...
Subject: [Moosefs-users] blank server charts

Hi, 

 

3 out of my 4 chunkserver graphs are all blank. 

 

Steve.

----------------------------------------------------------------------------
--
Throughout its 18-year history, RSA Conference consistently attracts the
world's best and brightest in the field, creating opportunities for
Conference
attendees to learn about information security's most important issues
through
interactions with peers, luminaries and emerging and established companies.
http://p.sf.net/sfu/rsaconf-dev2dev
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] mfs mount generates high trafic from chunks

[Eduard I. <edu...@di...>]

There are 2 chunks, goal 2 on all files, one active mount (the 
webserver): 3 virtual machines, each hosted on separate servers, nothing 
else on those servers. The master, metaloger and mfscgi are 3 virtual 
machines hosted on a 4th server. I attached some screenshots. I noticed 
the getattr chart resembles a lot those from traffic in chunk servers.

--
Eduard

Michał Borychowski wrote:
> Don't you have any additional replications between chunkservers? Does the
> webserver and one of chunkservers use the same machine?
>
>
> Kind regards
> Michał Borychowski 
> MooseFS Support Manager
>
>
>
> -----Original Message-----
> From: Eduard Iacoboaia [mailto:edu...@di...] 
> Sent: Friday, January 22, 2010 12:31 PM
> To: moo...@li...
> Subject: Re: [Moosefs-users] mfs mount generates high trafic from chunks
>
> Hi.
>
> This is the traffic incoming on port 9422 in the mount server:
>  --------------------------------------------------------
>  |  Rate  |              Average for                    |
>  --------------------------------------------------------
>  |########|    3 sec     |    15 sec     |    30 sec    |
>  --------------------------------------------------------
>  |  bits  |    154.2M    |     142.4M    |     145.7M   |
>  |  pkts  |     13.1K    |      12.1K    |      12.3K   |
>  --------------------------------------------------------
>
>
> This is the outgoing port 80 from the same server:
>  --------------------------------------------------------
>  |  Rate  |              Average for                    |
>  --------------------------------------------------------
>  |########|    3 sec     |    15 sec     |    30 sec    |
>  --------------------------------------------------------
>  |  bits  |      1.6M    |       1.2M    |       1.2M   |
>  |  pkts  |      171     |       129     |       127    |
>  --------------------------------------------------------
>
>
> I also attached a screenshot. If it's not what you needed, please let me
> know.
>
> Thanks,
> --
> Eduard
>
> Michał Borychowski wrote:
>   
>> And what do you have on graphs from chunkservers? Could you send us 
>> the screenshots from the cgi monitor?
>>
>> Is there really 35-70Mb transfer between mfsmount and chunkservers?
>>
>>
>> Kind regards
>> Michal
>>
>> -----Original Message-----
>> From: Eduard Iacoboaia [mailto:edu...@di...]
>> Sent: Thursday, January 21, 2010 11:29 AM
>> To: moo...@li...
>> Subject: [Moosefs-users] mfs mount generates high trafic from chunks
>>
>> Hi.
>>
>> I have a webserver that has the files (~1.1mil files and 280gb) 
>> mounted over mfs 1.6.11. Although the server generates about 2-3mbit 
>> of http trafic, it retrieves from chunk servers between 35 and 70mbit 
>> (5sec average). The mount has:
>>
>> mfscachefiles,mfsattrcacheto=10,mfsentrycacheto=10,mfsdirentrycacheto=
>> 20
>>
>> Is there anything I can do to lower the amount of traffic from chunks?
>>
>> Thanks,
>> --
>> Eduard
>>
>>
>> ----------------------------------------------------------------------
>> ------
>> --
>> Throughout its 18-year history, RSA Conference consistently attracts 
>> the world's best and brightest in the field, creating opportunities 
>> for Conference attendees to learn about information security's most 
>> important issues through interactions with peers, luminaries and 
>> emerging and established companies.
>> http://p.sf.net/sfu/rsaconf-dev2dev
>> _______________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>>
>>   
>>     
>
>   

Re: [Moosefs-users] mfs mount generates high trafic from chunks

[Michał B. <mic...@ge...>]

Don't you have any additional replications between chunkservers? Does the
webserver and one of chunkservers use the same machine?


Kind regards
Michał Borychowski 
MooseFS Support Manager



-----Original Message-----
From: Eduard Iacoboaia [mailto:edu...@di...] 
Sent: Friday, January 22, 2010 12:31 PM
To: moo...@li...
Subject: Re: [Moosefs-users] mfs mount generates high trafic from chunks

Hi.

This is the traffic incoming on port 9422 in the mount server:
 --------------------------------------------------------
 |  Rate  |              Average for                    |
 --------------------------------------------------------
 |########|    3 sec     |    15 sec     |    30 sec    |
 --------------------------------------------------------
 |  bits  |    154.2M    |     142.4M    |     145.7M   |
 |  pkts  |     13.1K    |      12.1K    |      12.3K   |
 --------------------------------------------------------


This is the outgoing port 80 from the same server:
 --------------------------------------------------------
 |  Rate  |              Average for                    |
 --------------------------------------------------------
 |########|    3 sec     |    15 sec     |    30 sec    |
 --------------------------------------------------------
 |  bits  |      1.6M    |       1.2M    |       1.2M   |
 |  pkts  |      171     |       129     |       127    |
 --------------------------------------------------------


I also attached a screenshot. If it's not what you needed, please let me
know.

Thanks,
--
Eduard

Michał Borychowski wrote:
> And what do you have on graphs from chunkservers? Could you send us 
> the screenshots from the cgi monitor?
>
> Is there really 35-70Mb transfer between mfsmount and chunkservers?
>
>
> Kind regards
> Michal
>
> -----Original Message-----
> From: Eduard Iacoboaia [mailto:edu...@di...]
> Sent: Thursday, January 21, 2010 11:29 AM
> To: moo...@li...
> Subject: [Moosefs-users] mfs mount generates high trafic from chunks
>
> Hi.
>
> I have a webserver that has the files (~1.1mil files and 280gb) 
> mounted over mfs 1.6.11. Although the server generates about 2-3mbit 
> of http trafic, it retrieves from chunk servers between 35 and 70mbit 
> (5sec average). The mount has:
>
> mfscachefiles,mfsattrcacheto=10,mfsentrycacheto=10,mfsdirentrycacheto=
> 20
>
> Is there anything I can do to lower the amount of traffic from chunks?
>
> Thanks,
> --
> Eduard
>
>
> ----------------------------------------------------------------------
> ------
> --
> Throughout its 18-year history, RSA Conference consistently attracts 
> the world's best and brightest in the field, creating opportunities 
> for Conference attendees to learn about information security's most 
> important issues through interactions with peers, luminaries and 
> emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>   

Re: [Moosefs-users] mfs mount generates high trafic from chunks

[Eduard I. <edu...@di...>]

Hi.

This is the traffic incoming on port 9422 in the mount server:
 --------------------------------------------------------
 |  Rate  |              Average for                    |
 --------------------------------------------------------
 |########|    3 sec     |    15 sec     |    30 sec    |
 --------------------------------------------------------
 |  bits  |    154.2M    |     142.4M    |     145.7M   |
 |  pkts  |     13.1K    |      12.1K    |      12.3K   |
 --------------------------------------------------------


This is the outgoing port 80 from the same server:
 --------------------------------------------------------
 |  Rate  |              Average for                    |
 --------------------------------------------------------
 |########|    3 sec     |    15 sec     |    30 sec    |
 --------------------------------------------------------
 |  bits  |      1.6M    |       1.2M    |       1.2M   |
 |  pkts  |      171     |       129     |       127    |
 --------------------------------------------------------


I also attached a screenshot. If it's not what you needed, please let me 
know.

Thanks,
--
Eduard

Michał Borychowski wrote:
> And what do you have on graphs from chunkservers? Could you send us the
> screenshots from the cgi monitor?
>
> Is there really 35-70Mb transfer between mfsmount and chunkservers?
>
>
> Kind regards
> Michal
>
> -----Original Message-----
> From: Eduard Iacoboaia [mailto:edu...@di...] 
> Sent: Thursday, January 21, 2010 11:29 AM
> To: moo...@li...
> Subject: [Moosefs-users] mfs mount generates high trafic from chunks
>
> Hi.
>
> I have a webserver that has the files (~1.1mil files and 280gb) mounted 
> over mfs 1.6.11. Although the server generates about 2-3mbit of http 
> trafic, it retrieves from chunk servers between 35 and 70mbit (5sec 
> average). The mount has:
>
> mfscachefiles,mfsattrcacheto=10,mfsentrycacheto=10,mfsdirentrycacheto=20
>
> Is there anything I can do to lower the amount of traffic from chunks?
>
> Thanks,
> --
> Eduard
>
>
> ----------------------------------------------------------------------------
> --
> Throughout its 18-year history, RSA Conference consistently attracts the
> world's best and brightest in the field, creating opportunities for
> Conference
> attendees to learn about information security's most important issues
> through
> interactions with peers, luminaries and emerging and established companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>