moosefs-users — Mailing list for MooseFS Users

Re: [Moosefs-users] [PVE-User] KVM on MFS

[Łukasz M. <l.m...@gm...>]

Dnia środa, 25 maja 2011 o 17:51:37 Giovanni Toraldo napisał(a):
> Hi Richard,
> 
> On 13/05/2011 19:26, Richard Chute wrote:
> > 5385  open("/mfs/pve/images/101/vm-101-disk-1.raw",
> > O_RDWR|O_DIRECT|O_CLOEXEC) = -1 EINVAL (Invalid argument)
> 
> I was wrong, I am using proxmox 1.7 with MooseFS. Today, after trying to
> ugrade to 1.8, I get the same error. :(
> 
> There is probably a regression (or some conflicts with MooseFS) on one
> of the latest proxmox updated packages:
> 
> [AGGIORNATO] ksm-control-daemon 1.0-4 -> 1.0-5
> [AGGIORNATO] libpve-storage-perl 1.0-16 -> 1.0-17
> [AGGIORNATO] proxmox-ve-2.6.32 1.7-30 -> 1.8-33
> [AGGIORNATO] pve-firmware 1.0-10 -> 1.0-11
> [AGGIORNATO] pve-kernel-2.6.32-4-pve 2.6.32-30 -> 2.6.32-33
> [AGGIORNATO] pve-manager 1.7-11 -> 1.8-17
> [AGGIORNATO] pve-qemu-kvm 0.13.0-3 -> 0.14.0-3
> [AGGIORNATO] qemu-server 1.1-28 -> 1.1-30
> [AGGIORNATO] vzctl 3.0.24-1pve4 -> 3.0.26-1pve4
> [AGGIORNATO] vzdump 1.2-10 -> 1.2-12
> [AGGIORNATO] vzprocps 2.0.11-1dso2 -> 2.0.11-2
> 
> The only thing I tried, it's to manually downgrade pve-qemu-kvm to 0.13,
> but nohting changed. After, I removed proxmox, modified my sources.list
> to point to pve1.7 version, and reinstall, and everything was working
> again.
> 
> I'm cc-ing to pve mailing list too.
> 
> Regards,

MFS is fuse based and fuse does not support O_DIRECT which is default since 
1.8, You need to add cache=writeback to disk defined in /etc/qemu-
server/$VID.conf
example:

virtio0: MFS_disks:101/vm-101-disk-1.raw,cache=writeback

or

ide0: MFS_disks:101/vm-101-disk-1.raw,cache=writeback

Łukasz Mierzwa

[Moosefs-users] corner case in disk for removal algorithm ?

[Laurent W. <lw...@hy...>]

Hi there,

I've got a chunkserver, say A, with 24 disks (JBOD).
12 have already been changed from 400 GiB to 2TiB. Load is finished.
Now, I marked for removal the 12 400 GiB that were remaining. The other
chunkservers are taking the chunks, but the 12 2 TiB disks on A are not
at all taking the load. So A, considering only the 12 2TiB disks, is
90% full, while the other chunkservers are now at 96%.
Could it be a forgotten corner case in chunks space load balance ?

Thanks,
-- 
Laurent Wandrebeck
HYGEOS, Earth Observation Department / Observation de la Terre
Euratechnologies
165 Avenue de Bretagne
59000 Lille, France
tel: +33 3 20 08 24 98
http://www.hygeos.com
GPG fingerprint/Empreinte GPG: F5CA 37A4 6D03 A90C 7A1D  2A62 54E6 EF2C
D17C F64C

[Moosefs-users] Non-Existent Chunk Message

[Robert D. <ro...@in...>]

Can we update the print for the nonexistent chunk message to show the
chunkserver ID?

 

Old Output:

 

daemon.log:May 24 14:36:29 mfsmaster mfsmaster[1798]: chunkserver has
nonexistent chunk (00000000007E98E1_00000001), so create it for future
deletion

 

New Output:

 

daemon.log:May 24 14:36:29 mfsmaster mfsmaster[1798]: chunkserver
(mfschunkserver1) has nonexistent chunk (00000000007E98E1_00000001), so
create it for future deletion

 

daemon.log:May 24 14:36:29 mfsmaster mfsmaster[1798]: chunkserver
(192.168.1.2) has nonexistent chunk (00000000007E98E1_00000001), so create
it for future deletion

 

Re: [Moosefs-users] Health Check

[Ricardo J. B. <ric...@da...>]

El Lunes 23 May 2011, Anh K. Huynh escribió:
> On Mon, 23 May 2011 16:18:04 -0300
>
> "Ricardo J. Barberis" <ric...@da...> wrote:
> > El Lunes 23 May 2011, Michal Borychowski escribió:
> > > The CGI monitor gives most comprehensive information about the
> > > state of the system.
> >
> > Is it safe to connect to the ports of master, metalogger and
> > chunckservers from an external host. to check if they're alive?
>
> Yes it is. You can read some examples in the source code of "mfs.cgi"
>
> > Could that cause any kind of problem?
> >
> > I'm thinking about monitoring those services like we do with others
> > (web, smtp, etc).
>
> Depends on how you interact with the master, logger :D

Well, I can check if the port is open. It's not a great check but it's better 
than nothing :)

Cheers,
-- 
Ricardo J. Barberis
Senior SysAdmin / ITI
Dattatec.com :: Soluciones de Web Hosting
Tu Hosting hecho Simple!

Re: [Moosefs-users] KVM on MFS

[Ólafur Ó. <osv...@ne...>]

Hi,
We have not done extensive testing within the domU's, we did before deploying do tests on the dom0's to compare against the SAN and iSCSI setups we have and found the speed was acceptable in comparison to those.

/Oli

On 13.5.2011, at 17:15, Kristofer Pettijohn wrote:

> Just curious - how has your performance been with that?  Have you ran any type of performance benchmarks from within any of the domU's?
> 
> ----- Original Message -----
> From: "Ólafur Ósvaldsson" <osv...@ne...>
> To: "Richard Chute" <ric...@ai...>
> Cc: moo...@li...
> Sent: Friday, May 13, 2011 10:05:54 AM
> Subject: Re: [Moosefs-users] KVM on MFS
> 
> Hi,
> Not sure if this is the same problem as yours, but we are running a decent Xen setup with MFS as the storage for the VM's, it works very well and the only problems we have had was when trying to use directio, if that is disabled it works fine.
> 
> /Oli
> 
> On 13.5.2011, at 14:24, Richard Chute wrote:
> 
>> Hello MFS Devs,
>>  I am evaluating MooseFS for use in various aspects of our business, 
>> and one of them is in use as a storage mechanism for virtual machines.  
>> Specifically, we're using proxmox for virtualization in our environment, 
>> and we seem to be hitting a snag when using KVM containers -- the 
>> virtual machines won't start.  We currently believe that this may be 
>> because MFS (or FUSE, possibly) does not allow mmap'ing of files.
>>  I am wondering if anyone can shed some light on this type of 
>> situation (using MFS for storage of KVM virtual machines) and/or if 
>> anyone has any known or possibly unknown issues with mmap'ing files on MFS.
>> 
>>  Thanks,
>>  Richard.
>> 
>> ------------------------------------------------------------------------------
>> Achieve unprecedented app performance and reliability
>> What every C/C++ and Fortran developer should know.
>> Learn how Intel has extended the reach of its next-generation tools
>> to help boost performance applications - inlcuding clusters.
>> http://p.sf.net/sfu/intel-dev2devmay
>> _______________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users
> 
> --
> Ólafur Osvaldsson
> System Administrator
> Nethonnun ehf.
> e-mail: osv...@ne...
> phone: +354 517 3400
> 
> 
> ------------------------------------------------------------------------------
> Achieve unprecedented app performance and reliability
> What every C/C++ and Fortran developer should know.
> Learn how Intel has extended the reach of its next-generation tools
> to help boost performance applications - inlcuding clusters.
> http://p.sf.net/sfu/intel-dev2devmay
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users

--
Ólafur Osvaldsson
System Administrator
Nethonnun ehf.
e-mail: osv...@ne...
phone: +354 517 3400

Re: [Moosefs-users] mount disapeared

[Papp T. <to...@ma...>]

On 05/23/2011 12:37 PM, Michal Borychowski wrote:
>
> Hi!
>
> Is this connected to the fact that you ran out of space at /var 
> partition? It's not easy to tell the exact cause. It looks like some 
> chunkservers where not connected while saving.
>
>


Yes I need more investigating on this. I still cannot say what caused what.

tamas

Re: [Moosefs-users] Health Check

[Michal B. <mic...@ge...>]

For the moment there are no special tools which allow you to monitor MooseFS
state from the command line. We plan to introduce them in the
'mfsmastertools', but I cannot tell when it could be ready.

 

 

Kind regards

Michal

 

From: Upendra Moturi [mailto:upe...@he...] 
Sent: Monday, May 23, 2011 12:38 PM
To: Michal Borychowski
Cc: Saurav Lahiri; moo...@li...
Subject: Re: [Moosefs-users] Health Check

 

Hi 
Thank you for your reply.
I would like to get the status of those through command line.

2011/5/23 Michal Borychowski <mic...@ge...>

Hi!

 

The CGI monitor gives most comprehensive information about the state of the
system.

 

 

Kind regards

Michał Borychowski 

MooseFS Support Manager

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gemius S.A.

ul. Wołoska 7, 02-672 Warszawa

Budynek MARS, klatka D

Tel.: +4822 874-41-00

Fax : +4822 874-41-01

 

 

From: Upendra Moturi [mailto:upe...@he...] 
Sent: Thursday, May 19, 2011 4:32 PM
To: moo...@li...; Saurav Lahiri
Subject: [Moosefs-users] Health Check

 

Hi



Is there a way to check the health of mfsmaster,mfsmetalogger and
mfschunkserver ?

-- 
Thanks and Regards,
Upendra.M




-- 
Thanks and Regards,
Upendra.M

[Moosefs-users] IPv6 ?

[Florent B. <fl...@co...>]

Hi everyone,

First of all, a big thank to Gemius team for their work on MooseFS, 
thank you!

Has anyone succeeded to run MooseFS using IPv6 network ? I'm trying but 
it seems not working ...

It could be a great update if it is not yet compatible.

By the way, could we have an idea of when 1.7 version could be released 
? (quotas management is really important :) )

Thank you a lot.
-- 


Florent Bautista

------------------------------------------------------------------------

Ce message et ses éventuelles pièces jointes sont personnels, 
confidentiels et à l'usage exclusif de leur destinataire.
Si vous n'êtes pas la personne à laquelle ce message est destiné, 
veuillez noter que vous avez reçu ce courriel par erreur et qu'il vous 
est strictement interdit d'utiliser, de diffuser, de transférer, 
d'imprimer ou de copier ce message.

This e-mail and any attachments hereto are strictly personal, 
confidential and intended solely for the addressee.
If you are not the intended recipient, be advised that you have received 
this email in error and that any use, dissemination, forwarding, 
printing, or copying of this message is strictly prohibited.

------------------------------------------------------------------------

30440 Saint Laurent le Minier
France

*Compagnie pour des Prestations Internet*

Téléphone : +33 (0)467 73 89 48
Télécopie : + 33 (0)9 59 48 06 27

Courriel : Fl...@Co... <mailto:fl...@co...>

------------------------------------------------------------------------

Re: [Moosefs-users] Health Check

[Anh K. H. <ky...@vi...>]

On Mon, 23 May 2011 16:18:04 -0300
"Ricardo J. Barberis" <ric...@da...> wrote:

> El Lunes 23 May 2011, Michal Borychowski escribió:
> >
> > The CGI monitor gives most comprehensive information about the
> > state of the system.
> 
> Is it safe to connect to the ports of master, metalogger and
> chunckservers from an external host. to check if they're alive?

Yes it is. You can read some examples in the source code of "mfs.cgi"

> Could that cause any kind of problem?
> 
> I'm thinking about monitoring those services like we do with others
> (web, smtp, etc).

Depends on how you interact with the master, logger :D

Regards,

-- 
Anh Ky Huynh @ ICT
Registered Linux User #392115

[Moosefs-users] Finding out what file is corrupt

[Tom E. <tom...@ot...>]

Hey guys,

I've just set up my first test cluster for trying out MooseFS, and when 
I restarted one of my chunkservers I ended up with one chunk in the 
'red' (zero copies).

How do I actually find out what file or files are missing chunks? I can 
see where the chunks are for each file using 'mfsfileinfo', but how can 
I get a quick summary of what the problem files are?

Is it the 'filesystem check info' at the bottom of the cgi page? Mine 
still just says 'no data'. Is there a way to trigger a 'file system 
check' or something like that?

Thanks!

	Tom

[Moosefs-users] Replication Priority?

[W K. <wk...@bn...>]

Last Night, we had one of our 4 chunkservers 'lock up' in some 
mysterious way.  The master started giving off these messages

May 22 22:37:50 mfs1master mfsmaster[2522]: (192.168.0.24:9422) chunk: 000000000023DB7F replication status: 22
May 22 22:37:51 mfs1master mfsmaster[2522]: (192.168.0.24:9422) chunk: 0000000000149C9D replication status: 22
May 22 22:37:51 mfs1master mfsmaster[2522]: (192.168.0.24:9422) chunk: 00000000002EB8F6 deletion status: 22
May 22 22:38:26 mfs1master mfsmaster[2522]: connection with ML(192.168.0.24) has been closed by peer
May 23 11:12:11 mfs1master mfsmaster[2522]: chunkserver disconnected - ip: 192.168.0.24, port: 9422, usedspace: 0 (0.00 GiB), totalspace: 0 (0.00 GiB)

MooseFS did the right thing, and kicked the chunkserver out. There was 
no interruption of service and we didn't even notice the problem until 
someone looked at
the CGI this morning and saw that we had a large number of undergoal 
(goal=2) files which moose was fixing (and had been fixing all night) 
the undergoal condition at a rate of about 2-4 chunks a second.

So we replaced the failed chunkserver  and continued on, quite content 
with how resiliant MooseFS was under a failure.

We then thought about it and decided that we had gone a long time with 
only 1 copy of a large number of chunks and that perhaps a goal of 3 
would have been safer. (i.e. if 1 of the 4 chunkservers dies, we still 
have 2 copies and could still lose a second chunkserver without harm).

So we reset the Goal from 2 to 3. We did this while were still in an 
undergoal position at goal=2 for about 10,000 chunks that hadn't yet 
been healed.

So now the CGI is showing 10,000+ chunks with a single copy (red), 2 
million+ chunks are now orange (2 copies) and the system is happily 
increasing the 'green' 3 valid copy column.

The problem is that it seems to be concentrating on the orange (2 copy) 
files and ignoring the 10,000+ red ones that are most at risk. In the 
last hour we've seen a few 'red' chunks
disappear but the vast majority of activity is occuring in the orange (2 
copy) column.

Shouldn't the replication worry about the single copy files first?

I also realize we could simply set the goal back to 2 let it finish that 
up and THEN switch it to 3 but I'm curious as to what the community says.

-WK

Re: [Moosefs-users] Health Check

[Ricardo J. B. <ric...@da...>]

El Lunes 23 May 2011, Michal Borychowski escribió:
> Hi!
>
> The CGI monitor gives most comprehensive information about the state of the
> system.

Is it safe to connect to the ports of master, metalogger and chunckservers 
from an external host. to check if they're alive?

Could that cause any kind of problem?

I'm thinking about monitoring those services like we do with others (web, 
smtp, etc).

Thank you,
-- 
Ricardo J. Barberis
Senior SysAdmin / ITI
Dattatec.com :: Soluciones de Web Hosting
Tu Hosting hecho Simple!

Re: [Moosefs-users] Health Check

[Steve <st...@bo...>]

The daemons all produce logging 

 

 

 

 

-------Original Message------- 

 

From: Upendra Moturi 

Date: 23/05/2011 12:04:58 

To: Michal Borychowski 

Cc: Saurav Lahiri; moo...@li... 

Subject: Re: [Moosefs-users] Health Check 

 

Hi 

Thank you for your reply. 

I would like to get the status of those through command line. 

 

 

2011/5/23 Michal Borychowski <mic...@ge...> 

 

Hi! 

 

The CGI monitor gives most comprehensive information about the state of the
system. 

 

 

Kind regards 

Michał Borychowski 

MooseFS Support Manager 

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

Gemius S.A. 

ul. Wołoska 7, 02-672 Warszawa 

Budynek MARS, klatka D 

Tel.: +4822 874-41-00 

Fax : +4822 874-41-01 

 

 

From: Upendra Moturi [mailto:upe...@he...] 

Sent: Thursday, May 19, 2011 4:32 PM 

To: moo...@li...; Saurav Lahiri 

Subject: [Moosefs-users] Health Check 

 

Hi 

 

 

Is there a way to check the health of mfsmaster,mfsmetalogger and
mfschunkserver ? 

 

-- 

Thanks and Regards, 

Upendra.M 

 

 

 

-- 

Thanks and Regards, 

Upendra.M 

 

 

 

 

 

-----------------------------------------------------------------------------
 

What Every C/C++ and Fortran developer Should Know! 

Read this article and learn how Intel has extended the reach of its 

next-generation tools to help Windows* and Linux* C/C++ and Fortran 

developers boost performance applications - including clusters. 

http://p.sf.net/sfu/intel-dev2devmay 

 

 

 

 

 

_______________________________________________ 

moosefs-users mailing list 

moo...@li... 

https://lists.sourceforge.net/lists/listinfo/moosefs-users 

 

Re: [Moosefs-users] Health Check

[Upendra M. <upe...@he...>]

Hi
Thank you for your reply.
I would like to get the status of those through command line.

2011/5/23 Michal Borychowski <mic...@ge...>

> Hi!
>
>
>
> The CGI monitor gives most comprehensive information about the state of the
> system.
>
>
>
>
>
> Kind regards
>
> Michał Borychowski
>
> MooseFS Support Manager
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Gemius S.A.
>
> ul. Wołoska 7, 02-672 Warszawa
>
> Budynek MARS, klatka D
>
> Tel.: +4822 874-41-00
>
> Fax : +4822 874-41-01
>
>
>
>
>
> *From:* Upendra Moturi [mailto:upe...@he...]
> *Sent:* Thursday, May 19, 2011 4:32 PM
> *To:* moo...@li...; Saurav Lahiri
> *Subject:* [Moosefs-users] Health Check
>
>
>
> Hi
>
>
> Is there a way to check the health of mfsmaster,mfsmetalogger and
> mfschunkserver ?
>
> --
> Thanks and Regards,
> Upendra.M
>



-- 
Thanks and Regards,
Upendra.M

Re: [Moosefs-users] mount disapeared

[Michal B. <mic...@ge...>]

Hi!

 

Is this connected to the fact that you ran out of space at /var partition?
It's not easy to tell the exact cause. It looks like some chunkservers where
not connected while saving.

 

 

Kind regards

Michal 

 

From: Papp Tamas [mailto:to...@ma...] 
Sent: Friday, May 20, 2011 11:16 AM
To: moo...@li...
Subject: [Moosefs-users] mount disapeared

 

hi! 

My server looks really weird. 

$ df |grep backup ;echo $? 
1 

$ df /data/backup 
Filesystem           1K-blocks      Used Available Use% Mounted on 
mfsmaster:9421               0         0         0   -  /data/backup 

$ mount |grep /data/backup 
mfsmaster:9421 on /data/backup type fuse.mfs
(rw,nosuid,nodev,allow_other,default_permissions) 


This is a drivish backup pool. There are success and unsuccess backups. 
I see rsync errors, like this: 

*** Execution cycle 0 *** 

rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: Broken
pipe (32) 
rsync: close failed on
"/data/backup/cluster/replicated/2011-05-20--04-00/tree/15_DY/40_2D/Admin/Ut
ilities/contact_sheet/.contact_sheet_render_CHK.nk.kQ7WgQ": No space left on
device (28) 
rsync error: error in file IO (code 11) at receiver.c(731) [receiver=3.0.7] 
rsync: connection unexpectedly closed (12188 bytes received so far) [sender]

rsync error: error in rsync protocol data stream (code 12) at io.c(601)
[sender=3.0.7] 

There are other errors too with no log file, dirvish job died somewehere
else... 


The chunk has free space ('cluster' has van chunk): 
/dev/sda6              10T  6.9T  3.1T  70% /mnt/mfschunk1 

Also now I see the the mount point: 
mfsmaster:9421         10T  6.9T  3.1T  70% /data/backup 


In the logs I don't see any special messages except it says, no metaloggers
connected, although metalogger is running. 

The system is Ubuntu Natty with lucid based packages from ppa. I still
didn't recompile them. No I will, but I'm not sure, does it cause this? 

Thank you, 

tamas 

Re: [Moosefs-users] out of space of metadata partition

[Michal B. <mic...@ge...>]

Hi

A new version of mfsmetarestore will be better in restoring metadata in such
situations. It will be available with next public release.


Kind regards
Michal


-----Original Message-----
From: Papp Tamas [mailto:to...@ma...] 
Sent: Saturday, May 21, 2011 11:40 PM
To: moo...@li...
Subject: [Moosefs-users] out of space of metadata partition


hi!

My server's /var partition run out of space.
Of course moosefs was not running correctly.
I killed the services and wanted to restore the metadata, but I was 
unsuccessful:

$ mfsmetarestore -a
loading objects (files,directories,etc.) ... ok
loading names ... ok
loading deletion timestamps ... ok
checking filesystem consistency ... ok
loading chunks data ... ok
connecting files and chunks ... ok
hole in change files (entries from 247260537 to 24247262063 are missing) 
- add more files


The metalogger server died with segfault, but after copying back the 
files I could recover the metadata successfully.
Now I'm happy, but this supposed to not happen if I'm right, am I?

Thank you,

tamas

----------------------------------------------------------------------------
--
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its 
next-generation tools to help Windows* and Linux* C/C++ and Fortran 
developers boost performance applications - including clusters. 
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] mfsmetarestore

[Papp T. <to...@ma...>]

On 05/23/2011 12:28 PM, Michal Borychowski wrote:
> Hi!
>
> It is really very strange that you need to 'kill -9' a chunkserver... And
> cgiserv is written in Python and killing it with '-9' means that the whole
> Python hung up (which is strange). Are you sure your hardware is for sure
> ok?
>

hi!

Yes, I'm absolutely sure.
There is no other sign of HW failure. Actually no it's working for 3 
days with now problem but of course I didn't try to stop mfs.

tamas

Re: [Moosefs-users] mfsmetarestore

[Michal B. <mic...@ge...>]

Hi!

It is really very strange that you need to 'kill -9' a chunkserver... And
cgiserv is written in Python and killing it with '-9' means that the whole
Python hung up (which is strange). Are you sure your hardware is for sure
ok?


Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01



-----Original Message-----
From: Papp Tamas [mailto:to...@ma...] 
Sent: Friday, May 20, 2011 4:33 PM
To: moo...@li...
Subject: [Moosefs-users] mfsmetarestore


hi!

I have some more problems.

Still Natty, and now with the recompiled ppa packages

1. the cgiservm and the chunkserver process cannot be stopped.
Only 'kill -9' helped.

2. After a hangup I'm trying to start the master server:

$ mfsmetarestore -a
loading objects (files,directories,etc.) ... loading node: read error: 
Unknown error
error
can't read metadata from file: /var/lib/mfs/metadata.mfs.back

$ mfsmetarestore -m metadata.mfs.back -o metadata.mfs changelog.*
loading objects (files,directories,etc.) ... loading node: read error: 
Unknown error
error
can't read metadata from file: metadata.mfs.back

Files are on this partition:
/dev/sda3 on /var type ext4 (rw,relatime)

Now I used the file metadata.mfs.back.tmp and it looks OK, but how can I 
be sure? Is there any way?

Thank you,

tamas


----------------------------------------------------------------------------
--
What Every C/C++ and Fortran developer Should Know!
Read this article and learn how Intel has extended the reach of its 
next-generation tools to help Windows* and Linux* C/C++ and Fortran 
developers boost performance applications - including clusters. 
http://p.sf.net/sfu/intel-dev2devmay
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Health Check

[Michal B. <mic...@ge...>]

Hi!

 

The CGI monitor gives most comprehensive information about the state of the
system.

 

 

Kind regards

Michał Borychowski 

MooseFS Support Manager

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gemius S.A.

ul. Wołoska 7, 02-672 Warszawa

Budynek MARS, klatka D

Tel.: +4822 874-41-00

Fax : +4822 874-41-01

 

 

From: Upendra Moturi [mailto:upe...@he...] 
Sent: Thursday, May 19, 2011 4:32 PM
To: moo...@li...; Saurav Lahiri
Subject: [Moosefs-users] Health Check

 

Hi

Is there a way to check the health of mfsmaster,mfsmetalogger and
mfschunkserver ?

-- 
Thanks and Regards,
Upendra.M

[Moosefs-users] out of space of metadata partition

[Papp T. <to...@ma...>]

hi!

My server's /var partition run out of space.
Of course moosefs was not running correctly.
I killed the services and wanted to restore the metadata, but I was 
unsuccessful:

$ mfsmetarestore -a
loading objects (files,directories,etc.) ... ok
loading names ... ok
loading deletion timestamps ... ok
checking filesystem consistency ... ok
loading chunks data ... ok
connecting files and chunks ... ok
hole in change files (entries from 247260537 to 24247262063 are missing) 
- add more files


The metalogger server died with segfault, but after copying back the 
files I could recover the metadata successfully.
Now I'm happy, but this supposed to not happen if I'm right, am I?

Thank you,

tamas

Re: [Moosefs-users] Two filesystems handled by one master?

[Thomas S H. <tha...@gm...>]

Sounds to me like you want 2 mfs masters. MooseFS is not great over slow
links, and if you have 2 masters then you have better distributed load
anyway. Your clients just need to specify what host they are connecting to
when they mount the FS

-Thomas S Hatch

[Moosefs-users] mfsmetarestore

[Papp T. <to...@ma...>]

hi!

I have some more problems.

Still Natty, and now with the recompiled ppa packages

1. the cgiservm and the chunkserver process cannot be stopped.
Only 'kill -9' helped.

2. After a hangup I'm trying to start the master server:

$ mfsmetarestore -a
loading objects (files,directories,etc.) ... loading node: read error: 
Unknown error
error
can't read metadata from file: /var/lib/mfs/metadata.mfs.back

$ mfsmetarestore -m metadata.mfs.back -o metadata.mfs changelog.*
loading objects (files,directories,etc.) ... loading node: read error: 
Unknown error
error
can't read metadata from file: metadata.mfs.back

Files are on this partition:
/dev/sda3 on /var type ext4 (rw,relatime)

Now I used the file metadata.mfs.back.tmp and it looks OK, but how can I 
be sure? Is there any way?

Thank you,

tamas

[Moosefs-users] mount disapeared

[Papp T. <to...@ma...>]

hi!

My server looks really weird.

$ df |grep backup ;echo $?
1

$ df /data/backup
Filesystem           1K-blocks      Used Available Use% Mounted on
mfsmaster:9421               0         0         0   -  /data/backup

$ mount |grep /data/backup
mfsmaster:9421 on /data/backup type fuse.mfs 
(rw,nosuid,nodev,allow_other,default_permissions)


This is a drivish backup pool. There are success and unsuccess backups.
I see rsync errors, like this:

*** Execution cycle 0 ***

rsync: writefd_unbuffered failed to write 4 bytes to socket [sender]: 
Broken pipe (32)
rsync: close failed on 
"/data/backup/cluster/replicated/2011-05-20--04-00/tree/15_DY/40_2D/Admin/Utilities/contact_sheet/.contact_sheet_render_CHK.nk.kQ7WgQ": 
No space left on device (28)
rsync error: error in file IO (code 11) at receiver.c(731) [receiver=3.0.7]
rsync: connection unexpectedly closed (12188 bytes received so far) 
[sender]
rsync error: error in rsync protocol data stream (code 12) at io.c(601) 
[sender=3.0.7]

There are other errors too with no log file, dirvish job died somewehere 
else...


The chunk has free space ('cluster' has van chunk):
/dev/sda6              10T  6.9T  3.1T  70% /mnt/mfschunk1

Also now I see the the mount point:
mfsmaster:9421         10T  6.9T  3.1T  70% /data/backup


In the logs I don't see any special messages except it says, no 
metaloggers connected, although metalogger is running.

The system is Ubuntu Natty with lucid based packages from ppa. I still 
didn't recompile them. No I will, but I'm not sure, does it cause this?

Thank you,

tamas

Re: [Moosefs-users] Two filesystems handled by one master?

[Christoph R. <c.r...@sc...>]

Am 18.05.2011 16:42, schrieb Thomas S Hatch:
> Hi,
>
> I have effectively 13 filesystems on one mfsmaster, but only one
> underlying "filesystem".
>
> The thing to do is to just have your clients mount subdirectories under
> the moosefs root directory, so the moosefs root directory looks like this:
>
> /stuff/<files>
> /morestuff/<files>
>
> and then on the client you mount like this:
>
> mfsmount <mountpoint> -S stuff # mounts the /stuff directory to the
> mountpoint instead of the whole moosefs mount
>
> I use this to partition out data for different environments and
> different types of data, so I have:
> /users
> /prod/media
> /qa/media
> etc.


Hi thx for your help.

But I'm looking for a solution with 2 FS... Because the 4 Server are 
located on two different serverrooms... and the bandwith between them is 
not very good (100 MBit).

And I don't want that Server 1+2 synchronize Data to Server 3+4... Thats 
why I want to have two differen Filesystems.


Someone any idea / information?

Regards

Christoph



>
> On Wed, May 18, 2011 at 2:02 AM, Christoph Raible
> <c.r...@sc... <mailto:c.r...@sc...>>
> wrote:
>
>     Hi
>
>     I have a question about configuration of the MooseFS-Filesystem. I got
>     the following infrastructure...
>
>     4 Chunkservers
>     1 Master
>     2 clients
>
>
>     Now I want 2 Filesystems (FS1 and FS2)
>     In FS1 is Server 1+2  and in FS2 is Server 3+4.
>
>     Both should be handled by the one Masterserver and the clients should
>     connect to both Filesystems...
>
>     Now I don't know if this is possible... or do I need one Master for a
>     Filesystem?
>
>
>     It would be great if someone can help me :)
>
>
>     Best regards,
>
>     Ch.Raible
>     --
>     Vorstand/Board of Management:
>     Dr. Bernd Finkbeiner, Dr. Roland Niemeier,
>     Dr. Arno Steitz, Dr. Ingrid Zech
>     Vorsitzender des Aufsichtsrats/
>     Chairman of the Supervisory Board:
>     Philippe Miltin
>     Sitz/Registered Office: Tuebingen
>     Registergericht/Registration Court: Stuttgart
>     Registernummer/Commercial Register No.: HRB 382196
>
>
>
>     ------------------------------------------------------------------------------
>     What Every C/C++ and Fortran developer Should Know!
>     Read this article and learn how Intel has extended the reach of its
>     next-generation tools to help Windows* and Linux* C/C++ and Fortran
>     developers boost performance applications - including clusters.
>     http://p.sf.net/sfu/intel-dev2devmay
>     _______________________________________________
>     moosefs-users mailing list
>     moo...@li...
>     <mailto:moo...@li...>
>     https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>

-- 
Vorstand/Board of Management:
Dr. Bernd Finkbeiner, Dr. Roland Niemeier, 
Dr. Arno Steitz, Dr. Ingrid Zech
Vorsitzender des Aufsichtsrats/
Chairman of the Supervisory Board:
Philippe Miltin
Sitz/Registered Office: Tuebingen
Registergericht/Registration Court: Stuttgart
Registernummer/Commercial Register No.: HRB 382196 

Re: [Moosefs-users] Question for using moosefs on post production

[Didi P. <di...@me...>]

Dear Thomas, Michal & Tamas,

Thanks for the response,

Actually it is in MB/s, because to edit 2k resolution (2048 x 1152) the
editor would need a throughput of around 350 MB/s, with edited file (dpx
file) size around 4TB (single file).

I would like to make a shared storage for 4 connected editing station. The
new info reduce the need of this shared storage size (for real time editing)
into 14 TB size, while I have to provide also 24-48 TB for temporary
archive.

I have to revise my calculation also in throughput, the total throughput
needed is around 900 MB/s because usually there are 2 client editing station
(autodesk lustre based on linux and davinci resolve based on mac)
simultaneously edit in this throughput (2 x 350 MB/s) while the other 200
MB/s I reserve for 2 other editing station connected to this shared storage.

If moosefs could not handle the function to be 'real time' editing storage,
then I guess I just able to use it for the temporary archive disk which not
needing so high throughput.

Best Regards,

Didi Pramujadi


2011/5/18 Thomas S Hatch <tha...@gm...>

> With MooseFS your biggest bottlenecks are chunkserver disk speed, and
> hitting the mfsmaster too often. like Michal said, if you have a lot of very
> small files you will place more load on the master. As for the disk speed, I
> use RAID 0 for my chunkservers in groups of 2-4 disks and we get 60-90MB
> throughput. Right now I have a ~500 TB MooseFS setup, so I think that a 24T
> setup should be no problem at all!
>
> -Thomas S Hatch
>
> 2011/5/18 Michal Borychowski <mic...@ge...>
>
>> Hi!
>>
>>
>>
>> I guess you mean 1200-1300 M*bits*/second? Which would make 150 M*bytes*
>> /second?
>>
>>
>>
>> Unfortunately I doubt if you can achieve such speeds with MooseFS. Unless
>> you use some SSD disks for chunkservers which would be very expensive... We
>> have write speeds of about 20-30 MiB/s and reads of 30-50MiB/s at our
>> environment with goal=2.
>>
>>
>>
>> On the other hand MooseFS would be perfect just for storing the content
>> (it is much better optimised for large files, not the small ones). For
>> editing purposes probably you should have separate machines and think of
>> good "flow" of the files.
>>
>>
>>
>>
>>
>> Kind regards
>>
>> Michał Borychowski
>>
>> MooseFS Support Manager
>>
>> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>>
>> Gemius S.A.
>>
>> ul. Wołoska 7, 02-672 Warszawa
>>
>> Budynek MARS, klatka D
>>
>> Tel.: +4822 874-41-00
>>
>> Fax : +4822 874-41-01
>>
>>
>>
>>
>>
>> *From:* Didi Pramujadi [mailto:di...@me...]
>> *Sent:* Wednesday, May 18, 2011 4:03 AM
>> *To:* moo...@li...
>> *Subject:* [Moosefs-users] Question for using moosefs on post production
>>
>>
>>
>> Dear Friends,
>>
>>
>> Hello everyone, I'm just joining this mailing list.
>>
>> I'm helping my boss to research the possibility of building a custom
>> shared storage for editing in his post production house. The requirement is
>> to provide a 24TB shared storage with troughput around 1200-1300 MB/s (for
>> simultaneous or concurrent editing of 2 station with 2k HD resolution + copy
>> file). I'm thinking of using iSCSI, Infiniband or FcOE for the transport.
>>
>> My questions is could moosefs utilized for this purpose ? Hows the details
>> hardware requirement for this?
>>
>> I'm sorry if this is too basic, thanks in advance.
>>
>> Best Regards,
>>
>> --
>> Didi Pramujadi
>>
>> Business Development
>> PT. Media Mozaic Indonesia
>>
>> www.mediamozaic.com
>> cellphone : +62811834579
>> Fax : +217408701
>> skype : didipramujadi
>>
>>
>>
>> ------------------------------------------------------------------------------
>> What Every C/C++ and Fortran developer Should Know!
>> Read this article and learn how Intel has extended the reach of its
>> next-generation tools to help Windows* and Linux* C/C++ and Fortran
>> developers boost performance applications - including clusters.
>> http://p.sf.net/sfu/intel-dev2devmay
>> _______________________________________________
>> moosefs-users mailing list
>>
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>>
>>
>


-- 
Didi Pramujadi

Business Development
PT. Media Mozaic Indonesia

www.mediamozaic.com
cellphone : +62811834579
Fax : +217408701
skype : didipramujadi