moosefs-users — Mailing list for MooseFS Users

[Moosefs-users] system high load when file has error

[Anh K. H. <anh...@gm...>]

Hi,

On the webUI, I can see the error

   currently unavailable chunk 0000000000139D7E
     (inode: 870852 ; index: 0)
   - currently unavailable file in trash 870852: foobar/bin/sam
     unavailable chunks: 1
     unavailable trash files: 1

As you can see, there is only one minor problem, but the cause is
weird / critical: The original file 'foobar/bin/sam' is inaccessible in
a loop-way: When my problem accesses the file, it waits forever, and I
have to press "^C" or to use "kill -9" to close the connection. This may
be possible, but if the program runs in background, system won't know
how to kill it, and system load will be very high . (In the mean time, I
delete the file so you see it reported in "trash", and get a new
version of file to use for my background tasks.)

My question is: Is there any way to force client to exit when they try
to access an error file in a MFS directory?

There is also a similar problem when the mfsmaster is down: All
mfsclients' system load will be (VERY) high. In an ideal condition, the
clients should be unmounted before the master is shut down, but for some
reasons, if the network / master has problem, the client should be
smart enough to disconnect from the master automatically.

Thank you very much.

Regards,

-- 
There seems no plan because it is all plan.
		-- C.S. Lewis

Re: [Moosefs-users] o_direct for chunkserver

[Michał B. <mic...@co...>]

Hi!

We already thought about O_DIRECT some time ago but even Linus Torvalds
advises against using it (citation below) and probably it would decrease the
chunkserver performance. 

On the other hand it would quite easy to implement O_DIRECT, but we would
need some strong arguments for doing this. Can you share some with us?


Citations:
On Thu, 11 Jan 2007, Aubrey wrote:
>
> Now, my question is, is there a existing way to mount a filesystem 
> with O_DIRECT flag? so that I don't need to change anything in my 
> system. If there is no option so far, What is the right way to achieve 
> my purpose?

The right way to do it is to just not use O_DIRECT. 

The whole notion of "direct IO" is totally braindamaged. Just say no.

	This is your brain: O
	This is your brain on O_DIRECT: .

	Any questions?

I should have fought back harder. There really is no valid reason for EVER
using O_DIRECT. You need a buffer whatever IO you do, and it might as well
be the page cache. There are better ways to control the page cache than play
games and think that a page cache isn't necessary.

So don't use O_DIRECT. Use things like madvise() and posix_fadvise()
instead. 

		Linus



From: Linus Torvalds [email blocked]
Subject: Re: O_DIRECT question
Date:	Wed, 10 Jan 2007 19:15:48 -0800 (PST)



On Wed, 10 Jan 2007, Linus Torvalds wrote:
> 
> So don't use O_DIRECT. Use things like madvise() and posix_fadvise() 
> instead.

Side note: the only reason O_DIRECT exists is because database people are
too used to it, because other OS's haven't had enough taste to tell them to
do it right, so they've historically hacked their OS to get out of the way.

As a result, our madvise and/or posix_fadvise interfaces may not be all that
strong, because people sadly don't use them that much. It's a sad example of
a totally broken interface (O_DIRECT) resulting in better interfaces not
getting used, and then not getting as much development effort put into them.

So O_DIRECT not only is a total disaster from a design standpoint (just look
at all the crap it results in), it also indirectly has hurt better
interfaces. For example, POSIX_FADV_NOREUSE (which _could_ be a useful and
clean interface to make sure we don't pollute memory unnecessarily with
cached pages after they are all done) ends up being a no-op ;/

Sad. And it's one of those self-fulfilling prophecies. Still, I hope some
day we can just rip the damn disaster out.

			Linus



Kind regards
Michał Borychowski 
MooseFS Support Manager


-----Original Message-----
From: Corin Langosch [mailto:in...@co...] 
Sent: Monday, March 26, 2012 5:23 PM
To: moo...@li...
Subject: [Moosefs-users] o_direct for chunkserver

Hi,

is it already possible to make chunkservers (latest version of moosefs) use
o_direct for all file operations? I couldn't find any configuration option
for this in the man pages.

Otherwise I'd like to suggest to support it in a future version :)

Corin


----------------------------------------------------------------------------
--
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] fs_writechunk returns status 11

[Michał B. <mic...@co...>]

Hi!

Error 11 means "chunk locked". It appears when several processes at
different computers try to write to the same file in parallel. You should
not be bothered by this message but it's wise to minimize occurrences of
this situation.


Kind regards
Michał Borychowski 
MooseFS Support Manager


-----Original Message-----
From: Brent A Nelson [mailto:br...@ph...] 
Sent: Tuesday, March 27, 2012 7:54 PM
To: moo...@li...
Subject: [Moosefs-users] fs_writechunk returns status 11

I've been using MooseFS 1.6.20 on 64-bit Ubuntu 10.04 for a year or so, and
it's been working rather well.  However, I get a certain type of error that
will fill my logging space from time-to-time:

Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 -
fs_writechunk returns status 11 Mar 23 07:02:01 somehost mfsmount[31484]:
file: 7214721, index: 0 - fs_writechunk returns status 11 Mar 23 07:02:01
somehost mfsmount[31484]: file: 7214723, index: 0 - fs_writechunk returns
status 11 Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214717, index: 0
- fs_writechunk returns status 11 Mar 23 07:02:01 somehost mfsmount[31484]:
file: 7214722, index: 0 - fs_writechunk returns status 11

This will repeat in long bursts, and then be quiet for awhile (hours or
days).  The machine on which this occurs is one where MooseFS is mounted for
my home directory (Linux/Gnome environment).  I have not noticed any
corresponding errors in the server logs.  The files referenced above are all
Google Chrome cache-related.  Perhaps all such errors have been from Google
Chrome files; most or all that I've looked at previously have been.

I'd like to move other users' home directories to MooseFS.  It's been doing
fine for non-home directories, but this error filling the logs (but not
having any other obvious impact) looks to be a small obstacle for home
directories.  There is also the Openoffice bug which cause it to break with
MooseFS, but I can get all the rest of my machines over to LibreOffice
without much trouble. We want to be on LibreOffice, anyway.

Does anyone know what causes the "fs_writechunk returns status 11" 
complaints? Is it already fixed in the new release? Is it specific to Google
Chrome (and perhaps its fault), or is it just that Google Chrome is active
more than anything else, at all times, and is therefore more prone to
glitches?

Thanks,

Brent Nelson
Director of Computing
Dept. of Physics
University of Florida

----------------------------------------------------------------------------
--
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] fox and bird

[Steve T. <sm...@cb...>]

On Thu, 29 Mar 2012, Brent A Nelson wrote:

> Have you tried just disabling the disk cache? It might eliminate the 
> slowness.  Web caches tend to consist of very large numbers of mostly small 
> files; that's not the most efficient situation, especially for typical 
> network filesystems...

Yes, turning off any caching does not really help very much. It appears to 
be the bookmarks cache (places.sqlite) that is a major source of 
bottleneck. At least for now I have the users off my back :)

Steve

Re: [Moosefs-users] moosefs-users Digest, Vol 27, Issue 24

[yishi c. <hol...@gm...>]

Re:fs_writechunk returns status 11 (Brent A Nelson)
     "fs_writechunk returns status 11"means the file you wanna write to is
locked(probably this file is written by the other mfsmount，so the mfs will
lock the file when there is mfsmount writting the exactly same file).you
can confirm these information by reviewing the
code(mfscommon/MFSCOMMUNICATION.h,mfsmaster/filesystem.c,mfsmaster/chunk.c,actually，the
11 means ERROR_LOCKED which indicate the chunk is written by other
mfsmount)， since I have no idea of the google chrome cache-file，no more
suggestion I can give to you.maybe the google chrome cache-file is
frequently writed.
      It is not commanded to store small but in large number file in
moosefs,the hash table to store the fsnode(inode) is not big enough,and the
network connection model of moosefs could be improved as same.


2012/3/29 <moo...@li...>

> Send moosefs-users mailing list submissions to
>        moo...@li...
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        https://lists.sourceforge.net/lists/listinfo/moosefs-users
> or, via email, send a message with subject or body 'help' to
>        moo...@li...
>
> You can reach the person managing the list at
>        moo...@li...
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of moosefs-users digest..."
>
>
> Today's Topics:
>
>   1. fs_writechunk returns status 11 (Brent A Nelson)
>   2. is rack awareness documented? (mARK bLOORE)
>   3. fox and bird (Steve Thompson)
>   4. Re: fox and bird (Steve Wilson)
>   5. Re: fs_writechunk returns status 11 (Steve Thompson)
>   6. Re: fox and bird (Brent A Nelson)
>   7. Re: fox and bird (Dr. Michael J. Chudobiak)
>   8. Re: fox and bird (Steve Wilson)
>   9. Re: fs_writechunk returns status 11 (Brent A Nelson)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 27 Mar 2012 13:53:39 -0400 (EDT)
> From: Brent A Nelson <br...@ph...>
> Subject: [Moosefs-users] fs_writechunk returns status 11
> To: moo...@li...
> Message-ID: <alp...@ca...>
> Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
>
> I've been using MooseFS 1.6.20 on 64-bit Ubuntu 10.04 for a year or so,
> and it's been working rather well.  However, I get a certain type of error
> that will fill my logging space from time-to-time:
>
> Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 -
> fs_writechunk returns status 11
> Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214721, index: 0 -
> fs_writechunk returns status 11
> Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214723, index: 0 -
> fs_writechunk returns status 11
> Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214717, index: 0 -
> fs_writechunk returns status 11
> Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214722, index: 0 -
> fs_writechunk returns status 11
>
> This will repeat in long bursts, and then be quiet for awhile (hours or
> days).  The machine on which this occurs is one where MooseFS is mounted
> for my home directory (Linux/Gnome environment).  I have not noticed any
> corresponding errors in the server logs.  The files referenced above are
> all Google Chrome cache-related.  Perhaps all such errors have been from
> Google Chrome files; most or all that I've looked at previously have been.
>
> I'd like to move other users' home directories to MooseFS.  It's been
> doing fine for non-home directories, but this error filling the logs (but
> not having any other obvious impact) looks to be a small obstacle for home
> directories.  There is also the Openoffice bug which cause it to break
> with MooseFS, but I can get all the rest of my machines over to
> LibreOffice without much trouble. We want to be on LibreOffice, anyway.
>
> Does anyone know what causes the "fs_writechunk returns status 11"
> complaints? Is it already fixed in the new release? Is it specific to
> Google Chrome (and perhaps its fault), or is it just that Google Chrome
> is active more than anything else, at all times, and is therefore more
> prone to glitches?
>
> Thanks,
>
> Brent Nelson
> Director of Computing
> Dept. of Physics
> University of Florida
>
>
>
> ------------------------------
>
> Message: 2
> Date: Tue, 27 Mar 2012 17:58:30 -0400
> From: mARK bLOORE <mb...@id...>
> Subject: [Moosefs-users] is rack awareness documented?
> To: moo...@li...
> Message-ID: <4F7...@id...>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> the file mfstopology.cfg describes how to set up server groups, but says
> nothing about how the
> groups are used.  is there any documentation on that?  all i have been
> able to find is the mention
> of that config file's existence in the release notes.
>
> --
> mARK bLOORE <mb...@id...>
> idee inc.
>
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 28 Mar 2012 15:33:37 -0400 (EDT)
> From: Steve Thompson <sm...@cb...>
> Subject: [Moosefs-users] fox and bird
> To: moo...@li...
> Message-ID:
>        <alp...@as...>
> Content-Type: TEXT/PLAIN; format=flowed; charset=US-ASCII
>
> MooseFS 1.6.20, Linux (Centos 5.7).
>
> I have discovered that neither firefox nor thunderbird (V10) will run
> correctly when launched from a MooseFS-based home directory; they just
> hang indefinitely. Does anyone else have this experience, or have I
> screwed up somewhere? Thanks,
>
> Steve
> --
>
> ----------------------------------------------------------------------------
> Steve Thompson, Cornell School of Chemical and Biomolecular Engineering
> smt AT cbe DOT cornell DOT edu
>   "186,282 miles per second: it's not just a good idea, it's the law"
>
> ----------------------------------------------------------------------------
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 28 Mar 2012 15:46:57 -0400
> From: Steve Wilson <st...@pu...>
> Subject: Re: [Moosefs-users] fox and bird
> To: moo...@li...
> Message-ID: <4F7...@pu...>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 03/28/2012 03:33 PM, Steve Thompson wrote:
> > MooseFS 1.6.20, Linux (Centos 5.7).
> >
> > I have discovered that neither firefox nor thunderbird (V10) will run
> > correctly when launched from a MooseFS-based home directory; they just
> > hang indefinitely. Does anyone else have this experience, or have I
> > screwed up somewhere? Thanks,
> >
> > Steve
>
> We're using MooseFS 1.6.20 for home directories in an Ubuntu
> environment.  We do get the occasional "fs_writechunk returns status 11"
> error noted earlier by Brent Nelson on this list but no problems like
> what you're seeing.  I know we have users running Firefox but I'm not
> sure if anyone is using Thunderbird.  We've been extremely pleased with
> MooseFS in the 12 months or so that we've been using it.
>
> Steve W.
>
>
>
> ------------------------------
>
> Message: 5
> Date: Wed, 28 Mar 2012 15:57:39 -0400 (EDT)
> From: Steve Thompson <sm...@cb...>
> Subject: Re: [Moosefs-users] fs_writechunk returns status 11
> To: Brent A Nelson <br...@ph...>
> Cc: moo...@li...
> Message-ID:
>        <alp...@as...>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
> On Tue, 27 Mar 2012, Brent A Nelson wrote:
>
> > Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 -
> > fs_writechunk returns status 11
>
> I've seen exactly one occurrence of this in the two months that I have
> been using MooseFS (CentOS 5.7, mfs 1.6.20). We don't use Google Chrome,
> so it wasn't that in my case. I don't know the cause.
>
> Steve
>
>
>
> ------------------------------
>
> Message: 6
> Date: Wed, 28 Mar 2012 16:00:00 -0400 (EDT)
> From: Brent A Nelson <br...@ph...>
> Subject: Re: [Moosefs-users] fox and bird
> To: Steve Wilson <st...@pu...>
> Cc: moo...@li...
> Message-ID: <alp...@ca...>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
> I, too, have encountered no issues with firefox under Ubuntu (8.04 or
> 10.04).  I haven't tried Thunderbird.  OpenOffice has an issue with the
> same effect as what you describe, but it's a known OpenOffice bug, and
> it's fixed in LibreOffice.
>
> Incidentally, for the "fs_writechunk returns status 11" messages, I've
> told google-chrome to disable its cache (-disk-cache-dir="/dev/null"),
> which I suspect will do the trick.  Chrome also now seems much faster...
>
> On Wed, 28 Mar 2012, Steve Wilson wrote:
>
> > On 03/28/2012 03:33 PM, Steve Thompson wrote:
> >> MooseFS 1.6.20, Linux (Centos 5.7).
> >>
> >> I have discovered that neither firefox nor thunderbird (V10) will run
> >> correctly when launched from a MooseFS-based home directory; they just
> >> hang indefinitely. Does anyone else have this experience, or have I
> >> screwed up somewhere? Thanks,
> >>
> >> Steve
> >
> > We're using MooseFS 1.6.20 for home directories in an Ubuntu
> > environment.  We do get the occasional "fs_writechunk returns status 11"
> > error noted earlier by Brent Nelson on this list but no problems like
> > what you're seeing.  I know we have users running Firefox but I'm not
> > sure if anyone is using Thunderbird.  We've been extremely pleased with
> > MooseFS in the 12 months or so that we've been using it.
> >
> > Steve W.
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > moosefs-users mailing list
> > moo...@li...
> > https://lists.sourceforge.net/lists/listinfo/moosefs-users
> >
>
>
>
> ------------------------------
>
> Message: 7
> Date: Wed, 28 Mar 2012 15:44:20 -0400
> From: "Dr. Michael J. Chudobiak" <mj...@av...>
> Subject: Re: [Moosefs-users] fox and bird
> To: moo...@li...
> Message-ID: <4F7...@av...>
> Content-Type: text/plain; charset=UTF-8; format=flowed
>
> On 03/28/2012 03:33 PM, Steve Thompson wrote:
> > MooseFS 1.6.20, Linux (Centos 5.7).
> >
> > I have discovered that neither firefox nor thunderbird (V10) will run
> > correctly when launched from a MooseFS-based home directory; they just
> > hang indefinitely. Does anyone else have this experience, or have I
> > screwed up somewhere? Thanks,
> >
> > Steve
>
> I'm running 1.6.20 on Fedora 16 servers, with several Fedora 16 clients
> (who have their home folders on the moosefs mount).
>
> Firefox and Thunderbird work just fine. No issues at all.
>
> LibreOffice was painfully slow, until I converted the chunkserver hard
> drives to SSDs. I think LibreOffice was fsyncing way too frequently
> (just a theory). Speedy Intel SSDs made that problem go away.
>
> - Mike
>
>
>
> ------------------------------
>
> Message: 8
> Date: Wed, 28 Mar 2012 16:04:26 -0400
> From: Steve Wilson <st...@pu...>
> Subject: Re: [Moosefs-users] fox and bird
> To: Brent A Nelson <br...@ph...>
> Cc: moo...@li...
> Message-ID: <4F7...@pu...>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> On 03/28/2012 04:00 PM, Brent A Nelson wrote:
> > I, too, have encountered no issues with firefox under Ubuntu (8.04 or
> > 10.04).  I haven't tried Thunderbird.  OpenOffice has an issue with
> > the same effect as what you describe, but it's a known OpenOffice bug,
> > and it's fixed in LibreOffice.
> >
> > Incidentally, for the "fs_writechunk returns status 11" messages, I've
> > told google-chrome to disable its cache (-disk-cache-dir="/dev/null"),
> > which I suspect will do the trick.  Chrome also now seems much faster...
> >
>
> We see the status 11 errors whenever multiple workstations attempt to
> simultaneously write to the same file in a MooseFS volume.
>
> Steve
>
>
> > On Wed, 28 Mar 2012, Steve Wilson wrote:
> >
> >> On 03/28/2012 03:33 PM, Steve Thompson wrote:
> >>> MooseFS 1.6.20, Linux (Centos 5.7).
> >>>
> >>> I have discovered that neither firefox nor thunderbird (V10) will run
> >>> correctly when launched from a MooseFS-based home directory; they just
> >>> hang indefinitely. Does anyone else have this experience, or have I
> >>> screwed up somewhere? Thanks,
> >>>
> >>> Steve
> >>
> >> We're using MooseFS 1.6.20 for home directories in an Ubuntu
> >> environment.  We do get the occasional "fs_writechunk returns status 11"
> >> error noted earlier by Brent Nelson on this list but no problems like
> >> what you're seeing.  I know we have users running Firefox but I'm not
> >> sure if anyone is using Thunderbird.  We've been extremely pleased with
> >> MooseFS in the 12 months or so that we've been using it.
> >>
> >> Steve W.
> >>
> >>
> ------------------------------------------------------------------------------
> >>
> >> This SF email is sponsosred by:
> >> Try Windows Azure free for 90 days Click Here
> >> http://p.sf.net/sfu/sfd2d-msazure
> >> _______________________________________________
> >> moosefs-users mailing list
> >> moo...@li...
> >> https://lists.sourceforge.net/lists/listinfo/moosefs-users
> >>
>
> --
> Steven M. Wilson, Systems and Network Manager
> Markey Center for Structural Biology
> Purdue University
> (765) 496-1946
>
>
>
>
> ------------------------------
>
> Message: 9
> Date: Wed, 28 Mar 2012 16:46:31 -0400 (EDT)
> From: Brent A Nelson <br...@ph...>
> Subject: Re: [Moosefs-users] fs_writechunk returns status 11
> To: Steve Thompson <sm...@cb...>
> Cc: moo...@li...
> Message-ID: <alp...@ca...>
> Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed
>
> By some near miraculous coincidence, my subject linked up with a thread
> from a year ago, which explained that the message is due to an attempt to
> write to a chunk that is already locked for writing by something else.
>
> I'm guessing that Google Chrome, with its highly threaded nature, kind of
> trips over itself when it comes to the cache, sometimes generating huge
> bursts of warnings from MooseFS (it really can quickly fill /var).
> Otherwise, you'd probably run into this when running two programs that try
> to write to the same file, which should be rare and not really an issue.
>
> It's a pity that Chrome doesn't have some file to edit so that you can set
> system-wide defaults, but the '--disk-cache-dir="/dev/null"' command-line
> option does look promising (and, as I mentioned in the Firefox thread, it
> seems to give Chrome a substantial speed boost when running on MooseFS).
>
> On Wed, 28 Mar 2012, Steve Thompson wrote:
>
> > On Tue, 27 Mar 2012, Brent A Nelson wrote:
> >
> >> Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 -
> >> fs_writechunk returns status 11
> >
> > I've seen exactly one occurrence of this in the two months that I have
> been
> > using MooseFS (CentOS 5.7, mfs 1.6.20). We don't use Google Chrome, so it
> > wasn't that in my case. I don't know the cause.
> >
> > Steve
> >
>
>
>
> ------------------------------
>
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
>
> ------------------------------
>
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>
>
> End of moosefs-users Digest, Vol 27, Issue 24
> *********************************************
>

Re: [Moosefs-users] fox and bird

[Brent A N. <br...@ph...>]

Have you tried just disabling the disk cache? It might eliminate the 
slowness.  Web caches tend to consist of very large numbers of mostly 
small files; that's not the most efficient situation, especially for 
typical network filesystems...

On Thu, 29 Mar 2012, Steve Thompson wrote:

> On Wed, 28 Mar 2012, Dr. Michael J. Chudobiak wrote:
>
>> I'm running 1.6.20 on Fedora 16 servers, with several Fedora 16 clients
>> (who have their home folders on the moosefs mount).
>>
>> Firefox and Thunderbird work just fine. No issues at all.
>
> I have further found that Thunderbird works well, but Firefox is so
> painfully slow (glacial) as to be unusable. For the time being, I have had
> to relocate the .mozilla directories to a non-MFS file system and replace
> them by symbolic links.
>
> Steve
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>

Re: [Moosefs-users] fox and bird

[Steve T. <sm...@cb...>]

On Wed, 28 Mar 2012, Dr. Michael J. Chudobiak wrote:

> I'm running 1.6.20 on Fedora 16 servers, with several Fedora 16 clients
> (who have their home folders on the moosefs mount).
>
> Firefox and Thunderbird work just fine. No issues at all.

I have further found that Thunderbird works well, but Firefox is so 
painfully slow (glacial) as to be unusable. For the time being, I have had 
to relocate the .mozilla directories to a non-MFS file system and replace 
them by symbolic links.

Steve

Re: [Moosefs-users] fs_writechunk returns status 11

[Giovanni T. <me...@gi...>]

2012/3/28 Brent A Nelson <br...@ph...>:
> It's a pity that Chrome doesn't have some file to edit so that you can set
> system-wide defaults, but the '--disk-cache-dir="/dev/null"' command-line
> option does look promising (and, as I mentioned in the Firefox thread, it
> seems to give Chrome a substantial speed boost when running on MooseFS).

On Debian/Ubuntu there is:

$ cat /etc/chromium/default
# Default settings for chromium. This file is sourced by /bin/sh from
# /usr/bin/chromium

# Options to pass to chromium
CHROMIUM_FLAGS="--password-store=detect
--disk-cache-dir=/dev/shm/$USER-chromium"

-- 
Giovanni Toraldo
http://gionn.net/about-me
http://it.linkedin.com/in/giovannitoraldo

Re: [Moosefs-users] fs_writechunk returns status 11

[Brent A N. <br...@ph...>]

By some near miraculous coincidence, my subject linked up with a thread 
from a year ago, which explained that the message is due to an attempt to 
write to a chunk that is already locked for writing by something else.

I'm guessing that Google Chrome, with its highly threaded nature, kind of 
trips over itself when it comes to the cache, sometimes generating huge 
bursts of warnings from MooseFS (it really can quickly fill /var). 
Otherwise, you'd probably run into this when running two programs that try 
to write to the same file, which should be rare and not really an issue.

It's a pity that Chrome doesn't have some file to edit so that you can set 
system-wide defaults, but the '--disk-cache-dir="/dev/null"' command-line 
option does look promising (and, as I mentioned in the Firefox thread, it 
seems to give Chrome a substantial speed boost when running on MooseFS).

On Wed, 28 Mar 2012, Steve Thompson wrote:

> On Tue, 27 Mar 2012, Brent A Nelson wrote:
>
>> Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 -
>> fs_writechunk returns status 11
>
> I've seen exactly one occurrence of this in the two months that I have been 
> using MooseFS (CentOS 5.7, mfs 1.6.20). We don't use Google Chrome, so it 
> wasn't that in my case. I don't know the cause.
>
> Steve
>

Re: [Moosefs-users] fox and bird

[Steve W. <st...@pu...>]

On 03/28/2012 04:00 PM, Brent A Nelson wrote:
> I, too, have encountered no issues with firefox under Ubuntu (8.04 or 
> 10.04).  I haven't tried Thunderbird.  OpenOffice has an issue with 
> the same effect as what you describe, but it's a known OpenOffice bug, 
> and it's fixed in LibreOffice.
>
> Incidentally, for the "fs_writechunk returns status 11" messages, I've 
> told google-chrome to disable its cache (-disk-cache-dir="/dev/null"), 
> which I suspect will do the trick.  Chrome also now seems much faster...
>

We see the status 11 errors whenever multiple workstations attempt to 
simultaneously write to the same file in a MooseFS volume.

Steve


> On Wed, 28 Mar 2012, Steve Wilson wrote:
>
>> On 03/28/2012 03:33 PM, Steve Thompson wrote:
>>> MooseFS 1.6.20, Linux (Centos 5.7).
>>>
>>> I have discovered that neither firefox nor thunderbird (V10) will run
>>> correctly when launched from a MooseFS-based home directory; they just
>>> hang indefinitely. Does anyone else have this experience, or have I
>>> screwed up somewhere? Thanks,
>>>
>>> Steve
>>
>> We're using MooseFS 1.6.20 for home directories in an Ubuntu
>> environment.  We do get the occasional "fs_writechunk returns status 11"
>> error noted earlier by Brent Nelson on this list but no problems like
>> what you're seeing.  I know we have users running Firefox but I'm not
>> sure if anyone is using Thunderbird.  We've been extremely pleased with
>> MooseFS in the 12 months or so that we've been using it.
>>
>> Steve W.
>>
>> ------------------------------------------------------------------------------ 
>>
>> This SF email is sponsosred by:
>> Try Windows Azure free for 90 days Click Here
>> http://p.sf.net/sfu/sfd2d-msazure
>> _______________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>>

-- 
Steven M. Wilson, Systems and Network Manager
Markey Center for Structural Biology
Purdue University
(765) 496-1946

Re: [Moosefs-users] fox and bird

[Dr. M. J. C. <mj...@av...>]

On 03/28/2012 03:33 PM, Steve Thompson wrote:
> MooseFS 1.6.20, Linux (Centos 5.7).
>
> I have discovered that neither firefox nor thunderbird (V10) will run
> correctly when launched from a MooseFS-based home directory; they just
> hang indefinitely. Does anyone else have this experience, or have I
> screwed up somewhere? Thanks,
>
> Steve

I'm running 1.6.20 on Fedora 16 servers, with several Fedora 16 clients 
(who have their home folders on the moosefs mount).

Firefox and Thunderbird work just fine. No issues at all.

LibreOffice was painfully slow, until I converted the chunkserver hard 
drives to SSDs. I think LibreOffice was fsyncing way too frequently 
(just a theory). Speedy Intel SSDs made that problem go away.

- Mike

Re: [Moosefs-users] fox and bird

[Brent A N. <br...@ph...>]

I, too, have encountered no issues with firefox under Ubuntu (8.04 or 
10.04).  I haven't tried Thunderbird.  OpenOffice has an issue with the 
same effect as what you describe, but it's a known OpenOffice bug, and 
it's fixed in LibreOffice.

Incidentally, for the "fs_writechunk returns status 11" messages, I've 
told google-chrome to disable its cache (-disk-cache-dir="/dev/null"), 
which I suspect will do the trick.  Chrome also now seems much faster...

On Wed, 28 Mar 2012, Steve Wilson wrote:

> On 03/28/2012 03:33 PM, Steve Thompson wrote:
>> MooseFS 1.6.20, Linux (Centos 5.7).
>>
>> I have discovered that neither firefox nor thunderbird (V10) will run
>> correctly when launched from a MooseFS-based home directory; they just
>> hang indefinitely. Does anyone else have this experience, or have I
>> screwed up somewhere? Thanks,
>>
>> Steve
>
> We're using MooseFS 1.6.20 for home directories in an Ubuntu
> environment.  We do get the occasional "fs_writechunk returns status 11"
> error noted earlier by Brent Nelson on this list but no problems like
> what you're seeing.  I know we have users running Firefox but I'm not
> sure if anyone is using Thunderbird.  We've been extremely pleased with
> MooseFS in the 12 months or so that we've been using it.
>
> Steve W.
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>

Re: [Moosefs-users] fs_writechunk returns status 11

[Steve T. <sm...@cb...>]

On Tue, 27 Mar 2012, Brent A Nelson wrote:

> Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 -
> fs_writechunk returns status 11

I've seen exactly one occurrence of this in the two months that I have 
been using MooseFS (CentOS 5.7, mfs 1.6.20). We don't use Google Chrome, 
so it wasn't that in my case. I don't know the cause.

Steve

Re: [Moosefs-users] fox and bird

[Steve W. <st...@pu...>]

On 03/28/2012 03:33 PM, Steve Thompson wrote:
> MooseFS 1.6.20, Linux (Centos 5.7).
>
> I have discovered that neither firefox nor thunderbird (V10) will run
> correctly when launched from a MooseFS-based home directory; they just
> hang indefinitely. Does anyone else have this experience, or have I
> screwed up somewhere? Thanks,
>
> Steve

We're using MooseFS 1.6.20 for home directories in an Ubuntu 
environment.  We do get the occasional "fs_writechunk returns status 11" 
error noted earlier by Brent Nelson on this list but no problems like 
what you're seeing.  I know we have users running Firefox but I'm not 
sure if anyone is using Thunderbird.  We've been extremely pleased with 
MooseFS in the 12 months or so that we've been using it.

Steve W.

[Moosefs-users] fox and bird

[Steve T. <sm...@cb...>]

MooseFS 1.6.20, Linux (Centos 5.7).

I have discovered that neither firefox nor thunderbird (V10) will run 
correctly when launched from a MooseFS-based home directory; they just 
hang indefinitely. Does anyone else have this experience, or have I 
screwed up somewhere? Thanks,

Steve
-- 
----------------------------------------------------------------------------
Steve Thompson, Cornell School of Chemical and Biomolecular Engineering
smt AT cbe DOT cornell DOT edu
   "186,282 miles per second: it's not just a good idea, it's the law"
----------------------------------------------------------------------------

[Moosefs-users] is rack awareness documented?

[mARK b. <mb...@id...>]

the file mfstopology.cfg describes how to set up server groups, but says nothing about how the 
groups are used.  is there any documentation on that?  all i have been able to find is the mention 
of that config file's existence in the release notes.

-- 
mARK bLOORE <mb...@id...>
idee inc.

[Moosefs-users] fs_writechunk returns status 11

[Brent A N. <br...@ph...>]

I've been using MooseFS 1.6.20 on 64-bit Ubuntu 10.04 for a year or so, 
and it's been working rather well.  However, I get a certain type of error 
that will fill my logging space from time-to-time:

Mar 23 07:02:01 somehost mfsmount[31484]: file: 4282810, index: 0 - 
fs_writechunk returns status 11
Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214721, index: 0 - 
fs_writechunk returns status 11
Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214723, index: 0 - 
fs_writechunk returns status 11
Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214717, index: 0 - 
fs_writechunk returns status 11
Mar 23 07:02:01 somehost mfsmount[31484]: file: 7214722, index: 0 - 
fs_writechunk returns status 11

This will repeat in long bursts, and then be quiet for awhile (hours or 
days).  The machine on which this occurs is one where MooseFS is mounted 
for my home directory (Linux/Gnome environment).  I have not noticed any 
corresponding errors in the server logs.  The files referenced above are 
all Google Chrome cache-related.  Perhaps all such errors have been from 
Google Chrome files; most or all that I've looked at previously have been.

I'd like to move other users' home directories to MooseFS.  It's been 
doing fine for non-home directories, but this error filling the logs (but 
not having any other obvious impact) looks to be a small obstacle for home 
directories.  There is also the Openoffice bug which cause it to break 
with MooseFS, but I can get all the rest of my machines over to 
LibreOffice without much trouble. We want to be on LibreOffice, anyway.

Does anyone know what causes the "fs_writechunk returns status 11" 
complaints? Is it already fixed in the new release? Is it specific to 
Google Chrome (and perhaps its fault), or is it just that Google Chrome 
is active more than anything else, at all times, and is therefore more 
prone to glitches?

Thanks,

Brent Nelson
Director of Computing
Dept. of Physics
University of Florida

Re: [Moosefs-users] Balancing of space with differently sized chunk servers

[Steve T. <sm...@cb...>]

On Tue, 27 Mar 2012, Jeff Williams wrote:

> I am happy to be pointed to some documentation, but I was wondering how 
> moosefs balances files between chunk servers with different available 
> space. Our case in point is we have dell servers with MD1200 disk 
> shelves attached. Some are full of 2TB disks, some with 3TB, so some 
> chuck servers will have extra 50% storage.

I haven't seen it documented, but the steady-state behavior in my setup is 
that each chunkserver is populated to the same fill percentage. So it 
seems to me that if you have only two chunkservers, one with n*2 TB and 
one with n*3 TB, and are using a goal of 2, you will never be able to use 
all the space and maintain the goal setting. In this situation, you will 
have to use more than two chunkservers. I have two chunkservers with 8 TB, 
two with 25 TB, and two with 10 TB (second one of these just being built).

Steve

[Moosefs-users] Balancing of space with differently sized chunk servers

[Jeff W. <je...@wh...>]

Hi,

I am happy to be pointed to some documentation, but I was wondering how moosefs balances files between chunk servers with different available space. Our case in point is we have dell servers with MD1200 disk shelves attached. Some are full of 2TB disks, some with 3TB, so some chuck servers will have extra 50% storage.

Regards,
Jeff

[Moosefs-users] o_direct for chunkserver

[Corin L. <in...@co...>]

Hi,

is it already possible to make chunkservers (latest version of moosefs) 
use o_direct for all file operations? I couldn't find any configuration 
option for this in the man pages.

Otherwise I'd like to suggest to support it in a future version :)

Corin

Re: [Moosefs-users] Download link new version

[Allen, B. S <bs...@la...>]

Hi Michal,

I think we all understand the need to collect customer information, and asking for that information before downloading is fine. However, adding a checkbox that says you can use my company's trademark and name is outside of the industry norm. If you'd remove this part of the form it would satisfy my (and I'm guessing others) issue with the change.

Thanks,

Ben

On Mar 24, 2012, at 2:51 AM, Michał Borychowski wrote:

> Hi!
> 
> MooseFS is GPLv3 and everyone can copy it. If you don't want to fill the
> form - all of you can simply use wget for downloading MooseFS (form is
> triggered as javascript onclick event only). 
> 
> The form is for those who download files by clicking on our webpage.
> 
> We will make all old releases (older than 1 month) available for downloading
> without filling the form. Also we will publish the code on sf.net and github
> of each next version of MooseFS with 1 month of delay.
> 
> We understand that some of you may think that this new form which you have
> to fill in when downloading the new version of MooseFS is quite disturbing
> :) We have to say that we really need these information to push MooseFS
> project forward and we hope for your understanding. 
> 
> Please share with us how can we do it better (have users/customers list of
> MooseFS and give easy download in the same time)?
> 
> Many thanks for your feedback.
> 
> 
> Kind regards
> Michal  
> 
> 
> 
> -----Original Message-----
> From: Atom Powers [mailto:ap...@di...] 
> Sent: Wednesday, March 21, 2012 7:10 PM
> To: moo...@li...
> Subject: Re: [Moosefs-users] Download link new version
> 
> On 03/21/2012 08:23 AM, Allen, Benjamin S wrote:
>> To the MooseFS folks, I work in an industry where I cannot agree to:
>> 
>> "To publish the name and trademark of the company, on behalf of which 
>> I act, on the MooseFS website or Core Technology website and to use 
>> the aforementioned name and trademark in order to present them to Core 
>> Technology's clients for marketing purposes only."
> 
> Same here. All contracts that come across my desk must have this kind of
> clause removed before I can sign them. As it stands, this clause will
> prevent me from upgrading.
> 
>> Also I would suggest you remove having to forcefully enter contact 
>> info to download. Typically this doesn't jive well with OSS folks. 
>> Make it voluntary.
> 
> I am less decisive on this point. If MooseFS was moving to a "freemium" 
> model I would understand but as an OSS project it doesn't make sense.
> 
> --
> --
> Perfection is just a word I use occasionally with mustard.
> --Atom Powers--
> Director of IT
> DigiPen Institute of Technology
> +1 (425) 895-4443
> 
> ----------------------------------------------------------------------------
> --
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here 
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
> 
> 
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here 
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] mfscgiserv 403 Forbidden

[陈钢 <yik...@gm...>]

Works~
Thank you.

2012/3/26 Ken <ken...@gm...>

> maybe:
>  chmod +x mfs.cgi
>
> -Ken
>
>
>
> On Mon, Mar 26, 2012 at 10:29 AM, 陈钢 <yik...@gm...> wrote:
> > After I restart mfscgiserv with cmd "python2.7 /usr/sbin/mfscgiserv -P
> 80".
> >
> > I got "HTTP/1.1 403 Forbidden",with "curl http://127.0.0.1 -i".
> >
> > Why?
> >
> >
> ------------------------------------------------------------------------------
> > This SF email is sponsosred by:
> > Try Windows Azure free for 90 days Click Here
> > http://p.sf.net/sfu/sfd2d-msazure
> > _______________________________________________
> > moosefs-users mailing list
> > moo...@li...
> > https://lists.sourceforge.net/lists/listinfo/moosefs-users
> >
>

Re: [Moosefs-users] mfscgiserv 403 Forbidden

[Ken <ken...@gm...>]

maybe:
  chmod +x mfs.cgi

-Ken



On Mon, Mar 26, 2012 at 10:29 AM, 陈钢 <yik...@gm...> wrote:
> After I restart mfscgiserv with cmd "python2.7 /usr/sbin/mfscgiserv -P 80".
>
> I got "HTTP/1.1 403 Forbidden",with "curl http://127.0.0.1 -i".
>
> Why?
>
> ------------------------------------------------------------------------------
> This SF email is sponsosred by:
> Try Windows Azure free for 90 days Click Here
> http://p.sf.net/sfu/sfd2d-msazure
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>

[Moosefs-users] mfscgiserv 403 Forbidden

[陈钢 <yik...@gm...>]

After I restart mfscgiserv with cmd "python2.7 /usr/sbin/mfscgiserv -P 80".

I got "HTTP/1.1 403 Forbidden",with "curl http://127.0.0.1 -i".

Why?

Re: [Moosefs-users] Download link new version

[Michał B. <mic...@co...>]

Hi!

MooseFS is GPLv3 and everyone can copy it. If you don't want to fill the
form - all of you can simply use wget for downloading MooseFS (form is
triggered as javascript onclick event only). 

The form is for those who download files by clicking on our webpage.

We will make all old releases (older than 1 month) available for downloading
without filling the form. Also we will publish the code on sf.net and github
of each next version of MooseFS with 1 month of delay.

We understand that some of you may think that this new form which you have
to fill in when downloading the new version of MooseFS is quite disturbing
:) We have to say that we really need these information to push MooseFS
project forward and we hope for your understanding. 

Please share with us how can we do it better (have users/customers list of
MooseFS and give easy download in the same time)?

Many thanks for your feedback.


Kind regards
Michal  



-----Original Message-----
From: Atom Powers [mailto:ap...@di...] 
Sent: Wednesday, March 21, 2012 7:10 PM
To: moo...@li...
Subject: Re: [Moosefs-users] Download link new version

On 03/21/2012 08:23 AM, Allen, Benjamin S wrote:
> To the MooseFS folks, I work in an industry where I cannot agree to:
>
> "To publish the name and trademark of the company, on behalf of which 
> I act, on the MooseFS website or Core Technology website and to use 
> the aforementioned name and trademark in order to present them to Core 
> Technology's clients for marketing purposes only."

Same here. All contracts that come across my desk must have this kind of
clause removed before I can sign them. As it stands, this clause will
prevent me from upgrading.

> Also I would suggest you remove having to forcefully enter contact 
> info to download. Typically this doesn't jive well with OSS folks. 
> Make it voluntary.

I am less decisive on this point. If MooseFS was moving to a "freemium" 
model I would understand but as an OSS project it doesn't make sense.

--
--
Perfection is just a word I use occasionally with mustard.
--Atom Powers--
Director of IT
DigiPen Institute of Technology
+1 (425) 895-4443

----------------------------------------------------------------------------
--
This SF email is sponsosred by:
Try Windows Azure free for 90 days Click Here 
http://p.sf.net/sfu/sfd2d-msazure
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users