moosefs-users — Mailing list for MooseFS Users

Re: [Moosefs-users] Chunks

[Steve <st...@bo...>]

What is the feature in moosefs that makes you choose it over say ext4 ? 

 

I'm not understanding this yet!

 

-------Original Message------- 

 

From: Michal Borychowski 

Date: 02/03/2011 16:08:20 

To: 'Heiko Schröter' 

Cc: 'moosefs-users' 

Subject: Re: [Moosefs-users] Chunks 

 

Hi Heiko! 

 

We undestand your point and can see that using RAID6 + goal=1 is a little 

bit more economical than RAW HDD + goal=2 but this is not a huge difference 

as you still need some disks for the RAID6. 

 

The main purpose of MooseFS system is security not the space savings. And 

solution with RAID6 is not that secure. We generally advise not to use any 

RAIDs and using at least goal=2. 

 

Module responsible for rebalancing chunks operates on chunks (independently 

of the files). Each chunk is treated individually while making different 

operations. So the requested change is not just a matter of providing a 

simple patch, it would be a change in the "philosophy" of the system and 

unfortunately won't be possible. 

 

 

Kind regards 

Michal 

 

 

-----Original Message----- 

From: Heiko Schröter [mailto:sch...@iu...] 

Sent: Monday, February 28, 2011 4:13 PM 

To: mic...@ge... 

Subject: Chunks 

 

Hi Michael, 

 

sorry for seeing your post to the list too late before posting my 

screenshots. 

I can see your point, but to us it would be very important to have a 

"chunkabilty" or striping in other fs of one. 

 

The reason: 

 

We receive a lot of large satellite data files per day. (400MB to 2GB per 

file). 

The storage space is limited (because of the governmental funding), so we 

need to keep risks to a certain minimum with the ressources given. 

 

We are running Hardware raid6 on our chunkservers. So there is some safety 

margin here. 

 

But we need to make sure that in case of a total breakdown of a chunkserver 

only some files are lost to 100%, and not all files beeing damaged to a 

certain extend and therefore irrecoverable. 

 

So if I could be of any help testing a patch I would very much appreciate 

it. 

 

Thanks for your time looking into this. 

 

Regards 

Heiko 

 

 

 

 

 

 

Hi Heiko! 

 

You are definitely right! I made a mistake writing all chunks of the file 

with goal=1 would reside just on one chunkserver. Each chunk of the file 

would go (more or less by random) to different chunkservers. 

 

On the other hand we again focus on the point that using goal=1 is almost 

useless. Unless these are some temporary, unimportant files. The expectation


of distributed file system is to keep at least two copies of the file :) 

 

Thank you for being conscious :) 

 

 

-----------------------------------------------------------------------------
 

Free Software Download: Index, Search & Analyze Logs and other IT data in 

Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 

generated by your applications, servers and devices whether physical,
virtual 

or in the cloud. Deliver compliance at lower cost and gain new business 

insights. http://p.sf.net/sfu/splunk-dev2dev 

_______________________________________________ 

moosefs-users mailing list 

moo...@li... 

https://lists.sourceforge.net/lists/listinfo/moosefs-users 

 

Re: [Moosefs-users] Large scale deletions

[Thomas S H. <tha...@gm...>]

Sure thing, I will give it a try, I have not yet spent much time looking at
chunk loops, but it looks like I can refine a lot of the backend processes
with it, I will have to play around with it!

2011/3/2 Michal Borychowski <mic...@ge...>

> Hi!
>
>
>
> Thomas, you can try to set CHUNKS_DEL_LIMIT option in mfsmaster.cfg to 10
> and see if it helps you for the moment. It depends if you delete lots of
> files once or you delete them “continuously”.
>
>
>
>
>
> Regards
>
> Michal
>
>
>
> *From:* Thomas S Hatch [mailto:tha...@gm...]
> *Sent:* Tuesday, March 01, 2011 12:46 AM
> *To:* moosefs-users
> *Subject:* [Moosefs-users] Large scale deletions
>
>
>
> I have run into a problem with Moosefs when I delete a LOT of files at
> once, in my environment we add hundreds of thousands of files, process them
> into smaller encoded files, and then delete the originals, when we delete
> the originals the deletion process can lock up our writes, and dramatically
> slows down our moosefs.
>
>
>
> Is there a way to make large scale deletions behave a little more nicely?
>
>
>
> -Thomas  S Hatch
>

Re: [Moosefs-users] fs_writechunk returns status 11

[Michal B. <mic...@ge...>]

Hi Steve!

Regarding your first question this means "CHUNK_LOCKED" - it may happen when
several clients try to write to the same file. The message doesn't mean
anything bad but it is better to avoid such situations.

Regarding your second question please have a look here:
http://www.moosefs.org/moosefs-faq.html#error-messages and the two following
answers.


Best regards
Michal




-----Original Message-----
From: Steve Wilson [mailto:st...@pu...] 
Sent: Tuesday, March 01, 2011 3:07 PM
To: moo...@li...
Subject: Re: [Moosefs-users] fs_writechunk returns status 11

On 03/01/2011 08:42 AM, Steve Wilson wrote:
> Hi,
>
> For a couple peak periods of time this morning, my logs were full of 
> messages like the following:
>     Mar  1 07:49:49 stanley mfsmount[1253]: file: 278364, index: 0 - 
> fs_writechunk returns status 11
>     Mar  1 07:49:49 stanley mfsmount[1253]: file: 278364, index: 1 - 
> fs_writechunk returns status 11
>
> They all reference the same file: 278364.  Does anyone know what this 
> means?  And, if so, what I should do about it?
>
> Thanks!
>
> Steve
>

A further look showed up a handful of other files but 99% of the 
messages refer to that one file.

Additionally, I saw several messages like the following from last 
night's logs:
     Mar  1 00:35:03 stanley mfsmount[1253]: file: 161779, index: 0, 
chunk: 310854, version: 1 - writeworker: connection with (80D2302F:9422) 
was timed out (unfinished writes: 5; try counter: 1)
     Mar  1 00:35:05 stanley mfsmount[1253]: file: 161779, index: 0, 
chunk: 310854, version: 1 - writeworker: connection with (80D2302F:9422) 
was timed out (unfinished writes: 5; try counter: 1)

Thanks,
Steve


----------------------------------------------------------------------------
--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 
generated by your applications, servers and devices whether physical,
virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Large scale deletions

[Michal B. <mic...@ge...>]

El Martes 01 Marzo 2011, Michal Borychowski escribió:
>> Hi Thomas!
>>
>> Do you use ext3fs? Deleting files on this system is on its own very very
>> slow. And we assume there is also some mechanism blocking other operations
>> on disk while deleting built into kernel. These are things we cannot do
>> anything about it.

>What filesystem would you recomend: ext4, xfs? (just curious, as I can't find 
>a recommendation on the website).
[MB] Unfortunately we do not have any recommendations for the filesystem on chunkservers. Generally speaking every filesystem is good. Some differences just appear in very big production environments. We use ext3 and are quite satisfied. We wait for your observations :)


Kind regards
Michal


> On the MooseFS side there are really some deletion limits which depend on
> the number of chunks necessary to delete. We'll think of introducing some
> option to configure this top limit.

Nice to know, as we also used to copy a directory with lots of small files, 
make a tarball and delete the copy.

As a workaround, instead of copying the files we symlink the origin directory 
and 'tar czhf' it.

Regards,
-- 
Ricardo J. Barberis
Senior SysAdmin / ITI
Dattatec.com :: Soluciones de Web Hosting
Tu Hosting hecho Simple!

------------------------------------------


------------------------------------------------------------------------------
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT data 
generated by your applications, servers and devices whether physical, virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Chunks

[Michal B. <mic...@ge...>]

Hi Heiko!

We undestand your point and can see that using RAID6 + goal=1 is a little
bit more economical than RAW HDD + goal=2 but this is not a huge difference
as you still need some disks for the RAID6.

The main purpose of MooseFS system is security not the space savings. And
solution with RAID6 is not that secure. We generally advise not to use any
RAIDs and using at least goal=2. 

Module responsible for rebalancing chunks operates on chunks (independently
of the files). Each chunk is treated individually while making different
operations. So the requested change is not just a matter of providing a
simple patch, it would be a change in the "philosophy" of the system and
unfortunately won't be possible.


Kind regards
Michal


-----Original Message-----
From: Heiko Schröter [mailto:sch...@iu...] 
Sent: Monday, February 28, 2011 4:13 PM
To: mic...@ge...
Subject: Chunks

Hi Michael,

sorry for seeing your post to the list too late before posting my
screenshots.
I can see your point, but to us it would be very important to have a
"chunkabilty" or striping in other fs of one.

The reason:

We recieve a lot of large satellite data files per day. (400MB to 2GB per
file).
The storage space is limited (because of the governmental funding), so we
need to keep risks to a certain minimum with the ressources given.

We are running Hardware raid6 on our chunkservers. So there is some safety
margin here.

But we need to make sure that in case of a total breakdown of a chunkserver
only some files are lost to 100%, and not all files beeing damaged to a
certain extend and therefore irrecoverable.

So if i could be of any help testing a patch i would very much appreciate
it.

Thanks for your time looking into this.

Regards
Heiko






Hi Heiko!

You are definitely right! I made a mistake writing all chunks of the file
with goal=1 would reside just on one chunkserver. Each chunk of the file
would go (more or less by random) to different chunkservers. 

On the other hand we again focus on the point that using goal=1 is almost
useless. Unless these are some temporary, unimportant files. The expectation
of distributed file system is to keep at least two copies of the file :)

Thank you for being conscious :)

Re: [Moosefs-users] Large scale deletions

[Michal B. <mic...@ge...>]

Hi!

 

Thomas, you can try to set CHUNKS_DEL_LIMIT option in mfsmaster.cfg to 10
and see if it helps you for the moment. It depends if you delete lots of
files once or you delete them "continuously".

 

 

Regards

Michal

 

From: Thomas S Hatch [mailto:tha...@gm...] 
Sent: Tuesday, March 01, 2011 12:46 AM
To: moosefs-users
Subject: [Moosefs-users] Large scale deletions

 

I have run into a problem with Moosefs when I delete a LOT of files at once,
in my environment we add hundreds of thousands of files, process them into
smaller encoded files, and then delete the originals, when we delete the
originals the deletion process can lock up our writes, and dramatically
slows down our moosefs.

 

Is there a way to make large scale deletions behave a little more nicely?

 

-Thomas  S Hatch

Re: [Moosefs-users] chunkserver priority?

[Giovanni T. <gt...@li...>]

Il 01/03/2011 14:21, Michal Borychowski ha scritto:
> This is quite an interesting suggestion of computers usage but not that easy
> to implement. We could think of "auxiliary chunkserver" but we should how to
> interpret goal value. Does goal=2 mean that we need to have 2 copies on the
> main chunkservers and one extra on the auxiliary one or one at the main and
> one at the auxiliary? How would the master know that auxiliary chunkserver
> would be permanently unavailable? 

The auxiliary chunkserver should be intended as extra copies for
lowering the I/O and network load on main chunkserver.

I have servers and main switches on Gigabit Ethernet and Fibre Channel,
but desktop clients are 10/100 on 10/100 switches, so the bandwidth is
capped to 10/100 in each room (5-6 clients on each room).

The reason I cannot use *now* the local desktop hd as auxiliary
chunckserver it's because I haven't the assurance that if all the
auxiliary chunckserver are down, all chunks are available on the main
chunkservers.

On the other hand, another solution could be to implement client-side
disk caching, like fs-cache, in the same way nfsv4 do (via cachefilesd).

> So for now we put it in the section "nice ideas" and we'll come back to it.

Thank you Michał.

-- 
Giovanni Toraldo
http://www.libersoft.it/

Re: [Moosefs-users] Large scale deletions

[Ricardo J. B. <ric...@da...>]

El Martes 01 Marzo 2011, Michal Borychowski escribió:
> Hi Thomas!
>
> Do you use ext3fs? Deleting files on this system is on its own very very
> slow. And we assume there is also some mechanism blocking other operations
> on disk while deleting built into kernel. These are things we cannot do
> anything about it.

What filesystem would you recomend: ext4, xfs? (just curious, as I can't find 
a recommendation on the website).

> On the MooseFS side there are really some deletion limits which depend on
> the number of chunks necessary to delete. We'll think of introducing some
> option to configure this top limit.

Nice to know, as we also used to copy a directory with lots of small files, 
make a tarball and delete the copy.

As a workaround, instead of copying the files we symlink the origin directory 
and 'tar czhf' it.

Regards,
-- 
Ricardo J. Barberis
Senior SysAdmin / ITI
Dattatec.com :: Soluciones de Web Hosting
Tu Hosting hecho Simple!

------------------------------------------

Re: [Moosefs-users] Non-existing chunks hang the mfsmaster

[Flow J. <fl...@gm...>]

This is nice but it would be great if the one week period is 
configurable (by reading the link below, it was 2 hrs?). Or can we have 
some native command to release the reserved files and clear nonexistent 
chunks? I'm asking this because last time we had the issue, all the 
chunks reported by cgiserver are healthy (has goal = 2), but the master 
server still claims non-existing chunks. So I had to find and delete 
them manually in the chunk server file system.

Thanks
Flow

On 03/01/2011 07:51 PM, Michal Borychowski wrote:
>
> Hi!
>
> In recent MooseFS versions session are cleaned up only after a week so 
> this is just a matter of time.
>
> Regards
>
> Micha?
>
> *From:*Flow Jiang [mailto:fl...@gm...]
> *Sent:* Monday, February 28, 2011 12:15 PM
> *To:* Stas Oskin
> *Cc:* moosefs-users
> *Subject:* Re: [Moosefs-users] Non-existing chunks hang the mfsmaster
>
> Hi,
>
> We had the similar issue recently and the symptom was it took a long 
> time for mfsmaster to start (but it eventually gets up and running, 
> after about 5mins). Here are what I did to make mfsmaster happy after 
> it starts again:
>
> 1. Use the script provided at 
> http://sourceforge.net/tracker/?func=detail&aid=3104619&group_id=228631&atid=1075722 
> <http://sourceforge.net/tracker/?func=detail&aid=3104619&group_id=228631&atid=1075722> 
> to release all reserved files. (Should comment the optional section in 
> it to speed up the process)
> 2. Delete all the nonexistent trunks on the trunk server.
>
> I'm not mfs expert but these steps do make our mfsmaster server happy 
> and it now loads has about 700M metadata in 5 seconds, no error in log 
> file.
>
> I'm also curious about the *official solution* from Michal :)
>
> Thanks
> Flow
>
> On 02/28/2011 03:25 AM, Stas Oskin wrote:
>
> Hi.
>
> We got a very strange that happened on our test cluster.
>
> After a power crash, the mfsmaster syslog is full of following errors:
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005313A2_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005393A2_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005493A3_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005293A3_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005513A3_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005313A3_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005113A3_00000001), so create it for future deletion
> Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent 
> chunk (00000000005093A3_00000001), so create it for future deletion
>
> These errors appears all the time, and practically hang the mfsmaster.
> mfscgi stops working (hangs), and mounts are aborting with following 
> error:
> error receiving data from mfsmaster: ETIMEDOUT (Operation timed out)
> error receiving data from mfsmaster: ETIMEDOUT (Operation timed out)
>
>
> Upgrading to .20 didn't help.
>
> Any idea what this could be and how to resolve it?
>
> Thanks.
>
>   
>   
> ------------------------------------------------------------------------------
> Free Software Download: Index, Search&  Analyze Logs and other IT data in
> Real-Time with Splunk. Collect, index and harness all the fast moving IT data
> generated by your applications, servers and devices whether physical, virtual
> or in the cloud. Deliver compliance at lower cost and gain new business
> insights.http://p.sf.net/sfu/splunk-dev2dev  
>   
>   
> _______________________________________________
> moosefs-users mailing list
> moo...@li...  <mailto:moo...@li...>
> https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Reserved files remain forever

[Flow J. <fl...@gm...>]

O.K. I think I lied in the last post. I changed all our configuration to 
use fstab mount the FS permanently and use "-fstype=bind" for AutoFS to 
link it to user home. But I still have more than 100 reserved files 
today, most of them look like temp files. (like 
.local|share|gvfs-metadata|home-8a63e833.log)

The interesting thing is that we have nightly builds running on 
non-UserHome MooseFS mounted directory, which also creates / deletes 
temp files, but non of them gets reserved. So I suspect this might be 
caused by the way applications write file.

So my question is that does any one has the similar issue when MooseFS 
is serving as home? Do you have reserved files remaining for a long 
time? And they will be deleted after 1 week, right?

BTW, the "-fstype=bind" work around do fixes the Core Dump issue.

Thanks
Flow

On 02/27/2011 01:20 AM, Flow Jiang wrote:
> Hi,
>
> I now know how to re-create the issue and it should also be related to 
> AutoFS.
>
> I noticed that if I make a new file with non-zero size and delete it 
> immediately, it will be set as reserved file and keep at this state 
> for a while (about 1 minute in our environment). If the underlying 
> MooseFS is unmounted within this short period, the behavior differs 
> between fstab mounted FS and AufoFS mounted FS.
>
> * If the FS is mounted by fstab, create and delete a file, then 
> unmount the FS, the file eventually gets deleted
> * If the FS is mounted by autofs, create and delete a file, then 
> unmount the FS, the file remains as reserved forever
>
> And unfortunately we are relying on the timeout feature of AutoFS so 
> unmount happens frequently. This is the reason we have so many cache 
> files (created then deleted during one mount session) remaining as 
> reserved files forever.
>
> Our AutoFS mount option is mentioned in the other thread 
> (http://sourceforge.net/mailarchive/forum.php?thread_name=4D68E18F.3000708%40gmail.com&forum_name=moosefs-users). 
> And the issue looks like the same:
>
> * If the FS is mounted by fstab, writing a file (> 1M) to some folder 
> then unmount it, works O.K.
> * If the FS is mounted by autofs, writing a file (> 1M) to some folder 
> then unmount it, core file gets dumped
>
> Increasing autofs timeout would help for the reserved file issue but 
> not for the core dump issue. And even if the timeout is increased, a 
> restart / power off of the system still won't leave enough time for 
> the file to be totally deleted.
>
> I do have a work around to fix the 2 issues temporarily, by mounting 
> the entire MFS system with fstab and use "-fstype=bind" option in 
> autofs to make auto mounting/unmounting happen. But this is 
> complicated and different mfs mount options can't be set with 
> different subfolders. So I do hope I can have native AutoFS supported 
> MooseFS mounts.
>
> Can any one help on this?
>
> Many Thanks
> Flow
>
> On 02/22/2011 12:03 AM, Flow Jiang wrote:
>> Hi,
>>
>> Just found another issue. I cleared about 10000 reserved files with 
>> the script provided at 
>> http://sourceforge.net/tracker/?func=detail&aid=3104619&group_id=228631&atid=1075722 
>> yesterday, and this morning I had 0 reserved file when started working.
>>
>> However, after one day development activity with 6 workstations, now 
>> we have 204 reserved files not deleted. I've noticed it's stated that 
>> "Each session after two hours is automatically closed and all the 
>> files are released." in above link, but seems it's not happening in 
>> our environment. We have CentOS 5.5 x86 servers and run mfsmount on 
>> Fedora 12 x64 workstations. Both servers and workstations run mfs 
>> 1.6.19. And mfs is serving as home with read / write access.
>>
>> Here are some example of the reserved files by reading the metadata:
>>
>> 00067856|UserHome|tompan|.mozilla|firefox|mk9e32d7.default|OfflineCache|index.sqlite-journal 
>>
>> 00067857|UserHome|tompan|.mozilla|firefox|mk9e32d7.default|OfflineCache|index.sqlite-journal 
>>
>>
>> Most of the 204 reserved files look like temp / journal files.
>>
>> Any ideas about the cause of the issue?
>>
>> BTW, OpenOffice fails to start if MFS serves as home directory. It 
>> should be a FS bug as stated on: 
>> http://qa.openoffice.org/issues/show_bug.cgi?id=113207. Would it be 
>> related to the issue above? And can we fix this OOo issue?
>>
>> Many Thanks
>> Flow
>>
>>
>>
>>
>>
>>
>>

Re: [Moosefs-users] Core Dumped from mfsmount with Autofs

[Flow J. <fl...@gm...>]

Michal,

Glad to know that this error could be simply solved by commenting out 
that line and will try tomorrow to see if it fixes this issue.

It does annoying since each core file takes about 170M and I tried to 
disable the core dump but failed. So hopefully we can have a better 
solution in the next release.

Thanks
Flow

On 03/01/2011 09:00 PM, Michal Borychowski wrote:
> Hi!
>
> This error is not a serious one. It may happen only upon exits. If these
> errors are annoying a quick solution is to comment out the
> "free(freecblockshead)" line, recompile mfsmount and run again. We'll
> prepare a better solution in the next release.
>
>
> Kind regards
> Michał
>
> -----Original Message-----
> From: Flow Jiang [mailto:fl...@gm...]
> Sent: Saturday, February 26, 2011 12:19 PM
> To: moo...@li...
> Subject: Re: [Moosefs-users] Core Dumped from mfsmount with Autofs
>
> I found a more easy way to re-create the issue.
>
> Just keep the auto.home as:
>
> * -fstype=fuse,mfssubfolder=UserHome/&  :mfsmount
>
> Then copy a file to an unmounted user folder, and run:
>
> service autofs stop
>
> The core file will be dumped.
>
> Flow
>
> On 02/26/2011 04:44 PM, Flow Jiang wrote:
>> Hi,
>>
>> After merging moosefs into our production environment for about 2
>> weeks, I now found there are a lot of core files dumped from mfsmount
>> and remaining in "/" directory. All the back traces look simulator,
>> which dies at free(freecblockshead) call in write_data_term (), when
>> mainloop() ends.
>>
>> Fedora 12 x64 (mfsmount is compiled from source):
>>
>> Core was generated by `mfsmount /home/fwjiang -o
>> rw,mfssubfolder=UserHome/fwjiang'.
>> Program terminated with signal 6, Aborted.
>> #0  0x00000039ab4327f5 in raise () from /lib64/libc.so.6
>> Missing separate debuginfos, use: debuginfo-install
>> fuse-libs-2.8.5-2.fc12.x86_64 glibc-2.11.2-3.x86_64
>> libgcc-4.4.4-10.fc12.x86_64
>> (gdb) bt
>> #0  0x00000039ab4327f5 in raise () from /lib64/libc.so.6
>> #1  0x00000039ab433fd5 in abort () from /lib64/libc.so.6
>> #2  0x00000039ab46fa1b in __libc_message () from /lib64/libc.so.6
>> #3  0x00000039ab475336 in malloc_printerr () from /lib64/libc.so.6
>> #4  0x000000000040eb12 in write_data_term () at writedata.c:906
>> #5  0x000000000041282d in mainloop (args=0x7fff49f484d0, mp=0x1bb72e0
>> "/tmp/autotAfzu1", mt=1, fg=0) at main.c:599
>> #6  0x0000000000412c48 in main (argc=<value optimized out>,
>> argv=0x7fff49f485f8) at main.c:819
>>
>> Centos 5.5 x86 (mfsmount is from DAG repository):
>>
>> Core was generated by `mfsmount /project/ui -o
>> rw,mfssubfolder=ProjectData/project/ui'.
>> Program terminated with signal 6, Aborted.
>> #0  0x00417410 in __kernel_vsyscall ()
>> (gdb) bt
>> #0  0x00417410 in __kernel_vsyscall ()
>> #1  0x00a8ddf0 in raise () from /lib/libc.so.6
>> #2  0x00a8f701 in abort () from /lib/libc.so.6
>> #3  0x00ac628b in __libc_message () from /lib/libc.so.6
>> #4  0x00ace5a5 in _int_free () from /lib/libc.so.6
>> #5  0x00ace9e9 in free () from /lib/libc.so.6
>> #6  0x08056cc3 in write_data_term ()
>> #7  0x0805a768 in mainloop ()
>> #8  0x0805ab37 in main ()
>>
>> The auto.home file to auto mount user home on Fedore 12 boxes look like:
>>
>> * -fstype=fuse,mfssubfolder=UserHome/&  :mfsmount
>>
>> All the server / clients run mfs 1.6.19. And all core files are dumped
>> from those mounts with Read/Write access. By reading time log of the
>> core dump listed above, I found it's dumped at when autofs timeouts
>> (the default timeout is 300s on CentOS 5.5).
>>
>> So I tried manually copy a file (about 80MB) to a user folder which
>> haven't been auto mounted, then wait 300s until the folder is auto
>> unmounted, the core was dumped as expected.
>>
>> Does anyone has the same issue? Am I doing the right thing to auto
>> mount with Moosefs?
>>
>> Thanks
>> Flow
> ----------------------------------------------------------------------------
> --
> Free Software Download: Index, Search&  Analyze Logs and other IT data in
> Real-Time with Splunk. Collect, index and harness all the fast moving IT
> data
> generated by your applications, servers and devices whether physical,
> virtual
> or in the cloud. Deliver compliance at lower cost and gain new business
> insights. http://p.sf.net/sfu/splunk-dev2dev
> _______________________________________________
> moosefs-users mailing list
> moo...@li...
> https://lists.sourceforge.net/lists/listinfo/moosefs-users
>

Re: [Moosefs-users] fs_writechunk returns status 11

[Steve W. <st...@pu...>]

On 03/01/2011 08:42 AM, Steve Wilson wrote:
> Hi,
>
> For a couple peak periods of time this morning, my logs were full of 
> messages like the following:
>     Mar  1 07:49:49 stanley mfsmount[1253]: file: 278364, index: 0 - 
> fs_writechunk returns status 11
>     Mar  1 07:49:49 stanley mfsmount[1253]: file: 278364, index: 1 - 
> fs_writechunk returns status 11
>
> They all reference the same file: 278364.  Does anyone know what this 
> means?  And, if so, what I should do about it?
>
> Thanks!
>
> Steve
>

A further look showed up a handful of other files but 99% of the 
messages refer to that one file.

Additionally, I saw several messages like the following from last 
night's logs:
     Mar  1 00:35:03 stanley mfsmount[1253]: file: 161779, index: 0, 
chunk: 310854, version: 1 - writeworker: connection with (80D2302F:9422) 
was timed out (unfinished writes: 5; try counter: 1)
     Mar  1 00:35:05 stanley mfsmount[1253]: file: 161779, index: 0, 
chunk: 310854, version: 1 - writeworker: connection with (80D2302F:9422) 
was timed out (unfinished writes: 5; try counter: 1)

Thanks,
Steve

Re: [Moosefs-users] chunkserver priority?

[Giovanni T. <gt...@li...>]

Il 01/03/2011 14:21, Michal Borychowski ha scritto:
> This is quite an interesting suggestion of computers usage but not that easy
> to implement. We could think of "auxiliary chunkserver" but we should how to
> interpret goal value. Does goal=2 mean that we need to have 2 copies on the
> main chunkservers and one extra on the auxiliary one or one at the main and
> one at the auxiliary? How would the master know that auxiliary chunkserver
> would be permanently unavailable? 

The auxiliary chunkserver should be intended as extra copies for
lowering the I/O and network load on main chunkserver.

I have servers and main switches on Gigabit Ethernet and Fibre Channel,
but desktop clients are 10/100 on 10/100 switches, so the bandwidth is
capped to 10/100 in each room (5-6 clients on each room).

The reason I cannot use *now* the local desktop hd as auxiliary
chunckserver it's because I haven't the assurance that if all the
auxiliary chunckserver are down, all chunks are available on the main
chunkservers.

On the other hand, another solution could be to implement client-side
disk caching, like fs-cache, in the same way nfsv4 do (via cachefilesd).

> So for now we put it in the section "nice ideas" and we'll come back to it.

Thank you Michał.

-- 
Giovanni Toraldo
http://www.libersoft.it/

[Moosefs-users] fs_writechunk returns status 11

[Steve W. <st...@pu...>]

Hi,

For a couple peak periods of time this morning, my logs were full of 
messages like the following:
     Mar  1 07:49:49 stanley mfsmount[1253]: file: 278364, index: 0 - 
fs_writechunk returns status 11
     Mar  1 07:49:49 stanley mfsmount[1253]: file: 278364, index: 1 - 
fs_writechunk returns status 11

They all reference the same file: 278364.  Does anyone know what this 
means?  And, if so, what I should do about it?

Thanks!

Steve

-- 
Steven M. Wilson, Systems and Network Manager
Markey Center for Structural Biology
Purdue University
(765) 496-1946

Re: [Moosefs-users] missing chunk problem

[Michal B. <mic...@ge...>]

Hi!

MooseFS returns information that a file is not available but after
relatively quite a long time. You can mount with "-o mfsioretries=1" option
- then MooseFS will inform about the unavailability of a file promptly. It
would return "EIO" (input/output error). And you have to check what
bahaviour would be on the part of nginx.


Kind regards
Michał Borychowski 
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01



-----Original Message-----
From: p g [mailto:pg0...@gm...] 
Sent: Thursday, February 24, 2011 10:06 AM
To: moo...@li...
Subject: [Moosefs-users] missing chunk problem

Dear Sir

We are using Moosefs currently, when users download a large file via nginx ,
if there are missing chunks inside that file due to the network
problem,  the nginx process will hang and become a zombie thread that
wait for the IO read.
is there any way to prevent this situation, for example, display "404
Not found" to user if missing chunk are found.

Regards
Pong

----------------------------------------------------------------------------
--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 
generated by your applications, servers and devices whether physical,
virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] chunkserver priority?

[Michal B. <mic...@ge...>]

Hi!

This is quite an interesting suggestion of computers usage but not that easy
to implement. We could think of "auxiliary chunkserver" but we should how to
interpret goal value. Does goal=2 mean that we need to have 2 copies on the
main chunkservers and one extra on the auxiliary one or one at the main and
one at the auxiliary? How would the master know that auxiliary chunkserver
would be permanently unavailable? 

So for now we put it in the section "nice ideas" and we'll come back to it.


Kind regards
Michał 

-----Original Message-----
From: Giovanni Toraldo [mailto:gt...@li...] 
Sent: Friday, February 25, 2011 3:41 PM
To: moo...@li...
Subject: [Moosefs-users] chunkserver priority?

Hi,

I was wondering about the ability to configure a priority flag for each 
available chunkserver:

on my setup I have two servers, a master+chunk and a metalogger+chunk, 
and a bunch of desktop client (now ~50, but they will probably increase 
in the future). Each client has a spare disk of 40/80/160 GB, that are 
mostly unused.

What if I use each desktop client as a small chunk server?

The problem is that even with an high number of goal, maybe some chunks 
will be placed on nodes that are currently offline on a particular 
moment of the day (mostly part-time worker here).

If I was able to configure as high priority the 2 big chunkserver and as 
low priority the small chunkservers, I would be pretty sure that at 
least the first two copies of a chunk are always on the servers.

Any comment is greatly appreciated.

Thanks.

-- 
Giovanni Toraldo
http://www.libersoft.it/

----------------------------------------------------------------------------
--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 
generated by your applications, servers and devices whether physical,
virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Core Dumped from mfsmount with Autofs

[Michal B. <mic...@ge...>]

Hi!

This error is not a serious one. It may happen only upon exits. If these
errors are annoying a quick solution is to comment out the
"free(freecblockshead)" line, recompile mfsmount and run again. We'll
prepare a better solution in the next release.


Kind regards
Michał 

-----Original Message-----
From: Flow Jiang [mailto:fl...@gm...] 
Sent: Saturday, February 26, 2011 12:19 PM
To: moo...@li...
Subject: Re: [Moosefs-users] Core Dumped from mfsmount with Autofs

I found a more easy way to re-create the issue.

Just keep the auto.home as:

* -fstype=fuse,mfssubfolder=UserHome/& :mfsmount

Then copy a file to an unmounted user folder, and run:

service autofs stop

The core file will be dumped.

Flow

On 02/26/2011 04:44 PM, Flow Jiang wrote:
> Hi,
>
> After merging moosefs into our production environment for about 2 
> weeks, I now found there are a lot of core files dumped from mfsmount 
> and remaining in "/" directory. All the back traces look simulator, 
> which dies at free(freecblockshead) call in write_data_term (), when 
> mainloop() ends.
>
> Fedora 12 x64 (mfsmount is compiled from source):
>
> Core was generated by `mfsmount /home/fwjiang -o 
> rw,mfssubfolder=UserHome/fwjiang'.
> Program terminated with signal 6, Aborted.
> #0  0x00000039ab4327f5 in raise () from /lib64/libc.so.6
> Missing separate debuginfos, use: debuginfo-install 
> fuse-libs-2.8.5-2.fc12.x86_64 glibc-2.11.2-3.x86_64 
> libgcc-4.4.4-10.fc12.x86_64
> (gdb) bt
> #0  0x00000039ab4327f5 in raise () from /lib64/libc.so.6
> #1  0x00000039ab433fd5 in abort () from /lib64/libc.so.6
> #2  0x00000039ab46fa1b in __libc_message () from /lib64/libc.so.6
> #3  0x00000039ab475336 in malloc_printerr () from /lib64/libc.so.6
> #4  0x000000000040eb12 in write_data_term () at writedata.c:906
> #5  0x000000000041282d in mainloop (args=0x7fff49f484d0, mp=0x1bb72e0 
> "/tmp/autotAfzu1", mt=1, fg=0) at main.c:599
> #6  0x0000000000412c48 in main (argc=<value optimized out>, 
> argv=0x7fff49f485f8) at main.c:819
>
> Centos 5.5 x86 (mfsmount is from DAG repository):
>
> Core was generated by `mfsmount /project/ui -o 
> rw,mfssubfolder=ProjectData/project/ui'.
> Program terminated with signal 6, Aborted.
> #0  0x00417410 in __kernel_vsyscall ()
> (gdb) bt
> #0  0x00417410 in __kernel_vsyscall ()
> #1  0x00a8ddf0 in raise () from /lib/libc.so.6
> #2  0x00a8f701 in abort () from /lib/libc.so.6
> #3  0x00ac628b in __libc_message () from /lib/libc.so.6
> #4  0x00ace5a5 in _int_free () from /lib/libc.so.6
> #5  0x00ace9e9 in free () from /lib/libc.so.6
> #6  0x08056cc3 in write_data_term ()
> #7  0x0805a768 in mainloop ()
> #8  0x0805ab37 in main ()
>
> The auto.home file to auto mount user home on Fedore 12 boxes look like:
>
> * -fstype=fuse,mfssubfolder=UserHome/& :mfsmount
>
> All the server / clients run mfs 1.6.19. And all core files are dumped 
> from those mounts with Read/Write access. By reading time log of the 
> core dump listed above, I found it's dumped at when autofs timeouts 
> (the default timeout is 300s on CentOS 5.5).
>
> So I tried manually copy a file (about 80MB) to a user folder which 
> haven't been auto mounted, then wait 300s until the folder is auto 
> unmounted, the core was dumped as expected.
>
> Does anyone has the same issue? Am I doing the right thing to auto 
> mount with Moosefs?
>
> Thanks
> Flow

----------------------------------------------------------------------------
--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 
generated by your applications, servers and devices whether physical,
virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Non-existing chunks hang the mfsmaster

[Michal B. <mic...@ge...>]

Hi!

 

In recent MooseFS versions session are cleaned up only after a week so this
is just a matter of time.

 

 

Regards

Michał 

 

From: Flow Jiang [mailto:fl...@gm...] 
Sent: Monday, February 28, 2011 12:15 PM
To: Stas Oskin
Cc: moosefs-users
Subject: Re: [Moosefs-users] Non-existing chunks hang the mfsmaster

 

Hi, 

We had the similar issue recently and the symptom was it took a long time
for mfsmaster to start (but it eventually gets up and running, after about
5mins). Here are what I did to make mfsmaster happy after it starts again:

1. Use the script provided at http://sourceforge.net/tracker/?func=detail
<http://sourceforge.net/tracker/?func=detail&aid=3104619&group_id=228631&ati
d=1075722> &aid=3104619&group_id=228631&atid=1075722 to release all reserved
files. (Should comment the optional section in it to speed up the process)
2. Delete all the nonexistent trunks on the trunk server.

I'm not mfs expert but these steps do make our mfsmaster server happy and it
now loads has about 700M metadata in 5 seconds, no error in log file.

I'm also curious about the *official solution* from Michal :)

Thanks
Flow

On 02/28/2011 03:25 AM, Stas Oskin wrote: 

Hi.

We got a very strange that happened on our test cluster.

After a power crash, the mfsmaster syslog is full of following errors:
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005313A2_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005393A2_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005493A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005293A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005513A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005313A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005113A3_00000001), so create it for future deletion
Feb 27 19:19:21 web1 mfsmaster[30654]: chunkserver has nonexistent chunk
(00000000005093A3_00000001), so create it for future deletion

These errors appears all the time, and practically hang the mfsmaster.
mfscgi stops working (hangs), and mounts are aborting with following error:
error receiving data from mfsmaster: ETIMEDOUT (Operation timed out)
error receiving data from mfsmaster: ETIMEDOUT (Operation timed out)


Upgrading to .20 didn't help.

Any idea what this could be and how to resolve it?

Thanks.

 
 
----------------------------------------------------------------------------
--
Free Software Download: Index, Search & Analyze Logs and other IT data in 
Real-Time with Splunk. Collect, index and harness all the fast moving IT
data 
generated by your applications, servers and devices whether physical,
virtual
or in the cloud. Deliver compliance at lower cost and gain new business 
insights. http://p.sf.net/sfu/splunk-dev2dev 
 
 
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users

Re: [Moosefs-users] Large scale deletions

[Michal B. <mic...@ge...>]

Hi Thomas!

 

Do you use ext3fs? Deleting files on this system is on its own very very
slow. And we assume there is also some mechanism blocking other operations
on disk while deleting built into kernel. These are things we cannot do
anything about it.

 

On the MooseFS side there are really some deletion limits which depend on
the number of chunks necessary to delete. We'll think of introducing some
option to configure this top limit.

 

 

Regards

-Michal

 

From: Thomas S Hatch [mailto:tha...@gm...] 
Sent: Tuesday, March 01, 2011 12:46 AM
To: moosefs-users
Subject: [Moosefs-users] Large scale deletions

 

I have run into a problem with Moosefs when I delete a LOT of files at once,
in my environment we add hundreds of thousands of files, process them into
smaller encoded files, and then delete the originals, when we delete the
originals the deletion process can lock up our writes, and dramatically
slows down our moosefs.

 

Is there a way to make large scale deletions behave a little more nicely?

 

-Thomas  S Hatch

[Moosefs-users] Mounting on Mac OS X

[Sebastian P. <sebastian.pierzgalski@p.lodz.pl>]

Hello,

We use Mac OS X. We have a problem with mounting the shares on the Mac. After installation when I want to create a file in nano, it hangs for about 3-4 mins and creates a file with 0 size. Can someone help us?

Pozdrawiam
Sebastian Pierzgalski
Grupa Utrzymania i Rozwoju Usług
Politechnika Łódzka Centrum Komputerowe

Re: [Moosefs-users] A bug in mfs

[Lin Y. <id...@gm...>]

Yes ~  What I  proposed doesn’t change anything.
Thanks for you answer :)

On Mon, Feb 28, 2011 at 7:57 PM, Michal Borychowski <
mic...@ge...> wrote:

> Hi!
>
>
>
> No, this is not an error, the code is good. The “tcptoread” function reads
> exactly “leng” bytes but it cannot wait for data longer than “msecto”
> milliseconds. The function returns number of bytes read.
>
>
>
> Appearance of EOF while reading is equivalent to an error. If we want to
> read 8 bytes and receive just 6 and the connection got broken it is not
> important for me if it is 6, 4 or 0 – it is important it’s less than 8. In
> other places in the code values returned by this function are compared to
> the length passed as a parameter and whenever value received is different,
> it is treated as an error. So to be honest the change you proposed doesn’t
> change anything.
>
>
>
> I hope the explanation is quite clear.) Thank you
>
>
>
>
>
> Kind regards
>
> Michał Borychowski
>
> MooseFS Support Manager
>
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
>
> Gemius S.A.
>
> ul. Wołoska 7, 02-672 Warszawa
>
> Budynek MARS, klatka D
>
> Tel.: +4822 874-41-00
>
> Fax : +4822 874-41-01
>
>
>
>
>
>
>
> *From:* Lin Yang [mailto:id...@gm...]
> *Sent:* Saturday, February 19, 2011 6:25 PM
> *To:* moo...@li...
> *Subject:* [Moosefs-users] A bug in mfs
>
>
>
> in mfscommon/sockets.c : line 381
>
>
>
> int32_t tcptoread(int sock,void *buff,uint32_t leng,uint32_t msecto) {
>
>       uint32_t rcvd=0;
>
>       int i;
>
>       struct pollfd pfd;
>
>       pfd.fd = sock;
>
>       pfd.events = POLLIN;
>
>       while (rcvd<leng) {
>
>             pfd.revents = 0;
>
>             if (poll(&pfd,1,msecto)<0) {
>
>                   return -1;
>
>             }
>
>             if (pfd.revents & POLLIN) {
>
>                   i = read(sock,((uint8_t*)buff)+rcvd,leng-rcvd);
>
>                   if (i<=0) {  //this code work only in Long Connection
> without EOF
>
>                         return i;
>
>                   }
>
>                   rcvd+=i;
>
>             } else {
>
>                   errno = ETIMEDOUT;
>
>                   return -1;
>
>             }
>
>       }
>
>       return rcvd;
>
> }
>
>
>
> -------------------------------------------------
>
> should be replaced with :
>
>
>
>                   if (i==0) { //EOF
>
>                         return rcvd;
>
>                   }
>
>                   if (i<0) {
>
>                         return i;
>
>                   }
>
>
>
>
> --
> 杨林
>
> 中科院计算技术研究所
>
>
>
> 15811038200
>
> id...@gm...
> http://idning.javaeye.com/
>
>
>



-- 
杨林
中科院计算技术研究所

15811038200
id...@gm...
http://idning.javaeye.com/

[Moosefs-users] Large scale deletions

[Thomas S H. <tha...@gm...>]

I have run into a problem with Moosefs when I delete a LOT of files at once,
in my environment we add hundreds of thousands of files, process them into
smaller encoded files, and then delete the originals, when we delete the
originals the deletion process can lock up our writes, and dramatically
slows down our moosefs.

Is there a way to make large scale deletions behave a little more nicely?

-Thomas  S Hatch

Re: [Moosefs-users] Chunks

[Heiko S. <sch...@iu...>]

Am Samstag 26 Februar 2011, um 14:27:34 schrieb Heiko Schröter:
> On Fri, 25 Feb 2011 11:21:33 +0100
> "Michal Borychowski" <mic...@ge...> wrote:
> 
> > Hi!
> > 
> > If you set goal=1, all chunks belonging to the same file will go to the same
> > chunk server.
> Supposed to, but it does not work this way.
> 
> In our test setup (one master, one client, two chunkservers) the chunks are beeing spread across the two chunkservers, even with "goal=1".
> df on on the chunksrevr clearly shows that the files are spread evenly across them.
> 
> mfsfileinfo shows that the chunks are placed alternatingly on the chunk servers.
> Incrementing "goal" only increases the number of copies.
> 
> Screenshots can be supplied on tuesday since our server room is under reconstruction over the weekend.
> 
> Heiko
> 

Ok here we go:

Installed OS,mfs on all systems:
2.6.36-gentoo-r5 x86_64
mfs-1.6.20
fuse-2.8.5

chunkserv1: 192.168.16.54
chunkserv2: 192.168.16.147

df on Chunkservers before cp:
chunkserv1 ~ # df -h /mnt/mfschunks*
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb              5.5T   97M  5.5T   1% /mnt/mfschunks1
/dev/sdc              5.5T   33M  5.5T   1% /mnt/mfschunks2
chunkserv2 ~ # df -h /mnt/mfschunks*
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb              2.8T   33M  2.8T   1% /mnt/mfschunks1
/dev/sdc              2.8T   33M  2.8T   1% /mnt/mfschunks2
/dev/sdd              2.8T   33M  2.8T   1% /mnt/mfschunks3



client ~ # mkdir /mnt/mfs/folder1   
client ~ # mfssetgoal -r 1 /mnt/mfs/folder1/
/mnt/mfs/folder1/:
 inodes with goal changed:                         0
 inodes with goal not changed:                     1
 inodes with permission denied:                    0

client ~ # dd if=/dev/urandom of=/tmp/ttt.dat bs=100M count=5
5+0 records in
5+0 records out
524288000 bytes (524 MB) copied, 83.5789 s, 6.3 MB/s
client ~ # cp /tmp/ttt.dat /mnt/mfs/folder1/ttt.dat

client ~ # mfsfileinfo /mnt/mfs/folder1/ttt.dat 
/mnt/mfs/folder1/ttt.dat:
        chunk 0: 0000000000000AC3_00000001 / (id:2755 ver:1)
                copy 1: 192.168.16.54:9422
        chunk 1: 0000000000000AC4_00000001 / (id:2756 ver:1)
                copy 1: 192.168.16.147:9422
        chunk 2: 0000000000000AC5_00000001 / (id:2757 ver:1)
                copy 1: 192.168.16.54:9422
        chunk 3: 0000000000000AC6_00000001 / (id:2758 ver:1)
                copy 1: 192.168.16.147:9422
        chunk 4: 0000000000000AC7_00000001 / (id:2759 ver:1)
                copy 1: 192.168.16.147:9422
        chunk 5: 0000000000000AC8_00000001 / (id:2760 ver:1)
                copy 1: 192.168.16.54:9422
        chunk 6: 0000000000000AC9_00000001 / (id:2761 ver:1)
                copy 1: 192.168.16.147:9422
        chunk 7: 0000000000000ACA_00000001 / (id:2762 ver:1)
                copy 1: 192.168.16.54:9422

client ~ # mfsgetgoal /mnt/mfs/folder1/ttt.dat 
/mnt/mfs/folder1/ttt.dat: 1
client ~ # mfsgetgoal /mnt/mfs/folder1         
/mnt/mfs/folder1: 1

df on Chunkservers after cp:
chunkserv1 ~ # df -h /mnt/mfschunks*
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb              5.5T  225M  5.5T   1% /mnt/mfschunks1
/dev/sdc              5.5T  161M  5.5T   1% /mnt/mfschunks2
chunkserv2 ~ # df -h /mnt/mfschunks*
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb              2.8T  149M  2.8T   1% /mnt/mfschunks1
/dev/sdc              2.8T   97M  2.8T   1% /mnt/mfschunks2
/dev/sdd              2.8T   97M  2.8T   1% /mnt/mfschunks3


Regards
Heiko

Re: [Moosefs-users] OpenOffice and MooseFS

[Steve W. <st...@pu...>]

On 02/28/2011 09:35 AM, Flow Jiang wrote:
> We see the same crash issue with MFS and OO 3.1 as described 
> http://openoffice.org/bugzilla/show_bug.cgi?id=113207&historysort=new. 
> And as all of our servers / workstations are CentOS 5.5 and Fedora 12, 
> it won't be an easy task to go OO 3.2.

 From what we've found, you would need to upgrade to OO 3.3 not just OO 3.2.

> A work around is to add following lines in 
> /usr/lib64/openoffice.org3/program/soffice:
>
> if [ -e /home3/$USER ] ; then
>     export HOME=/home3/$USER
> fi
>
> Our /home3 is the original NFS mounted home directory and it could 
> also be any other local file system directories.
>
> Flow
>
> On 02/28/2011 10:11 PM, Giovanni Toraldo wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Hi Steve,
>>
>> Il 25/02/2011 17:24, Steve Wilson ha scritto:
>>> I'm glad to hear that it's working for you but confused that a very
>>> similar configuration isn't working for us.
>> I've checked dmesg, and I found some:
>>
>> [262517.265151] oosplash.bin[6009]: segfault at b664bae0 ip 0804b5e5 sp
>> bf88f8a0 error 6 in oosplash.bin[8048000+6000]
>> [269154.491051] oosplash.bin[6693]: segfault at b65d2ae0 ip 0804b5e5 sp
>> bfc26060 error 6 in oosplash.bin[8048000+6000]
>> [269198.971353] oosplash.bin[6732]: segfault at b65beae0 ip 0804b5e5 sp
>> bfba7d80 error 6 in oosplash.bin[8048000+6000]
>>
>> however openoffice (1:3.2.0-7ubuntu4.2) seems working without any other
>> problem.
>>
>> Bye.
>>
>> - -- Giovanni Toraldo
>> http://www.libersoft.it/
>> -----BEGIN PGP SIGNATURE-----
>> Version: GnuPG v1.4.10 (GNU/Linux)
>> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>>
>> iQIcBAEBAgAGBQJNa6z1AAoJEGrHv689I8Z8UawQAJc/oRE2xNuEM4ZQ6ocwbp1Z
>> rCU0fshJ71TgYwTVrc5ASNCA37qa0RVydzeZAyEaONeUE+vfb+5q0VPgnPRSANDG
>> fXFOpFbf2sN9kkMlYtOhzSDLVu6UaoIbDkdRbMT6pr3Ux8q0hhqTMRN0/tI7zwkw
>> aqLdyi8pqctHNlEnGkOii7eV1oObNPKCHpNzWB2mMyOdY7kSpkbqPZ+rjZYc0EwK
>> e9YcEM7Tp3fceMsuQUIJnTNeFAnXlfIcRR+UvWXiRfeKW+RlV+CJ/N7+hJ4TQLJw
>> i4yJ9N+IWUoF3mlPLVnV8IjOz5aUoz+OzXuhBQZ+yJrXsGBxlm9zI1RoQHODx17L
>> psJzG88hOZzfmOebhq9CHWvFhDotHfMms3aYVmdvzezqarH3l031YePPXjsmtF6/
>> DHM30sloGUSNjM+qvapQpvgguGkshlggaBtg/fsoeuLPKTu3nbc/mEcK4GZ8hNio
>> toWPWh0QJ9yK58Hp5N+U/E3OP3ZWGGrWPtDF8BNz1LJQs9I8fPld/LQ3mIpQ2SGA
>> pd07Kck/FXsHaX+1VsE6HO6/EGIcGG/zOISsGgZVDTq0d21EgG/6Pk+hkEhi8+e8
>> LDOaeZUEbVS6NILpm0MtiyjEFMWx3oDjhRvzkbfoPMR9IDqqARDMgtF/Q/U9qXAN
>> bV6Z8u6q/VtZ01uKRM4/
>> =KFge
>> -----END PGP SIGNATURE-----
>>
>> ------------------------------------------------------------------------------ 
>>
>> Free Software Download: Index, Search&  Analyze Logs and other IT 
>> data in
>> Real-Time with Splunk. Collect, index and harness all the fast moving 
>> IT data
>> generated by your applications, servers and devices whether physical, 
>> virtual
>> or in the cloud. Deliver compliance at lower cost and gain new business
>> insights. http://p.sf.net/sfu/splunk-dev2dev
>> _______________________________________________
>> moosefs-users mailing list
>> moo...@li...
>> https://lists.sourceforge.net/lists/listinfo/moosefs-users

-- 
Steven M. Wilson, Systems and Network Manager
Markey Center for Structural Biology
Purdue University
(765) 496-1946

Re: [Moosefs-users] OpenOffice and MooseFS

[Giovanni T. <gt...@li...>]

Il 28/02/2011 15:35, Flow Jiang ha scritto:
> it won't be an easy task to go OO 3.2. A work around is to add following 
> lines in /usr/lib64/openoffice.org3/program/soffice:
> if [ -e /home3/$USER ] ; then
>      export HOME=/home3/$USER
> fi

On ubuntu, I added:
HOME="/tmp"
at the end of:
/etc/openoffice/soffice.sh

Bye.

-- 
Giovanni Toraldo
http://www.libersoft.it/