Menu
▾
▴
Re: [Moosefs-users] how many times chunckserver will retry when disconnecting
|
From: 丁赞 <di...@ba...> - 2010-12-15 01:57:24
|
In mfs_mount it is 31 times and I think the chunk would be the same value.
the mount reconnecting behavior would last about 30 minutes
Di...@ba...
-----邮件原件-----
发件人: moo...@li...
[mailto:moo...@li...]
发送时间: 2010年12月1日 16:06
收件人: moo...@li...
主题: moosefs-users Digest, Vol 12, Issue 1
Send moosefs-users mailing list submissions to
moo...@li...
To subscribe or unsubscribe via the World Wide Web, visit
https://lists.sourceforge.net/lists/listinfo/moosefs-users
or, via email, send a message with subject or body 'help' to
moo...@li...
You can reach the person managing the list at
moo...@li...
When replying, please edit your Subject line so it is more specific
than "Re: Contents of moosefs-users digest..."
Today's Topics:
1. Re: Problem with mfsmounts inside kvm (Thomas S Hatch)
2. about mfs_mount client performance (??)
3. Re: A problem of reading the same file at the same moment
(Micha? Borychowski)
4. Re: reset time count in trash ? (Micha? Borychowski)
5. Re: how many times chunckserver will retry when disconnecting
from metaserver ? (Micha? Borychowski)
----------------------------------------------------------------------
Message: 1
Date: Tue, 30 Nov 2010 09:34:23 -0700
From: Thomas S Hatch <tha...@gm...>
Subject: Re: [Moosefs-users] Problem with mfsmounts inside kvm
To: moosefs-users <moo...@li...>
Message-ID:
<AAN...@ma...>
Content-Type: text/plain; charset="iso-8859-1"
Ignore me :)
We had a problem with our routes, our vms could not see all of our chunk
servers.
On Tue, Nov 30, 2010 at 9:00 AM, Thomas S Hatch <tha...@gm...> wrote:
> I am experiencing problems with moosefs mounts inside of kvm virtual
> machines. There are a number of files which I cannot read on my mfs mount
> from inside a kvm virtual machine, (the read attempts hang and then issue
an
> IOError). But I am confident that the files are good because they can be
> read without problem from an mfsmount on a bare metal system. I am running
> moosefs 1.6.18 (a pre-release) on Ubuntu 10.04.
>
> Please let me know if there is any additional information I can send.
>
> -Thomas S Hatch
>
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
Message: 2
Date: Wed, 1 Dec 2010 12:05:12 +0800
From: ?? <di...@ba...>
Subject: [Moosefs-users] about mfs_mount client performance
To: <moo...@li...>
Message-ID: <086001cb910c$f40eb230$dc2c1690$@com>
Content-Type: text/plain; charset="gb2312"
Hi all
FUSE is a accept+fock module, which means if N clients(500+ httpd
threads in our enviroment) read data from mfs through one mfs_mount
simultaneously, FUSE could create N threads to handle the requests. This
mechanism spent much more time on context siwtch?which made apache
response time much more longer in our enviroment?SO anyone had tried to
promote the fuse or mfs_client performace? Using threads pool or user level
cache would work???
BTW?I am a new guy here??
Thanks all
DingZan @baidu.com
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
Message: 3
Date: Wed, 1 Dec 2010 08:53:36 +0100
From: Micha? Borychowski <mic...@ge...>
Subject: Re: [Moosefs-users] A problem of reading the same file at the
same moment
To: "'Laurent Wandrebeck'" <lw...@hy...>
Cc: moo...@li...
Message-ID: <000601cb912c$df7f94e0$9e7ebea0$@bor...@ge...>
Content-Type: text/plain; charset="utf-8"
Hi Laurent!
If master doesn't take full 100% of CPU it would work even better as one
thread. Generally speaking multithreading is often overestimated. Please
have a look at this article:
http://www.eecs.berkeley.edu/Pubs/TechRpts/2006/EECS-2006-1.pdf
Regards
Micha?
-----Original Message-----
From: Laurent Wandrebeck [mailto:lw...@hy...]
Sent: Friday, November 26, 2010 12:00 PM
To: moo...@li...
Subject: Re: [Moosefs-users] A problem of reading the same file at the same
moment
On Thu, 25 Nov 2010 10:55:03 +0100
Micha? Borychowski <mic...@ge...> wrote:
> Hi!
>
> As written on http://www.moosefs.org/moosefs-faq.html#goal increasing goal
may only increase the reading speed under certain conditions. You can just
try increasing the goal, wait for the replication and see if it helps.
I'm wondering if such a behaviour could be due to mfsmaster being a
monothread program. Thus, in high-load cases, the master being busy
answering a request kind of queues the others, being a performance
bottleneck by adding latency, preventing one than one request to be worked
on at the same time.
Comments ?
--
Laurent Wandrebeck
HYGEOS, Earth Observation Department / Observation de la Terre
Euratechnologies
165 Avenue de Bretagne
59000 Lille, France
tel: +33 3 20 08 24 98
http://www.hygeos.com
GPG fingerprint/Empreinte GPG: F5CA 37A4 6D03 A90C 7A1D 2A62 54E6 EF2C D17C
F64C
------------------------------
Message: 4
Date: Wed, 1 Dec 2010 08:59:39 +0100
From: Micha? Borychowski <mic...@ge...>
Subject: Re: [Moosefs-users] reset time count in trash ?
To: "'jose maria'" <let...@us...>
Cc: moo...@li...
Message-ID: <000701cb912d$b73dd630$25b98290$@bor...@ge...>
Content-Type: text/plain; charset="utf-8"
Hola Jose!
Yes, that's right - changing the goal for files in trash resets their timer.
We will look into it further and probably make a patch.
Saludos
Michal
-----Original Message-----
From: jose maria [mailto:let...@us...]
Sent: Saturday, November 27, 2010 4:23 PM
To: moo...@li...
Subject: Re: [Moosefs-users] reset time count in trash ?
El vie, 26-11-2010 a las 15:57 +0100, jose maria escribi?:
> * I have applied for testing, to the files of the cluster a trashtime of
> 12 hours, a script is executed every hour and reduces to 2 goals the
> files of the trash, for one week the number of files has been increasing
> in the trash and it continues without becoming stable, at present there
> is 400.000.-
>
> * ?Is it possible that on having applied setgoal 2, reset de time count?
>
> * The number of files in the reply of the secondary cluster with equal
> configuration of trashtime 12 hours is trivial, without execution of the
> script that applies mfssetgoal 2.
>
* Confirmed, I have disabled the cronjob that applies mfssetgoal to the
files in trash and in 12 hours the number of files has reduced from
375.000 to 1.500. ?any other idea to reduce goals of trashfiles? I need
1 week retained trashfiles ......
----------------------------------------------------------------------------
--
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users
------------------------------
Message: 5
Date: Wed, 1 Dec 2010 09:06:11 +0100
From: Micha? Borychowski <mic...@ge...>
Subject: Re: [Moosefs-users] how many times chunckserver will retry
when disconnecting from metaserver ?
To: "'kuer ku'" <ku...@gm...>
Cc: moo...@li...
Message-ID: <000b01cb912e$a0e50d80$e2af2880$@bor...@ge...>
Content-Type: text/plain; charset="utf-8"
Hi!
You have here some problem with starting the chunkserver. But for sure this
has no connection with the TIMEMODE_RUNONCE constant (it says that after
clock time change sth has to be run once and not 60 times).
We also had some very rare cases that while starting master server,
chunkserver got hung up. In 1.6.18 this problem should be eliminated.
Kind regards
Michal
From: kuer ku [mailto:ku...@gm...]
Sent: Monday, November 29, 2010 12:18 PM
To: moo...@li...
Subject: [Moosefs-users] how many times chunckserver will retry when
disconnecting from metaserver ?
Hi, all,
I deloyed a mfs-1.6.15 in my environment, today I found a problem. The
appearance is one of mfsmount (FUSE) complained that :
Nov 29 18:26:11 storage04 mfsmount[32233]: file: 43, index: 0 - can't
connect to proper chunkserver (try counter: 29)
I donot know which chunkserver cause this. ???
On web interface, I found storage01, one of chunkservers, is not in the
server list. and on storage01, there are some logs in /var/log/messages :
Nov 29 14:43:27 storage01 mfsmount[13155]: master: connection lost (1)
Nov 29 14:43:27 storage01 mfsmount[13155]: registered to master
Nov 29 14:44:12 storage01 mfschunkserver[11730]: Master connection lost
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
mfschunkserver found connection
lost, but there no logs indicate that mfschunkserver try to reconnect with
master
Nov 29 15:07:44 storage01 smartd[4268]: System clock time adjusted to the
past. Resetting next wakeup time.
# the following log happened because I restart chunkserver forcely
Nov 29 18:29:59 storage01 h*U?2[11730]: closing *:19722
Nov 29 18:30:13 storage01 mfschunkserver[6764]: listen on *:19722
Nov 29 18:30:13 storage01 mfschunkserver[6764]: connecting ...
Nov 29 18:30:13 storage01 mfschunkserver[6764]: open files limit: 10000
Nov 29 18:30:13 storage01 mfschunkserver[6764]: connected to Master
and in chunkserver/masterconn.c , I found codes :
1311 main_eachloopregister(masterconn_check_hdd_reports);
1312
main_timeregister(TIMEMODE_RUNONCE,ReconnectionDelay,0,masterconn_reconnect)
;
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
^^^^^^^^^^ it will try to reconnect once ??????
1313 main_destructregister(masterconn_term);
1314 main_pollregister(masterconn_desc,masterconn_serve);
1315 main_reloadregister(masterconn_reload);
I think chunkserver should re-connect to master again and again, until it
reachs master. but I does not find that in the code.
P.S. I remember that I adjust storage01 's time by ntpdate dateserver. does
this affect chunckserver so seriously ??
thanks
-- kuer
-------------- next part --------------
An HTML attachment was scrubbed...
------------------------------
----------------------------------------------------------------------------
--
Increase Visibility of Your 3D Game App & Earn a Chance To Win $500!
Tap into the largest installed PC base & get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
------------------------------
_______________________________________________
moosefs-users mailing list
moo...@li...
https://lists.sourceforge.net/lists/listinfo/moosefs-users
End of moosefs-users Digest, Vol 12, Issue 1
********************************************
|
