Re: [Moosefs-users] what should I do IF cannot shutdown master normally ??

[Michał B. <mic...@ge...>]

These messages (143616569: 1280322540|EMPTYRESERVED():0) are normal ones.
Functions run periodically delete free i-nodes, deleted and "reserved"
files. 0 means there was nothing to do.

 

It may happen that there are still some operations on the clients' side. If
nothing else happens you can always kill the master server process (in this
case it would be "kill -9 23172"), wait till the process ends, run
"mfsmetarestore -a" and run the master again by: "mfsmaster start".

 

Are there any other messages in syslog close to these operations?

 

 

Kind regards

Michał Borychowski 

MooseFS Support Manager

_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

Gemius S.A.

ul. Wołoska 7, 02-672 Warszawa

Budynek MARS, klatka D

Tel.: +4822 874-41-00

Fax : +4822 874-41-01  

 

 

 

From: kuer ku [mailto:ku...@gm...] 
Sent: Wednesday, July 28, 2010 3:22 PM
To: moo...@li...
Subject: Re: [Moosefs-users] what should I do IF cannot shutdown master
normally ??

 

hi, all,

 

I find some strange things in changelog,

 

 

$ tail -f changelog.0.mfs 

143616569: 1280322540|EMPTYRESERVED():0

143616570: 1280322600|FREEINODES():0

143616571: 1280322600|EMPTYRESERVED():0

143616572: 1280322600|EMPTYTRASH():0,0

143616573: 1280322660|FREEINODES():0

143616574: 1280322660|EMPTYRESERVED():0

143616575: 1280322720|FREEINODES():0

143616576: 1280322720|EMPTYRESERVED():0

143616577: 1280322780|FREEINODES():0

143616578: 1280322780|EMPTYRESERVED():0

 

 

what does this means, how to fix it ?

 

Jul 28 21:19:00 meta1 mfsmaster[23172]: chunkservers status:

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 1 (ip: 221.194.134.189, port:
19322): usedspace: 943864180736 (879.04 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 62.92%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 2 (ip: 221.194.134.187, port:
19322): usedspace: 957016182784 (891.29 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 63.79%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 3 (ip: 221.194.134.181, port:
19322): usedspace: 1898559021056 (1768.17 GiB), totalspace: 3000328257536
(2794.27 GiB), usage: 63.28%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 4 (ip: 221.194.134.186, port:
19322): usedspace: 940963352576 (876.34 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 62.72%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 5 (ip: 221.194.134.184, port:
19322): usedspace: 944276942848 (879.43 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 62.94%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 6 (ip: 221.194.134.190, port:
19322): usedspace: 1893327695872 (1763.30 GiB), totalspace: 3000328257536
(2794.27 GiB), usage: 63.10%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 7 (ip: 221.194.134.188, port:
19322): usedspace: 957261549568 (891.52 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 63.81%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 8 (ip: 221.194.134.185, port:
19322): usedspace: 957269495808 (891.53 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 63.81%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 9 (ip: 221.194.134.183, port:
19322): usedspace: 957314211840 (891.57 GiB), totalspace: 1500164128768
(1397.14 GiB), usage: 63.81%

Jul 28 21:19:00 meta1 mfsmaster[23172]: server 10 (ip: 221.194.134.182,
port: 19322): usedspace: 956960980992 (891.24 GiB), totalspace:
1500164128768 (1397.14 GiB), usage: 63.79%

Jul 28 21:19:00 meta1 mfsmaster[23172]: total: usedspace: 11406813614080
(10623.42 GiB), totalspace: 18001969545216 (16765.64 GiB), usage: 63.36%

 

there are still free space on chunkservers ( almost 40% free space )

 

what resources runs out ???

 

how to fix them ?

 

thanks

 

  -- kuer

 

 

On Wed, Jul 28, 2010 at 8:59 PM, kuer ku <ku...@gm...> wrote:

hi,

 

I just want to know why master stucked, but I find nothing wrong in
/var/log/messages.

 

>From /var/log/messages, it seems that master still work, but why it does NOT
exit when got SIGTERM ?

 

Are there any other way that I can find some useful messages ?

 

thanks 

 

    -- kuer

 

On Wed, Jul 28, 2010 at 8:52 PM, kuer ku <ku...@gm...> wrote:

hi, all

 

I cannot shutdown moosefs master normally. When shutdown, it shows :

 

working directory: /usr/local/moosefs/bin/master

sending SIGTERM to lock owner (pid:23172)

waiting for termination ... 10s 20s 30s 40s 50s give up

 

Something must be wrong, what should I do ?

 

thanks

 

   -- kuer