From: boblin [mailto:ai...@qq...]
Sent: Friday, April 15, 2011 5:00 PM
To: Michal Borychowski
Subject: Re:RE: [Moosefs-users] Questions about MooseFS before putting it
on the production env
Thanks for reply.
That article really helps me a lot , It's very very useful and most of my
puzzles are now gone ^_^ 。I've translated it into Chinese ,
How can I put it on the website to help others ?
[MB] That’s very nice! We are preparing a new version (1.6.21) and I would
have to make some updates to the article and then we could publish it in
Chinese, too.
In addition I'd like to test Thomas's patch , but it seems no any doc in
the tarball , I don't know how to use it ...
[MB] Thomas lately informed me that there is some error and the project
needs some time to get better.
Still have some sugguestions and questions , Here down below :
----------------------------------------------------------------------------
----------------------------
#### Questions
----------------------------------------------------------------------------
----------------------------
Question#1 : I can't still find any option to change the behavior for
dumping metadata in RAM to files in mfsmaster.cfg . -_-!
In that article , you said it's one hour by default .
[MB] Yes, this is one hour by default and there is no way to change it in
the config files. Honestly you do not need to change it as changelogs run
“online”.
Question#2 : file1's size is 1GB ,and file2's size is 1MB. Do they cost the
same memory for metadata in RAM ?
Is there any formula to estimate the memory usage ?
[MB] Yes, they use the same memory in RAM. 1 million files takes
approximately 300 MiB of RAM.
Question#3 : Does metalogger downloads just the difference of
metadata.mfs.back against the master's ( like Rsync ) ,
or simply the full file ?
[MB] Metalogger downloads full metadata.mfs file every 24hours and
constantly gets changelog files.
Question#4 : When client-A request file1 , can it read parallel from
multile chunkservers , just like RAID-0 ?
[MB] MooseFS could read files like this, but it doesn’t do this. It could
speed up single operations, but globally (due to lots of seeks on hard
drives) the performance of the system would reduce.
----------------------------------------------------------------------------
--------------------------------------------------------
#### Suggestions
----------------------------------------------------------------------------
--------------------------------------------------------
1、By now I can only get statis from mfscgiserv through web interface，Had
you ever considered to provide a cmd-line tool like
"squidclient mgr:info|5min" ， so I can just write a shell script and
send those data to my alarm system without watching the
graph "periodically & manually" ?
[MB] We have on our roadmap so called "mfstools” and we would introduce
such functionalities into them.
2、Why mfscgiserver can only just run on mfsmaster ? can It query the
master' metadata remotely ?
[MB] You can run mfscgiserver on any machine. In mfs.cgi you just need to
enter proper IP address of the master. Just running it on master is
convenient.
You can also run files from “PREFIX/share/mfscgi/” on any HTTP server
which supports CGI (Apache,Lighttpd,Nginx, etc.). See examples in
“index.html”
Kind regards
Michal
Best Regard
Boblin
---------------------------------------------------------------
Tencent Company ,
High-Tech park , Shenzhen , Guangdong Province
China
------------------ Original ------------------
From: "Michal Borychowski"<mic...@ge...>;
Date: Fri, Apr 15, 2011 08:18 PM
To: "'ailms'"<ai...@qq...>;
Cc: "'moosefs-users'"<moo...@li...>;
Subject: RE: [Moosefs-users] Questions about MooseFS before putting it on
the production env
Hi Bob!
My replies are in the text. And for start please read this article:
http://www.moosefs.org/news-reader/items/metadata-ins-and-outs.html
From: ailms [mailto:ai...@qq...]
Sent: Thursday, April 14, 2011 5:52 PM
To: moosefs-users
Subject: [Moosefs-users] Questions about MooseFS before putting it on the
production env
Dear guys,
We are plan to use moosefs as our store mechanism, which will be more than
5TB off-line command log.
But now we have someconcern, Please find my queries below:
-----
1. is there any plan about resolving the single-point failure of master ?
[MB] Please check a project developed by Thomas Hatch:
https://github.com/thatch45/mfs-failover
It still needs some improvements but is a great way to test the failovers.
And Thomas needs some beta testers.
2. The data will lost if master crash without dumping metadata from RAM
into disk,is it right?
[MB] Changelogs are saved continuously on the fly, there is very little
probability you would lose your data upon crash of the master server
3. How can I know which data were lost if the question #2 happend?
[MB] When you make a recovery process of metadata you would know if it was
successful or not
4. Dose the master only caches metadata read-only or will update the data
in RAM and flush to disk periodically?
[MB] The changes to metadata are saved to changelogs continuously on the fly
5. Are the change logs based on the metadata.mfs.back or the data in RAM ?
[MB] We can consider them as based on RAM
6. Can I change the default dumpling behaviour for master metadata from
per hour to user defined value? Such as 30 min?
[MB] As far as I remember, yes you can
7. How can we make sure no data lost at any time with HA（KeepAlived）+
DRBD + mfs solution ?
[MB] We have not done such tests. Please first check a solution given in
question 1.
8. The mfsmetaloggr didn't re-connect automatically even though I had
stoped and restarted the mfsmater, can only by manual?
[MB] Should have reconnected automatically. What was your environment?
9. which option in mfsmaster.cfg controls the rotation of changelog ?
according to the size ? or each time when mfsmaster dump the metadata ?
[MB] Please see "man mfsmaster.cfg” - you should have all the options
listed
Kind regards
Michał Borychowski
MooseFS Support Manager
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Gemius S.A.
ul. Wołoska 7, 02-672 Warszawa
Budynek MARS, klatka D
Tel.: +4822 874-41-00
Fax : +4822 874-41-01
Any help will be appreciated. Looking forward your answer.
Best Regard
Bob Lin
|
