Menu
▾
▴
Re: [Moosefs-users] ChunkServer in different Level
|
From: Ken <ken...@gm...> - 2012-01-16 01:06:10
|
Hi, Michał I push my scripts to github: https://github.com/pedia/moosefs/tree/master/failover-script It really work to me. Sorry for late reply, because weekend. Regards -Ken 2012/1/13 Michał Borychowski <mic...@ge...>: > Hi Ken! > > It would be great if you could also provide the group with your scripts for > auto recovery of the system. > > > Kind regards > Michał > > > -----Original Message----- > From: Ken [mailto:ken...@gm...] > Sent: Friday, January 13, 2012 8:59 AM > To: Davies Liu > Cc: moo...@li... > Subject: Re: [Moosefs-users] ChunkServer in different Level > > We also use ucarp, but changed a little from Thomas version. The auto > switching never failed. > > A friend of mine use DRBD and LVS. It also work fine, but I think it is much > smaller than the Douban's. > > -Ken > > > > On Fri, Jan 13, 2012 at 3:42 PM, Davies Liu <dav...@gm...> wrote: >> On Fri, Jan 13, 2012 at 3:25 PM, Ken <ken...@gm...> wrote: >>> I noticed the go-mfsclient in this mail-list before, we also write a >>> moose client in C++. ;-) It work find when mfsmaster failover. >> >> It's the most interesting part, how do you failover mfsmaster ? >> I have tried the method with ucarp, provided by Thomas S Hatch, It's >> seemed not stable enough, failed sometimes. >> >> Before a stable solution come up, we decide to do manually fail-over >> by ops, and do not deploy it in heavy online system. >> >>> We plan to build it as a preload dynamic library, and auto hook the file > API. >>> I think it is high availability enough. >>> >>> Thanks >>> >>> Regards >>> -Ken >>> >>> >>> >>> On Fri, Jan 13, 2012 at 2:52 PM, Davies Liu <dav...@gm...> wrote: >>>> On Fri, Jan 13, 2012 at 2:40 PM, Ken <ken...@gm...> wrote: >>>>>>> It's not good ideal to use moosefs as storage for huge amount of >>>>>>> small files >>>>> I agree. We combine some small files into one big file, and read >>>>> the small files with offset/length infomation. >>>> >>>> Is not safe to write to same file concurrently. >>>> We use this method to backup the original file user uploaded, with >>>> tar, when offline. >>>> Some times, some file will be broken. >>>> >>>> MFS is not good enough for online system, not high available, and >>>> some IO operations will be block when error in mfsmaster or > mfschunkserver. >>>> >>>> So we serve some video files (>10M) in MFS this way: >>>> Nginx -> nginx + FUSE -> MFS >>>> or >>>> Nginx -> go-mfsclient [1] -> MFS >>>> >>>> If there something wrong with MFS, it will not block the first Nginx >>>> and the whole site will not be affected. >>>> >>>> Davies >>>> >>>> [1] github.com/davies/go-mfsclient >>>> >>>>> Thanks. >>>>> >>>>> Regards >>>>> -Ken >>>>> >>>>> >>>>> >>>>> On Fri, Jan 13, 2012 at 2:32 PM, Davies Liu <dav...@gm...> > wrote: >>>>>> On Thu, Jan 12, 2012 at 5:28 PM, Ken <ken...@gm...> wrote: >>>>>>> hi, moosefs >>>>>>> >>>>>>> We plan to use moosefs as storage for huge amount photos uploaded by > users. >>>>>> >>>>>> It's not good ideal to use moosefs as storage for huge amount of >>>>>> small files, because the mfsmaster will be the bottle neck when >>>>>> you have more than 100M files. At that time, the whole size of >>>>>> files may be 1T (10k per file), can be stored by one local disk. >>>>>> >>>>>> Huge amount small files need other solutions, just like TFS [1] >>>>>> from taobao.com, or beansdb [2] from douban.com. >>>>>> >>>>>> [1] http://code.taobao.org/p/tfs/src/ [2] >>>>>> http://code.google.com/p/beansdb/ >>>>>> >>>>>>> Because of read operations of new files are very more than old >>>>>>> files, maybe write new files to SSD is a choice. >>>>>>> For strict safe reason, we must backup content to an other data > center. >>>>>>> And more features in maintain purpose are required. >>>>>>> >>>>>>> I don't think moosefs can work fine in these situation. We try to >>>>>>> implement these features several weeks ago. Till now, it's almost >>>>>>> done. >>>>>>> >>>>>>> Is there anyone interested in this? >>>>>>> >>>>>>> more detail: >>>>>>> # Add access_mode(none, read, write capability) to struct >>>>>>> matocserventry(matocserv.c). This value can be changed from >>>>>>> outside(maybe from the python cgi) # mfschunkserver.cfg add >>>>>>> 'LEVEL' config, if not, LEVEL=0 as normal. >>>>>>> ChunkServer report it to Master if need. >>>>>>> # Add uint32_t levelgoal into struct fsnode(filesystem.c). >>>>>>> # Add uint32_t levelgoal into sturct chunk(chunk.c). >>>>>>> As seen, uint32_t levelgoal = uint8_t levelgoal[4], implied >>>>>>> LEVEL should be 1,2,3 or 4. >>>>>>> [2,1,0,0] mean store 2 copies in level=1 ChunkServer, store 1 >>>>>>> copy in >>>>>>> level=2 ChunkServer. >>>>>>> # In chunk_do_jobs(chunk.c), send replicated command to ChunkServer. >>>>>>> This policy should be very complicated in future. >>>>>>> # Also, we add read/write levelgoal support in mfstools. >>>>>>> >>>>>>> We plan to put these trivial change into github or somewhere else. >>>>>>> >>>>>>> It's a very incipient prototype. We appreciate any advice from >>>>>>> the develop team and other users. >>>>>>> >>>>>>> Regards >>>>>>> -Ken >>>>>>> >>>>>>> ----------------------------------------------------------------- >>>>>>> ------------- >>>>>>> RSA(R) Conference 2012 >>>>>>> Mar 27 - Feb 2 >>>>>>> Save $400 by Jan. 27 >>>>>>> Register now! >>>>>>> http://p.sf.net/sfu/rsa-sfdev2dev2 >>>>>>> _______________________________________________ >>>>>>> moosefs-users mailing list >>>>>>> moo...@li... >>>>>>> https://lists.sourceforge.net/lists/listinfo/moosefs-users >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> - Davies >>>> >>>> >>>> >>>> -- >>>> - Davies >> >> >> >> -- >> - Davies > > ---------------------------------------------------------------------------- > -- > RSA(R) Conference 2012 > Mar 27 - Feb 2 > Save $400 by Jan. 27 > Register now! > http://p.sf.net/sfu/rsa-sfdev2dev2 > _______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > |
