Menu
▾
▴
Re: [Moosefs-users] ChunkServer in different Level
|
From: Michał B. <mic...@ge...> - 2012-01-13 09:20:01
|
Hi Ken! It would be great if you could also provide the group with your scripts for auto recovery of the system. Kind regards Michał -----Original Message----- From: Ken [mailto:ken...@gm...] Sent: Friday, January 13, 2012 8:59 AM To: Davies Liu Cc: moo...@li... Subject: Re: [Moosefs-users] ChunkServer in different Level We also use ucarp, but changed a little from Thomas version. The auto switching never failed. A friend of mine use DRBD and LVS. It also work fine, but I think it is much smaller than the Douban's. -Ken On Fri, Jan 13, 2012 at 3:42 PM, Davies Liu <dav...@gm...> wrote: > On Fri, Jan 13, 2012 at 3:25 PM, Ken <ken...@gm...> wrote: >> I noticed the go-mfsclient in this mail-list before, we also write a >> moose client in C++. ;-) It work find when mfsmaster failover. > > It's the most interesting part, how do you failover mfsmaster ? > I have tried the method with ucarp, provided by Thomas S Hatch, It's > seemed not stable enough, failed sometimes. > > Before a stable solution come up, we decide to do manually fail-over > by ops, and do not deploy it in heavy online system. > >> We plan to build it as a preload dynamic library, and auto hook the file API. >> I think it is high availability enough. >> >> Thanks >> >> Regards >> -Ken >> >> >> >> On Fri, Jan 13, 2012 at 2:52 PM, Davies Liu <dav...@gm...> wrote: >>> On Fri, Jan 13, 2012 at 2:40 PM, Ken <ken...@gm...> wrote: >>>>>> It's not good ideal to use moosefs as storage for huge amount of >>>>>> small files >>>> I agree. We combine some small files into one big file, and read >>>> the small files with offset/length infomation. >>> >>> Is not safe to write to same file concurrently. >>> We use this method to backup the original file user uploaded, with >>> tar, when offline. >>> Some times, some file will be broken. >>> >>> MFS is not good enough for online system, not high available, and >>> some IO operations will be block when error in mfsmaster or mfschunkserver. >>> >>> So we serve some video files (>10M) in MFS this way: >>> Nginx -> nginx + FUSE -> MFS >>> or >>> Nginx -> go-mfsclient [1] -> MFS >>> >>> If there something wrong with MFS, it will not block the first Nginx >>> and the whole site will not be affected. >>> >>> Davies >>> >>> [1] github.com/davies/go-mfsclient >>> >>>> Thanks. >>>> >>>> Regards >>>> -Ken >>>> >>>> >>>> >>>> On Fri, Jan 13, 2012 at 2:32 PM, Davies Liu <dav...@gm...> wrote: >>>>> On Thu, Jan 12, 2012 at 5:28 PM, Ken <ken...@gm...> wrote: >>>>>> hi, moosefs >>>>>> >>>>>> We plan to use moosefs as storage for huge amount photos uploaded by users. >>>>> >>>>> It's not good ideal to use moosefs as storage for huge amount of >>>>> small files, because the mfsmaster will be the bottle neck when >>>>> you have more than 100M files. At that time, the whole size of >>>>> files may be 1T (10k per file), can be stored by one local disk. >>>>> >>>>> Huge amount small files need other solutions, just like TFS [1] >>>>> from taobao.com, or beansdb [2] from douban.com. >>>>> >>>>> [1] http://code.taobao.org/p/tfs/src/ [2] >>>>> http://code.google.com/p/beansdb/ >>>>> >>>>>> Because of read operations of new files are very more than old >>>>>> files, maybe write new files to SSD is a choice. >>>>>> For strict safe reason, we must backup content to an other data center. >>>>>> And more features in maintain purpose are required. >>>>>> >>>>>> I don't think moosefs can work fine in these situation. We try to >>>>>> implement these features several weeks ago. Till now, it's almost >>>>>> done. >>>>>> >>>>>> Is there anyone interested in this? >>>>>> >>>>>> more detail: >>>>>> # Add access_mode(none, read, write capability) to struct >>>>>> matocserventry(matocserv.c). This value can be changed from >>>>>> outside(maybe from the python cgi) # mfschunkserver.cfg add >>>>>> 'LEVEL' config, if not, LEVEL=0 as normal. >>>>>> ChunkServer report it to Master if need. >>>>>> # Add uint32_t levelgoal into struct fsnode(filesystem.c). >>>>>> # Add uint32_t levelgoal into sturct chunk(chunk.c). >>>>>> As seen, uint32_t levelgoal = uint8_t levelgoal[4], implied >>>>>> LEVEL should be 1,2,3 or 4. >>>>>> [2,1,0,0] mean store 2 copies in level=1 ChunkServer, store 1 >>>>>> copy in >>>>>> level=2 ChunkServer. >>>>>> # In chunk_do_jobs(chunk.c), send replicated command to ChunkServer. >>>>>> This policy should be very complicated in future. >>>>>> # Also, we add read/write levelgoal support in mfstools. >>>>>> >>>>>> We plan to put these trivial change into github or somewhere else. >>>>>> >>>>>> It's a very incipient prototype. We appreciate any advice from >>>>>> the develop team and other users. >>>>>> >>>>>> Regards >>>>>> -Ken >>>>>> >>>>>> ----------------------------------------------------------------- >>>>>> ------------- >>>>>> RSA(R) Conference 2012 >>>>>> Mar 27 - Feb 2 >>>>>> Save $400 by Jan. 27 >>>>>> Register now! >>>>>> http://p.sf.net/sfu/rsa-sfdev2dev2 >>>>>> _______________________________________________ >>>>>> moosefs-users mailing list >>>>>> moo...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/moosefs-users >>>>> >>>>> >>>>> >>>>> -- >>>>> - Davies >>> >>> >>> >>> -- >>> - Davies > > > > -- > - Davies ---------------------------------------------------------------------------- -- RSA(R) Conference 2012 Mar 27 - Feb 2 Save $400 by Jan. 27 Register now! http://p.sf.net/sfu/rsa-sfdev2dev2 _______________________________________________ moosefs-users mailing list moo...@li... https://lists.sourceforge.net/lists/listinfo/moosefs-users |
