Menu
▾
▴
Re: [Moosefs-users] ChunkServer in different Level
|
From: Michał B. <mic...@co...> - 2012-05-04 09:04:01
|
Hi Ken! The idea would be very similar but we need to make our implementation of this and do some tests. Kind regards Michał From: Ken [mailto:ken...@gm...] Sent: Friday, May 04, 2012 10:23 AM To: Michał Borychowski Subject: Re: [Moosefs-users] ChunkServer in different Level Michal, I notice the blog Rack Awareness <http://www.moosefs.org/news-reader/items/rack-awareness.html> : The behaviour of choosing where to create the new chunks will be introduced as "level goal" functionality in an upcoming release. Is this 'level goal' same as 'ChunkServer in different Level'? Thanks -Ken On Mon, Jan 30, 2012 at 6:38 PM, Ken <ken...@gm...> wrote: hi, I am very glad to hear that. Patch attached, and web version via github: https://github.com/pedia/moosefs/commit/74bc2f498fd218ea6aa51ccd0779d0e154da 09ed And hope to help more if you need. Best Regards -Ken 2012/1/30 Michał Borychowski <mic...@ge...>: > Hi Ken > > Our developers took a closer look at your solution and it is very interesting. We'd like to incorporate your code (after some little changes) into our main branch - don't you have anything against it? If not, please send your changes in a form of a patch. > > Thank you > > > Kind regards > Michał Borychowski > MooseFS Support Manager > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > Gemius S.A. > ul. Wołoska 7, 02-672 Warszawa > Budynek MARS, klatka D > Tel.: +4822 874-41-00 > Fax : +4822 874-41-01 > > > -----Original Message----- > From: Ken [mailto:ken...@gm...] > Sent: Monday, January 16, 2012 2:06 AM > To: Michał Borychowski > Cc: moo...@li... > Subject: Re: [Moosefs-users] ChunkServer in different Level > > Hi, Michał > > I push my scripts to github: > https://github.com/pedia/moosefs/tree/master/failover-script > > It really work to me. > > Sorry for late reply, because weekend. > > Regards > -Ken > > > > 2012/1/13 Michał Borychowski <mic...@ge...>: >> Hi Ken! >> >> It would be great if you could also provide the group with your >> scripts for auto recovery of the system. >> >> >> Kind regards >> Michał >> >> >> -----Original Message----- >> From: Ken [mailto:ken...@gm...] >> Sent: Friday, January 13, 2012 8:59 AM >> To: Davies Liu >> Cc: moo...@li... >> Subject: Re: [Moosefs-users] ChunkServer in different Level >> >> We also use ucarp, but changed a little from Thomas version. The auto >> switching never failed. >> >> A friend of mine use DRBD and LVS. It also work fine, but I think it >> is much smaller than the Douban's. >> >> -Ken >> >> >> >> On Fri, Jan 13, 2012 at 3:42 PM, Davies Liu <dav...@gm...> wrote: >>> On Fri, Jan 13, 2012 at 3:25 PM, Ken <ken...@gm...> wrote: >>>> I noticed the go-mfsclient in this mail-list before, we also write a >>>> moose client in C++. ;-) It work find when mfsmaster failover. >>> >>> It's the most interesting part, how do you failover mfsmaster ? >>> I have tried the method with ucarp, provided by Thomas S Hatch, It's >>> seemed not stable enough, failed sometimes. >>> >>> Before a stable solution come up, we decide to do manually fail-over >>> by ops, and do not deploy it in heavy online system. >>> >>>> We plan to build it as a preload dynamic library, and auto hook the >>>> file >> API. >>>> I think it is high availability enough. >>>> >>>> Thanks >>>> >>>> Regards >>>> -Ken >>>> >>>> >>>> >>>> On Fri, Jan 13, 2012 at 2:52 PM, Davies Liu <dav...@gm...> wrote: >>>>> On Fri, Jan 13, 2012 at 2:40 PM, Ken <ken...@gm...> wrote: >>>>>>>> It's not good ideal to use moosefs as storage for huge amount of >>>>>>>> small files >>>>>> I agree. We combine some small files into one big file, and read >>>>>> the small files with offset/length infomation. >>>>> >>>>> Is not safe to write to same file concurrently. >>>>> We use this method to backup the original file user uploaded, with >>>>> tar, when offline. >>>>> Some times, some file will be broken. >>>>> >>>>> MFS is not good enough for online system, not high available, and >>>>> some IO operations will be block when error in mfsmaster or >> mfschunkserver. >>>>> >>>>> So we serve some video files (>10M) in MFS this way: >>>>> Nginx -> nginx + FUSE -> MFS >>>>> or >>>>> Nginx -> go-mfsclient [1] -> MFS >>>>> >>>>> If there something wrong with MFS, it will not block the first >>>>> Nginx and the whole site will not be affected. >>>>> >>>>> Davies >>>>> >>>>> [1] github.com/davies/go-mfsclient >>>>> >>>>>> Thanks. >>>>>> >>>>>> Regards >>>>>> -Ken >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Jan 13, 2012 at 2:32 PM, Davies Liu <dav...@gm...> >> wrote: >>>>>>> On Thu, Jan 12, 2012 at 5:28 PM, Ken <ken...@gm...> wrote: >>>>>>>> hi, moosefs >>>>>>>> >>>>>>>> We plan to use moosefs as storage for huge amount photos >>>>>>>> uploaded by >> users. >>>>>>> >>>>>>> It's not good ideal to use moosefs as storage for huge amount of >>>>>>> small files, because the mfsmaster will be the bottle neck when >>>>>>> you have more than 100M files. At that time, the whole size of >>>>>>> files may be 1T (10k per file), can be stored by one local disk. >>>>>>> >>>>>>> Huge amount small files need other solutions, just like TFS [1] >>>>>>> from taobao.com, or beansdb [2] from douban.com. >>>>>>> >>>>>>> [1] http://code.taobao.org/p/tfs/src/ [2] >>>>>>> http://code.google.com/p/beansdb/ >>>>>>> >>>>>>>> Because of read operations of new files are very more than old >>>>>>>> files, maybe write new files to SSD is a choice. >>>>>>>> For strict safe reason, we must backup content to an other data >> center. >>>>>>>> And more features in maintain purpose are required. >>>>>>>> >>>>>>>> I don't think moosefs can work fine in these situation. We try >>>>>>>> to implement these features several weeks ago. Till now, it's >>>>>>>> almost done. >>>>>>>> >>>>>>>> Is there anyone interested in this? >>>>>>>> >>>>>>>> more detail: >>>>>>>> # Add access_mode(none, read, write capability) to struct >>>>>>>> matocserventry(matocserv.c). This value can be changed from >>>>>>>> outside(maybe from the python cgi) # mfschunkserver.cfg add >>>>>>>> 'LEVEL' config, if not, LEVEL=0 as normal. >>>>>>>> ChunkServer report it to Master if need. >>>>>>>> # Add uint32_t levelgoal into struct fsnode(filesystem.c). >>>>>>>> # Add uint32_t levelgoal into sturct chunk(chunk.c). >>>>>>>> As seen, uint32_t levelgoal = uint8_t levelgoal[4], implied >>>>>>>> LEVEL should be 1,2,3 or 4. >>>>>>>> [2,1,0,0] mean store 2 copies in level=1 ChunkServer, store 1 >>>>>>>> copy in >>>>>>>> level=2 ChunkServer. >>>>>>>> # In chunk_do_jobs(chunk.c), send replicated command to ChunkServer. >>>>>>>> This policy should be very complicated in future. >>>>>>>> # Also, we add read/write levelgoal support in mfstools. >>>>>>>> >>>>>>>> We plan to put these trivial change into github or somewhere else. >>>>>>>> >>>>>>>> It's a very incipient prototype. We appreciate any advice from >>>>>>>> the develop team and other users. >>>>>>>> >>>>>>>> Regards >>>>>>>> -Ken >>>>>>>> >>>>>>>> ---------------------------------------------------------------- >>>>>>>> - >>>>>>>> ------------- >>>>>>>> RSA(R) Conference 2012 >>>>>>>> Mar 27 - Feb 2 >>>>>>>> Save $400 by Jan. 27 >>>>>>>> Register now! >>>>>>>> http://p.sf.net/sfu/rsa-sfdev2dev2 >>>>>>>> _______________________________________________ >>>>>>>> moosefs-users mailing list >>>>>>>> moo...@li... >>>>>>>> https://lists.sourceforge.net/lists/listinfo/moosefs-users >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> - Davies >>>>> >>>>> >>>>> >>>>> -- >>>>> - Davies >>> >>> >>> >>> -- >>> - Davies >> >> ---------------------------------------------------------------------- >> ------ >> -- >> RSA(R) Conference 2012 >> Mar 27 - Feb 2 >> Save $400 by Jan. 27 >> Register now! >> http://p.sf.net/sfu/rsa-sfdev2dev2 >> _______________________________________________ >> moosefs-users mailing list >> moo...@li... >> https://lists.sourceforge.net/lists/listinfo/moosefs-users >> > |
