Menu
▾
▴
Re: [Moosefs-users] ChunkServer in different Level
|
From: Ken <ken...@gm...> - 2012-05-04 09:18:07
|
Would you please publish more detail about it? or publish the source repository in developing? We very much hope will help to do some testing. -Ken On Fri, May 4, 2012 at 5:05 PM, Michał Borychowski < mic...@co...> wrote: > Hi Ken!**** > > ** ** > > The idea would be very similar but we need to make our implementation of > this and do some tests.**** > > ** ** > > ** ** > > Kind regards**** > > Michał **** > > ** ** > > ** ** > > *From:* Ken [mailto:ken...@gm...] > *Sent:* Friday, May 04, 2012 10:23 AM > *To:* Michał Borychowski > > *Subject:* Re: [Moosefs-users] ChunkServer in different Level**** > > ** ** > > Michal, > > I notice the blog Rack Awareness<http://www.moosefs.org/news-reader/items/rack-awareness.html>: > **** > > The behaviour of choosing where to create the new chunks will be > introduced as "level goal" functionality in an upcoming release.**** > > Is this 'level goal' same as 'ChunkServer in different Level'? > > > Thanks > -Ken > > > **** > > On Mon, Jan 30, 2012 at 6:38 PM, Ken <ken...@gm...> wrote:**** > > hi, > > I am very glad to hear that. > > Patch attached, and web version via github: > > https://github.com/pedia/moosefs/commit/74bc2f498fd218ea6aa51ccd0779d0e154da09ed > > And hope to help more if you need. > > Best Regards > -Ken > > > > 2012/1/30 Michał Borychowski <mic...@ge...>:**** > > > Hi Ken > > > > Our developers took a closer look at your solution and it is very > interesting. We'd like to incorporate your code (after some little changes) > into our main branch - don't you have anything against it? If not, please > send your changes in a form of a patch. > > > > Thank you > > > > > > Kind regards > > Michał Borychowski > > MooseFS Support Manager > > _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ > > Gemius S.A. > > ul. Wołoska 7, 02-672 Warszawa > > Budynek MARS, klatka D > > Tel.: +4822 874-41-00 > > Fax : +4822 874-41-01 > > > > > > -----Original Message----- > > From: Ken [mailto:ken...@gm...] > > Sent: Monday, January 16, 2012 2:06 AM > > To: Michał Borychowski > > Cc: moo...@li... > > Subject: Re: [Moosefs-users] ChunkServer in different Level > > > > Hi, Michał > > > > I push my scripts to github: > > https://github.com/pedia/moosefs/tree/master/failover-script > > > > It really work to me. > > > > Sorry for late reply, because weekend. > > > > Regards > > -Ken > > > > > > > > 2012/1/13 Michał Borychowski <mic...@ge...>: > >> Hi Ken! > >> > >> It would be great if you could also provide the group with your > >> scripts for auto recovery of the system. > >> > >> > >> Kind regards > >> Michał > >> > >> > >> -----Original Message----- > >> From: Ken [mailto:ken...@gm...] > >> Sent: Friday, January 13, 2012 8:59 AM > >> To: Davies Liu > >> Cc: moo...@li... > >> Subject: Re: [Moosefs-users] ChunkServer in different Level > >> > >> We also use ucarp, but changed a little from Thomas version. The auto > >> switching never failed. > >> > >> A friend of mine use DRBD and LVS. It also work fine, but I think it > >> is much smaller than the Douban's. > >> > >> -Ken > >> > >> > >> > >> On Fri, Jan 13, 2012 at 3:42 PM, Davies Liu <dav...@gm...> > wrote: > >>> On Fri, Jan 13, 2012 at 3:25 PM, Ken <ken...@gm...> wrote: > >>>> I noticed the go-mfsclient in this mail-list before, we also write a > >>>> moose client in C++. ;-) It work find when mfsmaster failover. > >>> > >>> It's the most interesting part, how do you failover mfsmaster ? > >>> I have tried the method with ucarp, provided by Thomas S Hatch, It's > >>> seemed not stable enough, failed sometimes. > >>> > >>> Before a stable solution come up, we decide to do manually fail-over > >>> by ops, and do not deploy it in heavy online system. > >>> > >>>> We plan to build it as a preload dynamic library, and auto hook the > >>>> file > >> API. > >>>> I think it is high availability enough. > >>>> > >>>> Thanks > >>>> > >>>> Regards > >>>> -Ken > >>>> > >>>> > >>>> > >>>> On Fri, Jan 13, 2012 at 2:52 PM, Davies Liu <dav...@gm...> > wrote: > >>>>> On Fri, Jan 13, 2012 at 2:40 PM, Ken <ken...@gm...> wrote: > >>>>>>>> It's not good ideal to use moosefs as storage for huge amount of > >>>>>>>> small files > >>>>>> I agree. We combine some small files into one big file, and read > >>>>>> the small files with offset/length infomation. > >>>>> > >>>>> Is not safe to write to same file concurrently. > >>>>> We use this method to backup the original file user uploaded, with > >>>>> tar, when offline. > >>>>> Some times, some file will be broken. > >>>>> > >>>>> MFS is not good enough for online system, not high available, and > >>>>> some IO operations will be block when error in mfsmaster or > >> mfschunkserver. > >>>>> > >>>>> So we serve some video files (>10M) in MFS this way: > >>>>> Nginx -> nginx + FUSE -> MFS > >>>>> or > >>>>> Nginx -> go-mfsclient [1] -> MFS > >>>>> > >>>>> If there something wrong with MFS, it will not block the first > >>>>> Nginx and the whole site will not be affected. > >>>>> > >>>>> Davies > >>>>> > >>>>> [1] github.com/davies/go-mfsclient > >>>>> > >>>>>> Thanks. > >>>>>> > >>>>>> Regards > >>>>>> -Ken > >>>>>> > >>>>>> > >>>>>> > >>>>>> On Fri, Jan 13, 2012 at 2:32 PM, Davies Liu <dav...@gm...> > >> wrote: > >>>>>>> On Thu, Jan 12, 2012 at 5:28 PM, Ken <ken...@gm...> wrote: > >>>>>>>> hi, moosefs > >>>>>>>> > >>>>>>>> We plan to use moosefs as storage for huge amount photos > >>>>>>>> uploaded by > >> users. > >>>>>>> > >>>>>>> It's not good ideal to use moosefs as storage for huge amount of > >>>>>>> small files, because the mfsmaster will be the bottle neck when > >>>>>>> you have more than 100M files. At that time, the whole size of > >>>>>>> files may be 1T (10k per file), can be stored by one local disk. > >>>>>>> > >>>>>>> Huge amount small files need other solutions, just like TFS [1] > >>>>>>> from taobao.com, or beansdb [2] from douban.com. > >>>>>>> > >>>>>>> [1] http://code.taobao.org/p/tfs/src/ [2] > >>>>>>> http://code.google.com/p/beansdb/ > >>>>>>> > >>>>>>>> Because of read operations of new files are very more than old > >>>>>>>> files, maybe write new files to SSD is a choice. > >>>>>>>> For strict safe reason, we must backup content to an other data > >> center. > >>>>>>>> And more features in maintain purpose are required. > >>>>>>>> > >>>>>>>> I don't think moosefs can work fine in these situation. We try > >>>>>>>> to implement these features several weeks ago. Till now, it's > >>>>>>>> almost done. > >>>>>>>> > >>>>>>>> Is there anyone interested in this? > >>>>>>>> > >>>>>>>> more detail: > >>>>>>>> # Add access_mode(none, read, write capability) to struct > >>>>>>>> matocserventry(matocserv.c). This value can be changed from > >>>>>>>> outside(maybe from the python cgi) # mfschunkserver.cfg add > >>>>>>>> 'LEVEL' config, if not, LEVEL=0 as normal. > >>>>>>>> ChunkServer report it to Master if need. > >>>>>>>> # Add uint32_t levelgoal into struct fsnode(filesystem.c). > >>>>>>>> # Add uint32_t levelgoal into sturct chunk(chunk.c). > >>>>>>>> As seen, uint32_t levelgoal = uint8_t levelgoal[4], implied > >>>>>>>> LEVEL should be 1,2,3 or 4. > >>>>>>>> [2,1,0,0] mean store 2 copies in level=1 ChunkServer, store 1 > >>>>>>>> copy in > >>>>>>>> level=2 ChunkServer. > >>>>>>>> # In chunk_do_jobs(chunk.c), send replicated command to > ChunkServer. > >>>>>>>> This policy should be very complicated in future. > >>>>>>>> # Also, we add read/write levelgoal support in mfstools. > >>>>>>>> > >>>>>>>> We plan to put these trivial change into github or somewhere else. > >>>>>>>> > >>>>>>>> It's a very incipient prototype. We appreciate any advice from > >>>>>>>> the develop team and other users. > >>>>>>>> > >>>>>>>> Regards > >>>>>>>> -Ken > >>>>>>>> > >>>>>>>> ---------------------------------------------------------------- > >>>>>>>> - > >>>>>>>> ------------- > >>>>>>>> RSA(R) Conference 2012 > >>>>>>>> Mar 27 - Feb 2 > >>>>>>>> Save $400 by Jan. 27 > >>>>>>>> Register now! > >>>>>>>> http://p.sf.net/sfu/rsa-sfdev2dev2 > >>>>>>>> _______________________________________________ > >>>>>>>> moosefs-users mailing list > >>>>>>>> moo...@li... > >>>>>>>> https://lists.sourceforge.net/lists/listinfo/moosefs-users > >>>>>>> > >>>>>>> > >>>>>>> > >>>>>>> -- > >>>>>>> - Davies > >>>>> > >>>>> > >>>>> > >>>>> -- > >>>>> - Davies > >>> > >>> > >>> > >>> -- > >>> - Davies > >> > >> ---------------------------------------------------------------------- > >> ------ > >> -- > >> RSA(R) Conference 2012 > >> Mar 27 - Feb 2 > >> Save $400 by Jan. 27 > >> Register now! > >> http://p.sf.net/sfu/rsa-sfdev2dev2 > >> _______________________________________________ > >> moosefs-users mailing list > >> moo...@li... > >> https://lists.sourceforge.net/lists/listinfo/moosefs-users > >> > >**** > > ** ** > |
