Re: [Moosefs-users] Solution of small file store

[Davies L. <dav...@gm...>]

On Tue, Apr 24, 2012 at 2:29 PM, Ken <ken...@gm...> wrote:

> On Tue, Apr 24, 2012 at 2:22 PM, Davies Liu <dav...@gm...> wrote:
> > Maybe leveldb + MooseFS is better.
> Is this your mean?
>  leveldb store url
>  moosefs store photo


No, store url and photo in leveldb, which use MooseFS as disks.
the performance may be not perfect, but enough.
If not, TFS from taobao.com may be the better choice.

>
> > On Tue, Apr 24, 2012 at 1:59 PM, Ken <ken...@gm...> wrote:
> >>
> >> We need to store tons of small files(photo), as noticed in faq, file
> >> count is limited in moosefs, I think bundle small files to a huge file
> >> maybe work.
> >>
> >> Write photos procedure like:
> >>   allocate a huge file
> >>   write head and photo content
> >>   return (huge file, position, size)
> >>   write another head and another photo
> >>   return (huge file, position, size)
> >>   ...
> >>
> >> Before read a photo, we should have enough information: huge file,
> >> position and length, the reading procedure is expected normally.
> >>
> >> To read a photo, we should provide an URL, like
> >> 'http://xxx.com/prefix/huge file/offset/size.jpg'
> >>
> >> And to be useful in WEB, build a fastcgi program for read/write access
> >> to the huge file.
> >>
> >> ps:
> >> *  The matchup information photo and url should store outside of
> mooosefs.
> >> *  Huge file size limited in 2G.
> >> *  Huge file maybe cause race condition.
> >>
> >> Is there anyone interested in this?
> >> or better solution?
> >>
> >> -Ken
> >>
> >>
> >>
> ------------------------------------------------------------------------------
> >> Live Security Virtual Conference
> >> Exclusive live event will cover all the ways today's security and
> >> threat landscape has changed and how IT managers can respond.
> Discussions
> >> will include endpoint security, mobile security and the latest in
> malware
> >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> >> _______________________________________________
> >> moosefs-users mailing list
> >> moo...@li...
> >> https://lists.sourceforge.net/lists/listinfo/moosefs-users
> >
> >
> >
> >
> > --
> >  - Davies
>



-- 
 - Davies