Menu
▾
▴
Re: [Moosefs-users] Solution of small file store
|
From: <ro...@mm...> - 2012-04-24 08:28:25
|
If you're on linux, i'd try a filesystem-in-a-file which would be used to store the photo and this file itselfs stored in a MooseFS volume. For example something along these lines: dd if=/dev/zero of=FILENAME bs=1024 count=1 seek=$((2*1024*1024-1)) mkfs -t ext3 FILENAME mount -t ext3 -o loop FILENAME MOUNTPOINT where MOUNTPOINT is a MooseFS volume. -Stathis Wang Jian wrote: > TFS has strict and random URL scheme, so it's difficult to do URL based > tuning. > > δΊ 2012/4/24 15:11, Davies Liu ει: >> On Tue, Apr 24, 2012 at 2:29 PM, Ken <ken...@gm... >> <mailto:ken...@gm...>> wrote: >> >> On Tue, Apr 24, 2012 at 2:22 PM, Davies Liu <dav...@gm... >> <mailto:dav...@gm...>> wrote: >> > Maybe leveldb + MooseFS is better. >> Is this your mean? >> leveldb store url >> moosefs store photo >> >> No, store url and photo in leveldb, which use MooseFS as disks. >> the performance may be not perfect, but enough. >> If not, TFS from taobao.com <http://taobao.com> may be the better >> choice. >> >> > >> > On Tue, Apr 24, 2012 at 1:59 PM, Ken <ken...@gm... >> <mailto:ken...@gm...>> wrote: >> >> >> >> We need to store tons of small files(photo), as noticed in faq, >> file >> >> count is limited in moosefs, I think bundle small files to a huge >> file >> >> maybe work. >> >> >> >> Write photos procedure like: >> >> allocate a huge file >> >> write head and photo content >> >> return (huge file, position, size) >> >> write another head and another photo >> >> return (huge file, position, size) >> >> ... >> >> >> >> Before read a photo, we should have enough information: huge >> file, >> >> position and length, the reading procedure is expected normally. >> >> >> >> To read a photo, we should provide an URL, like >> >> 'http://xxx.com/prefix/huge file/offset/size.jpg' >> >> >> >> And to be useful in WEB, build a fastcgi program for read/write >> access >> >> to the huge file. >> >> >> >> ps: >> >> * The matchup information photo and url should store outside of >> mooosefs. >> >> * Huge file size limited in 2G. >> >> * Huge file maybe cause race condition. >> >> >> >> Is there anyone interested in this? >> >> or better solution? >> >> >> >> -Ken >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Live Security Virtual Conference >> >> Exclusive live event will cover all the ways today's security and >> >> threat landscape has changed and how IT managers can respond. >> Discussions >> >> will include endpoint security, mobile security and the latest in >> malware >> >> threats. >> http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> >> _______________________________________________ >> >> moosefs-users mailing list >> >> moo...@li... >> <mailto:moo...@li...> >> >> https://lists.sourceforge.net/lists/listinfo/moosefs-users >> > >> > >> > >> > >> > -- >> > - Davies >> >> >> >> >> -- >> - Davies >> >> >> ------------------------------------------------------------------------------ >> Live Security Virtual Conference >> Exclusive live event will cover all the ways today's security and >> threat landscape has changed and how IT managers can respond. >> Discussions >> will include endpoint security, mobile security and the latest in >> malware >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ >> >> >> _______________________________________________ >> moosefs-users mailing list >> moo...@li... >> https://lists.sourceforge.net/lists/listinfo/moosefs-users > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. > http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ > moosefs-users mailing list > moo...@li... > https://lists.sourceforge.net/lists/listinfo/moosefs-users > |
