Menu
▾
▴
Re: [Moosefs-users] Question for using moosefs on post production
|
From: Didi P. <di...@me...> - 2011-05-19 18:40:05
|
Dear Thomas, Michal & Tamas, Thanks for the response, Actually it is in MB/s, because to edit 2k resolution (2048 x 1152) the editor would need a throughput of around 350 MB/s, with edited file (dpx file) size around 4TB (single file). I would like to make a shared storage for 4 connected editing station. The new info reduce the need of this shared storage size (for real time editing) into 14 TB size, while I have to provide also 24-48 TB for temporary archive. I have to revise my calculation also in throughput, the total throughput needed is around 900 MB/s because usually there are 2 client editing station (autodesk lustre based on linux and davinci resolve based on mac) simultaneously edit in this throughput (2 x 350 MB/s) while the other 200 MB/s I reserve for 2 other editing station connected to this shared storage. If moosefs could not handle the function to be 'real time' editing storage, then I guess I just able to use it for the temporary archive disk which not needing so high throughput. Best Regards, Didi Pramujadi 2011/5/18 Thomas S Hatch <tha...@gm...> > With MooseFS your biggest bottlenecks are chunkserver disk speed, and > hitting the mfsmaster too often. like Michal said, if you have a lot of very > small files you will place more load on the master. As for the disk speed, I > use RAID 0 for my chunkservers in groups of 2-4 disks and we get 60-90MB > throughput. Right now I have a ~500 TB MooseFS setup, so I think that a 24T > setup should be no problem at all! > > -Thomas S Hatch > > 2011/5/18 Michal Borychowski <mic...@ge...> > >> Hi! >> >> >> >> I guess you mean 1200-1300 M*bits*/second? Which would make 150 M*bytes* >> /second? >> >> >> >> Unfortunately I doubt if you can achieve such speeds with MooseFS. Unless >> you use some SSD disks for chunkservers which would be very expensive... We >> have write speeds of about 20-30 MiB/s and reads of 30-50MiB/s at our >> environment with goal=2. >> >> >> >> On the other hand MooseFS would be perfect just for storing the content >> (it is much better optimised for large files, not the small ones). For >> editing purposes probably you should have separate machines and think of >> good "flow" of the files. >> >> >> >> >> >> Kind regards >> >> Michał Borychowski >> >> MooseFS Support Manager >> >> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ >> >> Gemius S.A. >> >> ul. Wołoska 7, 02-672 Warszawa >> >> Budynek MARS, klatka D >> >> Tel.: +4822 874-41-00 >> >> Fax : +4822 874-41-01 >> >> >> >> >> >> *From:* Didi Pramujadi [mailto:di...@me...] >> *Sent:* Wednesday, May 18, 2011 4:03 AM >> *To:* moo...@li... >> *Subject:* [Moosefs-users] Question for using moosefs on post production >> >> >> >> Dear Friends, >> >> >> Hello everyone, I'm just joining this mailing list. >> >> I'm helping my boss to research the possibility of building a custom >> shared storage for editing in his post production house. The requirement is >> to provide a 24TB shared storage with troughput around 1200-1300 MB/s (for >> simultaneous or concurrent editing of 2 station with 2k HD resolution + copy >> file). I'm thinking of using iSCSI, Infiniband or FcOE for the transport. >> >> My questions is could moosefs utilized for this purpose ? Hows the details >> hardware requirement for this? >> >> I'm sorry if this is too basic, thanks in advance. >> >> Best Regards, >> >> -- >> Didi Pramujadi >> >> Business Development >> PT. Media Mozaic Indonesia >> >> www.mediamozaic.com >> cellphone : +62811834579 >> Fax : +217408701 >> skype : didipramujadi >> >> >> >> ------------------------------------------------------------------------------ >> What Every C/C++ and Fortran developer Should Know! >> Read this article and learn how Intel has extended the reach of its >> next-generation tools to help Windows* and Linux* C/C++ and Fortran >> developers boost performance applications - including clusters. >> http://p.sf.net/sfu/intel-dev2devmay >> _______________________________________________ >> moosefs-users mailing list >> >> moo...@li... >> https://lists.sourceforge.net/lists/listinfo/moosefs-users >> >> > -- Didi Pramujadi Business Development PT. Media Mozaic Indonesia www.mediamozaic.com cellphone : +62811834579 Fax : +217408701 skype : didipramujadi |
