Menu
▾
▴
Re: [MooseFS-Users] slow lookup
|
From: Eugene D. <it...@da...> - 2017-05-18 08:43:35
|
CPU Intel(R) Core(TM) i7-6700K CPU @ 4.00GHz RAM 32 GB with ECC 1 Gbit connection. load average: 0.00, 0.01, 0.05 Linux mfs 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u2 (2016-10-19) x86_64 GNU/Linux It is a totally new empty server. If I copy 1 gig file to mfs it tooks around 22 seconds, so it is couldn't be a network issue. The problem only with lookup operation. mfs metadata files are placed on ssd system drive. read/write test: dd if=/dev/zero of=/test bs=1024 count=5024000 5024000+0 records in 5024000+0 records out 5144576000 bytes (5.1 GB) copied, 19.4822 s, 264 MB/s dd if=/test of=/dev/null bs=1024 count=5024000 5024000+0 records in 5024000+0 records out 5144576000 bytes (5.1 GB) copied, 1.56791 s, 3.3 GB/s mfschukserver raid read/write local test (not thru mfs): dd if=/dev/zero of=/mnt/data/test bs=1024 count=5024000 5024000+0 records in 5024000+0 records out 5144576000 bytes (5.1 GB) copied, 6.66198 s, 772 MB/s dd if=/mnt/data/test of=/dev/null bs=1024 count=5024000 5024000+0 records in 5024000+0 records out 5144576000 bytes (5.1 GB) copied, 1.52796 s, 3.4 GB/s -- Best regards, Eugene > 18 мая 2017 г., в 10:51, Aleksander Wieliczko <ale...@mo...> написал(а): > > Hi, > metadata operations like ls, mostly depends on MooseFS master CPU speed and network latency. > > Would you be so kind and tell us something more about this parameters? > > > Best regards > Aleksander Wieliczko > Technical Support Engineer > MooseFS.com <x-msg://2/moosefs.com> > > On 18.05.2017 09:05, Ben Harker wrote: >> I've experienced weirdness when using various raid setups. It's stated in the documentation that a non raid setup is preferred - at least from what I've remembered. MFS is a fuse filesystem that's bound to have a performance penalty somewhere when it comes to huge numbers of small files due to it's distributed nature. >> >> I've mitigated this somewhat by introducing an SSD based tier of storage that holds files smaller than a certain size. This one's definitely helped, and you can also set it as the C (create) tier for when files get created if you get creative with your storage classes - the initial writes to SSD are much quicker and can make your setup feel much faster when writing files. >> >> Hope that's helpful! >> >> On May 18, 2017 07:06, Eugene Diatlov <it...@da...> <mailto:it...@da...> wrote: >> Hi, >> >> I have a hardware server with latest moosefs. HDD's are in a hardware RAID5. Debian 8. >> I have a lot of small files in folders (100k+). When I enter the folder I have a big delay while the system reading directory's files list. >> Using moosefs metrics I found that the speed of lookup operations is equal to 1,8k operations per minute at maximum. >> Why this could happen? I have another server with moosefs and the speed of lookup could be 300k per minute and higher. >> >> >> -- >> Best regards, >> Eugene >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> Check out the vibrant tech community on one of the world's most >> engaging tech sites, Slashdot.org! http://sdm.link/slashdot <http://sdm.link/slashdot> >> >> _________________________________________ >> moosefs-users mailing list >> moo...@li... <mailto:moo...@li...> >> https://lists.sourceforge.net/lists/listinfo/moosefs-users <https://lists.sourceforge.net/lists/listinfo/moosefs-users> > |
