[Moosefs-users] A MFS benchmark * Re: Moosefs for logs

[Anh K. H. <ky...@vi...>]

On Fri, 20 Aug 2010 12:58:40 +0200
Michał Borychowski <mic...@ge...> wrote:

> You can easily use MooseFS to store any kind of content, including
> log files. Performance should not be affected. But it depends on
> the logging mechanism, if for writing every line the file is
> continuously opened, data appended and the file closed, for sure it
> is not perfect way to save logs. 
> 
> And you have to remember about one thing. Different clients cannot
> write at the same moment to the same file located in MooseFS. So if
> you have several httpd servers, let them save the logs under
> different filenames (eg. access-192.168.0.1.log,
> access-192.168.0.2.log, etc.)

I share with you a small benchmark result. I set up a default apache server, then run
	ab -n 150000 -c <C> http://localhost/  # ab is provided by Apache

to get information about web serving. The '<C>' is the concurrency level (number of requests will be sent at the same time.)

I compared the output of data in two cases
 
(A) * Default apache setup
    * logs file will be written on local disk

(B) * Default apache setup on the same server as (A),
    * logs file will be written on a MFS disk (provided by another server.)

Here's the output

(A) http://gx.viettug.org/zen/kyanh/jobs/log_on_root.png.html
(B) http://gx.viettug.org/zen/kyanh/jobs/log_on_mfs.png.html

As you can see, when using MFS for logs, the number of requests per second decreases significantly. This number relies on the red curve of the graphs: the maximum value in (B) is 4500, which is lesser than the lowest value in (A).

Conclusion: MFS isn't suitable for high load logs service (esp. when WRITING is important.)

I don't know about the READ performance. Should make some other tests.

Regards,

> -----Original Message-----
> From: Anh K. Huynh [mailto:ky...@vi...] 
> Sent: Thursday, August 12, 2010 12:21 PM
> To: moo...@li...
> Subject: [Moosefs-users] Moosefs for logs
> 
> Hi,
> 
> I am a moosefs newbie. I am using EC2 instances and I intended to
> build a moosefs system to share EBS disks between instances.
> 
> My question is that: can I use moosefs for logs? My applications
> (web server, applications) need to write to logs files, but I don't
> know if there's any performance problem when logs are written to
> moosefs' disk.
> 
> Thank you for your helps,
> 
> Regards,
> 


-- 
Anh Ky Huynh