Re: [Dump-devel] [patch] Use sys_clone under Linux to share I/O contexts between dump processes

[Marc T. <ma...@dr...>]

Hi Jeff,

Patch applies cleanly to dump-0.4b41, but it doesn't then compile on my 
system (Slackware 12.2, kernel 2.6.27.7, CFQ scheduler):

tape.c: In function 'fork_clone_io':
tape.c:797: error: 'CLONE_IO' undeclared (first use in this function)
tape.c:797: error: (Each undeclared identifier is reported only once
tape.c:797: error: for each function it appears in.)

I'm no programmer, but I did find that changing the include for 
<sched.h> to <linux/sched.h> fixed that.

Performance is indeed improved, in my case from 11801 kB/s to 13434 kB/s 
dumping an LVM snapshot of the ext3 root filesystem on md-raid 
(mirrored) to another LVM volume in the same md-raid (same physical 
spindles).

I suspect the faster the destination device, the more apparent the 
performance improvement.

Stelian - were there any objections to including this patch?

BR,
Marc

On 17 Nov 08 Jeff Moyer wrote:
> Hi,
> 
> dump performs poorly when run under the CFQ I/O scheduler.  The reason
> for this is that the dump command interleaves I/O between two (or
> three?) cooperating processes.  This is about the worst case scenario
> you can get for CFQ, as the I/O access pattern within each process is
> sequential.  Thus, CFQ will idle for a number of milliseconds waiting
> for the current process to issue more I/O before switching to the next.
> 
> Now, this behaviour can be changed with tuning.  However, if the dump
> command simply shared I/O contexts between cooperating processes, CFQ
> could make more intelligent decisions about I/O scheduling.
> 
> So, here are the numbers, running under 2.6.28-rc3.
> 
> deadline    82241 kB/s
> cfq	    34143 kB/s
> cfq-shared  82241 kB/s
> 
> cfq-shared denotes that the dump utility was patched with the attached
> patch to share I/O contexts.  As you can see, with a very little bit of
> code change, we can drastically increase the performance of dump under
> CFQ (which is the default I/O scheduler used in a number of
> distributions).
> 
> For more information on the underlying problems, you can refer to the
> following kernel discussion:
>   http://lkml.org/lkml/2008/11/9/133
> 
> Comments are appreciated.
> 
> Cheers,
> 
> Jeff
>