[Moosefs-users] Why too slow in calling fdatasync()? (or futex())

[Jun C. P. <jun...@gm...>]

Hi,

I found a more specific case about the performance slowness issue that
I experienced before.

One of important commands in using KVM (kernel-based Virtual Machine)
is qemu-img that generates a special form of file (i.e., qcow2) as
follows.

# strace qemu-img convert -f qcow2 -O qcow2 ........

read(4, "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
128) = 128
pwrite(5, "\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1\0\1"...,
512, 196608) = 512
fdatasync(5)                            = 0
futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1
select(5, [4], [], NULL, NULL)          = 1 (in [4])
read(4, "\f\0\0\0\0\0\0\0\0\0\0\0\260r\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0"...,
128) = 128
pwrite(5, "\200\0\0\0\0\27\0\0\200\0\0\0\0\30\0\0\200\0\0\0\0\31\0\0\200\0\0\0\0\32\0\0"...,
512, 263168) = 512
fdatasync(5)
futex(0x63eec4, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0x63eec0, {FUTEX_OP_SET,
0, FUTEX_OP_CMP_GT, 1}) = 1

The problem of this command (unlike 'cp' where no calling of
fdatasync) is that, for every pwrite operation, fdatasync() and
futex() are called. Then, the MFS write performance is significantly
reduced from 60 M/s (with 'cp')  to 1M/s (with qemu-img). I noticed
via netstat with TIME_WAIT that, while the qemu-img command is
running, there are a lot of non-persistent TCP connections.

Is there any way to improve this situation?

Thanks,

-Jun