[Scst-devel] ib_srpt 2.2.0: Very slow SRP performance with MD RAID

[Sebastian R. <seb...@pr...>]

Hi list,

we've got 3.2.8 kernels on both SRP target and initiator. The one on the
target is patched with SCST 2.2.0. We use the ofa-kernel-modules from
intree and the matching OFED-1.5.4 user space. Our QDR HCAs are the
Mellanox MHQH19B-XTR (mlx4).

On the storage there is a MD RAID-10 (MD RAID-1 + MD RAID-0) with 512
KiB stripe size over 16 HDDs. We do a file copy benchmark on ext4
without any caching which is pretty fast (big chunk sizes) local on the
storage system. But over SRP it is much slower than with iSER and the
tgt target. We're exporting simple partitions via SRP.

On the initiator "iomin" is reported as 4 KiB and "ioopt" is reported as
512 KiB.

blktrace shows that nothing bigger than 64 KiB is in the queue on the
initiator as well as on the target with SRP. We've tried to increase
that by setting the ib_srpt parameter "srp_max_rdma_size=524288", but it
had no effect - still max. 64 KiB in the queue. We've also tried the
performance tuning described in the README - didn't have much effect.

How can we improve performance/increase chunk size there?

Cheers,
Sebastian


-- 
Sebastian Riemer
Linux Kernel Developer

ProfitBricks GmbH
Greifswalder Str. 207
10405 Berlin, Germany

Tel.:  +49 - 30 - 60 98 56 991 - 303
Fax:   +49 - 30 - 51 64 09 22
Email: seb...@pr...
Web:   http://www.profitbricks.com/

Sitz der Gesellschaft: Berlin
Registergericht: Amtsgericht Charlottenburg, HRB 125506 B
Geschäftsführer: Andreas Gauger, Achim Weiss