Performance issues between pvscsi 1.0.1.1 and 1.0.1.0-k?

[David H <isp...@gm...>]

Hi all, hope this reaches someone who might know the more intimate
details of the pvscsi driver's development.  I've been running
numerous centos 5 x86_64 installs on vmware ESXi 4.1 with EMC storage
on the back end.  Since the pvscsi is not built in, I only mount /var
on a pvscsi adaptor and hard disk after installing the
vmware-open-vm-tools-kmod-8.3.2-257589.el5.x86_64 and associated
packages, and the rest of the OS on a vmware LSI SAS hard disk that it
can see during install.  This package includes the pvscsi.ko version
1.0.1.1.

With this configuration, I typically see about 260 MB/sec write
speeds, 500 MB/sec read speeds and anywhere from 7000 to 11,000 IOPS
depending on the number of drives in the LUN on the EMC.

With CentOS 6, it comes with vmw_pvscsi.ko module version 1.0.1.0-k
which I assume is actually an older version than what the open tools
install.  In any case, with CentOS 6 x86_64 running on an identical
configuration (same vmware host, same EMC LUN, same size vmdk file,
same /var partition) I'm lucky to get 70 to 80 MB/sec write, 250
MB/sec read, 800 IOPS; so dramatically lower performance.

The only difference I could find in the pvscsi mod config is that
disable_msix defaults to on in 1.0.1.1 and off in 1.0.1.0-k.  I did
try turning it off via a boot flag, and verified it was off under
/sys, but the performance was not affected; still bad.

I tried to install the el5 rpm on centos 6 to get the other module
working but that just produces an error "pvscsi: no symbol version for
module_layout".  I also tried to manually rebuild a new initrd using
the pvscsi.ko file but have not been able to figure out the magic
combination of files, command, etc. to get a bootable machine yet.

Any ideas?

Thanks,

David