Re: [Aoetools-discuss] More AoE performance issues

[Jon N. <jne...@ja...>]

[Sorry for the repost from the correct address.....]


On Fri, Mar 21, 2008 at 3:31 AM, Tracy R Reed <tr...@ul...> wrote:
...
>  I can FTP a 118M file from the target to the initiator:
>
>  122880000 bytes received in 0.99 seconds (1.2e+05 Kbytes/s)

Much too short to really get an accurate measurement.

>  and get 120M/s. So I'm pretty sure my network is solid. No errors,
>  drops, etc on any of the interfaces in ifconfig.
>
>  The native disk performance seems quite good. On a single spindle with
>  no RAID or anything the Seagate 750G 7200 rpm disks I am using can write
>  at around 65MB/s:
>
>  [root@disk01 mnt]# dd if=/dev/zero of=foo bs=4096 count=4000000
>  4000000+0 records in
>  4000000+0 records out
>  16384000000 bytes (16 GB) copied, 249.371 seconds, 65.7 MB/s

You have to use oflag=fsync or if your dd doesn't support that find
some way - what you have above also includes the benefit of buffering
and thus is *NOT* timing how long it takes for data to get to disk.

>  And read performance is pretty impressive as well:
>
>  [root@disk01 mnt]# dd if=foo of=/dev/null bs=4096 count=4000000
>  4000000+0 records in
>  4000000+0 records out
>  16384000000 bytes (16 GB) copied, 148.729 seconds, 110 MB/s

Again, the data could be in buffers or in cache. Very very few
consumer drives are capable of more than about 80MB/s (with some
notable exceptions, the Seagate listed above not among them). Drop the
caches first with:

echo 3 > /proc/sys/vm/drop_caches

>  The box has 2G of RAM so I use very large reads/writes in testing to
>  minimize the impact of caching.

That helps certainly but at least in the read case it's insufficient -
if the last 1G of data fits into cache then all the numbers get thrown
off...

>  Is it just me or does ethtool have stupid command line options and
>  unclear documentation? -a is "show pause options" which I am assuming
>  refers to flow control. I am using the forcedeth driver because it turns

Aha. What kernel? On 2.6.22 performance of the nvidia (MCP55 in my
case) leaves quite a bit to be desired - in my case I suffer from lots
and lots of IRQ problems. 2.6.24+ improves things considerably!!
Earlier kernels have been hit or miss for me.

>  I got 86 retransmits and 76 unexpected rsp during that 1.6G dd which
>  came out at 20.7MB/s. Anyone know what can cause this when things seem
>  to work so well on the tcp level?

What does your ifconfig show for errors?
Try to get the latest kernel you can - I found a HUGE performance
increase with kernels at or later than the 2.6.24 mark. Several of my
gig-e interfaces were simply not usable in 2.6.22 and my (limited)
testing of 2.6.24 shows much improvement.

Hope this helps!!

-- 
Jon