Re: [E1000-devel] 82599 receive performance

[Waskiewicz J. P. P <pet...@in...>]

On Thu, 2012-06-28 at 08:55 +0000, Anna Fischer wrote:
> Hi, I have 2 VCPUs assigned to the VM. I thought more CPUs wouldn't
> really help because a single TCP connection doesn't span across cores
> anyway? I would have to run multiple connections simultaneously I
> think. 

Do you see 2 Tx and 2 Rx queues in ethtool -S ethX inside your VM or no?
The issue you can run into is head-of-line blocking of ACKs being sent
if you're using a single Tx and Rx queue.  TCP connections can't have
their ACKs sent or be received and processed since there's pending data
packets needing to be serviced.

-PJ

> I use the standard packet size, I don't do jumbo frames or anything. I
> don't tweak the OS either, I just want to see how it would perform by
> default.
> 
> I'm just surprised that CPU isn't maxed out at all. It must be because
> polling/NAPI is too slow fetching packets? It must be a timer thing
> like this? 
> Otherwise, is it that hardware resources are limited with just a
> single queue and the RX ring buffer cannot be filled/refilled quickly
> enough? Why else would it be slow if CPU resources are there.
> 
> When I say maxed out I mean the VCPU of course, so those cores are not
> running at full capability.
> 
> Anna
> 
> 
> ______________________________________________________________________
> From: "Waskiewicz Jr, Peter P" <pet...@in...>
> Date: Thu, 28 Jun 2012 08:45:23 +0000
> To: Fischer, Anna<ann...@hp...>
> Cc:
> e10...@li...<e10...@li...>
> Subject: Re: [E1000-devel] 82599 receive performance
> 
> 
> Hi Anna,
> 
> On Thu, 2012-06-28 at 08:32 +0000, Fischer, Anna wrote:
> > Hi, I'm running a 10G 82599 controller in PCI SR-IOV mode, and the
> VF is assigned to a VM (PCI pass-through). I only enable a single VF
> (max_vfs=1). On my machine I run KVM and just a single VM.
> > The machine has 4 Xeon X5550 cores.
> > 
> > I've tested RX and TX throughput using iperf (TCP). Basically I want
> to see how fast I can send and receive packets to/from the VM.
> 
> How many CPU's are provisioned to your VM? Also, what packet size are
> you using with iperf? In many cases, a single Rx/Tx queue pair is not
> enough to run 10GbE line rate.
> 
> -PJ
> 
> > TX performance comes to about 8.5Gbps. At that point, CPU is maxed
> out.
> > 
> > RX performance only comes to about 5.5Gbps. CPU isn't maxed out at
> all, and I wonder, if this is the expected RX throughput? I have to
> say that I only run a single TX queue and a single RX queue. (I am
> testing against something else that also just runs a single queue, so
> I need to keep it fair.) I haven't tuned any OS parameters, I run
> Fedora Core 16 and a 3.4 Linux kernel.
> > 
> > Thanks,
> > Anna
> > 
> >
> ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and 
> > threat landscape has changed and how IT managers can respond.
> Discussions 
> > will include endpoint security, mobile security and the latest in
> malware 
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > E1000-devel mailing list
> > E10...@li...
> > https://lists.sourceforge.net/lists/listinfo/e1000-devel
> > To learn more about Intel® Ethernet, visit
> http://communities.intel.com/community/wired
>