Re: [E1000-devel] i40e: Wrong hint on PCIE2.0 system

[Michal M. <mir...@re...>]

On Sat, Dec 03, 2016 at 05:39:08AM +0000, Duyck, Alexander H wrote:
> There is no need for that.  This is a Xeon E3-1220, note that there is no v2-v? on the end of it.  A quick look-up in ark.intel.com shows it has a gen2 PCIe connection with 20 lanes on the CPU itself.
> 
> In the case of the comment I would have to agree, another system might be an option that has to be considered if you are hoping to get a full 40 Gb/s out of the part.  If however you are only planning to use it for one or maybe two 10 Gb/s links you might be okay.  That is all that message is really meant to convey.  Basically in the current configuration you aren't going to be able to get optimal performance.
> 
> Also you should probably be aware that you are going to likely put some pretty serious stress on the memory system for the CPU.  I would advise making sure you are running DDR1333 and both channels populated as you max out at about 21GB/s of total memory bandwidth for the CPU.  With a 40Gb/s NIC it becomes pretty easy to saturate the memory bandwidth on a small processor.
> 
> There are other ways to work around the slot/bandwidth problem too.  In theory you could probably take a x16 PCIe gen3 capable switch and connect it to the system.  I've seen crazier things.  The switch would then negotiate down to x16 gen2 on the upstream ports and you could theoretically have x8 gen3 on the downstream ports.  Then again that would end up eating all but 4 of the 20 available PCIe lanes on the CPU and that assumes that a x16 slot is even available for something other than PCIe graphics.  

Dear Alex,

Thanks for your concern. This is a lab box for Q&A, so it's not that a problem
that it has only PCI-e 2.0.

>> [97789.169918] i40e 0000:01:00.3: PCI-Express: Speed 5.0GT/s Width x8
>> [97789.169923] i40e 0000:01:00.3: PCI-Express bandwidth available for this device may be insufficient for optimal performance.
>> [97789.169926] i40e 0000:01:00.3: Please move the device to a different PCI-e link with more lanes and/or higher transfer rate.

The thing is - the first line informs what the card got.
The second informs about observed limitation.
But the third is misleading - at least in this setup, and any other where
all the slots are the same.

Best Regards,
Michał Mirosław