Menu
▾
▴
Re: [E1000-devel] Bug report E1000 driver bonding in 802.3ad mode can not go beyond 1GB/s throughput
|
From: Ronciak, J. <joh...@in...> - 2009-03-27 18:39:44
|
So this same setup using the VLANs doesn't work on the 82546 NICs? Or has this not been tried? Cheers, John ----------------------------------------------------------- "...that your people will judge you on what you can build, not what you destroy.", B. Obama, 2009 >-----Original Message----- >From: Support Team [mailto:su...@ca...] >Sent: Friday, March 27, 2009 11:32 AM >To: Ronciak, John; 'Jay Vosburgh' >Cc: e10...@li... >Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >802.3ad mode can not go beyond 1GB/s throughput > >It is 82573 NIC built-in on the motherboard working with E1000E driver. >The 82573 board is powered by core 2 dual 1.6GHz. The 82546 >based MT server >adapter >is powered by Dual Xeon 3.6GHz. There are two VLANs on the bonded >interface, one VLAN >has the clients that generate the load, the other VLAN on the bonded >interfaces >talks to the servers. > >All the clients and servers and switches are same during the testing. > >-----Original Message----- >From: Ronciak, John [mailto:joh...@in...] >Sent: Friday, March 27, 2009 11:15 AM >To: su...@ca...; 'Jay Vosburgh' >Cc: e10...@li... >Subject: [work] RE: [E1000-devel] Bug report E1000 driver >bonding in 802.3ad >mode can not go beyond 1GB/s throughput > >OK one last question. When you say you are using the e1000e >driver what >NICs (devices) are being used? > >I'll ask around to see if any one here has done anything with bonding >lately. This could some how be releated to you trying to route packets >through a bonded interface. > >Cheers, >John >----------------------------------------------------------- >"...that your people will judge you on what you can build, not what you >destroy.", B. Obama, 2009 > > >>-----Original Message----- >>From: Support Team [mailto:su...@ca...] >>Sent: Friday, March 27, 2009 10:19 AM >>To: Ronciak, John; 'Jay Vosburgh' >>Cc: e10...@li... >>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>802.3ad mode can not go beyond 1GB/s throughput >> >>John, >> >>8257x device work fine on a less powerful HW system with same >>network cable, >>switch, clients and servers. >> >>"HW in between" is the Hw you wanted me to described. It is a >>motherboard >>with Intel NIC running in bond >>mode to act as a router. Both 82546 and 8257x are tested on >this "HW in >>between". 8257x based system >>is less powerful CPU and less memory but using E1000E driver. >It scales >>well: with 1GB load, it will >>pass 1GB, with 2GB load, it will pass 2GB. 82546 NIC based >>Intel Pro/1000 >>MT server adapter based HW >>has more powerful CPU and more memory. When we apply 1GB load, >>it will pass >>1GB. When we apply 2GB/s load, >>it still only pass 1GB/s. >> >>Both UUTs have exactly same kernel configuration, except one >>loaded with >>E1000E driver and the other loaded >>with E1000 driver. All NICs in bonding are full duplex >>reported by ethtool. >> >>Thanks, >>Wayne >> >>-----Original Message----- >>From: Ronciak, John [mailto:joh...@in...] >>Sent: Friday, March 27, 2009 9:55 AM >>To: su...@ca...; 'Jay Vosburgh' >>Cc: e10...@li... >>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>bonding in 802.3ad >>mode can not go beyond 1GB/s throughput >> >>So you are saying that the 8257x devices work fine on the same >>systems that >>the 82546 ones don't work in? Running the same kernel, HW, >>wiring, etc. >>I'm just trying to get a picture of what is going on. >> >>Also, can you please explain this comment: >>>Without this HW in between, we can get 4GB/s throughput from clients >>>directly to servers through the >>>switch. >>What HW in between? I can't seem to picture how you have >>things set up. >> >>Cheers, >>John >>----------------------------------------------------------- >>"...that your people will judge you on what you can build, >not what you >>destroy.", B. Obama, 2009 >> >> >>>-----Original Message----- >>>From: Support Team [mailto:su...@ca...] >>>Sent: Friday, March 27, 2009 9:27 AM >>>To: Ronciak, John; 'Jay Vosburgh' >>>Cc: e10...@li... >>>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>>802.3ad mode can not go beyond 1GB/s throughput >>> >>>John, >>> >>>The E1000 driver(V8.0.6) is the latest one from Intel web >>>site, which moved >>>out all the E1000E support to a >>>separate E1000E driver. The OS is Linux 2.4 with latest >>>kernel build from >>>kernel.org. >>>HW is dual Xeon processor with PCI-X buses on the north bridge. >>>Clients are four Load Runner machines and servers are four >>web servers. >>>Going through a Netgear switch. >>>Without this HW in between, we can get 4GB/s throughput from clients >>>directly to servers through the >>>switch. >>> >>>With the E1000 driver bonding together we expect to see the >>>traffic scale >>>up. But somehow hit a limit >>>at 1GB/s. Well your new E1000E driver works nicely on the >>>8257x chps on the >>>same test environment. >>> >>>Thanks for your help, >>>Wayne >>> >>>-----Original Message----- >>>From: Ronciak, John [mailto:joh...@in...] >>>Sent: Friday, March 27, 2009 9:07 AM >>>To: su...@ca...; 'Jay Vosburgh' >>>Cc: e10...@li... >>>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>>bonding in 802.3ad >>>mode can not go beyond 1GB/s throughput >>> >>>We are working on the support issue. I lead the team >>>responsible for the >>>e1000 driver. Since this is old HW and has been out in the >>>field for a very >>>long time I have some questions for you. >>>What is the OS and driver version being used? >>>What kind of system (HW-wise) is being used? >>>What is the networking setup like? (like clients, switches, etc.) >>> >>>So without bonding but using multiple interfaces, can you >run iperf or >>>netperf on each interface each with multiple streams so that >>>each interface >>>is running to a bandwidth of 1 gigabit? I have a hard time >>>believing that >>>this can't be done as this is a very simple test to do and has >>>been working >>>for years without problem. >>> >>>Running one adapter to 1 gigabit should not be taking 25% of >>>the CPU unless >>>this is on some very old HW. So something looks wrong right there. >>> >>> >>>Cheers, >>>John >>>----------------------------------------------------------- >>>"...that your people will judge you on what you can build, >>not what you >>>destroy.", B. Obama, 2009 >>> >>> >>>>-----Original Message----- >>>>From: Support Team [mailto:su...@ca...] >>>>Sent: Friday, March 27, 2009 8:52 AM >>>>To: Ronciak, John; 'Jay Vosburgh' >>>>Cc: e10...@li... >>>>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>>>802.3ad mode can not go beyond 1GB/s throughput >>>> >>>>John, >>>> >>>>Did you get my email about the person's name in Intel? I >>>>think finding his >>>>name is not that >>>>important. Finding the problem and solution is more important. >>>> >>>>RD >>>> >>>>-----Original Message----- >>>>From: Ronciak, John [mailto:joh...@in...] >>>>Sent: Friday, March 27, 2009 8:42 AM >>>>To: su...@ca...; 'Jay Vosburgh' >>>>Cc: e10...@li... >>>>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>>>bonding in 802.3ad >>>>mode can not go beyond 1GB/s throughput >>>> >>>>>Intel engineers told us that if not in 802.3ad mode, the >>>>>throughput will be >>>>>limited to 1GB/s. >>>>>But we are setting up the 802.3ad mode on these 82546 chips. >>>> >>>>Who at Intel are you talking to? >>>> >>>>Cheers, >>>>John >>>>----------------------------------------------------------- >>>>"...that your people will judge you on what you can build, >>>not what you >>>>destroy.", B. Obama, 2009 >>>> >>>> >>>>>-----Original Message----- >>>>>From: Support Team [mailto:su...@ca...] >>>>>Sent: Friday, March 27, 2009 8:32 AM >>>>>To: 'Jay Vosburgh'; su...@ca... >>>>>Cc: e10...@li... >>>>>Subject: Re: [E1000-devel] Bug report E1000 driver bonding in >>>>>802.3ad mode can not go beyond 1GB/s throughput >>>>> >>>>>Hi Jay, >>>>> >>>>>Our date stream to the bond interfaces are sending requests >>>>>and data replyes >>>>>at 4GB/s. >>>>>The xmit_hash_policy is L3+4. >>>>> >>>>>However, we noticed that if we just apply 1GB/s load, the CPU >>>>>usage is about >>>>>25%. Adding >>>>>more load will increase the CPU usage all the way to 100%, but the >>>>>throughput would not >>>>>go up at all. >>>>> >>>>>Intel engineers told us that if not in 802.3ad mode, the >>>>>throughput will be >>>>>limited to 1GB/s. >>>>>But we are setting up the 802.3ad mode on these 82546 chips. >>>>> >>>>>What else do you think can cause this limit? Just for make >>>>>this clear, we >>>>>use the >>>>>E1000E driver with 8257x chip can get throughput scale well, >>>with same >>>>>kernel configuration >>>>>and testing environment. >>>>> >>>>>Thanks! >>>>>Wayne >>>>> >>>>>-----Original Message----- >>>>>From: Jay Vosburgh [mailto:fu...@us...] >>>>>Sent: Friday, March 27, 2009 7:02 AM >>>>>To: su...@ca... >>>>>Cc: e10...@li... >>>>>Subject: [work] Re: [E1000-devel] Bug report E1000 driver >>>>>bonding in 802.3ad >>>>>mode can not go beyond 1GB/s throughput >>>>> >>>>>Support Team <su...@ca...> wrote: >>>>> >>>>>>Hello, >>>>>> >>>>>>Intel ada...@ma... recommended us to >>>>>open a ticket >>>>>>with you that >>>>>>your e1000 driver for 82546 chips has throughput limit. >>With 802.3ad >>>>>>bonding, the total >>>>>>throughput of 8 NIC is still 1GB/s, same as single NIC. >>>>> >>>>> How are you testing throughput? If you're only running a single >>>>>stream test, you'll only see the throughput of one adapter. >>>>This is by >>>>>design, the 802.3ad standard requires that a given >>>"conversation" (TCP >>>>>connection, stream of UDP packets to/from the same ports, >>>etc) be sent >>>>>across the same slave adapter. This is done to prevent >>reordering of >>>>>packets within the conversation. >>>>> >>>>> If you're running multiple streams, then you may want to set the >>>>>xmit_hash_policy option to layer3+4 or layer2+3. The layer3+4 >>>>>hash will >>>>>place multiple streams between the same two peer systems >on multiple >>>>>slaves (with a small risk of packet reordering if IP fragments are >>>>>generated); the layer2+3 won't, but will place all traffic >>>for a given >>>>>peer on the same slave (but balances better than the default layer2 >>>>>hash). >>>>> >>>>> The hashes are described in detail in the bonding.txt >>>>>documentation supplied with the kernel source. >>>>> >>>>> -J >>>>> >>>>>--- >>>>> -Jay Vosburgh, IBM Linux Technology Center, fu...@us... >>>>> >>>>> >>>>> >>>>>--------------------------------------------------------------- >>>>>--------------- >>>>>_______________________________________________ >>>>>E1000-devel mailing list >>>>>E10...@li... >>>>>https://lists.sourceforge.net/lists/listinfo/e1000-devel >>>>> >>>> >>>> >>> >>> >> >> > > |
