Menu
▾
▴
Re: [E1000-devel] Bug report E1000 driver bonding in 802.3ad mode can not go beyond 1GB/s throughput
|
From: Support T. <su...@ca...> - 2009-03-27 18:56:57
|
Yes, same VLAN setup in the software does not work for 82546 chip. The setup in software are identical, other than 82546 load E1000 driver and 82573 load E1000E driver, nothing else changed in software or test environment. If we can see 82546/MT adapter can get 2GB/s in bond mode, we will be happy. -----Original Message----- From: Ronciak, John [mailto:joh...@in...] Sent: Friday, March 27, 2009 11:39 AM To: su...@ca...; 'Jay Vosburgh' Cc: e10...@li... Subject: [work] RE: [E1000-devel] Bug report E1000 driver bonding in 802.3ad mode can not go beyond 1GB/s throughput So this same setup using the VLANs doesn't work on the 82546 NICs? Or has this not been tried? Cheers, John ----------------------------------------------------------- "...that your people will judge you on what you can build, not what you destroy.", B. Obama, 2009 >-----Original Message----- >From: Support Team [mailto:su...@ca...] >Sent: Friday, March 27, 2009 11:32 AM >To: Ronciak, John; 'Jay Vosburgh' >Cc: e10...@li... >Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >802.3ad mode can not go beyond 1GB/s throughput > >It is 82573 NIC built-in on the motherboard working with E1000E driver. >The 82573 board is powered by core 2 dual 1.6GHz. The 82546 >based MT server >adapter >is powered by Dual Xeon 3.6GHz. There are two VLANs on the bonded >interface, one VLAN >has the clients that generate the load, the other VLAN on the bonded >interfaces >talks to the servers. > >All the clients and servers and switches are same during the testing. > >-----Original Message----- >From: Ronciak, John [mailto:joh...@in...] >Sent: Friday, March 27, 2009 11:15 AM >To: su...@ca...; 'Jay Vosburgh' >Cc: e10...@li... >Subject: [work] RE: [E1000-devel] Bug report E1000 driver >bonding in 802.3ad >mode can not go beyond 1GB/s throughput > >OK one last question. When you say you are using the e1000e >driver what >NICs (devices) are being used? > >I'll ask around to see if any one here has done anything with bonding >lately. This could some how be releated to you trying to route packets >through a bonded interface. > >Cheers, >John >----------------------------------------------------------- >"...that your people will judge you on what you can build, not what you >destroy.", B. Obama, 2009 > > >>-----Original Message----- >>From: Support Team [mailto:su...@ca...] >>Sent: Friday, March 27, 2009 10:19 AM >>To: Ronciak, John; 'Jay Vosburgh' >>Cc: e10...@li... >>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>802.3ad mode can not go beyond 1GB/s throughput >> >>John, >> >>8257x device work fine on a less powerful HW system with same >>network cable, >>switch, clients and servers. >> >>"HW in between" is the Hw you wanted me to described. It is a >>motherboard >>with Intel NIC running in bond >>mode to act as a router. Both 82546 and 8257x are tested on >this "HW in >>between". 8257x based system >>is less powerful CPU and less memory but using E1000E driver. >It scales >>well: with 1GB load, it will >>pass 1GB, with 2GB load, it will pass 2GB. 82546 NIC based >>Intel Pro/1000 >>MT server adapter based HW >>has more powerful CPU and more memory. When we apply 1GB load, >>it will pass >>1GB. When we apply 2GB/s load, >>it still only pass 1GB/s. >> >>Both UUTs have exactly same kernel configuration, except one >>loaded with >>E1000E driver and the other loaded >>with E1000 driver. All NICs in bonding are full duplex >>reported by ethtool. >> >>Thanks, >>Wayne >> >>-----Original Message----- >>From: Ronciak, John [mailto:joh...@in...] >>Sent: Friday, March 27, 2009 9:55 AM >>To: su...@ca...; 'Jay Vosburgh' >>Cc: e10...@li... >>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>bonding in 802.3ad >>mode can not go beyond 1GB/s throughput >> >>So you are saying that the 8257x devices work fine on the same >>systems that >>the 82546 ones don't work in? Running the same kernel, HW, >>wiring, etc. >>I'm just trying to get a picture of what is going on. >> >>Also, can you please explain this comment: >>>Without this HW in between, we can get 4GB/s throughput from clients >>>directly to servers through the >>>switch. >>What HW in between? I can't seem to picture how you have >>things set up. >> >>Cheers, >>John >>----------------------------------------------------------- >>"...that your people will judge you on what you can build, >not what you >>destroy.", B. Obama, 2009 >> >> >>>-----Original Message----- >>>From: Support Team [mailto:su...@ca...] >>>Sent: Friday, March 27, 2009 9:27 AM >>>To: Ronciak, John; 'Jay Vosburgh' >>>Cc: e10...@li... >>>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>>802.3ad mode can not go beyond 1GB/s throughput >>> >>>John, >>> >>>The E1000 driver(V8.0.6) is the latest one from Intel web >>>site, which moved >>>out all the E1000E support to a >>>separate E1000E driver. The OS is Linux 2.4 with latest >>>kernel build from >>>kernel.org. >>>HW is dual Xeon processor with PCI-X buses on the north bridge. >>>Clients are four Load Runner machines and servers are four >>web servers. >>>Going through a Netgear switch. >>>Without this HW in between, we can get 4GB/s throughput from clients >>>directly to servers through the >>>switch. >>> >>>With the E1000 driver bonding together we expect to see the >>>traffic scale >>>up. But somehow hit a limit >>>at 1GB/s. Well your new E1000E driver works nicely on the >>>8257x chps on the >>>same test environment. >>> >>>Thanks for your help, >>>Wayne >>> >>>-----Original Message----- >>>From: Ronciak, John [mailto:joh...@in...] >>>Sent: Friday, March 27, 2009 9:07 AM >>>To: su...@ca...; 'Jay Vosburgh' >>>Cc: e10...@li... >>>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>>bonding in 802.3ad >>>mode can not go beyond 1GB/s throughput >>> >>>We are working on the support issue. I lead the team >>>responsible for the >>>e1000 driver. Since this is old HW and has been out in the >>>field for a very >>>long time I have some questions for you. >>>What is the OS and driver version being used? >>>What kind of system (HW-wise) is being used? >>>What is the networking setup like? (like clients, switches, etc.) >>> >>>So without bonding but using multiple interfaces, can you >run iperf or >>>netperf on each interface each with multiple streams so that >>>each interface >>>is running to a bandwidth of 1 gigabit? I have a hard time >>>believing that >>>this can't be done as this is a very simple test to do and has >>>been working >>>for years without problem. >>> >>>Running one adapter to 1 gigabit should not be taking 25% of >>>the CPU unless >>>this is on some very old HW. So something looks wrong right there. >>> >>> >>>Cheers, >>>John >>>----------------------------------------------------------- >>>"...that your people will judge you on what you can build, >>not what you >>>destroy.", B. Obama, 2009 >>> >>> >>>>-----Original Message----- >>>>From: Support Team [mailto:su...@ca...] >>>>Sent: Friday, March 27, 2009 8:52 AM >>>>To: Ronciak, John; 'Jay Vosburgh' >>>>Cc: e10...@li... >>>>Subject: RE: [E1000-devel] Bug report E1000 driver bonding in >>>>802.3ad mode can not go beyond 1GB/s throughput >>>> >>>>John, >>>> >>>>Did you get my email about the person's name in Intel? I >>>>think finding his >>>>name is not that >>>>important. Finding the problem and solution is more important. >>>> >>>>RD >>>> >>>>-----Original Message----- >>>>From: Ronciak, John [mailto:joh...@in...] >>>>Sent: Friday, March 27, 2009 8:42 AM >>>>To: su...@ca...; 'Jay Vosburgh' >>>>Cc: e10...@li... >>>>Subject: [work] RE: [E1000-devel] Bug report E1000 driver >>>>bonding in 802.3ad >>>>mode can not go beyond 1GB/s throughput >>>> >>>>>Intel engineers told us that if not in 802.3ad mode, the >>>>>throughput will be >>>>>limited to 1GB/s. >>>>>But we are setting up the 802.3ad mode on these 82546 chips. >>>> >>>>Who at Intel are you talking to? >>>> >>>>Cheers, >>>>John >>>>----------------------------------------------------------- >>>>"...that your people will judge you on what you can build, >>>not what you >>>>destroy.", B. Obama, 2009 >>>> >>>> >>>>>-----Original Message----- >>>>>From: Support Team [mailto:su...@ca...] >>>>>Sent: Friday, March 27, 2009 8:32 AM >>>>>To: 'Jay Vosburgh'; su...@ca... >>>>>Cc: e10...@li... >>>>>Subject: Re: [E1000-devel] Bug report E1000 driver bonding in >>>>>802.3ad mode can not go beyond 1GB/s throughput >>>>> >>>>>Hi Jay, >>>>> >>>>>Our date stream to the bond interfaces are sending requests >>>>>and data replyes >>>>>at 4GB/s. >>>>>The xmit_hash_policy is L3+4. >>>>> >>>>>However, we noticed that if we just apply 1GB/s load, the CPU >>>>>usage is about >>>>>25%. Adding >>>>>more load will increase the CPU usage all the way to 100%, but the >>>>>throughput would not >>>>>go up at all. >>>>> >>>>>Intel engineers told us that if not in 802.3ad mode, the >>>>>throughput will be >>>>>limited to 1GB/s. >>>>>But we are setting up the 802.3ad mode on these 82546 chips. >>>>> >>>>>What else do you think can cause this limit? Just for make >>>>>this clear, we >>>>>use the >>>>>E1000E driver with 8257x chip can get throughput scale well, >>>with same >>>>>kernel configuration >>>>>and testing environment. >>>>> >>>>>Thanks! >>>>>Wayne >>>>> >>>>>-----Original Message----- >>>>>From: Jay Vosburgh [mailto:fu...@us...] >>>>>Sent: Friday, March 27, 2009 7:02 AM >>>>>To: su...@ca... >>>>>Cc: e10...@li... >>>>>Subject: [work] Re: [E1000-devel] Bug report E1000 driver >>>>>bonding in 802.3ad >>>>>mode can not go beyond 1GB/s throughput >>>>> >>>>>Support Team <su...@ca...> wrote: >>>>> >>>>>>Hello, >>>>>> >>>>>>Intel ada...@ma... recommended us to >>>>>open a ticket >>>>>>with you that >>>>>>your e1000 driver for 82546 chips has throughput limit. >>With 802.3ad >>>>>>bonding, the total >>>>>>throughput of 8 NIC is still 1GB/s, same as single NIC. >>>>> >>>>> How are you testing throughput? If you're only running a single >>>>>stream test, you'll only see the throughput of one adapter. >>>>This is by >>>>>design, the 802.3ad standard requires that a given >>>"conversation" (TCP >>>>>connection, stream of UDP packets to/from the same ports, >>>etc) be sent >>>>>across the same slave adapter. This is done to prevent >>reordering of >>>>>packets within the conversation. >>>>> >>>>> If you're running multiple streams, then you may want to set the >>>>>xmit_hash_policy option to layer3+4 or layer2+3. The layer3+4 >>>>>hash will >>>>>place multiple streams between the same two peer systems >on multiple >>>>>slaves (with a small risk of packet reordering if IP fragments are >>>>>generated); the layer2+3 won't, but will place all traffic >>>for a given >>>>>peer on the same slave (but balances better than the default layer2 >>>>>hash). >>>>> >>>>> The hashes are described in detail in the bonding.txt >>>>>documentation supplied with the kernel source. >>>>> >>>>> -J >>>>> >>>>>--- >>>>> -Jay Vosburgh, IBM Linux Technology Center, fu...@us... >>>>> >>>>> >>>>> >>>>>--------------------------------------------------------------- >>>>>--------------- >>>>>_______________________________________________ >>>>>E1000-devel mailing list >>>>>E10...@li... >>>>>https://lists.sourceforge.net/lists/listinfo/e1000-devel >>>>> >>>> >>>> >>> >>> >> >> > > |
