Menu
▾
▴
Re: [E1000-devel] Possible bug in ixgbe-3.19.1 ixgbe_enumerate_functions()
|
From: Skidmore, D. C <don...@in...> - 2013-12-26 18:19:07
|
We may be able to detect when we get number of physical functions that can't be correct (i.e. not 1 or 2) and at least log a different message rather than the current warning about insufficient bandwidth.
Thanks,
-Don
> -----Original Message-----
> From: Rustad, Mark D [mailto:mar...@in...]
> Sent: Thursday, December 26, 2013 9:40 AM
> To: Tantilov, Emil S
> Cc: e10...@li...; Chaitanya Lala
> Subject: Re: [E1000-devel] Possible bug in ixgbe-3.19.1
> ixgbe_enumerate_functions()
>
> On Dec 24, 2013, at 2:30 PM, "Tantilov, Emil S" <emi...@in...>
> wrote:
>
> > You are correct - there is a bug in the function when used in a VM since all
> devices appear as part of the same bus, which leads to the driver counting all
> devices.
> >
> > I'm not sure if there is a good way around this. When the device is direct
> attached in a VM we may not be able to tell if it's part of a multiport NIC.
> >
> > Thanks,
> > Emil
>
> This is not the only problem in this area. In my experience, all functions of a
> NIC that are direct attached will have a subfunction number of 0 in a VM. This
> means that there is no way to make the NIC look like it should in the VM.
>
> I take the issues in this area to largely be the result of deficiencies in the
> hypervisor and that they really should be addressed there. Principally, there
> needs to be a hypervisor supporting a PCIe guest architecture.
>
> >> -----Original Message-----
> >> From: Chaitanya Lala [mailto:Cha...@ri...]
> >> Sent: Monday, December 23, 2013 5:19 PM
> >> To: Tantilov, Emil S; Chaitanya Lala; e1000-
> >> de...@li...
> >> Subject: Re: [E1000-devel] Possible bug in ixgbe-3.19.1
> >> ixgbe_enumerate_functions()
> >>
> >> Hi,
> >>
> >> This is how all devices look.
> >>
> >> 00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma]
> >> (rev
> >> 02)
> >> Subsystem: Red Hat, Inc Qemu virtual machine
> >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx-
> >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >>
> >> 00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA
> >> [Natoma/Triton II]
> >> Subsystem: Red Hat, Inc Qemu virtual machine
> >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx-
> >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=medium
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Latency: 0
> >>
> >> 00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE
> >> [Natoma/Triton II] (prog-if 80 [Master])
> >> Subsystem: Red Hat, Inc Qemu virtual machine
> >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx-
> >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Latency: 0
> >> Region 0: [virtual] Memory at 000001f0 (32-bit, non-
> >> prefetchable) [size=8]
> >> Region 1: [virtual] Memory at 000003f0 (type 3, non-
> >> prefetchable) [size=1]
> >> Region 2: [virtual] Memory at 00000170 (32-bit, non-
> >> prefetchable) [size=8]
> >> Region 3: [virtual] Memory at 00000370 (type 3, non-
> >> prefetchable) [size=1]
> >> Region 4: I/O ports at c000 [size=16]
> >> Kernel driver in use: ata_piix
> >>
> >> 00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
> >> Subsystem: Red Hat, Inc Qemu virtual machine
> >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx-
> >> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Interrupt: pin A routed to IRQ 9
> >> Kernel driver in use: piix4_smbus
> >> Kernel modules: i2c-piix4
> >>
> >> 00:02.0 VGA compatible controller: Cirrus Logic GD 5446 (prog-if 00
> >> [VGA
> >> controller])
> >> Subsystem: Red Hat, Inc Device 1100
> >> Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx-
> >> Status: Cap- 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Region 0: Memory at f0000000 (32-bit, prefetchable) [size=32M]
> >> Region 1: Memory at f2000000 (32-bit, non-prefetchable) [size=4K]
> >> Expansion ROM at f2010000 [disabled] [size=64K]
> >> Kernel modules: cirrusfb
> >>
> >> 00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
> >> Subsystem: Red Hat, Inc Device 0001
> >> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx-
> >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Latency: 0
> >> Interrupt: pin A routed to IRQ 11
> >> Region 0: I/O ports at c020 [size=32]
> >> Region 1: Memory at f2020000 (32-bit, non-prefetchable) [size=4K]
> >> Expansion ROM at f2030000 [disabled] [size=64K]
> >> Capabilities: [40] MSI-X: Enable+ Count=3 Masked-
> >> Vector table: BAR=1 offset=00000000
> >> PBA: BAR=1 offset=00000800
> >> Kernel driver in use: virtio-pci
> >>
> >> 00:04.0 Ethernet controller: Intel Corporation Ethernet Controller
> >> 10-Gigabit X540-AT2 (rev 01)
> >> Subsystem: Intel Corporation Ethernet Converged Network Adapter
> >> X540-T2
> >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx+
> >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Latency: 0, Cache Line Size: 64 bytes
> >> Interrupt: pin B routed to IRQ 10
> >> Region 0: Memory at f2200000 (32-bit, prefetchable) [size=2M]
> >> Region 4: Memory at f2400000 (32-bit, prefetchable) [size=16K]
> >> Expansion ROM at f2480000 [disabled] [size=512K]
> >> Capabilities: [a0] Express (v2) Endpoint, MSI 00
> >> DevCap: MaxPayload 512 bytes, PhantFunc 0,
> >> Latency L0s <512ns, L1 <64us
> >> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+
> >> FLReset-
> >> DevCtl: Report errors: Correctable- Non-Fatal-
> >> Fatal- Unsupported-
> >> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> >> MaxPayload 256 bytes, MaxReadReq 512 bytes
> >> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
> >> AuxPwr- TransPend-
> >> LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s
> >> L1, Latency L0 <1us, L1
> >> <8us
> >> ClockPM- Surprise- LLActRep- BwNot-
> >> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled-
> >> Retrain- CommClk+
> >> ExtSynch- ClockPM- AutWidDis- BWInt-
> >> AutBWInt-
> >> LnkSta: Speed 5GT/s, Width x8, TrErr- Train-
> >> SlotClk- DLActive- BWMgmt-
> >> ABWMgmt-
> >> DevCap2: Completion Timeout: Range ABCD,
> >> TimeoutDis+
> >> DevCtl2: Completion Timeout: 50us to 50ms,
> >> TimeoutDis-
> >> LnkCtl2: Target Link Speed: 5GT/s,
> >> EnterCompliance- SpeedDis-,
> >> Selectable De-emphasis: -6dB
> >> Transmit Margin: Normal Operating Range,
> >> EnterModifiedCompliance-
> >> ComplianceSOS-
> >> Compliance De-emphasis: -6dB
> >> LnkSta2: Current De-emphasis Level: -3.5dB
> >> Capabilities: [40] Power Management version 3
> >> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
> >> PME(D0-,D1-,D2-,D3hot-,D3cold-)
> >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0
> >> PME-
> >> Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
> >> Vector table: BAR=4 offset=00000000
> >> PBA: BAR=4 offset=00002000
> >> Capabilities: [50] MSI: Enable- Count=1/1 Maskable-
> >> 64bit-
> >> Address: 00000000 Data: 0000
> >> Kernel driver in use: ixgbe
> >> Kernel modules: ixgbe
> >>
> >> 00:05.0 Ethernet controller: Intel Corporation Ethernet Controller
> >> 10-Gigabit X540-AT2 (rev 01)
> >> Subsystem: Intel Corporation Ethernet Converged Network Adapter
> >> X540-T2
> >> Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV-
> >> VGASnoop- ParErr-
> >> Stepping- SERR- FastB2B- DisINTx+
> >> Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast
> >>> TAbort- <TAbort-
> >> <MAbort- >SERR- <PERR- INTx-
> >> Latency: 0, Cache Line Size: 64 bytes
> >> Interrupt: pin A routed to IRQ 10
> >> Region 0: Memory at f2600000 (32-bit, prefetchable) [size=2M]
> >> Region 4: Memory at f2800000 (32-bit, prefetchable) [size=16K]
> >> Expansion ROM at f2880000 [disabled] [size=512K]
> >> Capabilities: [a0] Express (v2) Endpoint, MSI 00
> >> DevCap: MaxPayload 512 bytes, PhantFunc 0,
> >> Latency L0s <512ns, L1 <64us
> >> ExtTag- AttnBtn- AttnInd- PwrInd- RBE+
> >> FLReset-
> >> DevCtl: Report errors: Correctable- Non-Fatal-
> >> Fatal- Unsupported-
> >> RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+
> >> MaxPayload 256 bytes, MaxReadReq 512 bytes
> >> DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq-
> >> AuxPwr- TransPend-
> >> LnkCap: Port #0, Speed 5GT/s, Width x8, ASPM L0s
> >> L1, Latency L0 <1us, L1
> >> <8us
> >> ClockPM- Surprise- LLActRep- BwNot-
> >> LnkCtl: ASPM Disabled; RCB 64 bytes Disabled-
> >> Retrain- CommClk+
> >> ExtSynch- ClockPM- AutWidDis- BWInt-
> >> AutBWInt-
> >> LnkSta: Speed 5GT/s, Width x8, TrErr- Train-
> >> SlotClk- DLActive- BWMgmt-
> >> ABWMgmt-
> >> DevCap2: Completion Timeout: Range ABCD,
> >> TimeoutDis+
> >> DevCtl2: Completion Timeout: 50us to 50ms,
> >> TimeoutDis-
> >> LnkCtl2: Target Link Speed: 2.5GT/s,
> >> EnterCompliance- SpeedDis-,
> >> Selectable De-emphasis: -6dB
> >> Transmit Margin: Normal Operating Range,
> >> EnterModifiedCompliance-
> >> ComplianceSOS-
> >> Compliance De-emphasis: -6dB
> >> LnkSta2: Current De-emphasis Level: -6dB
> >> Capabilities: [40] Power Management version 3
> >> Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
> >> PME(D0-,D1-,D2-,D3hot-,D3cold-)
> >> Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0
> >> PME-
> >> Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
> >> Vector table: BAR=4 offset=00000000
> >> PBA: BAR=4 offset=00002000
> >> Capabilities: [50] MSI: Enable- Count=1/1 Maskable-
> >> 64bit-
> >> Address: 00000000 Data: 0000
> >> Kernel driver in use: ixgbe
> >> Kernel modules: ixgbe
> >>
> >>
> >>
> >>
> >>
> >> On 12/23/13 4:13 PM, "Tantilov, Emil S"
> >> <emi...@in...> wrote:
> >>
> >>> Can you provide the output of lspci -vvv from the network
> >> device in the
> >>> VM?
> >>>
> >>> Thanks,
> >>> Emil
> >>>
> >>>> -----Original Message-----
> >>>> From: Chaitanya Lala [mailto:Cha...@ri...]
> >>>> Sent: Monday, December 23, 2013 4:03 PM
> >>>> To: e10...@li...
> >>>> Cc: Chaitanya Lala
> >>>> Subject: [E1000-devel] Possible bug in ixgbe-3.19.1
> >>>> ixgbe_enumerate_functions()
> >>>>
> >>>> Hello,
> >>>>
> >>>> I am testing ixgbe-3.19.1 with my dual port x540 10G Base-
> >> T
> >>>> card (PCIe
> >>>> passthrough into a Ubuntu 12.10 VM).
> >>>> I am seeing a warning like this -
> >>>>
> >>>> "This is not sufficient for optimal performance of this card. For
> >>>> optimal performance , at least 80 GT/s of bandwidth is required."
> >>>>
> >>>> This is not correct because I have a dual 10G port card
> >> and
> >>>> have a PCIe
> >>>> Gen2 based system and the card is in a x8 slot. So B/W should not
> >>>> be an issue On tracing the code it seems like a bug in ->
> >>>>
> >>>> static inline int ixgbe_enumerate_functions(struct ixgbe_adapter
> >>>> *adapter) {
> >>>> struct list_head *entry;
> >>>> int physfns = 0;
> >>>>
> >>>> /* Some cards can not use the generic count PCIe functions
> >>>> method,
> >>>> * because they are behind a parent switch, so we hardcode
> >>>> these to
> >>>> * correct number of ports.
> >>>> */
> >>>> if (ixgbe_pcie_from_parent(&adapter->hw)) {
> >>>> physfns = 4;
> >>>> } else {
> >>>> list_for_each(entry, &adapter->pdev-
> >>>>> bus_list) {
> >>>> <-------------- This counts all pcie devices in the system not just
> >>>> the ones matching this card/intel NIC #ifdef CONFIG_PCI_IOV
> >>>> struct pci_dev *pdev =
> >>>> list_entry(entry, struct pci_dev,
> >>>> bus_list);
> >>>>
> >>>> /* don't count virtual functions
> >> */
> >>>> if (pdev->is_virtfn)
> >>>> continue; #endif
> >>>>
> >>>> physfns++; <------ Should only be
> >>>> incremented when an Intel NIC is found
> >>>> }
> >>>>
> >>>> I can send a patch but not sure if what I am thinking is the
> >>>> correct way to solve this.
> >>>>
> >>>> Thanks,
> >>>> Chaitanya
> >>>>
> >>>>
> >>>> ----------------------------------------------------------
> >> --
> >>>> ------------------
> >>>> Rapidly troubleshoot problems before they affect your business.
> >>>> Most IT organizations don't have a clear picture of how
> >> application
> >>>> performance
> >>>> affects their revenue. With AppDynamics, you get 100% visibility
> >>>> into your Java,.NET, & PHP application. Start your 15-day FREE
> >>>> TRIAL of AppDynamics Pro!
> >>>> http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=
> >> /4
> >>>> 140/ostg.clktrk
> >>>> _______________________________________________
> >>>> E1000-devel mailing list
> >>>> E10...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/e1000-devel
> >>>> To learn more about Intel® Ethernet, visit
> >>>> http://communities.intel.com/community/wired
> >
> >
> > ----------------------------------------------------------------------
> > -------- Rapidly troubleshoot problems before they affect your
> > business. Most IT organizations don't have a clear picture of how
> > application performance affects their revenue. With AppDynamics, you
> > get 100% visibility into your Java,.NET, & PHP application. Start your
> > 15-day FREE TRIAL of AppDynamics Pro!
> > http://pubads.g.doubleclick.net/gampad/clk?id=84349831&iu=/4140/ostg.c
> > lktrk _______________________________________________
> > E1000-devel mailing list
> > E10...@li...
> > https://lists.sourceforge.net/lists/listinfo/e1000-devel
> > To learn more about Intel® Ethernet, visit
> > http://communities.intel.com/community/wired
>
> --
> Mark Rustad, Networking Division, Intel Corporation
|