[E1000-devel] Platform failed to complete resume from S3 with PCIE i210 WOL with IOMMU enabled

[Chuah, W. K. <wee...@in...>]

In current Intel under development project - EHL, we discovered an issue that i210 PCIe NIC wake-on-lan test will resultant in platform hang upon wake up S3(sleep) --> S0, with IOMMU enabled. If we disable the VT-d/IOMMU in BIOS, this feature WOL is working fine without any hang.

Version: igb: Intel(R) Gigabit Ethernet Network Driver - version 5.6.0-k

Somehow we managed to pull out the dmesg using off-the-shelve method with JTAG device. From the dmesg found that igb_up() not able to access the memory for some reason the address ffff88825f6dd101 - page fault.
igb driver not able to access the address, it requires permission. Need support from the Ethernet driver team, as this i210 Ethernet NIC Intel related product that should work in Intel platform which has IOMMU always enabled by default.

Please advise how do we proceed?

6a422f6d85:RIP: 0010:igb_up+0x4c/0x150 [igb]4000
6a422f7421:Code: d2 eb 16 f0 80 60 60 fe f0 80 60 60 f7 48 83 c2 01 39 93 14 02 00 00 76 13 48 8b 84 d3 08 0f 00 00 48 8b 48 60 83 e1 01 75 d9 <0f> 0b f6 83 11 02 00 00 20 0f 85 be 00 00 00 48 8b bb 08 0f 00 00mp_thermal kvm_int
6a422f777f:RSP: 0018:ffffc900003dbd30 EFLAGS: 000102467 48 83 c2 01 39 9
6a422f7c58:RAX: ffff888249751800 RBX: ffff88824249c840 RCX: 0000000000000000 02 00 00 76 13 48 8
6a422f7fa1:RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff888249752f00 02 00 00 76 13 48 8
6a422f8330:RBP: ffff88824249c000 R08: 00000000000000b1 R09: ffffc90000c13000 02 00 00 76 13 48 8
6a422f8690:R10: 0000000000220fe7 R11: 00000000002e6f74 R12: ffff88824249c840 02 00 00 76 13 48 8
6a422f89e8:R13: ffff8882611ce000 R14: ffff8882611ce028 R15: ffffc900003dbdb4 02 00 00 76 13 48 8
6a422fa0a6: igb_io_resume+0x2a/0x50 [igb]f8882611ce028 R15:

6a46d18c27:RIP: 0010:igb_up+0x4c/0x150 [igb]are boot completefff
6a46d19778:Code: d2 eb 16 f0 80 60 60 fe f0 80 60 60 f7 48 83 c2 01 39 93 14 02 00 00 76 13 48 8b 84 d3 08 0f 00 00 48 8b 48 60 83 e1 01 75 d9 <0f> 0b f6 83 11 02 00 00 20 0f 85 be 00 00 00 48 8b bb 08 0f 00 00mp_thermal kvm_int
6a46d19c69:RSP: 0018:ffffc900003dbd30 EFLAGS: 000102467 48 83 c2 01 39 9
6a46d1a48e:RAX: ffff888249751800 RBX: ffff88824249c840 RCX: 0000000000000000 02 00 00 76 13 48 8
6a46d1a886:RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff888249752f00 02 00 00 76 13 48 8
6a46d1ac61:RBP: ffff88824249c000 R08: 00000000000000b1 R09: ffffc90000c13000 02 00 00 76 13 48 8
6a46d1b063:R10: 0000000000220fe7 R11: 00000000002e6f74 R12: ffff88824249c840 02 00 00 76 13 48 8
6a46d1b3f7:R13: ffff8882611ce000 R14: ffff8882611ce028 R15: ffffc900003dbdb4 02 00 00 76 13 48 8
6a46d1ba1b:FS:  0000000000000000(0000) GS:ffff888264000000(0000) knlGS:00000000000000006 13 48 8
6a46d1be45:CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033000) knlGS:000000000
6a46d1c231:CR2: 000055da027c3ae0 CR3: 000000000240a000 CR4: 0000000000340ef00000
6a46d1f9e7:kernel tried to execute NX-protected page - exploit attempt? (uid: 0)
6a46d1fe80:BUG: unable to handle page fault for address: ffff88825f6dd101 uid: 0)
6a46d20401:#PF: supervisor instruction fetch in kernel modeff88825f6dd101uid
6a46d20780:#PF: error_code(0x0011) - permissions violationeff88825f6dd101uid
6a46d20c4e:PGD 3601067 P4D 3601067 PUD 3604067 PMD 800000025f6001e3 dd101uid
6a46d2193e:Oops: 0011 [#2] PREEMPT SMP NOPTI67 PMD 800000025f600


6a46d22ffa:RIP: 0010:0xffff88825f6dd101tion Elkhart Lake
6a47cd6122:RIP: 0010:igb_up+0x4c/0x150 [igb]---gx(O)
6a47cd6890:Code: d2 eb 16 f0 80 60 60 fe f0 80 60 60 f7 48 83 c2 01 39 93 14 02 00 00 76 13 48 8b 84 d3 08 0f 00 00 48 8b 48 60 83 e1 01 75 d9 <0f> 0b f6 83 11 02 00 00 20 0f 85 be 00 00 00 48 8b bb 08 0f 00 00mp_thermal kvm_int
6a47cd6ca0:RSP: 0018:ffffc900003dbd30 EFLAGS: 000102467 48 83 c2 01 39 9
6a47cd71de:RAX: ffff888249751800 RBX: ffff88824249c840 RCX: 0000000000000000 02 00 00 76 13 48 8
6a47cd754a:RDX: 0000000000000000 RSI: 0000000000000004 RDI: ffff888249752f00 02 00 00 76 13 48 8
6a47cd7908:RBP: ffff88824249c000 R08: 00000000000000b1 R09: ffffc90000c13000 02 00 00 76 13 48 8
6a47cd7c9c:R10: 0000000000220fe7 R11: 00000000002e6f74 R12: ffff88824249c840 02 00 00 76 13 48 8
6a47cd7fff:R13: ffff8882611ce000 R14: ffff8882611ce028 R15: ffffc900003dbdb4 02 00 00 76 13 48 8
6a47cd84f0:FS:  0000000000000000(0000) GS:ffff888264000000(0000) knlGS:00000000000000006 13 48 8
6a47cd88ea:CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033000) knlGS:000000000
6a47cd8c71:CR2: ffff88825f6dd101 CR3: 000000000240a000 CR4: 0000000000340ef00000
6a47cd90ac:Fixing recursive fault but reboot is needed!CR4: 000000000034
6a49c1c69d:Setting south_chicken1 SBCLK_RUN_REFCLK_DIS!CR4: 000000000034