Menu
▾
▴
Re: [Astlinux-users] APU2 keeps locking up
|
From: Lonnie A. <li...@lo...> - 2020-10-11 21:05:45
|
Thanks Michael for closing the loop on this issue. Additionally, AstLinux 1.4.0 should solve this issue without needing the workaround. Lonnie > On Oct 11, 2020, at 2:34 PM, Michael Knill <mic...@ip...> wrote: > > PS just letting the group know that I haven’t had any lockups since adding this workaround which is certainly a relief. > Thanks all for your help. > > Regards > Michael Knill > > On 21/9/20, 8:34 am, "Michael Knill" <mic...@ip...> wrote: > > Thanks Lonnie for all your help. > > Regards > Michael Knill > > On 21/9/20, 7:37 am, "Lonnie Abelbeck" <li...@lo...> wrote: > > Hi Michael, > > Considering this is a very obscure kernel bug that has been around for a long time, and AstLinux 1.4.x will have it fixed ... I personally would not get carried away implementing the workaround. > > BTW, the PHYETH_DISABLE_OFFLOAD workaround only applies to physical ethernet NICs. > > You understand the situation, so do what you think is best. > > Lonnie > > > > >> On Sep 20, 2020, at 3:45 PM, Michael Knill <mic...@ip...> wrote: >> >> Thanks Lonnie >> >> One last question; could I just blanket add this to all my systems or are there any that will certainly not be affected by this bug e.g. VM or virtual NIC? >> The plan is to have this command active by default and commented out if desired. >> >> Regards >> Michael Knill >> >> On 20/9/20, 9:58 pm, "Lonnie Abelbeck" <li...@lo...> wrote: >> >>> Do you think I should remove PHYETH_DISABLE_OFFLOAD and try a BIOS upgrade to see if it fixes the problem? >> >> No ... first make sure setting PHYETH_DISABLE_OFFLOAD solves the issue with your DLS provider and your APU2s. >> >> After that, AstLinux 1.4.0 will be the solution without setting PHYETH_DISABLE_OFFLOAD . >> >> Ideally, you could test a 1.4-pre-release on an APU2 with the new DLS provider to make certain we have identified the issue. >> https://www.astlinux-project.org/dev.html >> >> Lonnie >> >> >>> On Sep 20, 2020, at 5:24 AM, Michael Knill <mic...@ip...> wrote: >>> >>> Yes my BIOS is quite old: >>> 3999-IPCBuild-CM1 kd # dmesg | grep DMI >>> [ 0.000000] DMI: PC Engines APU, BIOS SageBios_PCEngines_APU-45 04/05/2014 >>> >>> Funny as it's a pretty new box. >>> >>> Do you think I should remove PHYETH_DISABLE_OFFLOAD and try a BIOS upgrade to see if it fixes the problem? >>> I don't think I will for existing sites as a BIOS upgrade looks pretty hard to do and I assume cannot be done remotely. >>> >>> Regards >>> Michael Knill >>> >>> On 20/9/20, 12:18 pm, "Lonnie Abelbeck" <li...@lo...> wrote: >>> >>> Hi Michael, >>> >>> Ahhh, very good ... looks like we are on to something. >>> >>> Add it to one/some of your APU2s and let us know how it goes. >>> >>> As far as my Qotom Q190G4N, I initially had to set PHYETH_DISABLE_OFFLOAD to keep it from locking-up with sustained high network traffic but then switched the RAM SO-DIMM with another brand and did not need PHYETH_DISABLE_OFFLOAD anymore. My comments probably got you started adding PHYETH_DISABLE_OFFLOAD. >>> >>> This is a very obscure kernel bug, as such it never got back-ported to Linux 3.16.x . >>> >>> For the APU2, the BIOS could play a role in how it initializes the NICs and whether this kernel bug is triggered. >>> >>> Lonnie >>> >>> >>> >>>> On Sep 19, 2020, at 7:49 PM, Michael Knill <mic...@ip...> wrote: >>>> >>>> Awesome thanks Lonnie. >>>> Yes its all making sense now. I already have this directive in my template against the Qotom Q190G4U for some reason (should I have?) >>>> I have two Qotoms connected to the problem provider, one had this directive set already and has not failed and one did not (I forgot to change when I changed hardware) which fails. >>>> All my APU's don't have this set so all have problems with this provider. >>>> >>>> I'm thinking we have finally solved this issue. >>>> Thanks so much for your help >>>> >>>> Regards >>>> Michael Knill >>>> >>>> On 20/9/20, 9:29 am, "Lonnie Abelbeck" <li...@lo...> wrote: >>>> >>>> I would try this first >>>> -- >>>> PHYETH_DISABLE_OFFLOAD="tso gso gro" >>>> -- >>>> and see if it fixes the problem. >>>> >>>> If by chance it does fix it, then it would not be needed in AstLinux 1.4.x. >>>> >>>> The PHYETH_DISABLE_OFFLOAD settings disable some of the "offload" features of the NICs in an effort to work around this (somewhat obscure) kernel bug. >>>> >>>> It all kind of makes sense that a particular provider is fragmenting packets in ways others do not, and hits this kernel bug. >>>> >>>> The PHYETH_DISABLE_OFFLOAD setting above is very "safe", only drawback is it slightly reduces network performance near the 1 Gbps level. >>>> >>>> BTW, if traffic shaping is enabled this PHYETH_DISABLE_OFFLOAD setting is already applied to external ethernet NIC(s). >>>> >>>> Lonnie >>>> >>>> >>>> >>>>> On Sep 19, 2020, at 6:09 PM, Michael Knill <mic...@ip...> wrote: >>>>> >>>>> Awesome thanks Lonnie. >>>>> >>>>> I will give it a try although I have no idea what it does! >>>>> I assume I can remove this when I go to Astlinux 1.4? >>>>> >>>>> Regards >>>>> Michael Knill >>>>> >>>>> Sent from my iPhone so please excuse my brevity. >>>>> >>>>>> On 19 Sep 2020, at 11:56 pm, Lonnie Abelbeck <li...@lo...> wrote: >>>>>> >>>>>> >>>>>> Hi Michael, >>>>>> >>>>>> Great info! >>>>>> >>>>>> Try this in your user.conf, and reboot. >>>>>> -- >>>>>> PHYETH_DISABLE_OFFLOAD="tso gso gro" >>>>>> -- >>>>>> >>>>>> If my hunch is correct, this kernel fix added in 4.1.17 may be related ... >>>>>> >>>>>> net: preserve IP control block during GSO segmentation >>>>>> https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/include/linux/skbuff.h?h=v4.1.17&id=abefd1b4087b9b5e83e7b4e7689f8b8e3cb2899c >>>>>> >>>>>> Lonnie >>>>>> >>>>>> >>>>>> >>>>>>> On Sep 18, 2020, at 10:56 PM, Michael Knill <mic...@ip...> wrote: >>>>>>> >>>>>>> Yay some progress on this problem. >>>>>>> >>>>>>> I had my 4th site lock up yesterday. It was a site I moved from one location to another. There were no changes to the Astlinux box at all other than PPPoE credentials but after a couple of hours it locked up. So realistically the only change is the internet provider which is a new one that I am trialling and is the same provider as two of the other sites that are failing. >>>>>>> >>>>>>> As we are also using this provider in our home office, I set up another box this morning and connected the serial port not expecting anything to happen but it locked up and we captured it. Yay! It is attached. >>>>>>> >>>>>>> I'm hoping it will help the resolution of this problem. >>>>>>> >>>>>>> Regards >>>>>>> Michael Knill >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> Astlinux-users mailing list >>>>>> Ast...@li... >>>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >>>>>> >>>>>> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >>>>>> <APU Crash.log> >>>>> _______________________________________________ >>>>> Astlinux-users mailing list >>>>> Ast...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >>>>> >>>>> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >>>> >>>> >>>> >>>> _______________________________________________ >>>> Astlinux-users mailing list >>>> Ast...@li... >>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >>>> >>>> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >>>> >>>> >>>> _______________________________________________ >>>> Astlinux-users mailing list >>>> Ast...@li... >>>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >>>> >>>> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >>> >>> >>> >>> _______________________________________________ >>> Astlinux-users mailing list >>> Ast...@li... >>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >>> >>> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >>> >>> >>> _______________________________________________ >>> Astlinux-users mailing list >>> Ast...@li... >>> https://lists.sourceforge.net/lists/listinfo/astlinux-users >>> >>> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >> >> >> >> _______________________________________________ >> Astlinux-users mailing list >> Ast...@li... >> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... >> >> >> _______________________________________________ >> Astlinux-users mailing list >> Ast...@li... >> https://lists.sourceforge.net/lists/listinfo/astlinux-users >> >> Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... > > > > _______________________________________________ > Astlinux-users mailing list > Ast...@li... > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... > > > _______________________________________________ > Astlinux-users mailing list > Ast...@li... > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... > > > _______________________________________________ > Astlinux-users mailing list > Ast...@li... > https://lists.sourceforge.net/lists/listinfo/astlinux-users > > Donations to support AstLinux are graciously accepted via PayPal to pa...@kr.... |
