|
From: Devil-Linux <dev...@js...> - 2011-05-05 15:01:46
|
Hi all, We just upgraded to latest software on one router/VPN/firewall. I had to change to ipsec-tools instead of *swan, but else all seemed fine from the 1.2.13 we had before. Anyways, as soon as we put trafik through (VPN) it it become totaly unresponcive, and top always hangs having either close to 100%si or close to 100% on ksoftirqd. The system is stale for a minute or so and then clears upp again, probably because the services runing trafic through it timeout. From watching /proc/interrupts it seem to be the network cards generating alot of interupts. We have tried changing networkcards from 3com to intel. We even bought a new machine, to change everything but still get the same problem. The new machine works better though as the IRQ only hog one CPU/core att a time so the system still answers. My next step is to try and recompile Devil-Linux with noapic and try that, but before I go ahead I thought I might ask here and see if you guys know anything else I could try. It might be just some silly mistake. Kind Regards Jacob Sandin |
|
From: Heiko Z. <he...@zu...> - 2011-05-05 21:35:01
|
Quoting Devil-Linux <dev...@js...>: > Hi all, > > We just upgraded to latest software on one router/VPN/firewall. > > I had to change to ipsec-tools instead of *swan, but else all seemed fine from the 1.2.13 we had before. > > Anyways, as soon as we put trafik through (VPN) it it become totaly unresponcive, and top always hangs having either close to 100%si or close to 100% on ksoftirqd. > > The system is stale for a minute or so and then clears upp again, probably because the services runing trafic through it timeout. > > From watching /proc/interrupts it seem to be the network cards generating alot of interupts. > > We have tried changing networkcards from 3com to intel. We even bought a new machine, to change everything but still get the same problem. > > The new machine works better though as the IRQ only hog one CPU/core att a time so the system still answers. > > My next step is to try and recompile Devil-Linux with noapic and try that, but before I go ahead I thought I might ask here and see if you guys know anything else I could try. It might be just some silly mistake. I asked google quickly and it seems you're not the only one with the issue. Unfortunately I couldn't find a post with a solution. -- Regards Heiko Zuerker http://www.devil-linux.org ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. |
|
From: Rudner, B. <br...@ba...> - 2011-05-06 06:35:16
Attachments:
smime.p7s
|
Hi Jacob, I remember that I also had this problem a while ago. I used heartbeat and apache on two servers with virtual-IP and loadbalancing. I also encountered the problem with the ksoftirq on 100%. I did not exactly remember what solved the problem but I have done some changes: 1. I switched from heartbeat to keepalived. 2. I have done some tweaks in the sysctl. I do not exactly know which of them solved the problem but you might try it. Here are the settings from my sysctl.conf that could be involved: kernel.panic=3 #net.ipv4.netfilter.ip_conntrack_max=65536 net.ipv4.tcp_ecn=1 net.ipv4.tcp_fin_timeout=30 net.ipv4.tcp_keepalive_time=120 net.ipv4.tcp_timestamps=0 net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=300 net.ipv4.netfilter.ip_conntrack_udp_timeout=60 net.ipv4.netfilter.ip_conntrack_udp_timeout_stream=180 # Set conntrack-stuff higher net.netfilter.nf_conntrack_max=3000000 Cheers, Björn Rudner Von: Devil-Linux [mailto:dev...@js...] Gesendet: Donnerstag, 5. Mai 2011 16:43 An: dev...@li... Betreff: [Devil-Linux-discuss] ksoftirqd 100% Hi all, We just upgraded to latest software on one router/VPN/firewall. I had to change to ipsec-tools instead of *swan, but else all seemed fine from the 1.2.13 we had before. Anyways, as soon as we put trafik through (VPN) it it become totaly unresponcive, and top always hangs having either close to 100%si or close to 100% on ksoftirqd. The system is stale for a minute or so and then clears upp again, probably because the services runing trafic through it timeout. >From watching /proc/interrupts it seem to be the network cards generating alot of interupts. We have tried changing networkcards from 3com to intel. We even bought a new machine, to change everything but still get the same problem. The new machine works better though as the IRQ only hog one CPU/core att a time so the system still answers. My next step is to try and recompile Devil-Linux with noapic and try that, but before I go ahead I thought I might ask here and see if you guys know anything else I could try. It might be just some silly mistake. Kind Regards Jacob Sandin |
|
From: Jacob S. <dev...@js...> - 2011-05-06 12:02:17
|
Hi, Björn, thank you for your sugestions, I tried adding the values to sysctl, but it sadly did not change the problem. Most similar problems I found on "google" seemed to be related to a faulty bios, kernel or drivers. I have changed both machine and nic's, only kernel left on that track. I also tried disable IPv6 but it did not change anything. Kind Regards Jacob On Fri, 6 May 2011 08:18:21 +0200, Rudner, Björn <br...@ba...> wrote: > Hi Jacob, > > I remember that I also had this problem a while ago. > > > > I used heartbeat and apache on two servers with virtual-IP and > loadbalancing. I also encountered the problem with the ksoftirq on 100%. > > > > I did not exactly remember what solved the problem but I have done some > changes: > > 1. I switched from heartbeat to keepalived. > > 2. I have done some tweaks in the sysctl. I do not exactly know > which of them solved the problem but you might try it. > > Here are the settings from my sysctl.conf that could be involved: > > > > kernel.panic=3 > > #net.ipv4.netfilter.ip_conntrack_max=65536 > > net.ipv4.tcp_ecn=1 > > net.ipv4.tcp_fin_timeout=30 > > net.ipv4.tcp_keepalive_time=120 > > net.ipv4.tcp_timestamps=0 > > net.ipv4.netfilter.ip_conntrack_tcp_timeout_established=300 > > net.ipv4.netfilter.ip_conntrack_udp_timeout=60 > > net.ipv4.netfilter.ip_conntrack_udp_timeout_stream=180 > > > > # Set conntrack-stuff higher > > net.netfilter.nf_conntrack_max=3000000 > > > > Cheers, > > > > Björn Rudner > > > > > > > > > > Von: Devil-Linux [mailto:dev...@js...] > Gesendet: Donnerstag, 5. Mai 2011 16:43 > An: dev...@li... > Betreff: [Devil-Linux-discuss] ksoftirqd 100% > > > > Hi all, > > We just upgraded to latest software on one router/VPN/firewall. > > I had to change to ipsec-tools instead of *swan, but else all seemed fine > from the 1.2.13 we had before. > > Anyways, as soon as we put trafik through (VPN) it it become totaly > unresponcive, and top always hangs having either close to 100%si or close > to 100% on ksoftirqd. > > The system is stale for a minute or so and then clears upp again, probably > because the services runing trafic through it timeout. > >>From watching /proc/interrupts it seem to be the network cards generating >>alot of interupts. > > We have tried changing networkcards from 3com to intel. We even bought a > new machine, to change everything but still get the same problem. > > The new machine works better though as the IRQ only hog one CPU/core att a > time so the system still answers. > > My next step is to try and recompile Devil-Linux with noapic and try that, > but before I go ahead I thought I might ask here and see if you guys know > anything else I could try. It might be just some silly mistake. > > Kind Regards > > Jacob Sandin |