FireHOL / Support Requests / #9 Problems abnormally slow and descending transfer rate

Jimisola Laursen - 2004-07-11

labels: 443836 -->
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

Logged In: YES
user_id=935333

Additional information:

I also get very strange (slow - usual is <50ms at least) ping
rates:

PING 194.16.57.126 (194.16.57.126) 56(84) bytes of data.
64 bytes from 194.16.57.126: icmp_seq=7 ttl=255 time=0.327
ms
64 bytes from 194.16.57.126: icmp_seq=1 ttl=255 time=6011
ms
64 bytes from 194.16.57.126: icmp_seq=2 ttl=255 time=5000
ms
64 bytes from 194.16.57.126: icmp_seq=3 ttl=255 time=4000
ms
64 bytes from 194.16.57.126: icmp_seq=4 ttl=255 time=3000
ms
64 bytes from 194.16.57.126: icmp_seq=5 ttl=255 time=2000
ms
64 bytes from 194.16.57.126: icmp_seq=6 ttl=255 time=1000
ms
64 bytes from 194.16.57.126: icmp_seq=8 ttl=255 time=8768
ms
64 bytes from 194.16.57.126: icmp_seq=9 ttl=255 time=7768
ms
64 bytes from 194.16.57.126: icmp_seq=10 ttl=255
time=6768 ms
64 bytes from 194.16.57.126: icmp_seq=11 ttl=255
time=5768 ms
64 bytes from 194.16.57.126: icmp_seq=12 ttl=255
time=4768 ms
64 bytes from 194.16.57.126: icmp_seq=13 ttl=255
time=3768 ms
64 bytes from 194.16.57.126: icmp_seq=14 ttl=255
time=2768 ms
64 bytes from 194.16.57.126: icmp_seq=15 ttl=255
time=1768 ms
64 bytes from 194.16.57.126: icmp_seq=16 ttl=255 time=768
ms
64 bytes from 194.16.57.126: icmp_seq=26 ttl=255
time=0.309 ms

Jimisola

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Costa Tsaousis - 2004-07-11

labels: --> Install Problem (example)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Costa Tsaousis - 2004-07-11

Logged In: YES
user_id=582393

Jimisola,

you mention 2 problems:

1. Slow traffic.
You are the first person reporting this. Normally FireHOL has
nothing to do with the response or the speed of iptables. It is
your kernel that runs the firewall. FireHOL just configures it.

Anyway, to check if iptables is the problem for the slow
response, could you please stop the firewall while you
experience the problem and see if the normal speed is
restored?

Other things to check:

a. the "protection" statement will block requests if the limit is
too low. I see that you have 10/s. Please understand that
this is extremely low if you have a lot of bandwidth.

b. if your machine is slow, avoid "masquerade" if you can and
use "snat" instead. Snat is by very light (in cpu consumption)
compared to masquerade. To use snat you have to have
static IPs.

c. The iptables connection tracker has various limits regarding
the maximum connections. Do:

sysctl -a | grep conntrack

to see the current settings. Pay special attention to
net.ipv4.netfilter.ip_conntrack_max which is the maximum
number of connections that may be open at any given time.

2. Inability to use NATed machines.
Does this happen when you have just booted your linux box,
or at random times?

It has been reported by a few users that when they
upgraded to v1.191 they could not use the firewall. The
reason is that v1.191 turns off FireHOL's startup at boot. If
you re-enable it, it will work.

If it happens at random times, something else is happening.

You can check the running firewall by:

iptables -nL

If you save this output when NATed machines work and you
re-save it when the have problems, and then you compare
the two files, do they differ? If they do, some other program
(not FireHOL) is changing your firewall. FireHOL activates and
forgets the running firewall. It does not change it while it is
running.

Anyway, do you have any logs while the NATed machines are
not operational? FireHOL always logs dropped traffic.

Costa

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

Logged In: YES
user_id=935333

Hi Costa!

Thank you for your reply and words of advice.

I am aware of the fact that FireHOL just configures iptables.
Reason for asking here is primary that I might have setup
FireHOL wrong (too
hard limits or likewise), and secondly, you or someone might
seen/heard about
this problems before.

Btw, I forgot to tell initally that I have tried shutting down
firewall,
iptables and actually removing the kernel module completely.
The problem is, I must say, very strange in the way that it
can work perfect for
months then suddenly it causes problems for a random period.

Anyway, I have gone through your "steps":

1. Slow traffic

I stopped a) Firehol, b) iptables and c) even removed iptables
module (did
modprobe -r iptables). No change.

a. I commented out the protection (10s) statement and still
no change.

b. I can't use SNAT since I don't have static IPs. I doubt that
lack of computer
power is the
issue since I'm running this on an AMD Athlon 800Mhz/512MB
(99.7% idle says
top) and as I said it do work for longer periods without any
problems.

c. sysctl -a | grep conntrack gave:

net.ipv4.ip_conntrack_max = 32760

I don't know if this is a high or low value. The firewall is used
at home and I
am the only user. Is there any chance that this is too low?
Can I check how many
connections are currently open?

Conclusion:

None of the steps gave any positive feedback. I have had
this problem for months
and I can't seem to figure out what it is.
I have searched extensively on the Internet without finding
anyone with a
similar problem.
Things to check I guess, are if there are any other services
that causes this
problem or if this could be a hardware failure.
Can it be a cache or something like that that is causing
problems? At first I
thought the problem had soemthing to do with change of IP
address on my public
interface, but that does not seem to be it either.

> 2. Inability to use NATed machines.
> Does this happen when you have just booted your linux box,
> or at random times?

> It has been reported by a few users that when they
> upgraded to v1.191 they could not use the firewall. The
> reason is that v1.191 turns off FireHOL's startup at boot. If
> you re-enable it, it will work.

It happens only when I boot up and I usually have to start up
FireHOL manually
which it a bit annoying.
I haven't been able to figure out why this happens as the
FireHOL service seems
to be starting up nicely.
Is there a solution to the problemm i.e. is there a way to get
around the manual
start up?

However, I do need to ask you something about the
interaction of iptables and
firehol.
I see know that I have BOTH iptables and firehol being
started as services
during boot.
Is this really the way it should be? I thought that either I
have firehol as a
service or I have iptables as a service (using the
saved /etc/sysconfig/iptables
file.

> Anyway, do you have any logs while the NATed machines
are
> not operational? FireHOL always logs dropped traffic.

Will attach /var/log/messages from today. I can't see any
traffic being rejected/dropped all-the-time from the
intranet/LAN interface (eth0) though.

Kind regards,
Jimisola

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

/var/log/messages

messages

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

Logged In: YES
user_id=935333

I just noticed something that I do not feel is correct. I
rebooted my machine and whoops: the ping rates are back to
normal. So, I did iptables -nL - all three default chains are
empty with default policy ACCEPT! So, firehol did not start up
during boot again. I therefore issued 'firehol restart' and after
that I still had my normal low ping rates (even from my
machines using NAT). As I said, I noticed something strange
and that is that ip_tables and ip_conntrack were loaded. One
would thing that ip_tables should have been loaded during
boot as it is a service and firehol who is also a service loads it
as well.

Jul 11 23:35:04 h kernel: ip_tables: (C) 2000-2002 Netfilter
core team
Jul 11 23:42:56 h firehol: FireHOL: Saving your old firewall to
a temporary file: succeeded
Jul 11 23:43:02 h firehol: FireHOL: Processing
file /etc/firehol/firehol.conf: succeeded
Jul 11 23:43:02 h kernel: ip_conntrack version 2.1 (4095
buckets, 32760 max) - 320 bytes per conntrack
Jul 11 23:43:08 h firehol: FireHOL: Activating new firewall
(697 rules): succeeded

Ping rates are now ~15ms for sites that I had 5-6000ms to
before.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

diff of saved iptables in working and non-working state (working was 1st parameter)

iptables.diff

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

Logged In: YES
user_id=935333

Added a file iptables.diff:
diff /etc/sysconfig/iptables.working /etc/sysconfig/iptables.NO
N-working > iptables.diff.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Costa Tsaousis - 2004-07-11

Logged In: YES
user_id=582393

Jimisola,

it has been reported by a user that the kernel iptables
modules fail to start at boot with certain kernel versions. For
this reason I have added (in v1.194) an "iptables -nL
>/dev/null" at the top of FireHOL, in order to initialize iptables
before sending commands to it.

You can get this version from
http://firehol.sf.net/firehol.tar.gz. Just replace
your /etc/init.d/firehol with the file "firehol.sh" found in this
archive.

Answers to your questions:
1. Can I check how many connections are currently open?

cat /proc/net/ip_conntrack | wc -l

net.ipv4.ip_conntrack_max = 32760
is not too low. Normally you should only have a dozen or two
active at time.

2. Diff of two firewalls

Please use the output of "iptables -nL", not the output of
iptables-save.
Do this during and after a problem. For example, do one just
after boot and one after you restart the firewall, or do one
when you have the slow response and one with the fast
response. Anyway, I have not found a single case where
FireHOL produces different firewalls at boot or at certain
conditions. If there is a difference, some other program
changed the firewall (have you installed other firewalling
solutions?).

I really suggest to keep only one firewalling solution starting
the firewall at boot time. Check all the services starting at
boot and keep only one.

3. For your slow response problem.
I guess it has something to do with your ethernet driver of
the ethernet hardware. Check "ifconfig" for CRC errors and
the logs for ethernet driver errors.

One thing to check is to ping the router in front of you when
you have the slow response of the remote hosts. Is this slow
too? If it is a problem of your linux it should be slow too. Use
traceroute too to figure out which host produces the problem.

From my experience:
If your link is a DSL, you may have problems with your
provider due to bandwidth overbooking and congestion during
peak hours.

If your router is a cheap DSL router you may have CPU
problems on this device. I have encountered such problems
with my DSL router when I had too many requests per second
(I could not even ping the router itself - I stopped the
program that generated the requests and I could ping the
router again full speed).

Costa

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-07-11

Logged In: YES
user_id=935333

Hi again,

I'll get that version of firehol.sh.

2. I don't have any other firewall solution on running machine
for sure. FireHOL is the only one for me :)

3. I'll check for driver problems with my card. I haven't found
any log messages about it, but next time the problem occurs
I'll check for CRC errors.

The ping is slow if I try to ping my gateway.

I don't use DSL, my landlord has a switched Ethernet all-over
town (100Mbit/s within each block and 1000Mbit/s between
blocks all over the city). Yes, it is VERY nice to see transfer
rates of 6Mb/s (at least when they don't drop slowly to
1000B7s) :)

Anyway, thanks for your help. I think we have made sure that
the transfer rate slowing down and the slow ping is not
caused by firehol.

I'll get that firehol.sh file and report back in case the problem
does not disappear.

Jimisola

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Costa Tsaousis - 2004-07-30

status: open --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jimisola Laursen - 2004-08-16

Logged In: YES
user_id=935333

FYI, seems as if it was the driver (via-rhine) that caused
the problem.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Problems abnormally slow and descending transfer rate

Group

Searches

Help

#9 Problems abnormally slow and descending transfer rate

Discussion