Shoreline Firewall (Shorewall)

On 05/03/2010 06:19 PM, sangprabv wrote:
> I have problem with my shorewall. We are now doing some stress test
> with a http application behind the shorewall. Firstly we send 10.000
> requests to a http based application with no firewall. It can
> received 100% requests. But when we put shorewall in front of it then
> it stats to loose requests. Is there any packet limitation from
> shorewall all it's about conntrack? Thanks for the reply

Shorewall itself imposes no limitations besides the 20% penalty imposed
by conntrack. But a stupid Shorewall configuration can certainly add
lots of delay and packet loss.

I suggest that you try:

a) clients -> server with no intermediate Linux Router.
b) clients -> Linux Router with no conntrack -> server
c) clients -> Linux Router with nf_conntrack loaded -> server.
d) clients -> Linux Router with Shorewall -> server.

If d) is substantially worse than c), then please submit the output of
'shorewall dump' collected after the test.

-Tom
--
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

What version of shorewall u r using

Mac

Sent from my iPhone

On 04-May-2010, at 6:49 AM, sangprabv <sangprabv@...> wrote:

> I have problem with my shorewall. We are now doing some stress test  
> with a http application behind the shorewall. Firstly we send 10.000  
> requests to a http based application with no firewall. It can  
> received 100% requests. But when we put shorewall in front of it  
> then it stats to loose requests. Is there any packet limitation from  
> shorewall all it's about conntrack? Thanks for the reply.
>
>
>
> sangprabv
> sangprabv@...
>
>
>
> --- 
> --- 
> --- 
> ---------------------------------------------------------------------
> _______________________________________________
> Shorewall-users mailing list
> Shorewall-users@...
> https://lists.sourceforge.net/lists/listinfo/shorewall-users

On 05/03/2010 06:42 PM, Tom Eastep wrote:
> On 05/03/2010 06:19 PM, sangprabv wrote:
>> I have problem with my shorewall. We are now doing some stress test
>> with a http application behind the shorewall. Firstly we send 10.000
>> requests to a http based application with no firewall. It can
>> received 100% requests. But when we put shorewall in front of it then
>> it stats to loose requests. Is there any packet limitation from
>> shorewall all it's about conntrack? Thanks for the reply
> 
> Shorewall itself imposes no limitations besides the 20% penalty imposed
> by conntrack. But a stupid Shorewall configuration can certainly add
> lots of delay and packet loss.
> 
> I suggest that you try:
> 
> a) clients -> server with no intermediate Linux Router.
> b) clients -> Linux Router with no conntrack -> server
> c) clients -> Linux Router with nf_conntrack loaded -> server.
> d) clients -> Linux Router with Shorewall -> server.
> 
> If d) is substantially worse than c), then please submit the output of
> 'shorewall dump' collected after the test.

It's also unclear what you mean by 'requests'. Shorewall rules fall into
multiple categories:

a) Those that are only applied for new connection requests. These include
   policies (including the LIMIT:BURST setting) and entries in the rules
   (NEW section), masq, nat and netmap files along with blacklisting
   entries (assuming that BLACKLISTNEWONLY=Yes).

b) Those that are applied to every packet. These include entries in the
   tcrules and route_rules file along with blacklisting entries when
   BLACKLISTNEWONLY=No and entries in the rules file ESTABLISHED section.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

We love shorewall! We're trying to implement our usual setup on the cloud, and 
it just ain't working though. Shorewall itself appears to be working, but
packets just don't arrive at the destination web server. I'm just fresh out of
diagnostic ideas and hope someone can think of some other things to try. 
Rackspace docs and support people all say what we're doing should work.

Simple setup. elkin and ronda are cloud instances.

     che.zorinco.com workstation: browse http://173.203.203.5
     64.81.168.12
         |
         | Internet
         |
     173.203.203.5
      elkin (shorewall)
     10.177.157.160
         |
         | (private net)
         |
     10.177.140.52
      ronda (web server)

If I try to browse http://173.203.203.5 from the Internet, I see 
a log line like this:

May  4 20:16:38 elkin kernel: [  872.168060] Shorewall:net_dnat:DNAT:IN=eth0 OUT= MAC=40:40:6e:d3:0b:4a:00:23:33:a0:fc:ff:08:00 SRC=64.81.168.12 DST=173.203.203.5 LEN=60 TOS=0x00 PREC=0x00 TTL=52 ID=56345 DF PROTO=TCP SPT=44862 DPT=80 WINDOW=5840 RES=0x00 SYN URGP=0

Running tcpdump on ronda (tcpdump -i eth1 tcp) shows nothing, and browser just gets timeout.

Running 'tcpdump -i eth1 tcp' on elkin, looking at private interface shows 
something like this while trying to browse to http://173.203.203.5 from 
outside:

20:22:25.854271 IP zorinco.com.43844 > ronda.http: S 3135179140:3135179140(0) win 5840 <mss 1460,sackOK,timestamp 1087987124 0,nop,wscale 6>


iptables on ronda is stopped (accept everything)

So, it appears that shorewall is doing it's job and 'trying' to forward packets to ronda, but they are not arriving on ronda.

On elkin, I can browse to ronda (links http://10.177.140.52) and see ronda's web output. ronda can ping 10.177.157.160, 
so basic connectivity works. Also, masquerading from dmz does not work (ronda cannot ping or browse the net), which might be a 
clue.
If, on ronda, I 'ping 98.137.149.56' (that's yahoo.com), while doing 'tcpdump -i eth1' on elkin, I see nothing. but if I ping the
gateway address (ping 10.177.157.160) then I do see output on tcpdump on elkin. Ronda works just fine if using the ISP supplied
gateway, as you'd expect (173.203.204.1).

One thing I do see on elkin is this message when shorewall is staretd, but it's not clear if this has anything to do with our problem.
     kernel: [81069.656706] ip_tables: connlimit match: invalid size 32 != 24

Shorewall dump output from elkin is attached.

>From Ronda:
[root@... chris]# iptables -L -n
 	Chain INPUT (policy ACCEPT)
 	target     prot opt source               destination

 	Chain FORWARD (policy ACCEPT)
 	target     prot opt source               destination

 	Chain OUTPUT (policy ACCEPT)
 	target     prot opt source               destination

[root@... chris]# netstat -nr
 	Kernel IP routing table
 	Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
 	173.203.204.0   0.0.0.0         255.255.255.0   U         0 0          0 eth0
 	10.177.128.0    0.0.0.0         255.255.224.0   U         0 0          0 eth1
 	169.254.0.0     0.0.0.0         255.255.0.0     U         0 0          0 eth1
 	0.0.0.0         10.177.157.160  0.0.0.0         UG        0 0          0 eth1


Very greatful for ideas!
Thanks for listening.
-C




--
Christopher Nielsen    chris@...    http://ZORINco.com

On 05/04/2010 01:32 PM, Christopher Nielsen wrote:

> 
> On elkin, I can browse to ronda (links http://10.177.140.52) and see
> ronda's web output. ronda can ping 10.177.157.160, so basic connectivity
> works. Also, masquerading from dmz does not work (ronda cannot ping or
> browse the net), which might be a clue.

If the server can't access the internet, you can't reasonably expect to
access the server from the internet!

> If, on ronda, I 'ping 98.137.149.56' (that's yahoo.com), while doing
> 'tcpdump -i eth1' on elkin, I see nothing. but if I ping the
> gateway address (ping 10.177.157.160) then I do see output on tcpdump on
> elkin. Ronda works just fine if using the ISP supplied
> gateway, as you'd expect (173.203.204.1).

Sounds like the routing on ronda is hosed; the default route doesn't go
through elkin. Note that this is prominently mentioned in the DNAT
troubleshooting instructions detailed in Shorewall FAQs 1a and 1b.

> 
> One thing I do see on elkin is this message when shorewall is staretd,
> but it's not clear if this has anything to do with our problem.
>     kernel: [81069.656706] ip_tables: connlimit match: invalid size 32
> != 24

It means that your iptables is incompatible with your kernel.

-Tom
-- 
Tom Eastep        \ When I die, I want to go like my Grandfather who
Shoreline,         \ died peacefully in his sleep. Not screaming like
Washington, USA     \ all of the passengers in his car
http://shorewall.net \________________________________________________

Tom Eastep wrote:

>> One thing I do see on elkin is this message when shorewall is staretd,
>> but it's not clear if this has anything to do with our problem.
>>     kernel: [81069.656706] ip_tables: connlimit match: invalid size 32
>> != 24
> 
> It means that your iptables is incompatible with your kernel.

I guess it's unpatched centos - there was that kind of bug in rhel-5
iptables which was fixed by iptables update.

-- 
Tuomo Soini <tis@...>
Foobar Linux services
+358 40 5240030
Foobar Oy <http://foobar.fi/>

On Wed, 5 May 2010, Tuomo Soini wrote:

> Tom Eastep wrote:
>
>>> One thing I do see on elkin is this message when shorewall is staretd,
>>> but it's not clear if this has anything to do with our problem.
>>>     kernel: [81069.656706] ip_tables: connlimit match: invalid size 32
>>> != 24
>>
>> It means that your iptables is incompatible with your kernel.
>
> I guess it's unpatched centos - there was that kind of bug in rhel-5
> iptables which was fixed by iptables update.


Thanks so much, both of you for that. I googled a bit for that, but 
couldn't determine if it was some feature-specific thing that could be 
ignored or more serious... It's clear now :-)

-C