Re: [Shorewall-users] shorewall + ipsec + ipvs : route back problem

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Hello,

I re-created a test platform with a lighter configuration.
Here are all the information.

I am facing difficulties with my chain :

  client - ipsec - shorewall - openswan - ipvs - Real servers.

It seems that the return packets never arrive to the clients.

Architecture :

client :10.44.0.254 
     |
    |
     \
+----+----+
| node A  |
|         |
+---+-----+
    |
    |
    |
    |
    |
+------+--------+
|    node B     |
|  shorewall    | 4.0.11
|   openswan    | 2.4.14
|    ipvs       | VIP: 10.4.0.30
+------X--------+
      -/\____
      /      \-
    -/         \
   -/           \
   /              \
RealServer1      RealServer2 	 	 	 	
10.0.1.60        

Ldirector configuration :

virtual=10.4.0.30:80
        real=10.0.1.60:80 masq
        service=http
        protocol=tcp
        checktype=on

 1. the ping: client -> 10.4.0.30 is working OK
 		Done with /etc/shorewall/rules

ACCEPT swan:10.44.0.0/24     fw     all

 2. The masq for real servers to exit with 10.4.0.30 is OK
 		Done with /etc/shorewall/masq

eth2::10.44.0.254       10.0.1.60       10.4.0.30       -       -

 3. The forward from ipvs to real server is OK
 when doing a : telnet 10.4.0.30 80
 I have the following tcpdump on Node B

Tcpdump from the shorewall :

15:36:27.558268 IP 10.44.0.254.49598 > 10.4.0.30.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947 0,nop,wscale 5> 
15:36:27.558310 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947 0,nop,wscale 5>
15:36:27.558312 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947 0,nop,wscale 5>
15:36:27.558426 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588542865 2991974947,nop,wscale 7> 
15:36:27.558816 arp who-has 10.44.0.254 tell 10.4.0.30
15:36:28.558764 arp who-has 10.44.0.254 tell 10.4.0.30
15:36:29.558589 arp who-has 10.44.0.254 tell 10.4.0.30
15:36:30.557790 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588545865 2991974947,nop,wscale 7> 
15:36:30.557797 IP 10.44.0.254.49598 > 10.4.0.30.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947 0,nop,wscale 5>
15:36:30.557826 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947 0,nop,wscale 5>
15:36:30.557828 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947 0,nop,wscale 5>
15:36:30.557930 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588545865 2991974947,nop,wscale 7>
15:36:36.557900 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588551865 2991974947,nop,wscale 7>
15:36:36.558100 IP 10.44.0.254.49598 > 10.4.0.30.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991983947 0,nop,wscale 5>
15:36:36.558148 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991983947 0,nop,wscale 5>

tcpdump from the Realserver :

15:36:27.509438 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991974947 0,nop,wscale 5>
15:36:27.509510 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588542865 2991974947,nop,wscale 7>
15:36:30.508811 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588545865 2991974947,nop,wscale 7> 
15:36:30.508944 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991977947 0,nop,wscale 5>
15:36:30.508950 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588545865 2991974947,nop,wscale 7>
15:36:36.508971 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588551865 2991974947,nop,wscale 7> 
15:36:36.509314 IP 10.44.0.254.49598 > 10.0.1.60.http: S 3659512001:3659512001(0) win 5840 <mss 1460,sackOK,timestamp 2991983947 0,nop,wscale 5>
15:36:36.509320 IP 10.0.1.60.http > 10.44.0.254.49598: S 3557705332:3557705332(0) ack 3659512002 win 5792 <mss 1460,sackOK,timestamp 2588551865 2991974947,nop,wscale 7>

However the return never arrives to the client. I don't seen any
drop/reject on the firewall. But I don't know what
is missing.

When I bypass the ipvs by a DNAT rules like this one :

DNAT:info			swan:10.44.0.254	 loc:10.0.1.60:80	 tcp 80 - 10.4.0.30

it works, but I am loosing the loadbalancer ipvs.

I am obviouly missing a rule to link packet from 
	loc -> ipvs -> shorewall -> openswan

but I don't know which one. Can someone help me ?

Thanks