[Ipsec-tools-devel] Figuring out how to make NAT-T work

[Brad R. <Brad@BradRubenstein.com>]

Dear IPSEC wizards:

I'm trying to find out why my two hosts (both behind NAT masquerading
firewalls) are not able to talk to each other via IPSEC+racoon.  They
negotiate phase 1 and phase 2 fine, but when host A tries to ping host B,
host B racoon says: "the length in the ISAKMP header is too big".

It seems that all packets, both NAT-T-IKE and regular ESP (e.g. an
encapsulated ICMP ping) get read from port 4500 by racoon on host B, and
racoon happily processes the encapsulated ping packet in isakmp_handler(),
where it is garbage (since it is not wellformed NAT-T-IKE).  I'm presuming
the packet should never have been handed to racoon, and the kernel should
have deencapsulated it and sent the appropriate ICMP response. No?

The SA's for hostA[4500] hostB[4500] esp-udp are emplaced by racoon (as can
be seen by the attached hostB/setkey.log.).  Could the packets not be
getting passed to the esp4 module? The esp4 module is loaded (according to
lsmod), how can I tell if it is getting and processing the encapsulated
packets?

Anyone have any hints on what I should be looking at?
As you can see, I'm sort of spinning my wheels... :-)  Any advice would be
appreciated.  Should I somehow try to make it work without racoon?

I'm attaching the racoon configuation and logs, and the SA and SP dumps, in
case anyone might like to eyeball them.  They're small.  The racoon.conf
files indicate the network topology I've got.

I'm running Fedora's kernel-2.6.7-1.494.2.2 I built ipsec-tools-0.3.3 from
source.

Brad Rubenstein