Hello all.
I'm stuck trying to set-up VPN tunnel from Nokia E70 smartphone to Gentoo 2006.1 distribution, running ipsec-tools 0.6.6
IMHO: Problem looks like ESP traffic, incapsulated in UDP, does not reach kernel, but instead still go to racoon.
Racoon provide error:
ERROR: the length in the isakmp header is too big.
Here is log for NAT-T session when login goes from GPRS session:
Hope ~ # uname -a
Linux Hope 2.6.18-gentoo-r6 #1 SMP Mon Jan 22 19:14:25 EET 2007 i586 AMD-K6(tm)-III Processor GNU/Linux
Hope ~ # racoon -F -v
Foreground mode.
2007-01-24 23:39:46: INFO: @(#)ipsec-tools 0.6.6 (http://ipsec-tools.sourceforge.net)
2007-01-24 23:39:46: INFO: @(#)This product linked OpenSSL 0.9.8d 28 Sep 2006 (http://www.openssl.org/)
2007-01-24 23:39:47: INFO: 83.243.92.24[4500] used as isakmp port (fd=5)
2007-01-24 23:39:47: INFO: 83.243.92.24[4500] used for NAT-T
2007-01-24 23:39:47: INFO: 83.243.92.24[500] used as isakmp port (fd=6)
2007-01-24 23:39:47: INFO: 83.243.92.24[500] used for NAT-T
2007-01-24 23:40:26: INFO: respond new phase 1 negotiation: 83.243.92.24[500]<=> 212.93.97.131[52461]
2007-01-24 23:40:26: INFO: begin Aggressive mode.
2007-01-24 23:40:26: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-03
2007-01-24 23:40:26: INFO: received Vendor ID: draft-ietf-ipsra-isakmp-xauth-06.txt
2007-01-24 23:40:26: INFO: received Vendor ID: CISCO-UNITY
2007-01-24 23:40:26: INFO: Selected NAT-T version: draft-ietf-ipsec-nat-t-ike-03
2007-01-24 23:40:27: INFO: Adding remote and local NAT-D payloads.
2007-01-24 23:40:27: INFO: Hashing 212.93.97.131[52461] with algo #2
2007-01-24 23:40:27: INFO: Hashing 83.243.92.24[500] with algo #2
2007-01-24 23:40:27: INFO: NAT-T: ports changed to: 212.93.97.131[52648]<->83.243.92.24[4500]
2007-01-24 23:40:27: INFO: Hashing 83.243.92.24[4500] with algo #2
2007-01-24 23:40:27: INFO: NAT-D payload #0 doesn't match
2007-01-24 23:40:27: INFO: Hashing 212.93.97.131[52648] with algo #2
2007-01-24 23:40:27: INFO: NAT-D payload #1 doesn't match
2007-01-24 23:40:27: INFO: NAT detected: ME PEER
2007-01-24 23:40:27: INFO: ISAKMP-SA established 83.243.92.24[4500]-212.93.97.131[52648] spi:4ed096fcd9f497b0:2e05cc5068d1d6e6
2007-01-24 23:40:27: INFO: Using port 0
2007-01-24 23:40:31: INFO: respond new phase 2 negotiation: 83.243.92.24[4500]<=>212.93.97.131[52648]
2007-01-24 23:40:31: WARNING: ignore REPLAY-STATUS notification.
2007-01-24 23:40:31: INFO: no policy found, try to generate the policy : 192.168.74.1/32[0] 0.0.0.0/0[0] proto=any dir=in
2007-01-24 23:40:31: INFO: Adjusting my encmode UDP-Tunnel->Tunnel
2007-01-24 23:40:31: INFO: Adjusting peer's encmode UDP-Tunnel(61443)->Tunnel(1)
2007-01-24 23:40:32: ERROR: the length in the isakmp header is too big.
2007-01-24 23:40:32: INFO: IPsec-SA established: ESP/Tunnel 212.93.97.131[52648]->83.243.92.24[4500] spi=90005672(0x55d60a8)
2007-01-24 23:40:32: INFO: IPsec-SA established: ESP/Tunnel 83.243.92.24[4500]->212.93.97.131[52648] spi=881681059(0x348d62a3)
2007-01-24 23:40:32: ERROR: such policy does not already exist: "192.168.74.1/32[0] 0.0.0.0/0[0] proto=any dir=in"
2007-01-24 23:40:32: ERROR: such policy does not already exist: "192.168.74.1/32[0] 0.0.0.0/0[0] proto=any dir=fwd"
2007-01-24 23:40:32: ERROR: such policy does not already exist: "0.0.0.0/0[0] 192.168.74.1/32[0] proto=any dir=out"
2007-01-24 23:40:34: ERROR: the length in the isakmp header is too big.
2007-01-24 23:40:40: ERROR: the length in the isakmp header is too big.
2007-01-24 23:41:28: ERROR: packet shorter than isakmp header size (5, 2102449591, 28)
2007-01-24 23:42:05: ERROR: the length in the isakmp header is too big.
2007-01-24 23:42:28: ERROR: packet shorter than isakmp header size (5, 2102449591, 28)
When no NAT-T is detected, I'm able to reach Apache2 web service, running in VPN box on another internal IP address.
 
I tested several kernel versions:
linux-2.6.14
linux-2.6.16-gentoo-r13
linux-2.6.18-gentoo-r6
linux-2.6.19.2
And all of them gave same result (ERROR: the length in the isakmp header is too big.).
 
ipsec-tools was compiled with following comand:
"./configure --prefix=/usr --host=i586-pc-linux-gnu --mandir=/usr/share/man --infodir=/usr/share/info --datadir=/usr/share --sysconfdir=/etc --localstatedir=/var/lib
--enable-hybrid --enable-dpd --enable-natt --enable-adminport --enable-frag --disable-idea --disable-rc5 --enable-ipv6 --with-readline --with-libpam
--build=i586-pc-linux-gnu --enable-natt-versions="00 01 02 03 04 05 06 07 08 rfc"
 
My racoon.conf is:
Hope ~ # cat /etc/racoon.conf
# $KAME: racoon.conf.in,v 1.18 2001/08/16 06:33:40 itojun Exp $

path include "/etc/racoon";
path pre_shared_key "/etc/racoon/psk.txt";

path certificate "/etc/cert";

padding
{
        maximum_length 20;      # maximum padding length.
        randomize on;           # enable randomize length.
        strict_check off;       # enable strict check.
        exclusive_tail off;     # extract last one octet.
}

listen
{
        isakmp 83.243.92.24 [500];
        isakmp 83.243.92.24 [4500];
}

timer
{
        # These value can be changed per remote node.
        counter 5;              # maximum trying count to send.
        interval 20 sec;        # maximum interval to resend.
        persend 1;              # the number of packets per send.

        phase1 10 sec;
        phase2 15 sec;
}

remote anonymous
{
        exchange_mode aggressive;
        doi ipsec_doi;
        situation identity_only;

        my_identifier address;                   
        mode_cfg on;
        nonce_size 16;
        initial_contact on;
        proposal_check obey;
        generate_policy on;
        ike_frag on;
        nat_traversal on;

        proposal {
                encryption_algorithm 3des;
                hash_algorithm sha1;
                authentication_method pre_shared_key;
                dh_group 2;
        }
}
        mode_cfg {
        auth_source system;
        conf_source local;
        accounting none;
        pool_size 254;
        network4 192.168.74.1;
        netmask4 255.255.255.0 ;
        dns4     192.168.1.3;
        pfs_group 2;
}

sainfo anonymous
{
        pfs_group 2;
        encryption_algorithm 3des;
        authentication_algorithm hmac_sha1;
        compression_algorithm deflate;
}
 
 
Please, advise me, where is problem?
P.S. I'm not a UNIX/Linux guru or programmer, so please don't blame me for lack of skills.
Thank you, Vladimir.