I have site to
site VPN between a Pix-Endian at the moment. I have read where if you use
certain non-standard characters in the PFS you will get the
Payload_Malformed error. If it is not already change the key to a simple
alphanumeric one. If it is already, try dumbing the encryption down to
simple DES to see what you get. I can not remember the exact error but I did
once have an issue where the current config would not work with 3DES-AES, I
had to do DES and build up from there. This was a few years ago and my
memory is not too good, but anyways currently I do have AES-256 enabled
A PIX will site-to-site with an Endian. With that
said, in my past experience I found that either using FreeBSD, FreeBSD based
firewalls (Monowall/PFSense) worked much better than Endian. I never tried
IPCOP, but have used several other Linux based firewalls. My main problem
with Endian is the VPN is not stable, meaning the connection dies out too
randomly. DPD does give an incorrect error message in that the
"R_U_THERE_ACK has invalid icookie" that can be ignored. Supposedly this is
Cisco not adhering to standards in regards to DPD, what ever the case the
Openswan list told me long ago that this is a non-issue.
What I have
found is that editing the ipsec.conf to try and change logging level to all
made no difference. The best advice I was able to get was to insure there
was a minimal constant flow of traffic through the tunnel. So I added some
machines on the Endian site to be monitored via SNMP at different intervals
along with using these machines to use NTP service from behind the PIX side.
This had no effect.
Below is the config for one of the VPNS which may
or may not be of assistance.
From: Daniel Barnett <email@example.com>
Date: Sun, 25 Mar 2007 11:55:00 -0500
Subject: [Efw-user] Endian to Pix Site to Site VPN.
Has anyone site to site VPN’d a pix and the
I keep getting PAYLOAD_MALFORMED errors on my
I also get a few with “invalid payloads”
The pix seems to say the pre shared keys don’t
but they are EXACTLY the same on both units.