[Ipsec-tools-devel] Phase 2 transform proposal mismatch

[Dan S. <dan...@uc...>]

Hello,

This is my first post (it might be a dupe because I sent the first  
before I was actually added to the mailing list, if this is a  
duplicate message please just ignore it).  I'm posting to the devel  
list because the users list doesn't look moderated or maintained.  If  
I'm posting in the wrong place could you please point me in the right  
direction?

So, here's my problem.  Im trying to build an IPSec tunnel between a  
RHEL5.2 linux box and a mainframe running z/OS 1.9.  I've successfully  
completed ISAKMP phase 1 but phase 2 keeps bombing out because of a  
transform proposal mismatch.  There are only two things that I notice  
that could be causing the problem; 1) the SPI is set to 0 on the  
racoon daemon and 2) it looks like the racoon daemon is proposing AH  
as well as ESP (although I don't think this would cause the problem I  
am seeing).  Could somebody possibly help me figure out why I am  
seeing this mismatch?  I've attached full debug output the the bottom  
of this email, but I think the code below illustrates the problem.

DEBUG: proposal #1: 1 transform
DEBUG: begin compare proposals.
DEBUG: pair[1]: 0x8236888
DEBUG:  0x8236888: next=(nil) tnext=(nil)
DEBUG: prop#=1 prot-id=ESP spi-size=4 #trns=1 trns#=1 trns-id=3DES
DEBUG: type=SA Life Type, flag=0x8000, lorv=seconds
DEBUG: type=SA Life Duration, flag=0x8000, lorv=14400
DEBUG: type=Group Description, flag=0x8000, lorv=1
DEBUG: type=Encryption Mode, flag=0x8000, lorv=Transport
DEBUG: type=Authentication Algorithm, flag=0x8000, lorv=hmac-sha
DEBUG: peer's single bundle:
DEBUG:  (proto_id=ESP spisize=4 spi=bf48855c spi_p=00000000  
encmode=Transport reqid=0:0)
DEBUG:   (trns_id=3DES encklen=0 authtype=hmac-sha)
DEBUG: my single bundle:
DEBUG:  (proto_id=AH spisize=4 spi=00000000 spi_p=00000000  
encmode=Transport reqid=0:0)
DEBUG:   (trns_id=SHA authtype=hmac-sha)
DEBUG:  (proto_id=ESP spisize=4 spi=00000000 spi_p=00000000  
encmode=Transport reqid=0:0)
DEBUG:   (trns_id=3DES encklen=0 authtype=hmac-sha)
ERROR: not matched
ERROR: no suitable policy found.
ERROR: failed to pre-process packet.

Thank-you,

Dan Sullivan