[OpenSTA-devel] [ opensta-Bugs-897856 ] Recording HTTPS fails because newer IE escapes URL

SourceForge Headquarters 225 Broadway Suite 1600 San Diego, CA 92101 +1 (858) 454-5900

Bugs item #897856, was opened at 2004-02-16 02:43
Message generated for change (Comment added) made by dansut
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=897856&group_id=10857

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: HTTP Capture
Group: Serious
Status: Closed
Resolution: Fixed
Priority: 5
Submitted By: Alain Ribault (ribault)
Assigned to: Daniel Sutcliffe (dansut)
Summary: Recording HTTPS fails because newer IE escapes URL

Initial Comment:
HTTPS recording :
With my browser (IE6, latest patches), when the opensta
receives an http page with https links, it transforms
the link like http://{ ..., which is correct by opensta
implementation. The problem is that IE escapes '{' by
'%7D' and the gateway does not understand http://%7D,
so the capture stops by 'page not found'.

Hisashi Iwashimizu wrote that it is after applying
MS04-004 
 security patch to IE6. 

I did some tests with Mozilla and it worked fine
(because no escape code). 

----------------------------------------------------------------------

>Comment By: Daniel Sutcliffe (dansut)
Date: 2005-12-18 13:54

Message:
Logged In: YES 
user_id=19748

This problem had NOTHING to do with script playback - it was
entirely a recording issue.  This fact does not matter
anyway, as the problem is fixed in the current release of
OpenSTA (1.4.3).

Also, please do not use this bug database as a type of
support forum - this just clutters up bug reports and makes
them more confusing.

----------------------------------------------------------------------

Comment By: kikibug (kikibug)
Date: 2005-12-07 04:42

Message:
Logged In: YES 
user_id=1314392

Should recording the script OK with '%7D' instead of '{'
affect playback? I have a problem that I do not know for
sure to be a bug, I have entered it in the mailing lists - I
cannot play back HTTPS script (which was recorded OK and
shows https:// in the requests) when going from the second
to the third HTTPS page in a row. So I am wondering if this
occurence with IE6 may affect playback in any way?

----------------------------------------------------------------------

Comment By: Lance (stampy)
Date: 2005-10-19 14:16

Message:
Logged In: YES 
user_id=1137295

hm, i too also get the %7B when using IE6. Im gathering from 
your previous response that its a problem in IE. Any other 
ideas?

im using osta 1.4.3.20

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2005-06-15 12:18

Message:
Logged In: YES 
user_id=19748

mvemers wrote:
> This problem seems to have re-occurred - IE is showing %7B
> in the location bar again:

This actually doesn't mean that this bug has re-occurred,
the fix to this problem was to allow the gateway to
understand the urlencoded form of the { character as well. 
It is only a problem if you cannot record HTTPS because of
this...

----------------------------------------------------------------------

Comment By: Mark Vevers (mvevers)
Date: 2005-06-15 11:26

Message:
Logged In: YES 
user_id=13500

This problem seems to have re-occurred - IE is showing %7B
in the location bar again:

gwhttp.dll: 1.4.3.6 
IE Version:  6.0.2900.2180.xpsp_sp2_gdr.050301-1519

Any suggestions?

Thanks

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2005-01-24 13:37

Message:
Logged In: YES 
user_id=19748

And addendum to this fix has been merged into the CVS HEAD.
The case were a Location: header was specified from the root
during a HTTPS redirect was not dealt with.  Thanks to Thierry
for spotting this.  This will be included in the OpenSTA 1.4.3
release.

gwhttp.dll: 1.4.3.5
OpenSTA: 1.4.3.18

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2004-12-20 17:29

Message:
Logged In: YES 
user_id=19748

The most recently mentioned fix has been merged into the CVS
HEAD.  It will become generally available in the OpenSTA 1.4.3
release.

gwhttp.dll: 1.4.3.4
OpenSTA: 1.4.3.13

----------------------------------------------------------------------

Comment By: Daniel Sutcliffe (dansut)
Date: 2004-12-20 14:33

Message:
Logged In: YES 
user_id=19748

I now have a "proper" fix for this that treats the URL encoded
version of { (which is %7b not %7d as Alain initially reported)
as equivilant to the {.

The fix had to be extended with a little fudge because it
uncovered and issue with IE and redirects.  This is best shown
by an example:  If I go to record https://osta.lan/mail/ I'll 
get http://{osta.lan/mail/ which the new "fixed" IE will rewrite
to http://%7Bosta.lan/mail/ - this will work fine, but if the
Web server then returns with a HTTP 302 and "Location:
src/login.php" then IE will get confused and try to redirect to
http://%7Bosta.lanan/mail/src/login.php - obviously the extra
2 chars and the domain name changing length confuse IE.  It
is unlikely that anyone in the "real world" will find this IE
buglet so I worked around it by having the gateway rewrite
the "Location:" header so that it is always fully specified for
SSL recordings (the recording .all gets the original value).

I was convinced this was something the gateway was doing
wrong for a long while but hours spent with a sniffer and
tests with other browsers showed that it was IE.

The fix also "enhances" the console and trace output from
the gateway.

This gateway code is ugly...

----------------------------------------------------------------------

Comment By: John L (johnleij)
Date: 2004-09-21 13:34

Message:
Logged In: YES 
user_id=999135

Hi all,

As I wasn't able to build OpenSTA due to lack of access to 
required third party tools, I binary patched the product. I'm 
not sure whether the license gives me the rights to distribute 
the patch, but I gather that it is for everybodys' best if I at 
least inform you of what I did.

I binary patched gwhttp.dll to make it work with SSL even 
after security patch MS04-004 on Internet Explorer 6.

The idea is to replace the ordinary "http://{" sequence 
with "http://-" so that IE doesn't destroy the URL when it is 
UrlEncoded. Note that this quick and dirty fix will make 
OpenSTA stop working with sites that begin with '-' (not very 
common though).

In gwhttp.dll I patched:
- Changed "http://{" to "http://-" (address ~00029380h)
- Changed "{" to "-" (address ~00029c40h)

I could write a long disclaimer here to inform you that I can't 
guarantee that the patch works, but I think you already know 
that. The patch surely seems to work for me.

----------------------------------------------------------------------

Comment By: jmrwnd (jmrwnd)
Date: 2004-09-21 09:23

Message:
Logged In: YES 
user_id=1125756

Has johnleij's fix been implemented anywhere, and is there 
any way to get this alternate version of the code?  I am 
unable to alter and build the source code myself and need a 
workaround for this problem ASAP.  I absolutely must test 
with IE6 or Netscape 6, but both escape the '{' character.

If all else fails, is there another similar tool that could record 
https traffic with these browsers that could run off of a 
laptop?  Any help is greatly appreciated.  Thanks.

----------------------------------------------------------------------

Comment By: John L (johnleij)
Date: 2004-03-16 09:28

Message:
Logged In: YES 
user_id=999135

A possible quick-fix could be to change the implementation of 
OpenSTA so that instead of using "http://{" to identify SSL, 
we could use "http://-". The latter isn't common on the 
internet and is not tampered with by MSIE.

A search through the source code gave me the idea to chage 
the value of constants CYRANO_SSL_URL and 
CYRANO_SSL_STR in HttpGateway\gwhttp\context.hpp to 
reflect this.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=110857&aid=897856&group_id=10857