[Filterproxy-devel] Re: Minor FilterProxy problem

[Bob M. <mce...@dr...>]

Colin J. Wynne [cw...@av...] wrote:
> Hi Bob,
>=20
> I just installed FilterProxy (on the recommendation of my friend,
> Danek Duvall, actually), and after some teething problems making it
> all work, it's up and running.

Suggestions on how to make the install easier would be appreciated.  ;)

> I have noticed one minor problem, though.  When I run an Internet
> Explorer browser through the proxy, I often find that the page loading
> indicators hang just before completion, even though a View Source
> shows me the whole source has come through and there is no indication
> of unfinished images on the visible page.  On the other hand, when I
> run lynx, the page loads right up, all nice and filtered, with no
> hesitations.

This has been a long-standing problem with Mozilla, and is a bug with
Mozilla.  IE is totally untested (I don't use any M$ products...), but
I'd be interested if this is a FP bug.  Personally, I've tested
Netscape, Mozilla, Galeon, Konqueror, and StarOffice.  The only one with
the page-hang problem is Mozilla, and there are bugs filed against
Mozilla for it.  (Note I think this is fixed in recent Mozilla
nightlies)

Ok, so which version of IE are you using?  Which version of FP?  (latest
is 0.29.1)

> One such has page has been spinning for the last five minutes, as I
> watch.  A strace of the corresponding FilterProxy process shows that
> is is stuck on
>=20
>     select(8, [6], NULL, NULL, {1558, 750000}

That basically means it's waiting for data...nothing unusual there.
FilterProxy thinks it's done.

> For the record, I am using a custom config.  I whacked all the rules
> out of the default config, and have put in just a handful of Rewrite
> rules which go after <img>, <src>, <script>, and <iframe> ads.

Shouldn't affect the unfinished-loading thing.  If you write some good
rules, please consider forwarding them to me or
fil...@li... and I'll include them in later
releases.  ;)  Someday I'll add a 'variables' facility to add things
like bad-hosts, to make the .* ADS rule less hideous.  (The thing is
that one complicated rule is a lot faster than several simple rules...)

Ok, so here's the lowdown:  FP tries to keep as many connections open to
the browser as possible, to speed things up.  It does this by using the
HTTP/1.1 Keep-Alive header.  For Netscape it uses the netscape-specific
Proxy-Connection header.  FilterProxy will, however, close the
connection if the browser tells it to do so (via the "Connection: close"
header).  The most likely cause of the unfinished-loading bug is a
connection that is left open, that IE thinks should be closed, or
improper headers sent by fp (assuming it's not an IE bug ;).

Questions:
1) Is IE a HTTP/1.1 client, or is it pretending to be Netscape?
    (HTTP/1.0 + Proxy-Connection)  Look in FilterProxy.log for:
        HTTP/1.0 proxy request for ...
2) Which connections are still open when IE hangs (ones for gifs?
    html?) -- look in log file and correlate pid's.  Log looks like:
        [<PID> <date>] Message...
    In Netscape/Mozilla it's useful to hit 'Esc' in the browser (or hit
    stop), which forces it to stop loading the page, and close all
    connections.  You can then look in the log to see which connections
    get closed, and scroll up in the log to find the last URL loaded by
    that pid.  Also useful is to 'kill -USR1 <filterproxy's pid>', which
    will cause FP to dump a list of open connections to the log.
3) Is Compress turned on?
4) Is IE sending the TE: header?

Turn on debug logging on the fp config page, and also turn on
header-dumping (hit 'Header' for the url '.*').  Please send me relevant
portions of your FilterProxy.log.

> **********************************************************************
>     /\           Colin J. Wynne      =20
>    (())    www.avtokrator.org/~cwynne/         ``Lunatic-at-Large''  =20
>   /____\      cw...@av...
>  /______\ =20
> /________\ ``[O]nce you have done away with the ability to make
>              judgments as to right and wrong [...] there's no real
>              culture left.  All that remains is clog dancing and
>              macrame.''                             ---Neal Stephenson
> **********************************************************************

Stevenson fan?  I'm almost done with Snow Crash.  He's quite a talented
author.

Cheers,
-- Bob

Bob McElrath (rsm...@st...)=20
Univ. of Wisconsin at Madison, Department of Physics