[Filterproxy-devel] Re: New module for FilterProxy

[John F W. <way...@WP...>]

Bob McElrath wrote:
>
> More rules will be happily accepted.  ;)  I know little about javascript
> at this point.

I have a passable knowledge of it, but you're right, we DO end up trying
to fake our way around the Halting Problem in trying to figure out if a
javascript block does something specific.

> I just added a big red warning if you try to disable Header.  In testing
> it, I saw it leaving numbers at the top (content-length in hex, BTW) and
> 0 at the bottom...when Header was disabled.

Ah, that was what I thought you said was a Mozilla bug.  Glad we have that
fixed.

> Ick, I really don't want to do that.  Frames are evil... Which browsers
> don't support javascript anyway?  If you want to use something like
> FilterProxy, you can enable javascript, and filter the bad stuff.  ;)

Good argument.

> A "control page" would be better, I think.  List the last several URL's
> loaded, and let you manipulate them.  (show source, show filtering, ...)
> This one's on my TODO list.

That works.

> I did a web search and ran across an HTML diff tool for "only" $149.95.
> I don't see anything in CPAN that would be useful.  Undoubtedly someone
> has written something similar using HTML::Parser, but I don't want to
> add HTML::Parser to this project, it has enough dependencies as it is,
> and HTML::Parser is a big one.

Ok, I've done some researching.  I think we want to write our own HTML
prettifier (I found a few, one that didn't use HTML::Parser even, but it
didn't fit our purposes).  I found a package, Algorithm::Diff, which does
exactly what we want: it allows traversal of a diff sequence.  If you're
ok with that one more single dependency, we could use this.

A thought occurs to me, because you're cringing about dependencies.  I
don't think it should be necessary for me to have installed Compress::Zlib
if I'm not using the Compress filterproxy module.  I think modules should
only be "use"d if they're in use.  How about load a module when you come
across a filter rule that uses it, except if it's already loaded (MODULES
contains it)?  Just a thought.

> I'm gonna try to write a Rewrite-changes-highlighter this weekend.

Ok.  I have a few ideas in my head, so let me know if/when you've
completed yours, so I'll know if I should write my own or look at yours :)

> Multiple ? could be OK, as long as we strip it off before sending it on.
> Another option might be to append #FilterProxy:gimmedasource to the end
> of the URL (which is a valid URI).

We've got not one, but two places where someone who's a stickler to the
URI RFC could screw us up.  Not only do we have to send a valid request to
the end server, from the proxy, but we also have to make a url that won't
gag any browsers.  The appending idea could work decently.

> Nod, pretty clever.  I hadn't thought of that until you sent your
> module.
>
> What about an .html file that had two frames, one showing some
> FilterProxy config stuff, and the other the source or hightlighted
> source of the page?  The frame with the source could still use your
> module...

True.  This helps let the true usefulness of a Rewrite change hiliter show
through, by allowing you to react to how your rule modified the page, and
change the rule accordingly.  This also allows the source viewer to work
standalone, to replace a browser's view-source capability, and for
browsers not supporting frames.  I like it. :)

> Another, far less elegant solution would be to have the proxy "tell
> itself" to grab the source.  i.e. load 2 urls in succession:
>     http://localhost:8888/Source.html?getnexturlsource=true
>     http://wherever.com...
> But then both are valid URI's...

Eew.  That's a race condition.

> or: http://localhost:8888/Source.html?getsourceof=http://.... since you
> can always encode nasties like ? in the second URL with %.  The module
> then just compares each URI to its internal variable $getsourceof...

That could work, but kind of requires new functionality written into the
core script.  I don't know about you, but I think I kind of want to avoid
that, it's a little inelegant.

> No, CGI scripts are so ugly.  I'm looking at HTML::Embperl and
> HTML::Mason.  If you have experience with either, I'd like to hear it...

Nope, sorry.

> > Ah, but the PORT is still changeable...  This'll kind of obfuscate the
> > source viewing URL, but at least it works with your server scheme, sorta.
> > Source.pm -10 would just have to snag a request to the hostname:8887 (or
> > whatever), and change it over, because the proxy would be in proxy mode.
>
> Hmm...I'll have to think about that...

It might be better to make viewsources sent to port 23 or somesuch, which,
while kind of kludgey, is CERTAIN (almost) to avoid masking a port on
which resides an actual HTTP server.