Heritrix: Internet Archive Web Crawler

Below is conversation from the list with Bjarne about
curly-brackets not getting encoded by IE. We should
probably do the same. Giving it priority 7 since issue
came in off the list.

Bjarne Andersen wrote:

> Thanks for the pointers - the URL I gave was not
complete - this ons is:
>
http://www.bs.dk/content.aspx?itemguid={31637766-92B4-4ACA-9A0D-5CFF042B151
E}
>
> URLs like this get encoded in the arc-files -> they
are encoded in my cdx-files -> my proxy-server can't
find them.
>
> :-)
> Bjarne
>
> stack wrote:
> > Bjarne Andersen wrote:
> >
> > > Which class does the URLencoding in heritrix ?
> >
> > org.archive.net.UURI. Study its superclasses
LaxURI and
> > commons-httpclient URI. Also see the
UURIFactory#fixup code.
> >
> > >
> > > it looks like URLs like:
> > > http://www.bs.dk/showfile.aspx?IdGuid={B0A}
> > > <http://www.bs.dk/showfile.aspx?IdGuid=%7BB0A%7D>
> > >
> > > gets encoded to:
> > > http://www.bs.dk/showfile.aspx?IdGuid=%7BBOA%7D
> > >
> > > my browser (IE) does not encode '{}'
automatically so when my browser
> > > wants access such URI's they are not found in my
archive unless I
> > > encode the braces myself. Because of this I want
to use the heritrix
> > > URLencoding Class to ensure consistency between
what heritrix has
> > > encoded and what my proxyserver encodes before
looking URIs up in my
> > > archive.
> >
> > Sounds like something we should just accomodate in
LaxURI (Doesn't
> > matter whether I use the encoded or non-encoded
version on live net in
> > FireFox, I get 'Server Error in '/' Application.'
Is that your
> > experience?).
> >
> > St.Ack
> >
> > >
> > > best
> > >
> > > ---
> > > Bjarne Andersen