#54 config redirect must expire at once

[Anonymous]

I'm running RH8 and Privoxy 3.0.0-2 which is included
with RH. I set it up to be chained from squid
(transparent setup). Both are up and running.
config.privoxy.org is accessible and I can see the
current supression ratio. However, I am unable to get
the status detection working. If I select any of the view
or edit options on the show-status page I get the well-
known response. The same goes for the bookmarklets,
action look-ups and so on. Basicly, when processing is
required, config.privoxy.org fails.

I read a couple of questions in this forum regarding the
problem and the solutions suggested. My configuration
is set as recommended. Squid is between the browser
and Privoxy; Privoxy is set as a parent proxy to Squid. I
tried to reset the squid cache and the local IE cache but
it had no effect on status detection.

I then tried to bypass Squid, first by disabling any local
redirections (transparency) and setting my IE client to
use gate:8118 as the proxy server. It works and in
addition there are no problems with status detection.

As far as I was able to see, the only difference to the
Privoxy configuration is the listen-address, which has to
be set to the local IP (127.0.0.1) when chaining from
squid.

Maybe if someone shed some light on what Privoxy's
embedded http scripts are actually doing to detect its
state it would be easier to solve this issue. I know it's all
in the source files but I would prefer to avoid having to
reverse-engineer the whole package in order to find out.

[Nobody/Anonymous]

Logged In: NO

Additional info: I didn't notice these entries before:

[root@gate00 privoxy]# tail -1000 logfile |grep -a privoxy.org
Mar 04 16:27:35 Privoxy(19374082) Request:
config.privoxy.org/ cgi call
Mar 04 16:27:35 Privoxy(19374082) Request:
config.privoxy.org/ crunch!
Mar 04 16:27:35 Privoxy(19382274) Request:
config.privoxy.org/send-stylesheet cgi call
Mar 04 16:27:35 Privoxy(19382274) Request:
config.privoxy.org/send-stylesheet crunch!
Mar 04 16:27:37 Privoxy(19390466) Request:
config.privoxy.org/show-status cgi call
Mar 04 16:27:37 Privoxy(19390466) Request:
config.privoxy.org/show-status crunch!
Mar 04 16:27:37 Privoxy(19398658) Request:
config.privoxy.org/send-stylesheet cgi call
Mar 04 16:27:37 Privoxy(19398658) Request:
config.privoxy.org/send-stylesheet crunch!

Looks like Privoxy is filtering its own output! Or what?
Adding ".privoxy.org" to the "unblocked" list doesn't do
anything. Any ideas?

[Andreas Oesterhelt]

Logged In: YES
user_id=78811

The "Privoxy is not working" page (which I have just
extended to include a more detailed explanation) is
probably still cached somewhere in your proxy chain
(i.e. in IE or squid).

Since you say that config.privoxy.org is accessible, (and
I assume you mean the "This is Privoxy 3.0.0..." UI home page
here) but that page comes with headers that prevent caching,
the alternative looks implausible.

To find out more, please shift-reload now. Do you see the
updated error page? Go to http://hwqeztjhgsdjsg.shdkjsh/
What do you see?

> Maybe if someone shed some light on what Privoxy's
> embedded http scripts are actually doing to detect its
> state it would be easier to solve this issue.

Actually, nothing. The "not working" (now more precisely:
"not being used") page is not generated by Privoxy. It is
a completely regular web page that is served from a machine
called config.privoxy.org.

During normal operation, i.e. when the browser sends all
requests through Privoxy, Privoxy intercepts requests for
config.privoxy.org and serves its web-based user intreface
instead. When Privoxy is, for whatever reason, bypassed,
you see the original, actual web page.

[Nobody/Anonymous]

Logged In: NO

According to FAQ the config.privoxy.org "crunches" are
normal operation. Never mind. The problem remains.

[Nobody/Anonymous]

Logged In: NO

I now see the "not being used" page. I was aware that this
page is not generated by the proxy. The
http://hwqeztjhgsdjsg.shdkjsh/ URL has problems with DNS
resolution :-)

I was not aware that it shows up because the proxy is
apparently ignoring the url for some reason, which is
somethin completely different from the situation where the
redirection occurs inside the proxy. The host you are talking
about is wf.zoneedit.com, right?

So, actually, if I look at the logfile, the problem lies with the
config.privoxy.org "crunches" that are -not- there. What could
be the reason for our special URL to pass "unnoticed" while
everything else is working just fine? I'm under the impression
that it is ignoring all URL's that have parameters (?file=...etc.)
BTW, http:\\p.p\ doesn't work either, with the
standard "Cannot find server or DNS Error"!

Just a thought: even in situations where the lost pages are
mis-cached it is quite impossible to reload them because the
browser gets redirected immediately... This might however be
irrelevant.

I tried adding debug log settings (2, 2048) but found nothing
interesting. You can probably have better suggestions about
what I should look for.

Thanks for your help!

[Andreas Oesterhelt]

Logged In: YES
user_id=78811

> The http://hwqeztjhgsdjsg.shdkjsh/ URL has problems
> with DNS resolution :-)

So much I guessed, but the point of the exercise was to
determine, by the exact source and text of the resulting error
message, where in your proxy chain the problem lies:

- Local IE error dialogue: IE doesn't use Squid->Privoxy
chain for the request. Check browser settings.

- Squid-generated error page: Squid doesn't forward
request to Privoxy; check Squid.conf, compare example
in Privoxy user manual, Chapter 7.5.3

- Privoxy-generated error page: Proxy chain OK, Caching
is the problem.

> I was not aware that it shows up because the proxy is
> apparently ignoring the url for some reason,

It doesn't. It can't. It just doesn't get asked in the first
place.

> the problem lies with the
> config.privoxy.org "crunches" that are -not- there

Exactly.

> Just a thought: even in situations where the lost pages are
> mis-cached it is quite impossible to reload them because the
> browser gets redirected immediately... This might however be
> irrelevant.

It is quite likely the reason why shift-reload doesn't help. The
only way for you to fix a caching problem (if the above test
indicates it is in fact a caching problem) is to manually clear
the browser's disk and memory cache, and, if that fails, a lso
the one of the intremediate proxy.

Note to devel list: Root cause of caching problem is
cacheability
of "not being used" page. Is there any way to set the
expiry for a
document served on SF.net's hosting service?

[Nobody/Anonymous]

Logged In: NO

The answer that I suspect will be appreciated by many future
squid/privoxy users is here:

There is a default Squid directive that makes all queries
bypass the cache and fetch the response directly from the
server:

acl QUERY urlpath_regex cgi-bin \?
no_cache deny QUERY

Leave these directives alone. What everybody needs to do is
the following:

acl Privoxy dstdomain .privoxy.org
never_direct allow Privoxy

and then:

cache_peer 127.0.0.1 parent 8118 7 no-query

Only after all these are set it makes sense to clear the cache
(if necessary).

Thanks to Andreas and his insight! Thanks to everybody
involved in running this newsgroup!

Damjan Pavlovec
IS management
CHS Ltd.

[Andreas Oesterhelt]

Logged In: YES
user_id=78811

Damjan, thanks for your squid conf suggestion and making us
aware of the problem!

I'd still prefer eliminating the root cause over providing hints
to our users how they can avoid the effects on different
browsers and proxies.

The redirect from config.privoxy.org to privoxy.org/config
needs to be changed to expire immediately (currently 24hrs).
I'll check with Zoneedit.com whether that's possible. If not,
we'll need another VHOST setup at SF.

oes@ratzfatz:~/projekte/privoxy/current > telnet
config.privoxy.org 80
Trying 64.251.66.3...
Connected to config.privoxy.org.
Escape character is '^]'.
GET / HTTP/1.1
Host: config.privoxy.org

HTTP/1.1 302 Found
Date: Wed, 05 Mar 2003 19:42:26 GMT
Server: Apache
Expires: Thu, 06 Mar 2003 19:42:39 GMT
location: http://privoxy.org/config
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html

cd
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<HTML><HEAD>
<TITLE>302 Found</TITLE>
</HEAD><BODY>
<H1>Found</H1>
The document has moved <A
HREF="http://privoxy.org/config">here</A>.<P>
</BODY></HTML>

0

Connection closed by foreign host.

[Andreas Oesterhelt]

Logged In: YES
user_id=78811

Turning into Devel-Todo item.

Pending on further feedback/action from ZoneEdit, whose 1st-level
support is -- as always -- a pleasure to deal with :-(

[Fabian Keil]

Logged In: YES
user_id=875547
Originator: NO

More recent Privoxy versions will redirect requests for
privoxy.org/config to config.privoxy.org which should
solve (meaning: work around) the problem.

It doesn't look like the expiration of the original
redirect will be modified any time soon, so we'll
just have to live with it.