#938 `Net::ReadTimeout with #<Socket:(closed)>` when using a connection pool

[Bráulio Barros de Oliveira]

I'm using Privoxy with Tor forwarding with Ruby's Mechanize.
In local single tests everything works fine.

But when in production with multiple threads for scraping, I get thousands of the following error:

 Net::ReadTimeout with #<Socket:(closed)>

Setting keep-alive-timeout to a higher value delays the problem a bit but it still occurs.

It seems related to Mechanize use of ConnectionPool ruby gem to reuse connections.

Any configuration or change that could help?

[Bráulio Barros de Oliveira]

replaced privoxy with https://github.com/oyyd/http-proxy-to-socks without any issues

[Fabian Keil]

A log excerpt would probably be helpful to debug this (if you still intend to use Privoxy).

https://www.privoxy.org/user-manual/contact.html

[Bráulio Barros de Oliveira]

I got the same error with http-proxy-to-socks so I think the problem is with Tor. I've updated with the version from their Ubuntu PPA and still got the issue.

Is there a way privoxy can recover from these socket closed error from Tor?

I'll collect more logs

[Fabian Keil]

Without a Privoxy log excerpt it's unclear to me what exactly happens.

Does setting keep-alive-timeout to 0 work around the problem?

[Bráulio Barros de Oliveira]

I've tried keep-alive-timeout with 0 and it didn't help
is debug 1024 enough? I'm enabling it now

[Fabian Keil]

That's not enough, please enable all the debug flags suggested at:
https://www.privoxy.org/user-manual/contact.html