#90 sr_subscribe freezes forever

[Rolf]

Occasionally (about once per week in my experience), sr_subscribe freezes when trying to download a file. I have turned on -debug mode and I see "2017-11-24 14:03:01,616 [DEBUG] sr_http open" as the last line. At that point, it freezes forever (I have left it for several hours). I have attached the last few thousand lines from the log just before the freeze.

I'm running:
Python 3.6.3 (default, Oct 6 2017, 08:44:35)
[GCC 5.4.0 20160609] on linux

Under Linux Mint:
$ uname -a
Linux endlisnis7 4.11.0-14-generic #20~16.04.1-Ubuntu SMP Wed Aug 9 09:06:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux

My complete configuration file is here:

broker amqp://anonymous:anonymous@dd.weather.gc.ca:5672/

subtopic observations.swob-ml.#
subtopic nowcasting.#

accept .*

mirror True
use_pika False

[Rolf]

It froze again at 8:45AM (EST) today. I've attached the log leading up to this freeze.

[Rolf]

It froze again today at 9:05pm. I've attached the log leading up to this freeze.

[psilva]

We've been working on exactly that sort of problem for the last few weeks. We have added a number of internal timers, and periodic checks of memory usage, so that it should catch such things and recover automatically. If you upgrade to 2.17.12a6 (made fresh today ;-) it may well fix the issue.

[psilva]

Hi, internal testing revealed some more hard loops in the recovery code. Upgrading to 2.17.12a8 would be advised, still trying to get to a stable version.

[Rolf]

I have not experienced a freeze since upgrading to 2.17.12a8. You can close this ticket now.

[psilva]

Will do. We are still finding issues in our own testing, so upgrades to later versions are advised.
In the version you are running, we had introduced logic to recover failed downloads which was
in flux. The recovery logic is still improving, so upgrading to the latest version is recommended ( v2.18.02a1 ) is certainly going to react better when there are failures. In ordinary operations, there won´t be a visible difference.