Menu ▾ ▴

#100 tv_grab_uk_tvguide: set required version for HTML::TreeBuilder

0.5.70
closed-fixed
Geoff
tv_grab_uk_tvguide (1)
5
2017-02-16
2017-02-15
Mike Parker
No

Attached patch ensures HTML::TreeBuilder version is at least v5.02.

As no calls are made to $tree-delete() in current grabber code, earlier versions of HTML::TreeBuilder consumed huge amounts of RAM as the Perl garbage collector failed to clean up contents of tree from memory. See http://search.cpan.org/~cjm/HTML-Tree-5.03/lib/HTML/Element.pm#Weak_References for details.

A days worth grab of ~42 channels data consumed 3GB+ RAM when using HTML::TreeBuilder v3.23!!

1 Attachments
HTML_TreeBuilder_version.patch

Discussion

  • Geoff

    Geoff - 2017-02-16
    • labels: --> tv_grab_uk_tvguide
    • status: open --> closed-fixed
    • Group: 0.5.69 --> 0.5.70
     

    • Mike Parker

      Mike Parker - 2017-02-16

      Hi Geoff,

      Thanks - it took me a while to determine that was the reason that my mythbox was running out of RAM whilst grabbing!!

      As no-one has taken any interest, would you mind test driving my other patch (#99)? It speeds up the retrieval of cached data considerably by removing the 1-5s delay between HTTP GET's for data coming from cache.

      Best regards,

      Mike

       

      Last edit: Geoff 2017-02-16

  • Geoff

    Geoff - 2017-02-16

    Thanks for the patch. I've modified this grabber (v1.22) to delete the tree after use.