#1038 Liferea Pulls Continuous Updates from TT-RSS

v1.8
closed-fixed
Lars Windolf
Networking (27)
7
2013-07-22
2012-11-01
Nathan Acks
No

When setting up a TT-RSS source, Liferea pulls continuous updates, downloading all of the feed data and then immediately downloading the data again. This cycle will continue until Liferea is closed, causing high load on the TT-RSS instance, and will immediately resume when Liferea is restarted. While this issue causes no noticable problems in Liferea's UI, if running TT-RSS on a small server it can cause enough server-side load to interfere with other webapps.

I see this behavior with both Liferea 1.8.7 and 1.8.10, both using TT-RSS 1.6.0 as a back end. However, this post from the liferea-devel mailing list seems to indicate that the issue has been present since at least Liferea 1.8.0 and TT-RSS 1.5.7: http://sourceforge.net/mailarchive/forum.php?thread_name=4F01473E.8060803%40exyr.org&forum_name=liferea-devel

A lightly sanitized log produced using liferea --debug-all is attached.

Discussion

<< < 1 2 (Page 2 of 2)
  • Lars Windolf
    Lars Windolf
    2013-05-08

    Hi Guys!

    First thing I need to say: I'm really sorry I didn't find time to support you earlier and give proper feedback.

    Actually I believe current git master has it already fixed. Can you please retest?

    The actual bug was in ttrss_source_auto_update () not behaving the same as the Google variant.

    The patch you guys created would also solve it, but I prefer the solution in git as it simplifies code and corrects invalid behaviour. I do not want to introduce a workaround instead. I hope you understand this.

    @Simon: As for the commit you mentioned. It is about simplifying the implementation of the TinyTinyRSS implementation. This commit makes it use feed metadata as all other node types do and removes the unnecessary timestamp handling. This is because each node source type relying on default updating shouldn't implement it's own handling.

     
  • Hi Lars, thanks for replying!

    I just did a pull and a recompile and unfortunately, no, I don't think git master has it fixed. If I run it with --debug-update, I see it pulling updates every 10 seconds.

    Using general mechanisms for handling feed metadata sounds great, but I don't quite understand how the ttrss_source_auto_update introduced in that commit, which is the same as in current git master could work:

    static void
    ttrss_source_auto_update (nodePtr node)
    {
            GTimeVal        now;
            ttrssSourcePtr  source = (ttrssSourcePtr) node->data;
    
            if (source->loginState == TTRSS_SOURCE_STATE_IN_PROGRESS) 
                    return; /* the update will start automatically anyway */
    
            g_get_current_time (&now);
    
            ttrss_source_update (node);
    }
    

    As far as my understanding goes, this is called every ten seconds by feedlist_auto_update. The only thing that would stop it from calling ttrss_source_update is if we're currently logging in. And ttrss_source_update calls subscription_update which starts pulling the updates.

    google_source_auto_update, by contrast, does the kind of timestamp checking that James' patch does, and that it seems ttrss_source_auto_update was doing before the above mentioned commit.

    Regards, Simon

     
    Last edit: Simon Kågedal Reimer 2013-05-09
  • Lars Windolf
    Lars Windolf
    2013-05-09

    Hi Simon,

    you are right ttrss_source_auto_update() is called every 10s because it should check every 10s if there is a pending update. If TinyTinyRSS is in state logged in it simply calls ttrss_source_update() which runs subscription_update() for each feed. Only subscription_update() performs the update interval check.

    This way we have the same code active for real feeds and TinyTinyRSS feeds.

    I did test with a 5min update interval so I see dozens of "ttrss_source_auto_update()" calls before the feeds are updated after 5min. Do you really see feed update processing handling after each ttrss_source_auto_update() for reach of the feeds in TinyTinyRSS?

    Cheers
    Lars

     
  • Hi again!

    Yes, I really see that. To be really sure it was nothing funny on my side I compiled a fresh git clone:

    git clone git://git.code.sf.net/p/liferea/code liferea
    cd liferea
    ./autogen.sh
    make
    sudo make install
    

    And cleaned up everything on the user side:

    rm -rf ~/.local/share/liferea
    rm -rf ~/.cache/liferea
    rm -rf ~/.config/liferea
    gsettings reset-recursively net.sf.liferea
    

    (I hope that gets everything)

    Then ran the freshly compiled Liferea with --debug-update. I added a ttrss source, a new local install of Tiny Tiny RSS 1.7.8 - and yes, updates every ten seconds.

    I attach the stdout+stderr output of running this for about a minute. You'll see several updates of the same feeds.

    I fail to see where in subscription_update there is timestamp checking. Shouldn't this involve subscription_auto_update? This seems to be where the timestamps are compared, but it is never reached through ttrss_source_auto_update.

    In ttrss_source_auto_update, I tried simply changing:

    ttrss_source_update (node);
    

    To:

    subscription_auto_update (node->subscription);
    

    This instead gives me no updates at all, even after timeout.

    Regards, Simon

     
  • Lars Windolf
    Lars Windolf
    2013-05-09

    Hi Simon,

    thanks for testing. Actually I messed up. I hadn't pushed the fix yet. It is online now (http://sourceforge.net/p/liferea/code/ci/master/tree/src/fl_sources/ttrss_source.c?diff=052111130e66067c29d885379f882cac92b1540a).

    It is as you tried just a replacement of ttrss_source_update() with subscription_auto_update() because this is the central function doing interval checking based on the updateState of a subscription.

    For me this works. I tried multiple times. I consider this to be solving the problem.

     
  • Allright! Mystery solved. :)

    I will test it again tonight.

     
  • Confirming that things work as expected in current git master!

    (If anyone in this thread has problems with this so that it doesn't run according to preferences, it might be that you've previously run Liferea with James' (or my) patch. This sets the update interval on ttrss subscriptions to once every day, saved in the database. Just remove the ttrss source and re-add it and things will work.)

     
  • Lars Windolf
    Lars Windolf
    2013-05-11

    • status: open --> open-fixed
    • assigned_to: Lars Windolf
    • Priority: 5 --> 7
     
  • Lars Windolf
    Lars Windolf
    2013-07-22

    Closing this

     
  • Lars Windolf
    Lars Windolf
    2013-07-22

    • status: open-fixed --> closed-fixed
     
<< < 1 2 (Page 2 of 2)