Data Dropouts in other Applications

Help
Ron Todd
2013-04-29
2013-05-02
  • Ron Todd
    Ron Todd
    2013-04-29

    Hi,

    When RSSOwl is updating feeds, another application intermittently loses data connections. Does anyone know how I can fix this?

    Here are the particulars:

    I have ~1,250 feeds which are automatically updated every 1 minute. Since these take ~1 minute to update, the updating is almost continuous. When RSSOwl runs on the same machine as a stock trading application (also running under Java), the stock application momentarily loses one or more data connections (it has 10) at an average rate of 1.5 losses/minute. Setting DmaxReloadJobs=10 when starting RSSOwl slows the update down so that it takes 3 to 5 minutes, but the data connection dropout rate remains essentially the same. When I run RSSOwl on a different machine than the stock application, but still on the same LAN, I get the same dropout rate (1.5/minute) as if it were running on the machine with the stock application. If I run RSSOwl (configured identically) on both machines, with one instance of the stock application also running on one of the machines (all are on the same LAN), dropouts in the stock application climb to an average of 4 per minute. I am nowhere near saturating my processing (<20% on a quad-core i7), RAM (3 out of 8 GB), or network bandwidth (network utilization ~1% of 130 Mbps). My computers are running Windows 7 home premium 64 bit, latest Java (version 7 update 21), latest RSSOwl (2.1.6).

    Any help would be appreciated,
    Thanks,
    Ron

     
  • genodeftest
    genodeftest
    2013-04-30

    I can reproduce your problem. I added ~200 feeds to RSSOwl and make it update every (!) second. Every time a update is scheduled RSSOwl adds all feeds to an internal list of feeds-to-be-updated and starts a new thread to download them all.
    Your ~1.250 feeds may (will) take more than 1 minute to update, but RSSOwl will start updating them again after 1 minute. Updating more feeds in parallel will slow down your internet connection a bit. After running RSSOwl for a while it will gather more and more "update all feeds" tasks, which will slow down your internet connection which will make RSSOwl update more feeds at the same time. At some point RSSOwl will update so many feeds in parallel that it consumes all your bandwith. At this time your other internet connections will time out.

    This is a extreme use case but uncovers a bug in RSSOwl: RSSOwl should only start updating all feeds when there is no other update running.

    As a workaround I recommend configuring RSSOwls feed update interval to at least ten times what it takes to update all feeds.

    Bug details:
    This bug basically leaks update threads. As a solution RSSOwl should add "update tasks" to a global (unique) queue handled by a Thread / thread pool instead of starting a new update thread for each and every time a feed update is triggered. The thread pool should have a fixed number of threads updating feeds.
    This bug can also be used to make RSSOwl run out of memory by spawning new worker threads before other worker threads have exited. I can reproduce this to make RSSOwl spawn >500 Worker threads and run out of memory under any condition (huge amounts of memory available, e.g. -Xms1G -Xmx8G) while listing >10000 threads to update in "Activities View".

     
  • Ron Todd
    Ron Todd
    2013-05-02

    Thank you for the analysis. That explains why I was seeing such large numbers of feeds being updated in the Downloads and Activity pane (I had assumed that this was just a cumulative count from when I started the program, not what was actually in the queue to be serviced!).

    I tried your suggested workaround, and that improves but does not resolve the interference. My stock trading application looks for a heartbeat every 10 seconds on each of its data connections or else it complains. I have RSSOwl updates set for every 5 minutes now (10 minutes gave the same result), and each update takes about 1 minute and 20 seconds. Near the end of the updating (usually over 80%) I occasionally get a data loss on the stock application, but data losses are far most numerous (5 or 6 connections down briefly) in the 45 seconds AFTER updates have completed! Any clue as to what RSSOwl may be doing in this interval that would prevent the other application from seeing the heartbeats? CPU, disk, and network activity in this interval are low.

    By the way, none of the other locally-executing RSS Readers that I have tried (BlogBridge, RSS Bandit, Awasu) exhibit this problem (using the same set of feeds and the same update interval). But I really like the features of RSSOwl over any of the others, so I'm anxious to have it working!

     
  • genodeftest
    genodeftest
    2013-05-02

    Maybe you chose to display an excerpt of the news (edit → preferences → notifications)?
    Are you using the newspaper layout?
    Both of them can make RSSOwl's internal browser loading the complete feed content (full text if available, images, style sheets, …). Since this is something the internal browser does it will not occur in RSSOwl's Activity view.

     
  • Ron Todd
    Ron Todd
    2013-05-02

    Unfortunately, that's not it. I'm using the program defaults, which for Tools -> Preferences -> Notifier; General has "Show an excerpt of the news content inside the notification" unchecked. I am also using View -> Layout -> Classic.

    I do have 4 saved filter feeds defined, though, and these are quite complex (2 of them have 172 search terms and the other 2 have 53). I deleted these and restarted RSSOwl. Unfortunately, this also did not resolve the interference.