My modificiations: Progress report

  • Jeffrey C. Jacobs

    I'm well on my way to implementing all of my suggestion to the ScaneRSS plugin.  When I'm done, I'd like the chance to go over with you each change for consideration of project integration.  Please, though, let's decide based on code impact since some changes my seem like a bad idea on paper but don't end up contributing much to the overall plugin and would be useful.

    So far, this is what I have:

    1) Parse RSSItem Description elements of the form:

    Foo: bar <br/>
    Boo: Baz <br/>

    The parser does not look for specific field names, and will ignore the description if it does not have that form.  (This is the way Demonoid formats its RSS Feed).

    Files changed: lbms/plugins/scanerss/main/

    2) Parse other elements found in an RSS item and categorize them as Meta-Data.  Only parse to first level and for multiple instance of the same tag, save only the last.  This effects all 3 RSS Parsers.

    Files changed: lbms/plugins/scanerss/main/
    Files changed: lbms/plugins/scanerss/main/

    3) Added code to display the meta-data stored within the RSSItem in the main ScaneRSS window GUI.

    Files changed: lbms/plugins/scanerss/main/

    4) Catch AccessControlException from scheduler.shutdown() so that the unload process can be completed.  See bug 1719333.




    try {
    catch ( e) {
        logChannel.log("Access Control Exception Encountered while shutting down threads!", e);

    In file lbms/plugins/scanerss/azureus/

    5) Fixed bug 1719442.  There is a bug in Java's implementation of the URL object (it has been submitted to Sun) where 2 solidus are removed from a valid file URL, breaking code that wishes to extract the path to the RSS XML when the URL uses the file schema (protocol).  Rather than using the broken toExternalForm() and removing the file:/// characters, the getFile method can be used to extract the embedded path and file name of the URL.  Specifically, in


    File f = new File(sourceURL.toExternalForm().substring(7));


    File f = new File(sourceURL.getFile());

    In file lbms/plugins/scanerss/main/

    6) Fixed bug (undocumented) where if neither a User Name, Password nor Cookie is set, but the HTTP-Referrer is set, it is ignored.  Replace:

    if (feed.getCookie() != null || (feed.getUsername() != null && feed.getPassword() != null)) {


    if (feed.getCookie() != null || (feed.getUsername() != null && feed.getPassword() != null) ||
        feed.getReferer() != null) {

    in file lbms/plugins/scanerss/azureus/

    7) Added code to allow the HTTP-Referrer to be set based on the URL associated with the item.  This requires use of the replace semantics for URLs but rather than allowing for /tor/ to be replaced with -- well, /tor/ in this case because you want the HTTP-Referrer to look like it came from the page the RSS Feed would have directed you to -- you have to specify a fully qualified URL.  I wanted the HTTP-Referrer to maintain existing functionality, so supplying /tor/ for the HTTP-Referrer would not work as it cannot be distinguished from a standard URL consistently.  One solution would have been to add a check-box switch to indicate whether the HTTP-Referrer should use the URL in the HTTP-Referrer box exactly as typed, or to have it act like the replaceAll box for the original link.  Instead, by using capturing groups in the ModLinkRegex, these can be extracted (from the first found match) in the HTTP-Referrer.  So, if the ModLinkRegex is "\d+)", your HTTP-Referrer can be "$1", where $1 is replaced by the digits following /tor/ in the URL.  The code requires the implementation of an expand function similar to the one in match.expand in Python.  A proposal has been submitted to Sun to add this method to the Matcher class.  For now, it is implemented manually in the View class.

    Files changed: lbms/plugins/scanerss/azureus/

    8) Added log messages for DownloadException, TorrentException and Bad URL exception.

    Files changed: lbms/plugins/scanerss/azureus/

    9) Added to do list.

    Files changed: lbms/plugins/scanerss/azureus/

    Still to do:

    1) Copy modifications to the other getTorrent/addTorrent methods (so far, only getTorrent(RSSItem) has been modified.

    2) Make the HTTP-Referrer expand method a generic function perhaps in a MatchResult-derived class while the method does not exist as part of the java.util.regex.Matcher class.

    3) Add meta-data from RSSItem to newly created torrent using custom properties defined for ScaneRSS.

    4) Devise strategy for Save Path Markup using defined (name, url, etc.) and meta-data fields.  Currently leaning towards "$(foo)" to get the property named foo in the directory path.  Obviously, "$(name)" and the like would have the obvious meaning, and anything not recognized would be looked up in the Description Parsed Strings and the Meta-Data, and if no match is found in any of those, maps to "".

    5) Save an automatic Character Encoding for new torrents so that downloads aren't delayed by a dialog asking for a selection.  Haven't figured out how to do this yet but it is quite annoying when the ScaneRSS identifies a new torrent for download and then ends up waiting until I visit my computer and click on ISO-Blah-Blah-Blah to tell it the Character Encoding for the given torrent before it even begins to download.  Very silly.

    6) When no default directory is provided, popup the standard Add Torrent dialog instead of dying.  This would require capturing and parsing the DownloadException to see if its message refers to a bad path.  The new log messages should make debugging this easier.

    7) Allow multiple values for a given field in the meta-data (i.e., Demonoid <category> should be a list of elements, rather than the last one.)

    8) Add Description Parser support for BitTurk.  This description format is VERY complicated so I may not do it.  Demonoid is easy so I think it worth keeping, but bitturk may become too esoteric.


    1) In the View constructor, in the implementation of the ActionProvider, in the method getTorrent, in the catch block for TorrentException, replace:

    throw new IOException(e);


    throw new IOException(e.getMessage());

    -- IOException objects could not be constructed from TorrentException objects, but Java 6 added support for construction from Throwable and so this code works in Java 6.  (I have closed the related bug.)

    2) Call the addDownload method overload with 4 parameters (middle 2 null) with the final one equal to initialState; verify that the code that sets it later is not needed.  Requires cast to DownloadManagerImpl; the DownloadManager interface does not support this overload of addDownload.

    3) Add better logging of errors such as "No Default Download Path" and bad replaceAll match.  Requires change to DownloadManagerImpl in azureus binary -- cannot be implemented in ScaneRSS.

    • Jeffrey C. Jacobs

      Oh, one To Do I forgot is:

      9) Allow the user to specify First, Last or x% for the initial leach position.

    • Leonard Brünings

      Hi Jeffrey,

      how is your modification going.

      I thought you wanted to send me the patches for ScaneRSS so that I could take a look.


Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:

No, thanks