#353 Will not grab feeds

v2.0
open
nobody
5
2005-05-08
2005-05-08
Geoff
No

I'm using iPodder v.2.0. When I check for new
podcasts, it scans the feeds, but doesn't download
anything.

The error log:

iPodder not grabbing subscriptions.
Uncaught exception!
Traceback (most recent call last):
File "iPodder.pyc", line 947, in start
File "iPodder.pyc", line 626, in download_enclosures
File "iPodder.pyc", line 586, in filter_enclosures
File "iPodder.pyc", line 538, in filter_enclosure
File "ipodder\feeds.pyc", line 225, in
get_target_filename
File "nturl2path.pyc", line 26, in url2pathname
IOError: Bad URL:
http://www.garageband.com/mp3/Storks-
Recuperating_Joan.mp3?|pe1|S8LTM0LdsaSnY1O0Zmw

I believe that someone else has already reported this
as well.

Discussion

  • Arik Baratz

    Arik Baratz - 2005-05-08

    Logged In: YES
    user_id=1274809

    I had the same happen to me. The problem is
    urllib.url2pathname() in Windows raises an IOError because
    it tries to parse
    Recuperating_Joan.mp3?|pe1|S8LTM0LdsaSnY1O0Zmw as a file
    name, and there are characters there illegal as a filename
    in Windows.

    I have fixed my iPodder by changing the get_target_filename to:

    def get_target_filename(self, enclosure):
    """Calculate the target filename for an enclosure
    referred to
    by this feed.

    enclosure -- either a URL or something with a .url
    attribute."""

    if hasattr(enclosure, 'url'):
    url = enclosure.url
    else:
    url = enclosure

    # clean up possible parameters after question-marks
    like in:
    # http://www.example.com/example.mp3?parameters
    if '?' in url:
    url=url[:url.find('?')]

    url = urllib.url2pathname(url) # decodes %20 etc
    filename = url.split(os.sep)[-1] # get last path
    component of URL
    result = os.path.join(self.target_directory, filename)
    # log.debug("%s -> %s", url, result)
    return result

    This removes everything after the first question-mark in the
    filename (not in the fetched URL)

    -- Arik

     
  • Geoff

    Geoff - 2005-05-09

    Logged In: YES
    user_id=1274808

    Ok, thanks for that help. However, how would I edit that
    value? Would I have to change the source code and compile
    it myself?

     
  • Arik Baratz

    Arik Baratz - 2005-05-09

    Logged In: YES
    user_id=1274809

    gklitt,

    1. Download the source code using CVS
    2. Run
    3. Make the change
    4. Run again

    It's written in Python which is compiled to byte-code on the
    fly, you don't need to recompile per-se. You do need the
    source code.

    -- Arik

     
  • Geoff

    Geoff - 2005-05-09

    Logged In: YES
    user_id=1274808

    Wow...sorry if I sound like an idiot, but that makes no
    sense to me. I think I have found CVS (Concurrent
    Versioning System?) and downloaded it, but I have no idea
    how to download the iPodder source code or even run CVS
    itself. I have tried to run CVS many times, to no avail.
    Also, do I need a seperate Python editor to edit the code,
    and do you have any recommendations? Any help would be
    greatly appreciated. Again, sorry that I don't know this
    stuff, you probably assumed that everyone submitting bugs
    for an open-source program would know about code editing.
    Anyway, if you have answers to any of those questions, it
    would be hugely helpful.

    Thanks again,

    gklitt

     
  • Geoff

    Geoff - 2005-05-09

    Logged In: YES
    user_id=1274808

    Also, is this caused by a certain podcast that I can just
    delete? It seems weird that a garageband song would be
    trying to get downloaded by iPodder.

     
  • Arik Baratz

    Arik Baratz - 2005-05-09

    Logged In: YES
    user_id=1274809

    Disclaimer: You really should know more about what you're
    doing. This might not work and might screw up your system,
    and I will not carry an iota of the blame.

    First you need a Python environment, with pyxml and
    wxPython. Download and install the following:

    Python
    ------

    http://www.activestate.com/Products/Download/Download.plex?id=ActivePython

    I suggest you take Python 2.3.5.236 and not 2.4.1, there's a
    parser bug in 2.4.1 that makes it unusable for me for other
    projects that I have. ipodder might be okay with 2.4.1 but I
    didn't try it.

    wxPython
    --------

    http://www.wxpython.org/download.php#binaries

    Take the unicode version for whichever Python you installed
    (2.3 or 2.4)

    pyxml
    -----

    http://sourceforge.net/project/showfiles.php?group_id=6473

    Take the latest version, the file which corresponds to your
    python version.

    After you installed all these, you need to install a CVS
    client and use it to download the ipodder sources. My client
    of choice for Windows is TortoiseCVS, from
    http://www.tortoisecvs.org/download.shtml.

    After the reboot (yes, it needs a reboot, it installs some
    heavy stuff in explorer) create a folder for the sources,
    right-click inside it, and select CVS Checkout.

    In the dialog box put the following CVS path:

    :pserver:anonymous@cvs.sourceforge.net:/cvsroot/ipodder

    Don't worry about the other parameters, except for the last
    one - module name - which should be "iSpider" (not iPodder).
    After you okay this, it should create a subfolder iSpider
    beneath that and download the source files into it.

    You should be able to run iPodder from source now by
    double-clicking on the file "iPodderGui.py" in that folder.
    If it works, great. If it doesn't, either you've done
    something wrong or I've missed something in my instructions
    - my machine is not clean and I might have something you
    might be missing. Oh, you have to quit the iPodder you're
    running before you can do it.

    If it runs okay, try to apply the change to the feeds.py
    file (find the function and replace it). If you did what I
    wrote you should have an editor installed by now -
    right-click the file and choose 'Edit in Pythonwin'.

    Python cares about indentation (==the space to the left of
    the code) so when you make the change, make sure the
    indentation stays the same as the original. In fact, don't
    delete the original before you make the change. oh yes, make
    a copy of the feeds.py files before you change it.

    The change can be made while ipodder is running, but
    obviously will not take effect until you shut it down and
    restart it, from the source. If you want to eliminate the
    ugly extra window that opens up, create a copy of
    iPodderGui.py and call it iPodderGui.pyw (notice the extra
    "w") and double-click that instead to run it.

    And to your other question: Yes, it is caused by a certain
    podcast, I think maybe OpenPodcast. It's not strange at all
    - in an RSS feed, one can put any URL for the enclosure (=
    the actual file to download). Someone elected to create an
    entry in an RSS file that points directly to the garageband
    site - perhaps the band trying to promote itself.

    -- Arik

     
  • Arik Baratz

    Arik Baratz - 2005-05-09

    Logged In: YES
    user_id=1274809

    I've found the culprit - it's the "Source Code Mini" at:

    http://feeds.feedburner.com/scm

    Anyway, the code needs to be change, so that the integrity
    of ipodder's operation won't be affected by any specific feed.

    -- Arik

     
  • Geoff

    Geoff - 2005-05-09

    Logged In: YES
    user_id=1274808

    Wow, that actually worked! I can't say how grateful I am to
    you for helping me fix this problem! Of course, I had 10
    podcasts to download after I fixed it, since I hadn't
    downloaded any for a while, but it worked anyway, and that's
    all that matters. Thank you so much.

    Note: perhaps I should send a note to Shooby over at the SCM
    telling him to fix that URL.

    Thanks again,
    gklitt

     
  • Arik Baratz

    Arik Baratz - 2005-05-09

    Logged In: YES
    user_id=1274809

    Wow! I have to say I'm pleasently surprised. But then I'm
    usually the pessimist. And all that info off the top of my
    head, without actually trying it.

    Just a few notes: If you want to upgrade your ipodder from
    now on, you can stay with the source version. Just go to the
    iSpider folder, right-click it and select 'CVS Update'. The
    down side is that you are getting code that has not made it
    into a release yet, so you may experience unique bugs.

    Your properly-installed ipodder and source ipodder use the
    same data (i.e. same database of podcasts, etc). If at some
    point of time in the future the database structure becomes
    incompatible, the properly-installed ipodder may behave in a
    strange way, not work or corrupt your data. My
    recommendation: use one or the other, but not both.

    IT's not Shooby's fault per se. He has given a URL to a
    file. The URL is valid. What ipodder was trying to do is
    decide on a proper filename to save it in. The algorithm for
    determining the name is simply: take everything after the
    last slash in the url. In Shooby's case, the URL included
    some characters that are not compatible with Windows' file
    system. Well, I think ipodder needs to handle that case, and
    that is the essence of my fix: remove everything after the
    question-mark.

    The view of the people who actually maintain iPodder
    (hello!) might be a little different, though.

    -- Arik

     
  • Sam Denton

    Sam Denton - 2005-05-10

    Logged In: YES
    user_id=96197

    This looks the same as bug 1197484, in both cases there are
    pipe symbols in the filename. It is also similar to 1194177
    (subbmitted by myself), except that the error occurs in a
    different module and the illegal characters are different.

    I dislike the proposed solution, because it is truncating
    everything after the '?', which works ok in this instance, but
    would fail for mine, where the name of the file to download
    occurs after that character. I don't program in Python, just
    Perl, so I don't really want to download the entire
    development kit, but I think that a better solution would be to
    scrub the filename using Python's version of the url-encode
    routine. This should create a filename usable on all platforms
    without losing important data in the url.

    Thanks!

     
  • Sam Denton

    Sam Denton - 2005-05-10

    Logged In: YES
    user_id=96197

    OK, I dug a bit more into Google and came up with this
    totally untested idea:

    def get_target_filename(self, enclosure):
    """Calculate the target filename for an enclosure
    referred to
    by this feed.

    enclosure -- either a URL or something with a .url
    attribute."""

    if hasattr(enclosure, 'url'):
    url = enclosure.url
    else:
    url = enclosure

    url = urllib.url2pathname(url) # decodes %20 etc
    filename = url.split(os.sep)[-1] # get last path
    component of URL

    # clean up possible parameters in the URL
    filename=urllib.quote(filename)

    result = os.path.join(self.target_directory, filename)
    # log.debug("%s -> %s", url, result)
    return result

     
  • Arik Baratz

    Arik Baratz - 2005-05-10

    Logged In: YES
    user_id=1274809

    You're right, but that would make every filename with a
    space very ugly.

    I think filtering after a question-mark is okay in any case,
    because a question mark separates the URL from GET
    parameters in the URL. Perhaps on Windows the filename need
    to be sanitized, removing or replacing the specific
    characters that are illegal on Windows (there's a way to get
    a complete list on MSDN).

    -- Arik

     
  • Nobody/Anonymous

    Logged In: NO

    You're right, but that would make every filename with a
    space very ugly.

    I think filtering after a question-mark is okay in any case,
    because a question mark separates the URL from GET
    parameters in the URL. Perhaps on Windows the filename need
    to be sanitized, removing or replacing the specific
    characters that are illegal on Windows (there's a way to get
    a complete list on MSDN).

    -- Arik

     

Log in to post a comment.

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks