[Freevo-tracker] [ freevo-Bugs-2994707 ] Fix for appletrailers plugin

[SourceForge.net <no...@so...>]

Bugs item #2994707, was opened at 2010-04-30 14:14
Message generated for change (Comment added) made by duncanwebb
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=446895&aid=2994707&group_id=46652

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: plugins
Group: 1.x svn (rel-1)
Status: Open
Resolution: Accepted
Priority: 5
Private: No
Submitted By: John Molohan (johnmolohan)
Assigned to: Duncan Webb (duncanwebb)
Summary: Fix for appletrailers plugin

Initial Comment:
Hi Duncan,

Simply setting the following in local_conf.py seems to fix the appletrailers plugin while not having any side effects. You might test it and consider adding it in svn.

MPLAYER_ARGS['video'] = "-cache 5000 -user-agent QuickTime/7.5"

John

----------------------------------------------------------------------

>Comment By: Duncan Webb (duncanwebb)
Date: 2010-05-11 17:52

Message:
Hi John,

HTML parsing is pretty simple using a combination of lxml and the firefox
plug-in firebug.

The trick is to use the xpath from lxml which is extremely powerful and
use firebug to find elements, firebug can select the xpath. Then you can
use relatively easy searches to get the elements.

Example code is in:
svn://svn.freevo.org/freevo/branches/rel-1/testing/Duncan/WIP/util/fxdimdb.py

I prefer lxml to BeautifilSoup as it uses the element tree API and is
still supported.

----------------------------------------------------------------------

Comment By: John Molohan (johnmolohan)
Date: 2010-05-11 00:21

Message:
Hi Duncan,

I've done a bit more on it since, see attached patch. I've added some
debugging info, expanded the plugin description and added some ui feedback
with the busy timer.

Unfortunately as you point out a lot of the directories are empty. It
seems that Apple have changed the way they structure their site. The
majority of the new trailer sites seem to have the links to the .mov files
embedded in javascript which the plugin fails to find. I can't see how to
adapt it to scrape that info, maybe you'd have an idea?

The other idea which could be easier I've mentioned in the plugin
description. Apple have a set of xml files with direct links to each
trailer in them, it should be easy (at least for someone who knows wha
tthey're doing :) ) to rewrite the plugin to use those instead. It looks
like my patch might have come too late but maybe it'll get someone
interested enough to do the rewrite.

John

----------------------------------------------------------------------

Comment By: Duncan Webb (duncanwebb)
Date: 2010-05-10 17:53

Message:
Hi John,

Seems to work, thanks for the patch. This keeps the setting in the
plug-in.

First time I've tried the apple trailers plug-in; there seem to be a lot
of empty directories, it this normal?

Patch applied to svn at r11672.

----------------------------------------------------------------------

Comment By: John Molohan (johnmolohan)
Date: 2010-05-06 23:31

Message:
Hi Duncan,

Can you test this patch so:

---
/usr/lib/python2.6/site-packages/freevo/video/plugins/appletrailers.py.mdkorig
     2010-05-06 21:15:22.551413116 +0100
+++ /usr/lib/python2.6/site-packages/freevo/video/plugins/appletrailers.py
     2010-05-06 22:18:59.864389286 +0100
@@ -30,6 +30,9 @@
 import os
 import urllib

+import socket
+socket.setdefaulttimeout(5)
+
 import config
 import plugin
 import menu
@@ -61,7 +64,8 @@
     fn = os.path.join(cachedir, fn)

     if not os.path.exists(fn):
-        urllib.urlretrieve(url, fn)
+        urllib.FancyURLopener.version = 'QuickTime/7.5'
+        urllib.urlretrieve(url,fn)

     return fn

@@ -115,6 +119,7 @@
         VideoItem.__init__(self, url, parent)
         self.name = name
         self.description = "URL: " + url
+        self.mplayer_options = '-user-agent QuickTime/7.5'

 class Trailer(Item):
     def __init__(self, name, title, trailer, parent):


----------------------------------------------------------------------

Comment By: Duncan Webb (duncanwebb)
Date: 2010-05-06 18:20

Message:
Hi John,

This will only work for mplayer, it would be better to fix
video.appletrailers. A quick look and it is using urllib instead of
urllib2. With the latter you can specify the headers like user-agent.

Sorry but I will have to reject this as it is not really a general
solution.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=446895&aid=2994707&group_id=46652