Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

Replacing the JavaScript Engin with Python

alex sayle
2011-09-17
2013-05-30
<< < 1 2 3 > >> (Page 2 of 3)
  • chowjok
    chowjok
    2011-09-20

    Now I'm getting this:

    ../src/content_manager.cc:60:34: fatal error: layout/py_layout.h: No such file or directory
    
     
  • alex sayle
    alex sayle
    2011-09-20

    … woops. commited and synced. 

     
  • chowjok
    chowjok
    2011-09-20

    Got an error on autoreconf because of missing script(s) so I used autoreconf -i and it worked fine.

    Compiled successfully on Fedora 15. Nice work! I'll try doing some fancy stuff with python now. ;)

    Are there any other modifications that you plan to do?

     
  • alex sayle
    alex sayle
    2011-09-20

    Super, at least I'm not dreaming this up :)

    With in the MediaTomb runtime context there's a module called "mediatomb" which can be imported.

    The next step is to enhance that to duplicate the functionality of the JS.

    Now that it can be tested be other people, I'll take in any suggestions for API usage.

    Currently I'm thinking that the code should look like

    import mediatomb
    media = mediatomb.MediaTomb()
    # do magic with media 
    mediatomb.log("media (%s) has been processed into virtual path (%s)" % (media.location  , media.path) )
    
     
  • chowjok
    chowjok
    2011-09-20

    To be honest, I'm not a python programmer, so I can't really recommend anything about the API. That being said, what you suggest looks fine to me. I'm just glad that I now have something else to play with instead of JavaScript, and am no longer bound to the previous limitations.

    I'm just interested in the addCdsObject() and createContainerChain() functions and whatever syntax you choose is okay with me.

     
  • alex sayle
    alex sayle
    2011-09-20

    okie dokie. I'm hoping I can knock out the data retrieval for the media today , don't know if I'll get to the Cds side. either way I'll keep you posted. now back to the day job ..

     
  • genisis300
    genisis300
    2011-10-05

    i managed to get this to compile but i've been unable to get much further, when i run mediatomb with python enabled it run's tells me that python was loaded then segfaults.

    please help i'm really interested in seeing if i can help with this.

    Regards
    Matthew

     
  • alex sayle
    alex sayle
    2011-10-06

    yep, that's about as far as the release on github goes currently.

    I'm slowly hooking up the media's meta data to the python object so its coming along.

    If you're willing to hack python code in C/C++ with me I'll clone what I have to github.

     
  • genisis300
    genisis300
    2011-10-06

    I'm not sure how much help i will be but I'd love to take a look,  if i can work out whats going on i can have a crack at hacking some of the code with you.

    Regards
    Matthew

     
  • alex sayle
    alex sayle
    2011-10-06

    okie dokie. I'll branch the code and uploade it.

     
  • alex sayle
    alex sayle
    2011-10-08

    a quick update on this topic,

    I think I've hammered out the exposure of the media data in to python. you can now play around with the data as it coes in.
    However, you can't do anything with it yet :)  that part is next.

     
  • fleshTH
    fleshTH
    2011-10-10

    I'll preface to say that i absolutely HATE python. it is a disgusting language. I absolutely fell in love with mediatomb once i found out it had JS support. I was was less enthused, however, when i found out it really didn't have any expansion to the basic JS, such as CommonJS modules.

    So, with that said, I understand the want to switch to something a little more robust. One of the things i wanted to be able to do with MT was create online services through uPnP with virtual directories, similar to how youtube is supposed to work only this would be handled completely by script.

     
  • alex sayle
    alex sayle
    2011-10-10

    fleshth,

    I find your comment interesting so I'll bite. Plus I appreciate you made an account just post that comment.

    I've only very recently picked up python and frankly I'm language agnostic. Its the best tool for the job principal that led me to python and not ill wish against JS, I simply find it lacking for the task at hand.

    so that makes you feel so violently against python? the lack of braces? You even mention yourself that you consider python more 'robust'.

    I share your sentiment that the lack of expansion support is a pain. What drove me to place a python runtime was the lack of capability for MediaTomb to interact with the file system, or for that matter escape the sand box. For this reason I consider JS to be the wrong tool for the job.

    I did consider Lua as a scripting language as well, its a nice DSL but I decided that a large capability to interact with the world would be better suited for MT than a DSL.

    what I would like to be able to do is to fetch meta data for the media being classified from external sources if needs be, a folder that show which month a movie was released or classify movies by the leader actors kevin bacon score ( ok that's a joke but it'd be funny).  There are libraries to fetch such metadata from imdb, tvdb and the likes in python.  Sure, you _could_ do it in JS… but that doesn't mean one _should_.  To me the same applies for fetching such data in C++ and exposing it to JS.  _could_, yes, _should_, no.

    I would speculate that what you'd like to do could be archived using  the richer library selection of python than the limited very basic JS.

    I personally have no need for JS, but I haven't ripped it out, I understand that some may wish to continue to use JS. I'm simply providing options - for my own personal gain, and I've been very clear about that -

    As I now know the MT code base a little bit better, if you'd like to point me towards a fully featured javascript engine, I could take a hack at adding support for it. Mainly for the LOLz.

     
  • chowjok
    chowjok
    2011-10-10

    Now that we're talking about the language, I just wanted to say I prefer Perl for things like this but I don't mind learning a bit of any language if it'll allow me to do what I want with Mediatomb. My goal is to sort the movies by the director and cast on imdb and based on other threads I've read, this isn't possible with the current JS implementation. This will be possible easily with your plan to add Python because I could just load external modules that do what I want. If you decide to use another language instead, I'd support that too.

     
  • cmcardle75
    cmcardle75
    2011-10-10

    Is the python version usable yet? Ubuntu no longer allows the JS version, so a python version would be great! Due to a server upgrade, I'm left without a media server (mediatomb without scripting is like a car without wheels).

    I'm a bit worried that you haven't stripped JS out, though. Is it possible to compile with python, but no JS so it works on Ubuntu?

    I love the idea about cross referencing with imdb.

     
  • alex sayle
    alex sayle
    2011-10-10

    chowjok,

    I've coded far more perl than python ( and a little less than C ) and I can categorically say that embedding perl into another language is not fun, not easy, and well a bad idea™.  Hence my choice of python.

    And your imdb idea is exactly what I'm after you your wishes will come true soon.

    cmcardle75,

    if you define usable as "does everything that JS does" then the answer is "No".  if you define it by "can play with it" then yes.
    The current status as of yesterday is that I have all the basic attributes exposed into Python and you can acces / modify them.
    You can start playing with it and construct the logic for imdb lookups but you can't tell MT about it just yet.

    What's currently missing is the ability to inform the runtime about the cdsobject.  Parts of import.js and common.js also need to be translated into python ( which I've done the bare minimum of ).

    its a evening or two's worth of work and requires me not to go to the pub. If there's interest I'll see what I can do but this is my hobby so social calls will take precedent.

    Now as for your ubuntu concerns, my build platform is ubuntu so it works on ubuntu (11.04). By not ripping it out I mean that I haven't taken the code out. What I've done is add a -enable-scripting=   flag in the configure script so that you can select one or the other. When python is specified as a scripting engine it'lll disable JS and not look for it nor will it attempt to compile against it, thus enabling (python) scripting. 

    Those users whom wish to continue to use JS can use -enable-scripting=javascript and go on their merry way, I do not wish to stand in there way :)

     
  • chowjok
    chowjok
    2011-10-10

    I never mentioned Perl before because I usually end up getting flamed

    I don't have any experience with embedding a perl interpreter into anything, so I'm not sure why you say it's a bad idea (SWIG?) because I've seen it used in irssi and it seems to work very well.

    But anyway, like I said, I'm perfectly okay with Python or any language and I'm really excited about what you're doing. You're headed in the right direction with this project.

     
  • cmcardle75
    cmcardle75
    2011-10-10

    We used swig in a previous project, which is good for avoiding getting flamed. On the other hand, I'd rather have python this weekend (pub not withstanding) than swig in a month!

     
  • alex sayle
    alex sayle
    2011-10-10

    chowjok,

    SWIG makes bindings, not embedding the interpreter inside apps. at least that's my understanding.
    The MT scripting interface isn't a simple matter of wrapping a C/C++ interface in an external language its a bit more rough and ready than that.

    I've looked into putting perl inside another app before and the C interface to perl is not at neat/friendly as that of pythons.
    So yes it can be done and as you say irssi does a superb job. I simply believe perl would be more work for the same gain and I'm lazy. Beyond that my only real reason is that I'd done lua embedding before, tried perl so I thought I might as well give python a go :) plus flexget is written in python which is where I intended to grab the imdb lookup code from :D

     
  • alex sayle
    alex sayle
    2011-10-10

    cmcardle75,

    oh yes, swig is a great tool for "interface generating" and not getting flamed. What's occouring here is more glue. There's internal logic written in C++ to process the media items, the idea is to replace that loop with JS/Python and thus the layer that is getting exposed isn't a simple object or interface.  I don't believe swig is the right tool for the job here.

    My approch is to create a python runtime with in mediatomb in-place of the JS engine, then provide a mediatomb moedule / class with in the python runtime. the JS engine provided a few static functions that could be called, the python version will do that via a module.

    anyhow, my day job calls upon my attention.. must go hack.

     
  • chowjok
    chowjok
    2011-10-10

    I guess I misunderstood what SWIG really does. I tried znc which has its api exposed to perl and they use SWIG as a dependency so I assumed it lets you embed interpreters and create bindings.

    I'll have a look at flexget. As far as the imdb lookup code is concerned, there is a private json api (see http://api.imdb.com) used by IMDb for the Android and iOS apps. It works over SSL so it's a bit of a headache to reverse engineer it. I tried a simple MITM server but that didn't work because the Android app seems to be verifying the SSL certificate, so I've decompiled the app and I'm going through the code to figure out the params needed. I will eventually have something useful. I'll post as soon as I find out more about their API.

     
  • fleshTH
    fleshTH
    2011-10-10

    cmcardle75,

    Use this for JS support: https://launchpad.net/~jinto/+archive/mediatomb-js.

    crankcube,
    Yes, part of the reason i dislike python is because of the 'no braces' thing. It's very annoying to me. But there are other things that i remember not liking when i coded a script for XBMC. i was cussing and fighting my way through that script.
    The basic JS doesn't really do anything. And it's not supposed to do anything. It's just a basic ECMA-626, but that's where CommonJS comes into play. Other implementations based of javascript engines, such as Nodejs, v8cgi, and some of the other ones written with spidermonkey. (i prefer v8), work with CommonJS.

    I just wish there was something that allowed you to access outside of MT… even a shell exec would have been great. My 1st goal was to use TheTVDB for correctly naming episodes. Which was mostly impossible to do with just MT + spidermonkey. i say mostly because i had an idea of downloading TheTVDB's info and maintaining it in mysql, then using triggers, but i got a headache just thinking about it. Right now i have it set up with my import.js parsing both tv shows and movies with the minimally provided info contained in the filename. Then i use djmount on my HTPC which is read by XBMC which uses it's parsers to retrieve the information.

    I am honestly trying to think of an implementation of JS that would be good to use, but i'm coming up blank. The best would be NodeJS, but that's not really stable enough. So i guess python is the best choice for wide functionality…

     
  • alex sayle
    alex sayle
    2011-10-10

    chowjok,

    what Swig does is it takes C/C++ interface and generates code to marshall and unmarshall Python objects.
    in C/C++ sense, Python references are all PyObject * as they come in and out and if you know what the interface is, the code generation isn't rocket science.  the PyQt binding for instance is done mostly using swig and a little bit glue. ( and that's mostly license related code I believe )  the same goes for boost.python. Neither should generate code that includes
    Py_Main() or Py_Run*() calls that actually kick off an interpreter.

     
  • alex sayle
    alex sayle
    2011-10-10

    fleshth,

    yeah, I like braces too. and I'm totally with you on ECMA-626. I got that far and went "nice knowing you JS, please get out of my memory stack"  and showed it the door. ECMAScript is great for in-browser sandboxs, not the real world.

    In a sense allowed external access breaks what ECMAScript is attempting to do and provide so I don't disagree with their views and cosider NodeJs and friends to be a clunky hacks, but then I think a lot of things are clunky hacks :D call me old-school.

    I understand that people use MT on odd little NAS boxes and other rather strange places, JS runtime may have its place in such place, I simply don't know.

    … lets see. bum v8 isn't a drop in replacement to libmoz, that puts the hassle factor right up there … maybe once I'm done with python I'll take a look when I'm re-factoring code. For now you'll have to live with the lack of braces.

     
<< < 1 2 3 > >> (Page 2 of 3)