Mediatomb and P2P TV

jeff
2007-03-29
2013-05-30
  • jeff
    jeff
    2007-03-29

    Hi,
       I have a DSM-320 and watch p2p TV on linux for quite a time, such as the http://www.sopcast.com, now what I want is:

    1. show the program list on DSM-320 and watch those movies

    is it possible to do it the following way? I'm going to try it when I got the time

    a: sopcast has a CLI, and can be setup to watch using VLC at http://localhost:port/
       1. so I can using perl to dynamically retrieve TV list and set up stuff needed by sopcast
          before the actual http-get device request
    b: I have the apache setup on the box also, so I can add a virtual Item (external URL) to trigger the step a script, let's call it 'SOPCAST' under VIDEO container
    c: when SOPCAST was attemped from renderer, step a will be triggered, it will get the TV list and then populated/update the mt_cds_object table for every channel on the TV list (which I think will be external URL links)

    Jin, do you think it is theoretically fine?

    You mentioned there is a 'active item' help link in UI, but I can't find it, I am using the 0.9.0 release

    Thanks!

     
    • jeff
      jeff
      2007-03-30

      Jin,
          I did find any information about active item (except mentioned a user-defined script will be triggered when renderer try the 'http-get' request), js engine is so nice for user to organize their own data stuture,

      I think it might make mediatomb more flexible if provide some well-defined interface for user to plug-in their own stuff at event such as:

      referer try to
      get the content (active item did this already)
      stop playing (so we can do some cleanup stuff)
      ...

      I did some research on uPnP software for linux, it is sooooooooooooo good, Kudos to you guys!

      if possible, can you direct me to the document regarding to 'active item'?

      Thanks!

       
    • Jin
      Jin
      2007-04-01

      Hi everyone,

      sorry for the late response, I'm away for two weeks, just got myself a slow GPRS connection here yesterday :)

      Well... unfortunately active items are not well documented :) In the 0.8 series noone seemed to be interested in this too much, so this feature had only very little attention from our side when preparing 0.9.0. It does work, though :)

      Now, first thing to mention: import scripts and active items are not reall realted; that means - an active item script can be anything that will be processed by your operating system. The only important thing is that your script accepts an XML on stdin and prints the resulting/changed XML on stdout. The rest is really up to you.. so you can use perl, python, write your own C program or anything you want.

      allthough it will probably work, we do not recommend messing with the database directly; we plan to add playlist support in the future releases, but well - I guess noone can prevent you from hacking :)

      Basically you could either precache your TV list and populate the database, this is somewhat risky because you really have to know what you are doing, and you have to know the "rules" of populating the database. If you break something, things might stop working at all and you will have to recreate the DB.

      The second possibility - you could write a simple UI-fake application (Nik did that for shoutcast streams. As you know our web UI uses AJAX to communicate with the server. All parameters directed to the server are encoded in the URL and a GET request is issued; the response from the server is always an XML.
      So, you could write a small APP that would open a UI session, and simply add new objects to the server by issuing appropriate HTTP GET requests and reading the XML responses; so you would be acting as an alternative UI. The benefit of that solution is, that all safeguards that we put in place will work, so it is unlikely that you will break the DB or do something that may produce unexpected results.

      The third way is probably not exactly what you want, because it allows you to manipulate only the current item. That's what active items are for; so theoretically you could implement something cyclic like "Channel Up" using active items. The "problem" with active items is, that it only works nicely if your renderer listens to CDS container updates. Meaning: if something changes in the database, we let the renderers know via a UPnP-specced mechanism, however, this part of the spec is optional and not all renderers implement that. A renderer that supports it will immediately refresh the current container, if something has changed; else you will have to go out and browse it again to get the changes.

      What follows is a paste from a mail that I sent out to one of our users, who was also interested in ActiveItmes:
      The idea is simple: when you issue an HTTP GET request on some item (i.e. -
      press play on your renderer), the server will execute a script of your choice.

      This script takes an XML fragment, which describes the item, as input, it can
      modify the XML to change the items properties, and then feeds that
      XML back to the server. Of course this external script - written in the
      language of your choice - can also call some programs.

      I once wrote a demonstrator, where I used MediaTomb in combination with an
      SL300i renderer to turn on and off lights in the house (light control was done
      via an rfid dongle), MT executed scripts to control the dongle.

      The important thing in this case is, to have a renderer that supports
      ContentDirectory Service ContainerUpdates, because then the device will
      refresh the view automatically when items in a container change.

      So the operation goes like this:

      You have your active item, titled "Turn Light On", the item points to an
      image of a lightbulb that is on and has a thumbnail that displays the
      bulb in the off state - the current state.

      Now, on your renderer you "play" this item, the image with a glowing
      lightbulb is being displayed. At the same time MT executes your script;
      the script, changes the location of the object to a non-glowing lightbulb,
      and changes the title to "Turn Light Off", the non-glowing lightbulb image
      has a thumbnail which displays the "on" state.

      So each time you hit play on this item, the title and the image change
      accordingly. If your renderer - like the Streamium SL300i - supports
      ContainerUpdates, then you do not even need to refresh the view, the title
      will update automatically.

      Take a look at the demo_toggle.py script that is located in the src/scripts
      folder in our distribution.

      -----

      So indeed, take a look at demo_toggle.py, I think it gives a pretty good explanation of how things work.

      I hope the above made sense, just ask if you have more questions.

      Greetings,
      Jin

       
    • jeff
      jeff
      2007-04-01

      Jin,
           I heartly appreciate your help here, you are right I should not meddle with database directly since there is a alternative method (the xml and AJAX, request and reponse with mediatomb server), I will change to that method after making P2P TV to work

      from your description, active item is so cool, seems should solve my problem, I will update you guys and probably a little document about the steps I did to make it working.

      still two things need to get done before it working:
      asf --> xvid transcoding
      DSM-320 chinese character thing,

      Jin, you mention some guy update the DSM-320 firmware to support their native lanaguage, can you point me that information, if it is too much hassle, don't bother!

      Thanks!

       
    • Jin
      Jin
      2007-04-02

      Hi Jeff,

      this is the post about the custom firmware:
      http://forum.dlink.ru/viewtopic.php?t=36592

      unfortunately it is in russian, so I am not sure if it will be of any help to you. The guy who managed to modify the image is "Sharo", he is the one who started the thread. Maybe you can PM him or write him on the forum or write him a mail or something. I am on a slow GPRS connection now, so I can not read through the whole thread; however it seems he did not post the information on how he did it, he just made the image available.

      I hope this helps!

      Greetings,
      Jin

       
    • jeff
      jeff
      2007-04-04

      Jin,
         I almost got it working (need to put the encoding part after this resolved), it is too slow to watch, after some packet sniffing, find out that:

      1. dsm-320 made some duplicated and unsequential requests (such as)
      Host: 192.168.1.11:80^M
      Range: bytes=10258-^M
      User-Agent: dma/1.0 redsonic^M

      Host: 192.168.1.11:80^M
      Range: bytes=65540-^M
      User-Agent: dma/1.0 redsonic^M

      unsenqutial is OK, is there a way to reduce the dumplcated request through the upnp message

      2. the http 'Range: bytes=16388-' is too small, so it called the script too many times, too much overhead
      seems the dsm-320 only allow the buffering about 16883 bytes, is there a way to work around that, or is it a hardware limit

      I was thinking to use mod_perl to speed up the script, but if dsm-320 only ask for less than 17k data per request, seems it will not help too much, I try to feed more data than requested, it seems trying to ignore

       
    • Jin
      Jin
      2007-04-04

      Hi Jeff,

      have you added the custom dsm-320 settings (the redsonic http header) to your config.xml? As far as I know it will tell the DSM to use keep-alive, this may help a little. However, for the requests: well yes, I also saw on ethereal that the DSM requests data in small chunks and basically floods the server with GET requests. I do not know a way on how to prevent that - that's in the DSM firmware.

      We do plan to add transcoding support, my idea there was to introduce a cache/buffer that would store a part of the transcoded data. I guess that you may need something similar, a way to precache portions of your online stream, so the DSM makes it's tiny requests to the cache (which should be fast enough), and the rest of the data is downloaded as needed to keep the cache full. I guess this is only possible for data formats that do not require seeking (like MPEG1 or MPEG2).

      Greetings,
      Jin

       
    • jeff
      jeff
      2007-04-05

      Hi Jin,

      I totally rewrote the script to use mod_perl and now it is fast enough (I did put the trancoding part in yet),

      however it always stop at the same place (about 2 - 3 minutes), and dsm320 give me the end of video messages (I know it is not the end)

      Thanks!

      Host: 192.168.1.11:80^M
      Connection: Keep-Alive^M
      Range: bytes=10216674-10233057^M
      User-Agent: dma/1.0 redsonic^M
      ^M
      ¸ð^TF^[(^B^@~R^K^@^@~R^K^@^@^@^O=ßH¶^@À~_%Ãe^H^@E^@^K~DÕ~\@^@@^FÖkÀ¨^A^KÀ¨^A^P^@P^D^Wê±r~V + + +    QÔê~@^P^GÓ~Nâ^@^@^A^A^H
      ^Ds2~C^@^@:-HTTP/1.1 200 OK^M
      Date: Thu, 05 Apr 2007 12:51:04 GMT^M
      Server: Apache/2.2.3 (Fedora)^M
      Content-Length: 16384^M
      Last-Modified: Sat, 31 Mar 2007 20:30:23 GMT^M
      Content-Range: bytes 10216674-10233057/367482880^M
      Accept-Ranges: bytes^M
      X-User-Agent: redsonic^M
      Keep-Alive: timeout=15, max=23^M
      Connection: Keep-Alive^M
      Content-Type: video/x-msvideo^M
      Content-Language: en^M

      GET /view_tv?url=sop://broker.sopcast.com:3912/8301 HTTP/1.0^M
      Host: 192.168.1.11:80^M
      Connection: Keep-Alive^M
      Range: bytes=10234338-10234869^M
      User-Agent: dma/1.0 redsonic^M
      ^M
      ¸ð^TF^HÐ^E^@½^C^@^@½^C^@^@^@^O=ßH¶^@À~_%Ãe^H^@E^@^C¯Õ¸@^@@^FÞ$À¨^A^KÀ¨^A^P^@P^D^Wê±æ^E  QØ + + +  K~@^X^GÓ~G^M^@^@^A^A^H
      ^Ds2¿^@^@:E
      HTTP/1.1 200 OK^M
      Date: Thu, 05 Apr 2007 12:51:04 GMT^M
      Server: Apache/2.2.3 (Fedora)^M
      Content-Length: 532^M
      Last-Modified: Sat, 31 Mar 2007 20:30:23 GMT^M
      Content-Range: bytes 10234338-10234869/367482880^M
      Accept-Ranges: bytes^M
      X-User-Agent: redsonic^M
      Keep-Alive: timeout=15, max=18^M
      Connection: Keep-Alive^M
      Content-Type: video/x-msvideo^M
      Content-Language: en^M

       
    • jeff
      jeff
      2007-04-12

      Here, I like to express my appreciation to Jin who help me a lot along the way.

      please let me advise you first, please don't even try it if you would not consider yourself a hands-on guy on the linux!

      After 2 weeks of later night work and constant complains from wife and kids (they needed my attention), finally it is working for me (well, still need to make it perfect)

      my setup is like this:

      1. installed mod_perl and apache for the script (let's called VIEW_TV) (invoked by dsm320 as external link through UPDATE_TV_LIST script)
      2. setup a transcoding daemon process (call it asfMPEG) to start retrieving P2P program stream (ASF format) and do the transcoding upon request from VIEW_TV (contact asfMPEG if the mpeg file isn't there)
      3. the asfMPEG wait for request on a socket connect from VIEW_TV with a timeout (if no request coming in, it just do some maintenance job, such as UPDATE_TV_LIST or CLEAN_UP),
      4. UPDATE_TV_LIST is a routine to retrieve tv_program list from web and insert into mediatomb mt_cds_object so you can see updated tv list on DSM320
      5. CLEAN_UP is just a routine to clear up the mess left over by viewing tv (temporary asf file, transcoded mpeg file and might some lingering processes)
      6. I am using 'mplayer/mencoder' to do the transcoding in real time, modified the mplayer/mencoder source code and compile it for blocking/waiting when the source stream/file is reached end of file (in this case, it might be slow network for P2P TV)

       
    • Jin
      Jin
      2007-04-13

      Cool! :)

      Allthough I hope that transcoding will be made easy once we offer it as a feature in a future release :)

       
    • YuChung Wang
      YuChung Wang
      2007-07-26

      From this great work, we know that the major components in the transcoding is

      * Streaming server: This can be the mediatomb web server itself.
      * Transcoder: This can be ffmpeg/mplayer/transcode/....
      * EPG generator: This is the perl script which can read the media object list and organize them into a tree.

      The EPG generator will generate URL which can be parsed and handled by the mediatomb web server and call the transcoder.

       
    • Jin
      Jin
      2007-07-26

      Well, this EPG thing goes into the direction that we already had in mind: we want to offer an easy way to import and organize online content. I am also interested in the sopcast thing - I tried it and it is actually pretty cool. Well, we will finish the transcoding feature first, then we will take care of the online content.