Jpeg Frames

2008-04-21
2013-01-01
  • Hi,

    I'm trying to send jpeg-frames to the Movino server. There is one thing I don't understand in the documentation: Do I need to separate the jpeg header from the jpeg data and send one at a time? I mean, if I send only the header for the first frame with one package, and the just send the jpeg frame data for the following, wouldn't this screw up the decoding since the tables may differ? Or is the most common approach using some kind of standard tables in the header?

    The easiest for me would be to send the jpeg data "as is". Is this possible?

    Regards,
    Joachim

     
    • Hi Joachim,

      Great question!

      No, you don't need to separate the header from the jpeg data. This is just a small optimisation used; when using libjpeg directly, one can set it into a mode to create "abbreviated jpegs", that is, jpegs consisting of only the data with a constant set of tables. In this mode, it first outputs the header and the all subsequent images are headerless using the same tables as the initial one.

      Conversely, the receiver actually feeds both packet types into the jpeg decoder, with the only difference that the header blocks are fed to the decoder without expecting any output image from it.

      If unsure, just send complete, ordinary jpeg files as the "jpeg data" type.

      May I, out of pure interest, ask about what kind of application you're trying to build, feeding data to the movino server? :-)

      // Martin

       
    • I'm creating an live video streaming application for commercial use. I'm not sure what the L-GPL says about commercial use (I read it through but only became confused). Anyway, we need an application that differ from the Movino client anyway. My guess is that it's ok, for commercial use, to build our own client and use the server as it is. Is this correct?

      By trying both libjpeg, the built-in jpeg encoder, and a combination using both I get best performance using only the built-in encoder and setting everything up in a certian way (at least on the phone models we target).

       
      • What LGPL says is, in principle, that you're free to use, modify and distribute the modified versions. However, everything which is a derived work must still be licensed according to the LGPL.

        As in this case, you're free to use and modify the movino client, but then you're bound to the LGPL license. Starting with the Movino codebase but extending/modifying it is exactly the same; even though you've extended the code, that doesn't remove my copyright from it, and thus you'd be bound by my license.

        Reimplementing a compatible client from the ground up just using the protocol specs is not a derived work, and in that case you're free to use whatever license you want to, since you're the only copyright owner.

        What license the server has doesn't matter at all for you if you build a compatible client. You're also free to do whatever modifications to the server; the LGPL only demands that you publish the source to the modified version if you distribute it.

        So, to answer your question - yes, it's ok to build your own client licensed any way you want and use the server as it is, as long as your proprietary code isn't directly derived from (that is, "developed on top of"/extending) my  code.

        // Martin

         
    • Great! Thanks for your help!

      I'm trying to get the format right. If I get "bad block size" in daemon.log, is this because the data pay load size given in the header is too big/too small?

       
      • Yes, that's correct. This is set at line 159 in core/demuxer.cpp, and the limit currently is 1000000 bytes. This is to avoid buffering of huge blocks which wouldn't probably be valid data anyway.

        // Martin

         
    • I thought I solved the problem. I saw that I forgot to add 4 bytes for the timestamp to the data payload data size. I fixed this and looking in daemon.log it looks like this:

      new source
      [stream 7238fe75a3a5d4c9] started

      which looks ok. But then I discovered that the server is terminated and I have to restart it. So something goes terribly wrong! Any ideas?

       
      • Hi,

        I don't have any guess off-hand what could be going wrong. Could you try to debug the server and see where it crashes?

        Start the server by running "gdb movino-daemon" and then call "run <parameters>". Then connect to it as normally with your client. When the server crashes, do "bt" in the gdb prompt to get a call stack dump. That'll help me sort out the issue, and probably also pinpoint what you're doing wrong.

        By the way, a simple example on sending data to the movino server can be found in movino-suite/test/test-tcp-jpeg.cpp. But please do debug the server so I can fix the crash.

        // Martin

         
    • Ok, I ran it in debug mode. It responded diffently. I got a new errors on the phone app (client) - "esock_client 14" which means that it's problems with the descriptor parameter with the data to send. This is most likely a local error, however I can't figured out why it appeared now and not before.
      Anyway, suddenly some video appered on the server from one out of three attempts. After a while the server crashed, and this was after I closed the client. Strange. Anyway here's the output if it helps you:

      Program received signal SIGABRT, Aborted.

      [Switching to Thread -1218860144 (LWP 16850)]

      0x007d1402 in __kernel_vsyscall ()

      (gdb) bt

      #0  0x007d1402 in __kernel_vsyscall ()

      #1  0x00510ba0 in raise () from /lib/libc.so.6

      #2  0x005124b1 in abort () from /lib/libc.so.6

      #3  0x00546dfb in __libc_message () from /lib/libc.so.6

      #4  0x0054eaa6 in _int_free () from /lib/libc.so.6

      #5  0x00551fc0 in free () from /lib/libc.so.6

      #6  0x0805fce9 in ~ImageController (this=0x9033848) at image.cpp:98

      #7  0x0805ce5c in ~DeMuxer (this=0x900d720) at demuxer.cpp:117

      #8  0x08051c01 in ServerThread::main (this=0x90287f0) at serverthread.cpp:281

      #9  0x08051c75 in ServerThread::threadFunc (data=0x90287f0)

          at serverthread.cpp:137

      #10 0x006bf45b in start_thread () from /lib/libpthread.so.0

      #11 0x005b624e in clone () from /lib/libc.so.6

      I beleave I format the data correctly:
      1 byte = type (4 = jpeg frame)   \ next 4 bytes = payload size      /  header
      next 4 bytes = timestamp  \ jpeg data                 /  payload data

      I been looking at your code and it looks like you'r doing the same thing,
      except that the header is "sent" first (don't know if it's actually sent
      in the same moment calling send() though) and the payload data later.

       
      • Hmm, ok. I'll see if I find anything to fix in image.cpp for that case. Could you try running the server through valgrind (just "valgrind ./movino-daemon <parameters>"), with your current client code, and see if you find some other error?

        The format for the packet seems ok. I assume that you've noticed that the payload size in the header should include the 4 timestamp bytes, too? If not, that would explain the situation, which probably is that the server notices that the client sends bad data and is trying to close the connection.

        // Martin

         
    • I had that problem before, now I include the extra 4 bytes for the timestamp. I've checked the data I send and the first 9 bytes is header type + payload size + timestamp. The rest is a jpegfile.

      I do handshake with the server and then it takes a few seconds before I start the sending of jpeg frames. That's it. I only send jpeg frames (type 4) at this stage. When the connection closes down I assume everything else will close down correctly.

      Have I missed something?

      The Movino server seems to hang, and after a while it terminate. I haven't been able to run it with valgrind yet since I don't have direct access to the server myself.

       
    • This problem was investigated, the problem turned out to be that the client sent jpeg frames with a width not divisble by 16. This hadn't been tested earlier, and the code interfacing with libjpeg needed to allocate a bit more memory for padding for the rows. This error has been corrected in SVN rev 144.

      // Martin