Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

latency and sync

Help
peter
2007-02-20
2013-05-14
  • peter
    peter
    2007-02-20

    greetings!

    i am attempting to use xjadeo to do audio post for video, and am wondering about how to ensure accurate sync. it appears that, when using xjadeo with a jack-transport-aware application such as ardour, the sync between video and audio is dependant on the jack server latency. i.e. on playback, the audio is earlier, relative to xjadeo's video, when the jack server has been started with a lower latency. so, video latency seems constant, while audio latency can be changed independant of that.

    does this imply that the only truly accurate sync would occur with a 0 latency, or is there actually some video latency as well. if there is video latency, how can we determine what it is, in order to compensate the audio latency accurately?

    thanks!

    .pltk.

     
    • Robin Gareus
      Robin Gareus
      2007-02-20

      Hi plutek!

      > i am attempting to use xjadeo to do audio post for video, and am wondering about
      > how to ensure accurate sync.

      xjadeo can not assure synchronization while JACK-transport is rolling (the xj-5 branch tries to solve some of that issues, but you don't want to use that in a production system just yet..)

      The time displayed on qjackctl or ardour timeline should match the SMPTE (or frame) of xjadeo's On Screen Display: It always rounds down to the last video-frame!

      There is a time-stamp-movie-maker in xjadeo's contrib/ which I used to assure that xjadeo displays the correct frame when the transport is stopped. (AFAIR I've only tested 25fps and 24 fps)

      > it appears that, when using xjadeo with a
      > jack-transport-aware application such as ardour, the sync between video and
      > audio is dependant on the jack server latency.

      yes and no. JACK-transport is "correct by definition". The audio-app has to play the sound earlier to compensate for latencies (sound-card, effects, etc) - ardour2 does this very well!

      when I play a file with ardour2 and monitor the audio-data on the jack-bus, the audio is ahead in time by one-hardware-audio-buffer (it takes that long to get out of the computer to the speakers.)

      However some Plugins do *not* announce their latencies correctly or ignore to forward output port latencies. - I've experienced  Audio-phasing before I could notice A/V sync issues!

      > i.e. on playback, the audio is
      > earlier, relative to xjadeo's video, when the jack server has been started with
      > a lower latency. so, video latency seems constant, while audio latency can be
      > changed independant of that.

      correct. but you should not change the audio-latency, just use the values appropriate for your hardware.

      xjadeo only allows to offset video-frames not audio-frames from the jack-transport position! - I did not yet see a need for it - but one more command line arg (+ LASH setting + rc-config file + remote control command) won't hurt, will it?

      If you want to dig : ardour always uses the max. latency of it's output ports: connect ardour's output thru some application that forwards the audio, but reports zero port latency... play the same audio file phase-inverted with ecasound-> phase shifted instead of muted sound. go one more step and add some Plugins to ardour - toggle their bypass to discover some bugs in ardour and mute the signal!

      There's recent development on this (libjack 0.102 supports VideoFrames/AudioFrames and Video Offset in jack_transport_t , which are not yet in xjadeo-0.4 )

      > does this imply that the only truly accurate sync would occur with a 0 latency,
      > or is there actually some video latency as well.

      There is *random* video latency! (the time it takes to decode the video-frame) http://xjadeo.sourceforge.net/doc/ar01s05.html#sync_info

      but we're working on fixing this for the next version.

      >if there is video latency,
      > how can we determine what it is, in order to compensate the audio latency
      > accurately?

      you cant. - current plan for xj5: decode ahead, double-buffer. flip buffers on X-servers notice by (re) programming an V-sync IRQ in each jack callback. I've got some framebuffer prototype but lack the time.. wanna join in coding then we'll get there sooner...

      IMHO this has only applications in live video performance (VJ) or synchronizing multiple screens;  unless you are using very old hardware or xjadeo+X11 have more than 60% CPU usage during playback, I would not worry about video latencies - but it's a good practice to align visible audio-peaks (eg. closing doors, footsteps, etc.) while the transport is stopped anyway.

      BTW. I once tried a complete cycle: export, mutliplex A/V (mencoder) mpeg/vob (transcode/mencoder) - burn to DVD (dvdauthor), play and re-import (pro-tools, dvd-rip/transcode) - and it worked just fine for at 24fps.

      #robin