Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo


LiVES programming notes (2) encoder plugins

  • Salsaman

    In the last article, I showed how LiVES can communicate with an RFX type plugin. In this article, I will describe how to code an encoder plugin for LiVES.

    Encoder plugins in LiVES take a sequence of jpeg (and optionally png) files, and optionally a raw or wav audio file, and produce a file output. Encoder plugins can be written in any language. They can also produce several different formats if required and in the current CVS version (what will be, they can prompt the user for extra parameters (e.g. a quality setting).

    Most encoder plugins are written in Perl (this has some advantages, the variables are parsed and named for you), although this is not necessary. However, for this example we will use Perl for the code and take the current CVS version of the mjpegtools_encoder. For other languages, the code will be similar, however you will need to parse the variables yourself.


    LiVES encoder plugins use a system of "directives". That is, the plugin is called (sometimes via an intermediary program) with a string of parameters. The first parameter is the "directive", and the program should act and produce different output depending on this directive.

    Lets take a look at the example encoder_plugin. After some starting comments, we have:

    33 #######################################################################
    35 if (!defined($standalone)) {
    36 my $smogrify=which smogrify;

    37 chomp($smogrify);
    38 require $smogrify;
    39 }
    40 else {
    41 $command=$ARGV[0];

    42 }

    This code is only required for Perl plugins which want to make use of the LiVES backend (smogrify). This will create a non-standalone plugin (unless $standalone is set, e.g. for testing purposes).

    Otherwise, and for non-perl languages, we just parse the commandline and get the first parameter (the directive or command). This will create a standalone plugin.

    One very important thing for standalone and non-Perl plugins:

    you MUST trap "SIGHUP", and if this signal is received, stop encoding as soon as possible, and proceed to the "clear" directive (this is explained more fully below). If you write a perl plugin which includes the above code, this is done for you by LiVES. Note that the SIGHUP is also propogated to any subprocesses; in many cases this is enough to stop them (for example, mencoder will exit when it receives a SIGHUP). This happens for example when the user presses "Cancel" during encoding.

    45 #######################################################################
    46 $tmpvid="tmpvid.m1v";
    47 $tmpaud="tmpaud.mpa";


    Here we just set some variables for temporary files.

    49 if ($command eq "version") {

    50 print "mjpegtools encoder plugin v0.96\n";
    51 exit 0;
    52 }

    OK, here we have the first directive "version". We must print out the plugin version to stdout, and exit with a code of 0.

    55 if ($command eq "init") {
    56 # perform any initialisation needed

    57 # On error, print error message and exit 1
    58 # otherwise exit 0
    59 if ($img_ext eq "png") {
    60 if (&location("png2yuv") eq "") {

    61 print "png2yuv was not found. Please install it and try again.";
    62 exit 1;
    63 }
    64 }

    65 else {
    66 if (&location("jpeg2yuv") eq "") {
    67 print "jpeg2yuv was not found. Please install it and try again.";

    68 exit 1;
    69 }
    70 }
    71 if (&location("mpeg2enc") eq "") {

    72 print "mpeg2enc was not found. Please install it and try again.";
    73 exit 1;
    74 }
    75 if (&location("mp2enc") eq "") {

    76 print "mp2enc was not found. Please install it and try again.";
    77 exit 1;
    78 }
    79 if (&location("mplex") eq "") {

    80 print "mplex was not found. Please install it and try again.";
    81 exit 1;
    82 }
    83 if (&location("yuvscaler") eq "") {

    84 print "yuvscaler was not found. Please install it and try again.";
    85 exit 1;
    86 }

    88 # end init code
    89 print "initialised\n";
    90 exit 0;
    91 }

    Here we see the second directive, "init". This is called when the user selects this plugin for output. In this section we can perform any initialisation and checks. If there is an error, we can print out an error message and exit with a code of 1. Otherwise we can exit with a code of 0.

    96 if ($command eq "get_capabilities") {
    97 # return capabilities - this is a bitmap field

    98 # bit 0 - takes extra parameters (using RFX request)
    99 # bit 1 - unused
    100 # bit 2 - can encode png
    101 # bit 3 - not pure perl

    102 print "5\n";
    103 exit 0;
    104 }

    This is the third directive, "get_capabilities". Here we print out a flag of the plugin's capabilities, and exit with a code of 0.

    Bit 0 can be set if the plugin requires extra parameters (this is described below, and in the last article). Bit 1 is currently unused, bit 2 means the plugin can encode png images as input as well as jpeg, and bit 3 should be set if the plugin is not written in Perl, or if it is a standalone plugin (see above).

    In this case, we can encode png images, and we want extra parameters, so we print out "5" and exit with a code of 0.

    108 if ($command eq "get_formats") {
    109 # for each format: -
    110 # return format_name|display_name|audio_types|restrictions|extension|

    112 # audio types are: 0 - cannot encode audio, 1 - can encode using
    113 # mp3, 2 - can encode using pcm, 3 - can encode using pcm and mp3

    115 #mpeg 2
    116 print "mpeg2|mpeg2 high quality|4|arate=32000;44100;48000,fps=24000:1001;24;25;30000:1001;30;50;60000:1001;60,aspect=1:1;4:3;16:9;2.21:1,hblock=8,vblock=8|mpg|\n";

    118 # mpeg1 may be wrong, aspect (and maybe fps) comes from PAL/NTSC - TODO
    119 # needs testing
    120 print "mpeg1|mpeg1|4|arate=32000;44100;48000,fps=24000:1001;24;25;30000:1001;30;50;60000:1001;60,aspect=1:1;4:3;16:9;2.21:1,hblock=8,vblock=8|mpg|\n";

    122 #vcd
    123 print "vcd_pal|vcd (PAL)|4|arate=32000;44100;48000,fps=25,aspect=1:1;4:3;16:9;2.21:1,hblock=16,vblock=16|mpg|\n";
    124 print "vcd_ntsc|vcd (NTSC)|4|arate=32000;44100;48000,fps=30000:10001,aspect=1:1;4:3;16:9;2.21:1,hblock=16,vblock=16|mpg|\n";

    126 #svcd
    127 print "svcd_pal|svcd (PAL)|4|arate=32000;44100;48000,fps=25,aspect=1:1;4:3;16:9;2.21:1,hblock=16,vblock=16|mpg|\n";
    128 print "svcd_ntsc|svcd (NTSC)|4|arate=32000;44100;48000,fps=30000:1001,aspect=1:1;4:3;16:9;2.21:1,hblock=16,vblock=16|mpg|\n";

    130 # dvd
    131 print "dvd_pal|dvd (PAL)|4|arate=32000;44100;48000,fps=25,aspect=1:1;4:3;16:9;2.21:1,hblock=16,vblock=16|mpg|\n";
    132 print "dvd_ntsc|dvd (NTSC)|4|arate=32000;44100;48000,fps=30000:1001,aspect=1:1;4:3;16:9;2.21:1,hblock=16,vblock=16|mpg|\n";

    134 # custom
    135 print "custom2|custom mpeg2|4|arate=32000;44100;48000,fps=24000:1001;24;25;30000:1001;30;50;60000:1001;60,aspect=1:1;4:3;16:9;2.21:1,hblock=8,vblock=8|mpg|\n";

    137 exit 0;
    138 }

    The next directive is "get_formats". Here the plugin prints out details of the formats it can handle, their requirements, and exits with a code of 0.

    Each format contains the following fields:


    and should end with a newline (\n).

    Format - the short name of the video format, used to identify it
    Description - user description of the format
    audio_formats - this is a bitmask of audio formats that the video format can produce, it is a bitmask with the current values:

    bit 1 - mp3
    bit 2 - PCM
    bit 3 - mp2
    bit 4 - vorbis
    bit 5 - ac3
    bit 6 - AAC
    bit 7 - AMR_NB

    restrictions - this is a list of restrictions for the video format. The list is separated by semi-colon (,), and can have all, some or none of the following:

    none - no restrictions
    arate=x;y;z - limit audio rate (Hz) to x,y, or z
    fps=a;b:c;d - limit frames per second to a, b:c or d
    aspect=e:f;g:h - limit (frame) aspect ratio to e:f or g:h
    hblock=i - width (in pixels) must be a multiple of i
    vblock=j - height (in pixels) must be a multiple of j
    size=kxl - frame size is fixed at k pixels width x l pixels height

    Obviously, some combinations don't make any sense, for example "none" cannot be combined with anything else; size=100x100 could not be combined with hblock=16. In the case of a contradiction like this, the host action is undefined.

    The next directive in our plugin is "get_rfx". This is an optional section, which is only called if the plugin sets bit 0 in "get_capabilities". This is explained further in the last LiVES programming notes: https://sourceforge.net/forum/forum.php?thread_id=1921281&forum_id=777668

    This directive also receives the standard parameters. This is explained below in the "save" directive.

    In this section we can print print out an RFX scrap. See the LiVES RFX guide for more details on the format of this:

    An RFX scrap (as opposed to a full RFX plugin), uses only the following RFX sections:
    <define>,<language_code>,<params>, and optionally <param_window> and/or <onchange>

    The next directive is "get_format_request". Here the plugin tells the host how it would like frames and (optionally) audio presented to it. This is a bitmap flag which the plugin should print out before exiting.

    The bitmap uses the following bits:
    bit 0 - if this is unset, the host delivers raw audio, if set, the host will add a wav file header to it
    bit 1 - if this is unset, the host will deliver all audio, and allow the plugin to clip it itself (because the plugin might be
    asked to encode only a range of frames and audio)
    bit 2 - if this is unset, the host will deliver all frames and the plugin will be expected to encode only the frames requested; if it is set, the host will renumber all the frames so that the requested range always starts at frame 1

    In our plugin we would have something like:

    397 if ($command="get_format_request") {
    398 # return the code for how we would like audio and video delivered
    399 # this is a bitmap field composed of:
    400 # bit 0 - unset=raw pcm audio; set=pcm audio with wav header
    401 # bit 1 - unset=all audio; set=clipped audio
    402 # bit 2 - unset=all frames; set=frames for selection only
    404 print "3\n"; # clipped pcm wav, all frames
    405 }

    However, for non-standalone plugins (see the start of the article), this is implemented as a subroutine which simply returns the value:

    397 sub get_format_request {
    398 # return the code for how we would like audio and video delivered
    399 # this is a bitmap field composed of:
    400 # bit 0 - unset=raw pcm audio; set=pcm audio with wav header
    401 # bit 1 - unset=all audio; set=clipped audio
    402 # bit 2 - unset=all frames; set=frames for selection only
    404 return 3; # clipped pcm wav, all frames
    405 }

    The next directive is "encode", and here the plugin does the main part of its work. The "encode" directive is followed by the standard parameters plus any extra parameters which were requested in "get_rfx".

    For a non-standalone plugin, the LiVES code sets various variables ready for you, for standalone/non-perl plugins, the plugin needs to parse these parameters itself. The variable names are shown here, but for a standalone plugin these would be second parameter, third parameter, etc. (recall that the first parameter is the directive, in this case "encode").

    Here are the standard parameters:

    $fps : the framerate
    $nfile : the file name to save to, enclosed in quotes
    $start : the number of the first frame to encode (if "renumbered frames" was requested in "get_format_request", this will always be 1).
    $end : the number of the last frame to encode
    $img_ext : the file extension of the image format this is normally .jpg, but if your plugin can handle png, it could also be .png
    $otype : this is the output (video) format which the user requested, as defined in "get_formats".
    $atype : this is a number corresponding to the audio type the user requested (see "get_formats")
    $hsize : the horizontal (width) size of the frames in pixels
    $vsize : the vertical (height) size of the frames in pixels
    $DB_ENCODERS : if set to 1, the plugin should provide as much debug output as possible on STDERR
    $arate : the audio rate in Hz (if the plugin or the user requested "no audio", this will be set to 0)
    $achans : number of audio channels (e.g. 1 for mono, 2 for stereo)
    $asamps : audio sample size in bits (e.g. 8 or 16)
    $ssigned : signedness of audio (1 for signed, 0 for unsigned)

    Following these standard parameters are any extra parameters that may have been requested in "get_rfx". These parameters start at the 14th parameter for non-standalone plugins, or at the 16th parameter for standalone/non-perl plugins.

    These extra parameters have strings enclosed in quotes, with any internal quotes escaped as \&quot;.

    Non-standalone plugins also get some extra variables:
    $title : title of clip
    $author : author of clip
    $comment : comment about clip

    The plugin must encode the image sequence and audio (if applicable) which will be present in the current directory, and produce the output file whose name is given. The images are numbered as %8d$img_ext, for example:


    The name of the audio file is just "audio" if you requested raw audio. If you requested wav file format, the name will be "audiodump.wav". For non-standalone plugins, the audio file name is set in $audiofile.
    Note: if the audio rate is 0, you MUST NOT encode audio, only video.

    Also, remember the note above about catching SIGHUP for standalone plugins. If unsure, go back and read it again. The "encode" section is where your plugin could receive this SIGHUP.

    Finally, to indicate that encoding is complete, you MUST create a file named .status (yes, that starts with a ".") in the current directory, and it must contain only the word "completed" (without the quotes). You can then exit with a code of 0. (For non-standalone plugins, you can simply call &sig_complete).

    Following encoding, the next directive is "clear". This is called by the host after "encode" (whether it completed or was aborted by SIGHUP). For standalone plugins, the parameters following the directive are the start and end frame numbers. For non-standalone plugins, the are set as $start and $end.

    Again for this directive, when we are done we need to create a file called .status in the current directory, containing only the word "completed".

    367 if ($command eq "clear") {
    368 # this is called after "encode"
    369 # note that encoding could have been stopped at any time
    370 if (-f $tmpvid) {
    371 unlink $tmpvid;
    372 }
    373 if (-f $tmpaud) {
    374 unlink $tmpaud;
    375 }

    Write "completed" to .status:

    376 &sig_complete;
    377 exit 0;
    378 }

    The final directive is "finalise". This is called when the plugin is deselected by the user. Here you can clean up any temporary files, etc.

    381 if ($command eq "finalise") {
    382 # do any finalising code
    384 # ...
    386 # end finalising code
    387 print "finalised\n";
    388 exit 0;
    389 }

    And that's it !

    Hopefully this will inspire people to get working on other types of encoders. Remember, you can write your encoder plugin in any language, as all communication with the host is done through the commandline and stdout.

    Happy hacking !