Salsaman - 2008-01-23

I have just added support for advanced encoder plugins in LiVES, so I thought I would post a series of articles related to plugins/encoders and LiVES coding in general.

This article will detail the changes in the LiVES core, and show how communication is made with the plugin, also an intro. to RFX.

OK, so what is an encoder plugin in LiVES ? Well, it is just a commandline program that can take a numbered sequence of jpeg (and optionally png) images and (optionally) a wav or raw audio track, and produce some kind of file output. The plugin must be written in such a way that it can communicate with LiVES (via commandline and stdout) about what formats it can produce, what restrictions each format has ( e.g. a certain aspect ratio or set of frame rates), the name of the program, version number, etc. The nice thing in LiVES is, encoder plugins can be written in any language - this will be covered in more depth in the next article.

The current change is to allow the encoder plugin to request extra parameters from the host (lives-exe).

This is achieved as follows:
- the plugin sets a special flag bit to indicate it uses RFX
- the plugin implements a new "directive" get_rfx, which prints out an RFX scrap
- in the host, if the flag bit it set, it calls the get_rfx directive and sends the output to a file (an RFX script file)
- the RFX script file is compiled into an RFX plugin
- the host calls the "directives" in the RFX plugin to get the RFX parameters, layout hints, and triggers
- the host uses this to create a lives_rfx_t
- host uses the lives_rfx_t to create and run a parameter window
- when the window is closed, the parameter values are marshalled ready for sending to the encoder

This may seem complicated, but it is done this way to allow plugins to be written in any language. LiVES mainly uses Perl for encoder plugins, but there is also at least one in python.

Here is an explanation of the terms above.

RFX is the LiVES standard for creation of parameter windows. Such windows are (mostly) not hardcoded in LiVES, but rather are generated dynamically using some guesswork, and some hints from the plugin.

A "directive" is the first parameter passed to plugin, for example if my plugin is called foo, then calling:

foo bar 1 2 3

the "directive" is "bar". Within the plugin, the directive needs to be parsed and the plugin takes appropriate action.

OK, so, let's take a look at the steps in more detail.

1) the plugin sets a special flag bit to indicate it uses RFX

within the plugin there should be handling for the directive "get_capabilities". The plugin should print out a number (bitmask) and exit.

You can see this here for the mjpegtools_encoder, at line 96:

http://lives.cvs.sourceforge.net/lives/lives-plugins/plugins/encoders/mjpegtools_encoder?revision=1.7&view=markup

2) the plugin implements a new "directive" get_rfx, which prints out an RFX scrap

this can be seen at line 142:

http://lives.cvs.sourceforge.net/lives/lives-plugins/plugins/encoders/mjpegtools_encoder?revision=1.7&view=markup

The format of what is sent back here is an RFX "scrap". A full RFX script is used to generate a rendered effect plugin for example. A "scrap" contains just the information for the parameter window part of a plugin.

The RFX guide is here:

http://lives.cvs.sourceforge.net/lives/lives/RFX/RFX.spec?view=markup

For a "scrap", not all of the sections are required, only the following:

  • define [mandatory]
  • params [mandatory]
  • param_window [optional]
  • onchange [optional]

When get_rfx is called in the mjpegtools_encoder, if prints out the following RFX scrap:

<define>
|1.7
</define>
<language_code>
0xF0
</language_code>
<params>
vbitrate|_Video bitrate|num0|3800|1152|4096
abitrate|_Audio bitrate|num0|224|128|256
cbr|Use _Constant Bitrate|bool|0|0
quantisation|_Quantisation (only for Variable bitrate)|num0|8|1|31
vbuff|Video _buffer (Kb)|num0|2048|46|4096
msearch|Motion search _radius|num0|24|16|32
hfhand|_High frequency handling|string_list|1|reduce|normal|keep
</params>
<param_window>
layout|p0|p1|
</param_window>
<onchange>
</onchange>

That is all we need in LiVES to create the parameter window, run it, get our parameter values from the user, and pass them to the plugin later (in the "encode" directive).

Here is a screenshot of the window which is created in LiVES from this:

http://lives.rm.org/examples/aencex.png

Now let's leave plugins for the next notes, and look at the code in lives-exe.

There is only one function in lives-exe for encoding files, and that is the function save_file() in saveplay.c
It takes two paramters, a boolean "existing" (which we set to TRUE for "encode", and "FALSE" for "encode_as"), and the name of the output file. It will save the current file (cfile) using the user defined settings for encoder, format, etc.

You can see it here, starting at line 515:
http://lives.cvs.sourceforge.net/lives/lives/src/saveplay.c?revision=1.174&view=markup

The new code we will add starts at line 697:

696 // get extra parameters for saving

697 if (prefs->encoder.capabilities&HAS_RFX
) {

prefs (preferences) has a substruct which is an encoder struct. The capabilities field of this contains a copy of the plugin's flags.

698 // here we create an rfx script from some fixed values and values from the plugin; we will then compile the script to an rfx scrap and use the scrap to get info

699 // about additional paramters, and create the parameter window
700
701 // this is done like so to allow use of plugins written in any language; they need only output an RFX scrap when called with get_rfx as the first parameter
702
703 FILE sfile;
704 lives_rfx_t
rfx=(lives_rfx_t *)g_malloc(
sizeof(lives_rfx_t));

Here we created a lives_rfx_t, which is a struct used internally for all effects (in this case we are using the parameter window part of effect handling, so we only need a very minimal lives_rfx_t).

705 gchar string;
706 gchar
rfx_scrapname=g_strdup_printf("rfx.%d"
,getpid());
707 gchar *rfxfile=g_strdup_printf ("%s/.%s.script"
,prefs->tmpdir,rfx_scrapname);
708 int res;
709
710 string=g_strdup_printf("<name>\n
%s\n</name>\n",rfx_scrapname);
711 sfile=fopen(rfxfile,"w");

712 fputs(string,sfile);
713 fclose(sfile);
714 g_free(string);

We create an RFX script file, and fill the <name> field with the name of the RFX scrap we are going to generate. Now we let the plugin fill in the rest of the RFX scriptlet. We call the get_rfx directive, via first the "save" directive.

This will be explained in more detail in the next article, but basically we do it this way to allow the plugin access to other variables regarding the file to be encoded, for example which format the user selected.

715
716 if (prefs->encoder.capabilities&ENCODER_NON_NATIVE
) {
717 com=g_strdup_printf("smogrify save get_rfx %s \&quot;
%s%s%s/%s\&quot; %s \&quot;%s\&quot; %d %d %d %d %d %d %.4f %.4f %s >>%s"
,cfile->handle

,prefs->prefix_dir,PLUGIN_DIR,PLUGIN_ENCODERS
,prefs->encoder.name,fps_string,full_file_name

,1
,cfile->frames,arate,cfile->achans,cfile->
asampsize,asigned,aud_start,aud_end,extra_params,rfxfile);
718 }
719 else {
720 com=g_strdup_printf("%s%s%s/%s save get_rfx %s \&quot;\&quot;
%s \&quot;%s\&quot;

%d %d %d %d %d %d %.4f %.4f %s >>%s",
prefs->prefix_dir,PLUGIN_DIR,PLUGIN_ENCODERS,prefs->encoder.name
,cfile->handle,fps_string,full_file_name,1,cfile
->frames,arate,cfile->achans,cfile->asampsize,asigned
,aud_start,aud_end,extra_params,rfxfile);
721 }
722 system(com);
723 g_free(com);
724

725 // OK, we should now have an RFX fragment in a file, we can compile it, then build a parameter window from it
726

The following system command tells the RFX script compiler to compile a scrap for us from the script in rfxfile, and put the output in the temp
directory:

727 com=g_strdup_printf("%s %s %s >/dev/null",RFX_BUILDER
,rfxfile,prefs->tmpdir

);
728 res=system(com);
729 g_free(com);
730
731 unlink(rfxfile);
732 g_free(rfxfile);
733
734

We executed the command, and cleaned up, removing the intermediate script file. We need to check the return result of system() to see if
the script file compiled properly.

735 if (res==0) {
736 // the script compiled correctly
737
738 // now we pop up the parameter window, get the values of our parameters, and marshall them as extra_params
739

In the following lines we initialise our lives_rfx_t and prepare to fill the parameters in it:

Note we should read the RFX field delimiter from the RFX scrap file; however for testing we just assume the default "|".

740 // first create a lives_rfx_t from the scrap

741 rfx->name=g_strdup(rfx_scrapname);
742 rfx->action_desc=rfx->extra=NULL;
743 rfx->status=RFX_STATUS_SCRAP;
744 g_snprintf(rfx->delim,2,
"|"); // TODO - read from file
745 rfx->menu_text=g_strdup_printf(_(
"%s advanced settings"),prefs->encoder.of_desc);
746 rfx->is_template=FALSE;
747
748 render_fx_get_params(rfx,rfx_scrapname,RFX_STATUS_SCRAP);

That last function gets the parameters from the script file...

Now we build a param window from our lives_rfx_t. This is done in the function on_render_fx_pre_activate(). This is actually a GTK+
callback which we are calling manually (historically this was used only from the Effects menu). The first parameter is a menuitem (in this

case, NULL, as we are calling it manually), the second parameter is our lives_rfx_t.

The function creates (static) fx_dialog[1]. (fx_dialog[0] is used for other purposes).

749
750 // now we build our window and get param values
751 on_render_fx_pre_activate(NULL,rfx);
752
753 gtk_window_set_transient_for(GTK_WINDOW(fx_dialog
1
),GTK_WINDOW(mainw->LiVES));
754 gtk_window_set_modal (GTK_WINDOW (fx_dialog
1
), TRUE);
755
756 gtk_dialog_run(GTK_DIALOG(fx_dialog[1]));
757
758 // marshall our params for passing to the plugin
759 extra_params=param_marshall(rfx,FALSE);

The function param_marshall() takes the values of parameters in a lives_rfx_t, and concatenates them into a string, separated by spaces.

String parameters are enclosed in quotes, and any existing quotes within strings are escaped as \&quot;. The second parameter to param_marshall()
tells the function if we want min and max values in the output. In this case we don't.

Then we clean up, freeing the lives_rfx_t and removing the RFX scrap.

760
761 rfx_free(rfx);
762 g_free(rfx);
763 rfxfile=g_strdup_printf ("%s/%s",prefs
->tmpdir,rfx_scrapname);
764 unlink(rfxfile);
765 g_free(rfx_scrapname);
766 g_free(rfxfile);
767 }
768 }

Having tested all of this, I decided to put all of this code in a function, since it could be useful in other parts of the program where we wish to build a parameter window for an external component. The most logical place for this is in plugins.c

The new function is called plugin_run_param_window(). It takes one gchar * parameter, a system command which should be called to produce an RFX scriptlet on stdout. The function will return a marshalled string of parameter values (this string will be empty if the scriptlet was missing/malformed).

In addition, I changed the button layout elsewhere, so that the user can click "OK" or "Cancel". In the case that the user clicks cancel, a NULL will be returned by function.

Also, the delimiter is now properly read from the RFX scrap.

The new function is now in plugins.c, line 1671:

http://lives.cvs.sourceforge.net/lives/lives/src/plugins.c?revision=1.100&view=markup

and it is called from saveplay.c, around line 694:

http://lives.cvs.sourceforge.net/lives/lives/src/saveplay.c?revision=1.175&view=markup

I hope you all enjoyed this little LiVES coding example. If anybody has any questions, feel free to respond.

In the next article I will take a more in-depth look at encoder plugins and how to create one.

Happy hacking !

Salsaman,
http://lives.sf.net