Home

See [QuickStart] for instructions on how to get up and running.
See [Modules] for a complete reference on various modules.

[ Major Design Objectives ]

- Support motion detection across multiple cameras simultaneously
- Modular design : segregation of input, processing, output, etc
- Multi-core support by using multiple threads
- Automatic motion masking (ie, ignore areas with continuous motion)

[ To-do ]

- Automatic noise level calibration
- Support Pan/Tilt cameras (ie, 1 camera, but multiple camera instances)

[ Implementation ]

This is the life cycle of a typical camera instance.

- Initialize data structures
- Enter the program main loop
  - request a camera frame
  - auto update motion heat map
  - write current frame to a file (keeps overriding this file)
  - if motion is detected, save image to disk

The configuration of each camera instance is as follows :

  camera {
    name <...>

    input {
      module <...>
      params {
        <...>
      }
    }
    noise {
      module <...>
      params {
        <...>
      }
    }
    motion_mask {
      module <...>
      params {
        <...>
      }
    }
    detection {
      module <...>
      params {
        <...>
      }
    }
    output {
      module <...>
      params {
        <...>
      }
    }
  }

From the above camera{..} block, we see that various components within a
camera instance are built from user configurable modules. Each module may be
supplied parameters relevant to it. The following example illustrates how a
single IP camera may be configured :

  camera {
    name "my backyard"
    input {
      module "input_http.so"
      params {
        url "http://admin:cAmeRa1@192.168.1.50/snapshot.cgi"
      }
    }
    noise {
      module "std_noise.so"
    }
    motion_mask {
      module "std_motion_mask.so"
    }
    detection {
      module "std_motion_detect.so"
      params {
        sensitivity 0.15
      }
    }
    output {
      module "output_jpeg.so"
      params {
        save_dir "/data/cameras/mybackyard"
      }
    }
  }

Each module may receive configuration that is appropriate for it in the
params{..} block. Statements here must be in the format :

  <key> <value>

where value's supported data types are (note, strings must be enclosed within
double-quotes) :

  integer (eg, 1234)
  floating point (eg, 12.34)
  string (eg, "1234")

[ Camera Instance Data Structures ]

From the above configuration, you can see that each camera instance has the
following configuration stanzas :

  1. name - a label
  2. input - where/how to get an image
  3. noise - for detecting the camera noise level and motion heat maps
  4. motion_mask - an algorithm for determining non-motion sensitive areas
  5. detection - an algorithm for determining if motion (of interest) occured
  6. output - a means of writing output (logs, images, etc)

Each configuration stanza consists of exactly 1 module and an optional
"params" sub-stanza, which consists of an arbituary number of key/value pairs.

This data structure is created while parsing the config file. The parsing of
the config file is primarily driven by "grammar.y" with tokenization performed
by "lex.l". This is essentially an event driven model as "grammar.y" and
"lex.l" fire off function calls as parsing progresses. Thus, this gives rise
to the following events that are of interest :

  Lex: RW_CAMERA - new camera block
  Par: name statement - the name of this camera instance
  Lex: RW_INPUT - now in input{..} stanza context
  Lex: RW_OUTPUT - now in output{..} stanza context
  Lex: RW_NOISE - now in noise{..} stanza context
  Lex: RW_MOTIONMASK - now in motionmask{..} stanza context
  Lex: RW_DETECTION - now in detection{..} stanza context
  Par: module statement - the name of the shared object to load
  Lex: RW_PARAMS - now in params{} sub-stanza
  Lex: RW_KEY(foo) - encountered an attribute key named "foo"
  Par: value<type>(bar) - encountered attribute's value of "bar"
  Par: input{} stanza complete
  Par: output{} stanza complete
  Par: noise{} stanza complete
  Par: motion_mask{} stanza complete
  Par: detection{} stanza omplete
  Par: camera{} block complete

[ Modules And Function Invocation ]

For each work unit within a camera instance (ie, input, noise, detection,
etc), this program will open the corresponding shared object and attempt to
locate the function called "handler()". This function is expected to have the
following prototype :

  int handler (Camera *cam, Stanza *stanza, char *stage, Core_Runtime *rt)

Where:

  cam    - pointer to the camera instance that invoked this module. A module
           uses this to access camera instance wide variables and structures.
  stanza - a pointer to the current module stanza that is executing. A module's
           handler() uses this to find out more about itself (eg, its own
           configuration, loop counter, etc).
  stage  - a string which indicates the current invocation stage, this
           takes on the values "input", "noise", "motion_mask", "detection"
           or "output".
  rt     - pointer to the shared jmotion runtime data structures.

The handler() function is expected to return 1 to indicate successful
operation, otherwise 0 to indicate that something went wrong. At the same
time, an optional cleanup() function is called when jmotion terminates.

Each module instance is allowed to track its own (opaque) data structure in
"stanza->dptr". The data at this pointer are the module's sole responsibility.
Thus, the following code listing illustrates the layout of such a module :

  #include "jmotion.h"

  int handler (Camera *cam, Stanza *stanza, char *stage, Core_Runtime *rt)
  {
    if (stanza->loop == 0)
    {
      /* do my own init */
      stanza->dptr = my_data_structure ;
    }
    /* do work */
    return (1) ;
  }

  int cleanup (Camera *cam, Stanza *stanza, char *stage, Core_Runtime *rt)
  {
    /* clean up stuff */
    return (1) ;
  }

The above code may be compiled using :

  % gcc -fPIC -shared -rdynamic -o foo.so foo.c

[ Modules and Multi-threading Notes ]

Since jmotion supports multiple camera instances, we may end up with multiple
modules being instantiated concurrently. Since modules are developed as third
party components, we need some mechanism for ensuring synchronization of
non-thread safe APIs. Take for example 2 modules which may be unrelated to
each other (for illustration only):

  input_http.so - uses HTTP GET to obtain an image from an ip camera
  output_s3.so  - saves an image to amazon S3 via REST

In the above example, we have 2 modules, developed by unrelated parties. Each
however, has chosen to use libcurl to implement http transactions. Now libcurl
requires exactly 1 call to curl_global_init(), but each thread also needs to
call curl_easy_init() to obtain its own handle. Since all module threads get
instantiated at program start, jmotion provides 2 shared variables in the
shared Core_Runtime data structure for ensuring serial execution of such
initialization routines.

1) "global_lock" - this mutex may be used by modules to prevent concurrent
   thread execution in critical sections. This is similar to the legacy linux
   kernel's big kernel lock (BKL).

2) "symbol_list" - a null terminated list of strings, storing process-wide
   initializations that have been performed. In our above example, if
   "input_http.so" gets to run first, it acquires "global_lock", it sees that
   "curl_global_init" has not been called, so it calls "curl_global_init()",
   adds "curl_global_init" to "symbol_list" and releases "global_lock". When
   "output_s3.so" runs, it doesn't call "curl_global_init()" because sees the
   symbol already exists in the list.

Note that jmotion does NOT make use of the 2x above variables. They are
provided solely for modules. Incidentally, each stanza (ie, module instance)
has a "loop" counter. This counter is incremented each time jmotion invokes
the module's handler() function. Thus, the module can use this counter to
perform one time initialization.

To simplify interaction with "symbol_list", jmotion provides the following
functions, they all return 1 on success, otherwise 0 :

  int symbol_list_get (char *symbol) ; // check if symbol is present
  int symbol_list_set (char *symbol) ; // adds the specified symbol
  int symbol_list_del (char *symbol) ; // removes the specified symbol

Note that the caller should still acquire "global_lock" when calling any of
the routines.

[ Module Operation ]

Each camera instance is essentially a dedicated thread executing a main loop.
The loop consists of executing modules (if present) in each stage, in the
following order :

  1. input
  2. noise
  3. motion_mask
  4. detection
  5. output

Each stage may be comprised of zero or more modules. If more than 1 module
is present in a particular stage, they are invoked in the order that they were
declared in the configuration file.

At the start of each loop, the first input module is expected to acquire a
fresh frame from the camera. This module is responsible for placing the newly
acquired frame in "Camera->frame_buf". If this pointer is NULL, the module
is expected to allocate a suitably sized buffer, saving the size of this
buffer into "Camera->frame_total". The actual amount of data used should be
reflected in "Camera->frame_used". This frame buffer should then be treated as
read-only by subsequent modules. Typically, we do not want to keep reallocating
this buffer to fit the frame we place here, so we only grow "Camera->total"
when needed, and set "Camera->frame_used" to reflect the current frame length.

[ Stages, and what's expected of them ]

The "input" stage

- Writes a freshly captured frame to "cam->frame_buf", the buffer's size is
  "cam->frame_total"
- "cam->frame_used" indicates the actual number of data bytes used (in case
  subsequent frames shrink).
- This module is expected to poll/pause as required. For example, if we want
  to capture a fresh frame every 2 seconds, then this module is responsible
  for sleep()'ing as necessary.
- This module is expected to decompress the (jpeg) image. The raw pixels are
  placed in "cam->img_buf". The size of this buffer is "cam->img_total".
- The decompressed image is "cam->img_width" by "cam->img_height", and its
  color depth is "cam->depth".

The "noise" stage

- Compare "cam->prev_buf" with "cam->img_buf", build/update a motion heatmap
  in "cam->heatmap". To determine motion change, a pixel is considered a
  change if its neighboring pixels all exhibit change.
- The motion heatmap is maintained at cam->heatmap, which is expected to be
  an array of double, with values between 0.0 and 1.0.

The "motion_mask" stage

- Certain kinda of motion artifacts ought to be ignored. For example, on IR
  illuminated cameras, rain appears as white streaks. This is where custom
  motion masking algorithms may be applied.

The "detection" stage

- Inspect the motion heatmap, determine which pixels will be used for motion
  detection. From that point, determine how much change has occured and set
  the "cam->motion_detected" flag if so.

The "output" stage

- Save the original camera image "cam->frame_buf", of length "cam->frame_used"
  to a file.
- If "cam->motion_detected" is set, save the original camera image to the
  "motion_dir" directory.

(end of cycle)

- The thread engine automatically copies the contents of "cam->img_buf" to
  "cam->prev_buf". The dimensions and depth of the subsequent frames are
  expected to remain the same.

- Set "cam->motion_detected" to 0.

[ Logging ]

Whether events originate from within jmotion, or from modules, logging is
configured in the main config file, using the following statements :

  global
  {
    log_basedir "<dirname>"
    log_maxsize <bytes>
    log_rotations <num>
    log_level <num>
  }

Modules are encouraged to use the jmotion function which returns 1 on success,
otherwise 0.

  int f_log (char *ident, int level, char *format, ...)

Where,

  ident       - the identity of the caller (typically the macro "__func__")
  level       - LOG_DEBUG to LOG_CRIT, see syslog(3)
  format,...  - the event string and subsequent arguments

Eg,

  f_log ("output_jpeg", LOG_ERR, "Cannot write to %s", myfile) ;

Thus, a module running under camera instance name "foo" will have its events
delivered to the file "/path/to/basedir/foo.log". Incidentally, events which
originate from the core jmotion engine are logged to "jmotion.log". The
initial log level can be set via the "LOG_LEVEL" environment variable.

[ Minimal Configuration ]

The following configuration is the simplest possible working setup, which
pulls an image from a usb camera and writes it periodically to a file. It
does not perform any processing (ie, no motion detection, no image archiving,
etc).

  camera {
    name "mycam1"
    input {
      module "input_v4l2.so"
    }
    output {
      module "output_jpeg.so"
      params {
        cur_frame "/tmp/mycam1.jpg"
      }
    }
  }

[ Signal Handler ]

The jmotion program supports the following signals :

  SIGHUP  - dumps the current runtime info
  SIGUSR1 - decrements the log level
  SIGUSR2 - increments the log level
  SIGINT  - performs an orderly shutdown
  SIGTERM - performs an orderly shutdown
  SIGALRM - watchdog timer (for internal use only)

Project Members:

• James Tay (admin)