Release Notes and Documentation NoteTakerPen

Purpose

PegaTech's NoteTaker is a digital pen which consists of two devices.
One pen with a real ballpoint (and some additional infrared device)
and a receiver, that can be connected to your PC via USB.
You can use the pen like a regular pen with regular paper -- the
receiver will in addition record the strokes you make.
See Pegasus Technologies web site

The tools allow you to retrieve the Notes stored on the receiver and
to erase the memory. You can also retrieve notes live. The notes can
be saved in raw format (sets of coordinates with pen-up events inserted)
or be converted to SVG.

Structure

The software consists of two python files.

• Peg_NoteTaker.py is the lowlevel driver that knows how to talk to
  the pen via USB and understands the data packets it receives.
  It can save the raw data and optionally erase the receiver's
  memory. Alternatively it can be used by the next program and pass
  a list of notes and paths to it.
  This driver works for the Pegasus NoteTaker (M210) and rebranded
  versions of it. (My device is from TCM, which stands for Tchibo
  Certified Merchandise.)
  Peg_NoteTaker.py implements the protocol documented at
  Pegasus web site
  using python-usb (pyusb) 1.0. It has been tested on Linux.
• Note2SVG.py converts a list of notes and paths (as produced by
  Peg_NoteTaker.py -- other drivers could be implemented) and produces
  a nice SVG file from it. It uses Bezier curves to smooth the handwriting
  and does some magic to filter out possible recording issues, such as
  missing or spurious pen-up events.

Requirements

In order to use this software, you need

• The hardware (PegaTech's NoteTaker, USB 0e20:0101)
• A Linux system
• The python-usb (pyusb) library version 1.0 and a supported backend driver
  (I work with libusb-1_0 and have successfully tested with libusb-0_1 as
  well.)
• An SVG Viewer (e.g. modern web browsers)

Documentation

There's currently no man page.
However, both files have a reasonable help functions.
Call Peg_NoteTaker.py -h and Note2SVG.py -h to see usage information.

To understand the values in brackets in the help function: The first value is the
default value; the second one is a suggestion if you want to make the effect of
this more pronounced (but it's not the maximum, you can do more if you really
want).

SVG creation options

Why SVG?

SVG has been chosen, as it's a simple, standardized and well-supported format
for vector graphics. It's an obvious choice for handwriting. As it stores
a path, it's indeed scalable, resulting in smooth lines even at high
magnification.

Postprocessing

If you just connect the recorded points with straight lines, the result does
not look very nice. Use Note2SVG.py -B -s to check the result.
The reason is that the pen has a somewhat limited precision
and more importantly, does only record a limited number of points per second.
So in order for the SVG path to resemble your writing better, some optimizations
are done. The most important one is using Bezier Curves rather than straight
lines.

Here's a list of steps that are taken by Note2SVG.py after acquiring the data
from the device or a saved file through the driver:

1. Translation and/or Rotation is applied if requested by the user (options -T and -R)
2. If we find a very short path (one or two points) and they are very close to an adjacent
   path, it is assumed that the paths actually should be connected and that the pen-up
   event should not have happened.
   Option -c controls this (by default this is disabled).
3. If two adjacent points within a path are very close to each other, the pair is replace
   with a single point (in the middle of the two points) to simplify processing later
   on. This is controlled by the -d option. The default is small enough that you will
   not be able to see the difference at reasonable magnification levels.
4. If adjacent points within a path are very far apart from each other, we assume that
   a pen-up event has been missed and the paths really should be disconnected.
   This is controlled by -C, the default only disconnects points really far apart.
5. The direction vectors in a path are followed. Three adjacent directions are
   calculated. If the curvature changes twice, we assume this is an unwanted
   shaking and we move the middle point a bit to prevent this. With -s 1 we
   would fully compensate it; the default is lower, so the shaking is reduced
   visibly but not fully compensated. This is a tradeoff between making the
   writing look nicer (especially for lines that should be straight) and rare
   situations where the moving was indeed intended.
6. There is an optional step, where we create cubic Bezier curves from every
   other point and then move the intermediate points to get closer to this
   curve. -S 1 would move them all the way to fall on the Bezier curve and
   thus make for very round curves; in reality a lower value should be applied.
   By default, this is not enabled.
7. Similar to the previous step we construct Bezier curves disregarding one
   point. Rather then using this to correct the shapes, we check whether
   the point would be predicted well with the curve and if so, we can drop
   it, simplifying further processing. This is controlled by the -D
   option.
8. The bounding box is calculated, which can be overridden with option -p.
9. We output Bezier curves, creating smooth paths such that the tangents
   of two Bezier splines are identical at the connecting point. You can
   avoid using Bezier curves with option -B.
   1. There is an option to extend the last pieces of a path by a bit
      to compensate for a pen to report pen-down too late and pen-up
      too early. This is option -x and -x 0.1 means that the ends
      are extended by 10% with straight lines. (The extensions are
      drawn lighter to reflect the fact that the pressure on the
      pen was lower here.)
   2. In live recording, we record one additional point before
      pen-down and after pen-up, with a similar reasoning to 9.1;
      with -g (automatically set in live mode), those last
      pieces of the path are also drawn lighter.
   3. The option -y varies the thickness and opacity a bit
      depending of the velocity with which a path has been
      drawn. This reflects that some pens draw darker or
      thicker lines when moving it more slowly.

You can visualize the postprocessing optimizations done here. With option -v,
all retained points are drawn with darkred circles; the dropped points from steps
3 and 7 are drawn in yellow. When points have been moved in step 5 and 6, the
original points are drawn in lightblue. Connections that have been made in
step 2 are drawn with a blue line. Disconnected paths (step 4) are connected
with a red line.
With option -V, in addition the control points for the Bezier curves (step 9)
are drawn as green circles.

Note2SVG.py will report something like this:

Save note 4 to Note-20130921_175608-004.svg
2116 pts: (1 sing, drop 53 dup + 59 smooth, 642 moved in lowpass)
10 paths: (0 conn, 1 disconn)

The numbers reported here refer to the number of (retained) points in a note,
the number of points that are singular (not connected), the number of points
dropped in steps 3 and 7 and the points moved in step 5. Finally the number
of paths in a note is reported as well as the number of connections that
has been made (step 2) and disconnections (step 4).

A note on the units for the options -c, -d, -C: My device reports
coordinates as 16bit values at 1200dpi; the
calculations are done in these units and the parameters for divers options are
in these units (or squares of the units). When we start to support other devices with
different resolutions, we'll probably have to change the interface and internal
math to work with real (device-independent) units such as e.g. mm.

Substantial work has gone into the SVG optimization; you'll need to remember
your geometry lessons from school (how to calculate normal vectors, how
to use dot products to do projections, etc.) and read up on Bezier
curves to fully appreciate the code.

License and disclaimer

I'm not associated with PegaTech.
I publish this purely out of joy as an open source contribution, hoping that
this might be useful for others and hoping that others may contribute with
useful suggestions, patches, bug reports and the like.

The code is published under the GNU General Public License (GPL) version 2 or
version 3 (at your option).

I can not take responsibility for anything that goes wrong when you use this
software. It may not be fit for purpose and there is a remote possibility that
you may destroy your hardware or lose important data. There's no warranty
whatsoever.

Version history

Version 1.1 (2013-09-21):

• Wrote up this document ...
• Don't silently overwrite files unless the -force is used
• Fixed a typo which broke live note retrieval

Version 1.0 (2013-09-20):

• Initial release
• After the code has existed and was functional for two years now,
  I had finally found some time to clean it up and make it
  publishable. I also found out how to make it work without
  a kernel patch to drivers/usb/core/devio.c.