Release Notes and Documentation NoteTakerPen

Purpose

PegaTech's NoteTaker is a digital pen which consists of two devices. One pen with a real ballpoint (and some additional infrared device) and a receiver, that can be connected to your PC via USB. You can use the pen like a regular pen with regular paper -- the receiver will in addition record the strokes you make. See Pegasus Technologies web site

The tools allow you to retrieve the Notes stored on the receiver and to erase the memory. You can also retrieve notes live. The notes can be saved in raw format (sets of coordinates with pen-up events inserted) or be converted to SVG.

Structure

The software consists of two python files.

• Peg_NoteTaker.py is the lowlevel driver that knows how to talk to the pen via USB and understands the data packets it receives. It can save the raw data and optionally erase the receiver's memory. Alternatively it can be used by the next program and pass a list of notes and paths to it. This driver works for the Pegasus NoteTaker (M210) and rebranded versions of it. (My device is from TCM, which stands for Tchibo Certified Merchandise.) Peg_NoteTaker.py implements the protocol documented at Pegasus web site using python-usb (pyusb) 1.0. It has been tested on Linux.
• Note2SVG.py converts a list of notes and paths (as produced by Peg_NoteTaker.py -- other drivers could be implemented) and produces a nice SVG file from it. It uses Bezier curves to smooth the handwriting and does some magic to filter out possible recording issues, such as missing or spurious pen-up events.

Requirements

In order to use this software, you need

• The hardware (PegaTech's NoteTaker)
• A Linux system
• The python-usb (pyusb) library version 1.0 and a supported backend driver (I work with libusb-1_0 and have successfully tested with libusb-0_1 as well.)
• An SVG Viewer (e.g. modern web browsers)

Documentation

There's currently no man page. However, both files have a reasonable help functions. Call Peg_NoteTaker.py -h and Note2SVG.py -h to see usage information.

To understand the values in brackets in the help function: The first value is the default value; the second one is a suggestion if you want to make the effect of this more pronounced (but it's not the maximum, you can do more if you really want).

SVG creation options

Why SVG?

SVG has been chosen, as it's a simple, standardized and well-supported format for vector graphics. It's an obvious choice for handwriting. As it stores a path, it's indeed scalable, resulting in smooth lines even at high magnification.

Postprocessing

If you just connect the recorded points with straight lines, the result does not look very nice. Use Note2SVG.py -B -s to check the result. The reason is that the pen has a somewhat limited precision and more importantly, does only record a limited number of points per second. So in order for the SVG path to resemble your writing better, some optimizations are done. The most important one is using Bezier Curves rather than straight lines.

Here's a list of steps that are taken by Note2SVG.py after acquiring the data from the device or a saved file through the driver:

1. Translation and/or Rotation is applied if requested by the user (options -T and -R)
2. If we find a very short path (one or two points) and they are very close to an adjacent path, it is assumed that the paths actually should be connected and that the pen-up event should not have happened. Option -c controls this (by default this is disabled).
3. If two adjacent points within a path are very close to each other, the pair is replace with a single point (in the middle of the two points) to simplify processing later on. This is controlled by the -d option. The default is small enough that you will not be able to see the difference at reasonable magnification levels.
4. If adjacent points within a path are very far apart from each other, we assume that a pen-up event has been missed and the paths really should be disconnected. This is controlled by -C, the default only disconnects points really far apart.
5. The direction vectors in a path are followed. Three adjacent directions are calculated. If the curvature changes twice, we assume this is an unwanted shaking and we move the middle point a bit to prevent this. With -s 1 we would fully compensate it; the default is lower, so the shaking is reduced visibly but not fully compensated. This is a tradeoff between making the writing look nicer (especially for lines that should be straight) and rare situations where the moving was indeed intended.
6. There is an optional step, where we create cubic Bezier curves from every other point and then move the intermediate points to get closer to this curve. -S 1 would move them all the way to fall on the Bezier curve and thus make for very round curves; in reality a lower value should be applied. By default, this is not enabled.
7. Similar to the previous step we construct Bezier curves disregarding one point. Rather then using this to correct the shapes, we check whether the point would be predicted well with the curve and if so, we can drop it, simplifying further processing. This is controlled by the -D option.
8. The bounding box is calculated, which can be overridden with option -p.
9. We output Bezier curves, creating smooth paths such that the tangents of two Bezier splines are identical at the connecting point. You can avoid using Bezier curves with option -B.
   1. There is an option to extend the last pieces of a path by a bit to compensate for a pen to report pen-down too late and pen-up too early. This is option -x and -x 0.1 means that the ends are extended by 10% with straight lines. (The extensions are drawn lighter to reflect the fact that the pressure on the pen was lower here.)
   2. In live recording, we record one additional point before pen-down and after pen-up, with a similar reasoning to 9.1; with -g (automatically set in live mode), those last pieces of the path are also drawn lighter.
   3. The option -y varies the thickness and opacity a bit depending of the velocity with which a path has been drawn. This reflects that some pens draw darker or thicker lines when moving it more slowly.

You can visualize the postprocessing optimizations done here. With option -v, all retained points are drawn with darkred circles; the dropped points from steps 3 and 7 are drawn in yellow. When points have been moved in step 5 and 6, the original points are drawn in lightblue. Connections that have been made in step 2 are drawn with a blue line. Disconnected paths (step 4) are connected with a red line. With option -V, in addition the control points for the Bezier curves (step 9) are drawn as green circles.

Note2SVG.py will report something like this:

Save note 4 to Note-20130921_175608-004.svg
2116 pts: (1 sing, drop 53 dup + 59 smooth, 642 moved in lowpass)
10 paths: (0 conn, 1 disconn)

The numbers reported here refer to the number of (retained) points in a note, the number of points that are singular (not connected), the number of points dropped in steps 3 and 7 and the points moved in step 5. Finally the number of paths in a note is reported as well as the number of connections that has been made (step 2) and disconnections (step 4).

A note on the units for the options -c, -d, -C: My device reports coordinates as 16bit values at 1200dpi; the calculations are done in these units and the parameters for divers options are in these units (or squares of the units). When we start to support other devices with different resolutions, we'll probably have to change the interface and internal math to work with real (device-independent) units such as e.g. mm.

Substantial work has gone into the SVG optimization; you'll need to remember your geometry lessons from school (how to calculate normal vectors, how to use dot products to do projections, etc.) and read up on Bezier curves to fully appreciate the code.

License and disclaimer

I'm not associated with PegaTech. I publish this purely out of joy as an open source contribution, hoping that this might be useful for others and hoping that others may contribute with useful suggestions, patches, bug reports and the like.

The code is published under the GNU General Public License (GPL) version 2 or version 3 (at your option).

I can not take responsibility for anything that goes wrong when you use this software. It may not be fit for purpose and there is a remote possibility that you may destroy your hardware or lose important data. There's no warranty whatsoever.

Version history

Version 1.1 (2013-09-21):

• Wrote up this document ...
• Don't silently overwrite files unless the -force is used
• Fixed a typo which broke live note retrieval

Version 1.0 (2013-09-20):

• Initial release
• After the code has existed and was functional for two years now, I had finally found some time to clean it up and make it publishable. I also found out how to make it work without a kernel patch to drivers/usb/core/devio.c.