STMX Files

STMX is a high-performance implementation of composable Transactional Memory (TM) for Common Lisp. TM is a concurrency control mechanism aimed at making concurrent programming easier to write and understand. Instead of traditional lock-based programming, one programs with atomic memory transactions, which can be composed together to make larger atomic memory transactions.

A memory transaction gets committed if it returns normally, while it gets rolled back if it signals an error (and the error is propagated to the caller).

Finally, memory transactions can safely run in parallel in different threads, are re-executed from the beginning in case of conflicts or if consistent reads cannot be guaranteed, and effects of a memory transaction are not visible from other threads until it commits.

STMX gives freedom from deadlocks, automatic roll-back on failure, and it aims at resolving the tension between granularity and concurrency.

Latest news, 31st August 2013

From version 1.9.0, STMX also supports hardware memory transactions in addition to classic software ones. It requires 64-bit SBCL and uses Intel TSX (Transactional Synchronization Extensions) instructions available on Core i5 4570, Core i7 4670, Core i7 4770 and possibly other CPUs.

Supported systems

STMX is currently tested on the following Common Lisp implementations:

• SBCL version 1.1.9 (x86_64) on Debian GNU/Linux 7.0 (x86_64)
• SBCL version 1.0.55.0 (x86) on Ubuntu Linux 12.04LTS (x86)
• ABCL version 1.1.1 with OpenJDK 6b27-1.12.5-2 (x86_64) on Debian GNU/Linux 7.0 (x86_64)
• CCL version 1.9-r15769 (x86_64) on Debian GNU/Linux 7.0 (x86_64)
• CCL version 1.9-r15769M (x86) on Debian GNU/Linux 7.0 (x86_64)
• CCL version 1.9-dev-r15475M-trunk (LinuxARM32) on Raspbian GNU/Linux (armhf) Raspberry Pi
• CMUCL version 20d Unicode (x86) on Debian GNU/Linux 7.0 (x86_64)
• ECL version 13.5.1 (x86_64) on Debian GNU/Linux 7.0 (x86_64)

It will probably work on several other Common Lisp implementations as long as they support log4cl, closer-mop, bordeaux-threads and trivial-garbage, but the author gives no guarantees.

Installation and loading

Stable version - from Quicklisp

STMX is now available from Quicklisp. The simplest way to obtain it is to first install Quicklisp then run these commands from REPL:

CL-USER> (ql:quickload "stmx")
;; lots of output...
CL-USER> (use-package :stmx)

If all goes well, this will automatically download and install the stable branch of STMX and its dependencies:

• log4cl
• closer-mop
• bordeaux-threads
• trivial-garbage

Since STMX was added to QuickLisp quite recently (15 June 2013), it may happen that your Quicklisp installation can't find it. In such case, you need to first update your QuickLisp installation as described here - search for "To get updated software" in the page.

Note: as of August 2013, the STMX version available from Quicklisp does not yet contain support for hardware memory transactions - to get them, download the latest version from GitHub (see below) or wait for the next Quicklisp libraries update.

Latest version - from GIT

In case you want to use the "latest and greatest" version directly from the author, in order to get the newest features - most notably hardware memory transactions - improvements, bug fixes, and occasionally new bugs, you need to download it into your Quicklisp local-projects folder. Open a shell and run the commands:

$ cd ~/quicklisp/local-projects
$ git clone git://github.com/cosmos72/stmx.git

then proceed as before - load a REPL and run:

CL-USER> (ql:quickload "stmx")
;; lots of output...
CL-USER> (use-package :stmx)

If all goes well, it will automatically load STMX and its dependencies.

Note: unless you know what you are doing, do not try to load different STMX versions one after the other from the same REPL - strange things may happen.

Troubleshooting

In case you get errors:

• check that Quicklisp is installed correctly, for example by executing at REPL:

  CL-USER> (ql:quickload "closer-mop")
  

• if you tried to download the stable version from Quicklisp, check that your quicklisp is updated and knows about STMX:

  CL-USER> (ql:system-apropos "stmx")
  

should print something like

    #<SYSTEM stmx / stmx-stable-7e68763b-git / quicklisp 2013-06-15>
    #<SYSTEM stmx.test / stmx-stable-7e68763b-git / quicklisp 2013-06-15>

If it doesn't, you need to update Quicklisp as described here - search for "To get updated software" in the page.

• if you tried to download the latest version from GIT, check that you downloaded STMX creating an stmx/ folder inside your Quicklisp local-projects folder, usually ~/quicklisp/local-projects

Testing that it works

After loading STMX for the first time, it is recommended to run the test suite to check that everything works as expected. From the REPL, run:

CL-USER> (ql:quickload "stmx.test")
;; lots of output...
CL-USER> (fiveam:run! 'stmx.test:suite)
;; even more output...
 Did 7133 checks.    
    Pass: 7133 (100%)
    Skip: 0 ( 0%)
    Fail: 0 ( 0%)

Note: (ql:quickload "stmx.test") intentionally works only after (ql:quickload "stmx") has completed successfuly.

The test suite should report zero Skip and zero Fail; the number of Pass may vary. You are welcome to report any failure you get while running the test suite, please include in the report: - operating system name and version (example: Debian GNU/Linux x86_64 version 7.0) - Common Lisp implementation and version (example: SBCL 1.0.57.0.debian, x86_64) - EXACT output produced by the test suite - any other relevant information

See "Contacts, help, discussion" below for the preferred method to send the report.

Documentation

The source code contains detailed documentation, that can also be viewed online on the STMX web page on GitHub

For background information, Composable Memory Transactions is a very good - though a bit technical - explanation of memory transactions and how they are used and combined. For the interested reader, it also goes in deep detail on how to actually implement them.

Contacts, help, discussion

As long as the traffic is low enough, GitHub Issues can be used to report test suite failures, bugs, suggestions, general discussion etc.

If the traffic becomes high, more appropriate discussion channels will be set-up.

The author will also try to answer support requests, but gives no guarantees.

Status

As of August 2013, STMX is being written by Massimiliano Ghilardi and is considered by the author to be stable.

STMX is a full rewrite of CL-STM, which has been developed by Hoan Ton-That for the Google Summer of Code 2006.

Legal

STMX is released under the terms of the Lisp Lesser General Public License, known as the LLGPL.