[Exist-development] DSL for testing Transactions

[Adam R. <ad...@ex...>]

I just wanted to bring to the attention of the developers a Transaction DSL
(Domain Specific Language) I created for writing Transaction Tests for
eXist-db. I am using this in Granite, but feel that it has value for
eXist-db as well, so am contributing it back.

I needed a way of being able to very simply and clearly describe a schedule
of operations across transactions, and then have that schedule executed
where each transaction is a distinct thread.

To solve this I created the TransactionTestDSL which you can find here:
https://github.com/adamretter/exist/blob/fbbb5f61d9220203dda410838a044737cd4c9211/test/src/org/exist/test/TransactionTestDSL.java#L54

The purpose of the TransactionTestDSL is to allow the user to describe an
absolute schedule of transactional operations using as little plain-english
like code as possible. The idea being that a user or developer should be
able to just glance at the coded schedule and clearly understand the
transactions involved and order of execution.

At present the DSL provides a transaction scheduler that will allow you to
work with two concurrent transactions (BiTransactionSchedule). If required
this could be extended in future to work with more 2+n concurrent
transactions. So far, in my experience, two concurrent transactions of
which you can control the operational scheduling of, is often enough to
show ACID semantics or violations and/or concurrency issues.

An example of using the DSL for a test where two transactions retrieve a
document:

@Test
public void getDocuments() throws ExecutionException, InterruptedException {
  final String documentUri = "/db/test/hamlet.xml";

  final Tuple2<DocumentImpl, DocumentImpl> result = biSchedule()
      .firstT1(getDocument(documentUri))

 .andThenT2(getDocument(documentUri))
      .andThenT1(commit())
                                               .andThenT2(commit())
      .build()
    .execute(existEmbeddedServer.getBrokerPool(), EXECUTION_LISTENER);

  // check results
  assertNotNull(result);
  assertNotNull(result._1);
  assertNotNull(result._2);

  assertEquals(documentUri, result._1.getURI().getCollectionPath());
  assertEquals(documentUri, result._2.getURI().getCollectionPath());
}

There are various utility operations that you can schedule on a transaction
in org.exist.test.TransactionTestDSL.TransactionOperation. As the API is
functional, these operations can also be composed with other existing
operations or with your own functions. You can of course also write your
own TransactionOperation functions to be scheduled or composed with the
provided operations.

For example, if you want to get more than one document, you could write a
TransactionOperation like andGetDocument:

static <T> TransactionOperation<T, Tuple2<T, DocumentImpl>>
andGetDocument(final String uri) {
  return (broker, txn, listener, t) -> new Tuple2<>(t,
getDocument(uri).apply(broker, txn, listener, t));
}

which you could compose with getDocument like so:

getDocument(uri1).andThen(andGetDocument(uri1))

However, a better approach for this trite example would be to use the more
generic TransactionOperation#with, which would compose like so:

getDocument(uri1).with(getDocument(uri2))

I hope that this DSL proves useful to other eXist-db developers as well. I
plan to add many more tests around transactions in eXist-db in an effort to
help both document the transaction properties of eXist-db, and assist me in
completing my locking rewrite. As I do so, I will add any missing
functionality to the TransactionTestDSL.

If you would like any help understanding or using the
TransactionTestDSL please just ask :-)

-- 
Adam Retter

eXist Core Developer
{ United Kingdom / United States }
ad...@ex...