moddiutils.async

modditutils.async

The async subpackage of moddiutils contains the implementation of an asyncronous execution model for Python3, compatible with version 3.4 (and maybe earlier).

Asyncronous WTF?

In a normal program, when you trigger some action by calling a function (syncronously), that function won’t return until that action is done, and in the meantime your program will block. In simple cases this is perfectly OK, but especially in a GUI context it is a strict no-go – while your code is chugging away processing some files or communicating over the network the GUI must still be able to repaint the screen, display the progress bar, handle clicks on the “Abort” button, you get the idea.

There are two ways of solving this, and both cover part of it. In the olden times’ absence of threads (i.e. in tkinter) everything must be implemented in a way that it won’t block, ever, but instead arranges to be signalled in some way when it can proceed. A central event loop will collect the signalization (and possibly wait for any to happen) and dispatch to the associated code. This actually is very robust, has a low overhead and is the way to go if you have the means to replace blocking by the appropriate signalization. The trouble with this programming style is that what used to be a single function disintegrates into a bunch of minimal steps possibly needing to share state, i.e. some means to express that pile of minijobs in the form of something that looks like a function would be really helpful, especially when implementing non-trivial protocols with several steps and decisions.

The other way is of course to do it in another thread, which means that you need at least some way to pass some “finished”-signal and maybe a result back to the invoker. The ability of exchanging some “abort” commands and exceptions between threads would be helpful, too.

Asyncronous how?

moddiutils.async supports mixing both styles. It helps you forwarding jobs, aborts, results and exceptions between threads. And it allows functions submitted to worker threads to suspend their operation waiting for another function’s result or an event being emitted, while not blocking the worker thread.

As of 2017-12-15 moddiutils.async contains:

identifier	what it does
moddiutils.async.Thread	moddiutils.threading.Thread that aborts the whole program on exceptions
moddiutils.async.Gate	moddiutils.threading.Gate that can submit calls to some moddiutils.async.executor.Executor
moddiutils.async.Worker	Worker thread, i.e. a moddiutils.async.executor.Executor that runs in its own thread
moddiutils.async.context.Context	Context handler maintaining a stack of callbacks for functions finishing, throwing or yielding and providing a backtrace to point of creation (to augment later exception backtraces with cross-thread information)
moddiutils.async.context.catch, moddiutils.async.context.then, moddiutils.async.context.process	Convenient way to set callbacks for functions finishing, throwing or yielding
moddiutils.async.control.GOTO, moddiutils.async.control.CALL, moddiutils.async.control.PARALLEL	Things to yield from a function invoked asyncronously by a moddiutils.async.executor.Executor in order to spawn another asyncronous function replacing, preempting or running concurrent to the yielding function
moddiutils.async.executor.Executor	A queue of submitted asyncronous jobs and the engine to call them, preempt them upon yield, forward results and exceptions to their invoker (possibly cross-thread). Can run continuously within its own thread (=moddiutils.async.Worker) or can run on demand within some other main loop (e.g. moddiutils.tk.app.App).
moddiutils.async.server.Server	Thing that can emit callbacks to subscribers, asyncronously and cross-thread
moddiutils.async.server.PushmePullyou	Queue with asyncronous wait