Multi-threading_and_OWLNext

Multi-threading and OWLNext

This article details our advice for using multi-threading in OWLNext applications.

• Multi-threading and OWLNext
  • Introduction
  • Current state of OWLNext
  • Best practice for multi-threaded OWLNext applications
    • Rules of thumb
    • Waking up the main thread
    • Keeping the application responsive
  • Examples of multi-threaded OWLNext applications
    • OWLMaker
    • FolderSize
  • Conclusion
  • References

---

Introduction

Since the turn of the century, there has been an increasing focus on parallelism in the computing industry. Processors in mainstream PCs and consumer products have gone from single-core to multi-core. At the same time single-thread performance has stagnated. We no longer enjoy the the exponential performance improvements we were used to see between processor releases [1].

With this in mind, OWLNext users may be looking to better exploit the parallel computing resources in modern hardware. Adventurous users may have peeked into the OWLNext source code and seen what looks like multi-threading support; locks around shared structures, thread-local storage classes, etc.

Unfortunately, the multi-threading support in OWLNext is horribly broken, and running more than one thread through the OWLNext code is a risky game of Russian roulette.

But this does not mean you should shelve all your plans for using multi-threading in your OWLNext applications; just do not multi-thread OWLNext itself. Below we will summarize our experiences and best advice for how to take your OWLNext applications into the multi-threaded era.

---

Current state of OWLNext

OWLNext is simply not designed and implemented for multi-threading. The application encapsulation, windows encapsulation and event handling code assumes that just a single thread is running through it.

Also, we performed a light non-expert review of the 6.32 code with respect to multi-threading, and many components were found to be broken. For example, we found a data-corrupting bug in the thread-level storage management class TTlsContainer [bugs:#196]. This bug is consistent and makes any multi-threaded use of the class unsafe. TTlsContainer is widely used throughout the internal implementation of OWLNext for application data, GDI caches, etc.

While some components have since been repaired, the fixes have not been thoroughly reviewed and tested, and issues are known to remain. Check out the bug list in our tracker [2] for the known issues and their status. Tip: Filter the list on the "multi-threading" category.

---

Best practice for multi-threaded OWLNext applications

Practice multi-threading safely in your OWLNext programs by adhering to the following guidelines.

Rules of thumb

Do not use the OWLNext multi-threading components.

A good multi-threading library is all about safety-guarantees and trust. The multi-threading components in OWLNext, such as TThread, TCriticalSection and TTlsContainer, are poorly documented and implemented, and our review of the code has shown fundamental issues. Instead, look towards well-defined and actively maintained alternatives, e.g. the C++ standard library or Boost.

Do not run multiple threads through the OWLNext code, not even if you make them take turns.

Much of the code in OWLNext assumes that there is only one thread running through it (e.g. see the implementation of TWindow::ReceiveMessage). Also, due to internal bugs, simultaneous execution of OWLNext code can cause race conditions and contention for shared resources, such as synchronization problems, lock-ups and undefined behaviour. Even if you carefully synchronize each thread so that only one runs at a time through any critical section, you may still encounter issues due to these bugs.

Dedicate the main thread as the sole thread allowed to run OWLNext code.

The main thread runs program start-up and shut-down. This is when critical initialization and clean-up of global OWLNext components happen. Safe multi-threaded access to these global resources (singletons) can not be guaranteed. This is another reason why it is necessary to leave OWLNext execution to the main thread.

Use message passing between threads to update the UI from secondary threads.

If you spawn extra threads, let them work on pure application tasks, i.e. no execution of OWLNext or UI code. When the worker threads complete work that needs a UI update, then send a message to the main thread and let it handle the update. While message passing is the simplest way to safely synchronize tasks, you may of course use more sophisticated synchronization, such as shared data locking or lock-less algorithms. But beware that these techniques may be tricky to get right, with all the challenges of potential data races, dead-locks and other multi-threading headaches.

Waking up the main thread

As stated in the guidelines above, message passing is the preferred way to communicate with the main thread. Sending a message, e.g. when a worker thread has finished some work, ensures that the main thread is woken up and handles the event in a timely manner. However, if you use other means of communicating with the main thread, you need to ensure that the main thread is woken up.

The main thread will usually go to sleep if it has nothing to do. If TApplication::IdleAction returns false (meaning it has nothing to do), then TApplication goes to sleep by calling the Windows API function MsgWaitForMultipleObjects within TApplication::MessageLoop. This causes the main thread to sleep until another message arrives, or any of the specified synchronization objects are signalled. You can add wait handles that refer to such synchronization objects (mutex, event, semaphore, etc.) by calling TApplication::WaitOnObject. For example, the function std::mutex::native_handle gives you a handle to wait on, or you can use some native synchronization primitive in the Windows API, e.g. an Event Object. For example, see OWLMaker, which uses an event for the stop token.

However, note that in versions prior to OWLNext 8, "TWindow::IdleAction is not called after awakening from sleep" [bugs:#510].

A simple but somewhat dirty solution for waking up the main thread, is to use a Timer. The timer events do not have to be handled. The timer messages just cause the main thread to be woken up at regular intervals, which in turn ensures that your override of IdleAction is called regularly, in which you can deal with any work passed from a worker thread. That said, we recommend using WaitOnObject instead, so that the main thread can get proper sleep, when there is nothing to do.

Note: Although you can prevent the main thread from ever going to sleep, e.g. by always returning true in your override of IdleAction, you should avoid doing so. Allow the main thread to sleep. Going to sleep is the intended Windows application behaviour, reducing the load on the system, and allowing it to idle in a lower power state.

Keeping the application responsive

If the main thread is given too much work to do in your event handlers, or within your overrides of TMsgThread::IdleAction and TWindow::IdleAction, if any, the application UI may become sluggish and unresponsive. Hence it is important that you reduce or slice up any substantial work on the main thread, so that the response time of your handlers, and a single iteration of IdleAction, is below a certain threshold (e.g. 33 ms for 30 fps).

Typically, you achieve responsiveness by slicing up any time-consuming work and performing it piecemeal in IdleAction. For example, see the FolderSize example. Also, if the time-consuming work does not need to use OWLNext, consider off-loading the work to a helper thread.

Note: Do not use TMsgThread::PumpWaitingMessages to achieve responsiveness. This function was often used as a simple solution in single-threaded Windows applications to process waiting messages while performing time-consuming work within an event handler. However, this creates a message loop within a message loop (remember the handler was itself called within a message loop), causing re-entrant calls to the OWLNext message dispatch machinery and the application event handlers. This may easily lead to havoc. For example, the user may shutdown Windows (WM_ENDSESSION), which leads to application shutdown (TXEndSession) in the middle of your handler. Using PumpWaitingMessages recursively like this, also risks running out of stack space. Rather than use PumpWaitingMessages, refactor your code following the advice above.

---

Examples of multi-threaded OWLNext applications

The following sections demonstrate how you can use multi-threading safely in your OWLNext applications.

OWLMaker

OWLMaker Build 4204 and later is a multi-threaded application that can launch multiple build commands asynchronously to be executed in parallel in multiple threads and child processes. The OWLMaker source code shows how you can use std::async to simply and safely launch asynchronous tasks on worker threads, while the main GUI thread continues to run OWLNext unhindered, thus providing a responsive user interface while work is ongoing. To avoid any concurrency issues within OWLNext, the worker threads make no calls to OWLNext code. Results are communicated back to the main thread automatically and safely through std::future objects returned by std::async, and the main thread is notified when results are ready by posting a message using the plain Windows API. The std::future objects also capture and store any exception that may occur in the worker threads, transporting it safely to the main thread, where it is rethrown when the corresponding result is accessed.

For more details, see the source code.

FolderSize

FolderSize is one of the examples shipping with OWLNext. It is a handy utility that scans the selected folder and calculates the total size of all the files within the folder. A folder tree is displayed in the left pane with color coding and sizes. Clicking on a folder will display the files from it in the right pane and on a separate tab are displayed statistics of file sizes grouped by type. The processing of the folder is done in a background thread, and the UI is updated periodically, in order to keep it responsive. The background thread does not call any OWLNext code. Instead it works on an internal shared data structure synchronized by locking and atomic operations. When otherwise idle, the main thread extracts data from the shared data structure and updates the program window. The data is processed in batches, each of a suitable time slice to keep the UI responsive.

See the source code for implementation details.

---

Conclusion

The OWLNext library itself is not safe for multi-threading. Use only the main thread to execute OWLNext code.

The auxiliary multi-threading utilities are deprecated and will likely be removed in the future. As a replacement, we recommend looking at the C++ standard library [3] or Boost [4] for any concurrency needs you have in your own code.

Follow our guidelines stated in this article to practice multi-threading safely in your OWLNext applications.

---

References

1. The Free Lunch is Over, Herb Sutter, Dr. Dobb's Journal, 30(3), March 2005.
2. OWLNext Multi-threading Bugs, SourceForge Bug Tracker.
3. C++11 Memory Model and Multi-threading Facilities, Wikipedia.
4. Boost.Thread, Boost C++ Libraries.