so-5.5.10 Factories for MPSC event queue locks

Several SObjectizer's dipatchers use Multi-Producer/Single-Consumer event queues: so_5::disp::one_thread, so_5::disp::active_obj, so_5::disp::active_group, so_5::disp::prio_one_thread::strictly_ordered, so_5::disp::prio_one_thread::quoted_round_robin, so_5::disp::prio_dedicated_threads::one_per_prio. Those dispatchers provide just one working thread for some group of agents and agents from that group will work only on that thread (agents cannot migrate from one working thread to another, like with thread-pool dispatchers). This working thread is a single consumer of the corresponding event queue.

Until v.5.5.10 combined locking scheme with spinlock and mutex/condition_variable was used. This locking scheme works as described below.

There is a busy waiting part:

• event producer acquires spinlock and stores new event into the queue. Then event producer releases spinlock;
• consumer thread does an busy waiting loop. On each iteration consumer acquires spinlock and checks queue. If queue is empty then consumer releases spinlock and calls std::this_thread::yield method.

Busy waiting works great when message passing is intensive. But if there are some pauses in generation of events then locking scheme switches to usage of mutex and condition variable:

• consumer thread breaks busy waiting loop, acquires mutex and goes to sleep on conditon variable;
• event producer acquires spinlock and see that event queue is empty and consumer is sleeping on condition variable. Producer stores new event into the queue, then acquires mutex and sets up condition variable;
• consumer thread wakes up and returns to event processing with busy waiting loop for new events.

There is a time limit for busy waiting. If there is no new events during some time then consumer thread switches from busy waiting on spinlock to ordinary waiting on mutex/condition_variable. This time limit was one millisecond.

This scheme was implemented as combined_lock abstraction and was used in all dispatchers util v.5.5.10.

Locking with combined_lock is efficient if application is working under the heavy load. But not effiecient on some specific load profiles.

Lets imagine active agent which initiates several periodic messages. All actions are performed by that agent only on arrival of periodic messages. All other time agent do nothing and its working thread is sleeping on empty event queue.

If an application uses just one such active agent the overhead cost of busy waiting is relative small and could be ignored. But if there are several dozens of such agents the overhead cost could be relative high: 3-4% of CPU usage even if application do nothing and all working threads just do busy waiting periodically. Situation could be more dramatic if there are several such application on the same server.

To solve this problem v.5.5.10 introduces concept of lock factoris for MPSC queues. An user can specify lock factory during the creation of the dispatcher. Dispatcher will use a lock created by that factory.

There are two lock factories:

• combined_lock_factory which creates combined_lock (described above);
• simple_lock_factory which creates very simple lock with mutex and condition_variable without any complex schemes with busy waiting or something else.

New factories can be added in the future versions of SObjectizer.

combined_lock_factory is still used by default. If this locking scheme is not appropriate for your application it is possible to specify different locking factory (or to specify combined_lock_factory with different busy waiting time).

To specify lock factory it is necessary to use params_t object and the corresponding create_disp or create_private_disp functions for dispatcher creation. Since v.5.5.10 there are appropriate definitions of params_t types is the dispatcher's namespaces.

There are also namespaces queue_traits with definitions of lock factory functions and other queue-related stuff in dispatchers' namespaces (like so_5::disp::one_thread::queue_traits or so_5::disp::prio_one_thread::strictly_ordered::queue_traits). Technicaly speaking those queue_traits namespaces are just an alias for so_5::disp::mpsc_queue_traits namespace.

Because of that the preparation of params_t for a dispatcher look similar for different dispatcher types. For example that is for so_5::disp::one_thread dispatcher:

using namespace so_5::disp::one_thread;
auto disp = create_private_disp( env, params_t{}.tune_queue_params(
        []( queue_traits::params_t & queue_params ) {
            queue_params.lock_factory( queue_traits::simple_lock_factory() );
        } ) );

And this is for so_5::disp::active_obj dispatcher:

using namespace so_5::disp::active_obj;
auto disp = create_private_disp( env, params_t{}.tune_queue_params(
        []( queue_traits::params_t & queue_params ) {
            queue_params.lock_factory( queue_traits::simple_lock_factory() );
        } ) );        

Please note that lock_factory can be specified only at the moment of the creation of a dispatcher. Lock cannot be changed after the creation of a dispatcher.

Lock factory can be specified for the default dispatcher too. That dispatcher is created automatically by SObjectizer Environment. To specify lock factory for the default dispacher it is necessary to use so_5::rt::environment_params_t:

so_5::launch( []( so_5::rt::environment_t & env ) {...},
    []( so_5::rt::environment_params_t & env_params ) {
        using namespace so_5::disp::one_thread;
        env_params.default_disp_params( params_t{}.tune_queue_params(
            []( queue_traits::params_t & queue_params ) {
                queue_params.lock_factory( queue_traits::simple_lock_factory() );
            } ) );
    } );

At mentioned above the combined_lock_factory is still used by default. Default waiting time for busy waiting is specified by default_combined_lock_waiting_time() function. In v.5.5.10 it is one millisecond. It is possible to set different waiting time by using combined_lock_factory(duration) function:

using namespace so_5::disp::active_group;
auto disp = create_private_disp( env, params_t{}.tune_queue_params(
    []( queue_traits::params_t & queue_params ) {
        // Set up combined_lock with 0.5 second busy waiting time.
        queue_params.lock_factory( queue_traits::combined_lock_factory(
            std::chrono::milliseconds(500) ) );
    } ) );

NOTE. There are two dispatchers which use Multi-Producer/Multi-Consumer event queues: so_5::disp::thread_pool and so_5::disp::adv_thread_pool. Those dispatchers also use combined locking scheme inside. But there is no way to change that scheme in v.5.5.10. Implementation of this feature for MPMC event queues is postponed for v.5.5.11.