cppcms-users

[Cppcms-users] Synchronous app calls io_service.post() and blocks itself

[CN <cn...@fa...>]

My understanding is that all requests sent to booster::aio::io_service
are processed "sequentially", meaning that these requests must wait in
line for service from the single instance of booster::aio::io_service.
This characteristic of "processing requests one at a time" refrains
requests from racing issue. Locking is not required in such use case
when aio is used instead.

This is why my synchronous application calls io_service.post() in order
to ensure no racing:

async_app->service().get_io_service().post(boost::bind(&my_class::my_method,this));

Now I want this synchronous application not to respond to clients and
return immediately. Instead, this synchronous application will wait
until my_class::my_method completes before it continues.

Is such design correct? If it is, how do I implement this? If it is not,
what alternatives do I have?

Regards,
CN

-- 
http://www.fastmail.fm - Or how I learned to stop worrying and
                          love email again

Re: [Cppcms-users] Synchronous app calls io_service.post() and blocks itself

[Artyom B. <art...@ya...>]

Hello,

Excellent question.

It is both correct and isn't correct, depends on what you do.

See my notes

 
> My understanding is that all requests
> sent to booster::aio::io_service
> are processed "sequentially", meaning that these requests
> must wait in
> line for service from the single instance of
> booster::aio::io_service.
> This characteristic of "processing requests one at a time"
> refrains
> requests from racing issue. Locking is not required in such
> use case
> when aio is used instead.

Yes
 
> This is why my synchronous application calls
> io_service.post() in order
> to ensure no racing:
> 
> async_app->service().get_io_service().post(boost::bind(&my_class::my_method,this));
> 

Now there is a slight problem...

(a) what is "this"
(b) what thread "this" lives in

if this is an async application living in the event-loop thread - no problem.
if it is some persistent class (not a sync app!!!) that is only accessed from the event loop, it is ok as well.

this is a synchronous app that it isn't correct (in most of cases) because:

1. the synchronous application "lives" in the thread pool so the my_method would be called from a different thread (event-loop thread)
2. the synchronous application may not even exist when my_method is called because it was removed from the application cache

So it is better to pass the relevant data to the async_app withing the callback, i,e,

struct mydata {
   // all data needed
   void operator()() const // the callback
   {
      async_app->handle(this); 
   }
};

 mydata info_for_async_thread;
 async_app->service().get_io_service().post(info_for_async_thread);

> Now I want this synchronous application not to respond to
> clients and
> return immediately. Instead, this synchronous application
> will wait
> until my_class::my_method completes before it continues.
> Is such design correct? If it is, how do I implement this?
> If it is not,
> what alternatives do I have?

Now the case when you actually make the sync app to wait for the response...

technically you can implement this by for example waiting for a conditional variable.
that would be changed in the event loop:

Caller
------------------

this->resp_is_ready=false

async_app->service().get_io_service().post(boost::bind(&my_class::my_method,this));

while(true){
    booster::unique_lock guard(this->resp_is_ready_mutex)
    this->resp_is_ready_cond.wait(guard)
    if(this->resp_is_ready)
      break;
} 
-------------
Handler
---------------------

my_class:my_method()
{
   // do something with async app/withing the event loop

    booster::unique_lock guard(this->resp_is_ready_mutex);
    this->resp_is_ready = true
    this->resp_is_ready_cond.notify_one()
  
}
--------------------------------------------------------

Now the question what do you need this for...

If you need to do a fast query from the event loop for some data without running too many mutexes
it is ok. However it is still somewhat borderline. I'd suggest to keep some data in some global object
with very short and fast accesses protected by mutexes.

But... if you want to put the thread on hold for a long time (for example to wait for some other event),
it is very bad idea as you may run out of threads in the pool very fast.

If you want to postpone the response of the synchronous application you may do the following steps:

(a) detach the http::context (like in async app)
(b) change the response().io_mode() to asynchronous
(c) send the detached http::context to the async_app to handle it asynchronously vio post()...

So basically sync app tells I don't want to handle it and give it away to the async app.

(this probably should go to wiki :-) )


Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

Re: [Cppcms-users] Synchronous app calls io_service.post() and blocks itself

[CN <cn...@fa...>]

Dear Artyom,

Many thanks for your swift and thorough clarification! It will take me
some time to digest your responses as I feel they contain so many
invaluable concepts.

> this is a synchronous app that it isn't correct (in most of cases)

Unfortunately "this" falls to this category. It refers to a descendant
of synchronous cppcms::application that is to be invoked by
cppcms::applications_factory.

> Now the case when you actually make the sync app to wait for the
> response...
> 
> technically you can implement this by for example waiting for a
> conditional variable.
> that would be changed in the event loop:
> 
> Caller
> ------------------
> 
> this->resp_is_ready=false
> 
> async_app->service().get_io_service().post(boost::bind(&my_class::my_method,this));
> 
> while(true){
>     booster::unique_lock guard(this->resp_is_ready_mutex)
>     this->resp_is_ready_cond.wait(guard)
>     if(this->resp_is_ready)
>       break;
> } 
> -------------
> Handler
> ---------------------
> 
> my_class:my_method()
> {
>    // do something with async app/withing the event loop
> 
>     booster::unique_lock guard(this->resp_is_ready_mutex);
>     this->resp_is_ready = true
>     this->resp_is_ready_cond.notify_one()
>   
> }

By the way, I started to investigate std::future and std::promise. I am
not quite sure, yet, if they can be incorporated into cppcms framework.
Then I feel the last approach you offered (see below) is the most
elegant way for me to go.

> If you need to do a fast query from the event loop for some data without
> running too many mutexes
> it is ok. However it is still somewhat borderline. I'd suggest to keep
> some data in some global object
> with very short and fast accesses protected by mutexes.

Such hand made "cache" seems to be unable to fit my case. I need to use
booster::aio::io_service as a synchronizer, like a mutex, to 
*sequentially* execute cppdb operations and manipulate in-memory data.

> But... if you want to put the thread on hold for a long time (for example
> to wait for some other event),
> it is very bad idea as you may run out of threads in the pool very fast.
> 
> If you want to postpone the response of the synchronous application you
> may do the following steps:
> 
> (a) detach the http::context (like in async app)
> (b) change the response().io_mode() to asynchronous
> (c) send the detached http::context to the async_app to handle it
> asynchronously vio post()...
> 
> So basically sync app tells I don't want to handle it and give it away to
> the async app.

I feel this (the last 3 steps) is the most elegant solution of all, and
easy to follow.

Best Regards,
CN

-- 
http://www.fastmail.fm - A fast, anti-spam email service.

Re: [Cppcms-users] Synchronous app calls io_service.post() and blocks itself

[Artyom B. <art...@ya...>]

>
> Such hand made "cache" seems to be unable to fit my case. I
> need to use
> booster::aio::io_service as a synchronizer, like a mutex, to
> 
> *sequentially* execute cppdb operations and manipulate
> in-memory data.
 
One VERY important point to alarm you.

There is ONE even loop that handles LOTS of operations:
incoming connections, uploads, transfers, request parsing,
etc.

It MUST not be blocked. As it basically blocks the entire CppCMS
service from doing its job.

If you call some heavy query to cppdb FROM the event loop
or async app. You'll block the entire application.

It is bad idea to abuse the even loop for such a purpose.
Any operations in the async application or event loop must
be as fast as possible and what is important -  never block.

If you run for example an SQL query that access the disk
you may delay your entire cppcms application up to 6ms
required for a physical disk to seek to the non-cached data.

If you need to perform blocking/heavy operation by the async-app,
inside event loop - post it to the thread pool and get the
results "posted" back.

If you want some "job queue" for sequential execution you'll probably want to
create some additional execution queue,

you may create your own booster::aio::io_service and run it in its own thread
that would act like a dispatch queue (although it is most likely overkill)

Or write your own simple job queue (which is quite a textbook tasks)

Artyom

Re: [Cppcms-users] Synchronous app calls io_service.post() and blocks itself

[CN <cn...@fa...>]

Thank you for your kind reminder! I think I get your points.

> > Such hand made "cache" seems to be unable to fit my case. I
> > need to use
> > booster::aio::io_service as a synchronizer, like a mutex, to
> > 
> > *sequentially* execute cppdb operations and manipulate
> > in-memory data.
>  
> One VERY important point to alarm you.
> 
> There is ONE even loop that handles LOTS of operations:
> incoming connections, uploads, transfers, request parsing,
> etc.
> 
> It MUST not be blocked. As it basically blocks the entire CppCMS
> service from doing its job.
> 
> If you call some heavy query to cppdb FROM the event loop
> or async app. You'll block the entire application.
> 
> It is bad idea to abuse the even loop for such a purpose.
> Any operations in the async application or event loop must
> be as fast as possible and what is important -  never block.
> 
> If you run for example an SQL query that access the disk
> you may delay your entire cppcms application up to 6ms
> required for a physical disk to seek to the non-cached data.
> 
> If you need to perform blocking/heavy operation by the async-app,
> inside event loop - post it to the thread pool and get the
> results "posted" back.

I have two questions if I opt for this approach:

(1) Can I simply use
cppcms::thread_pool &thread_pool()
in the central service or must I create my own instance of
cppcms::thread_pool?

(2) Synchronous applications require below technique you mentioned
earlier, or std::future or the likes, to get the results "posted" back
by the jobs running in thread pool. Correct?

**begin technique**
technically you can implement this by for example waiting for a
conditional variable.
that would be changed in the event loop:

Caller
------------------

this->resp_is_ready=false

async_app->service().get_io_service().post(boost::bind(&my_class::my_method,this));

while(true){
    booster::unique_lock guard(this->resp_is_ready_mutex)
    this->resp_is_ready_cond.wait(guard)
    if(this->resp_is_ready)
      break;
} 
-------------
Handler
-------------

my_class:my_method()
{
   // do something with async app/withing the event loop

    booster::unique_lock guard(this->resp_is_ready_mutex);
    this->resp_is_ready = true
    this->resp_is_ready_cond.notify_one()
  
}
**end technique**
 
> If you want some "job queue" for sequential execution you'll probably
> want to
> create some additional execution queue,
> 
> you may create your own booster::aio::io_service and run it in its own
> thread
> that would act like a dispatch queue (although it is most likely
> overkill)
> 
> Or write your own simple job queue (which is quite a textbook tasks)

Regards,
CN

-- 
http://www.fastmail.fm - Accessible with your email software
                          or over the web

Re: [Cppcms-users] Synchronous app calls io_service.post() and blocks itself

[Artyom B. <art...@ya...>]

> I have two questions if I opt for this approach:
> 
> (1) Can I simply use
> cppcms::thread_pool &thread_pool()
> in the central service or must I create my own instance of
> cppcms::thread_pool?
 
Yes you can use thread pool of the central service  - it is designed for such job.

> (2) Synchronous applications require below technique you
> mentioned
> earlier, or std::future or the likes, to get the results
> "posted" back
> by the jobs running in thread pool. Correct?
> 
> **begin technique**
> ...
> **end technique**

Basically it is a way to wait for something to happen in the current thread...

  

Artyom