Re: [Cppcms-users] Consumer producer problem

[Christian G. <chr...@gm...>]

2013/3/13 Artyom Beilis <art...@ya...>:
> ----- Original Message -----
>
>> From: Christian Gmeiner <chr...@gm...>
>> To: cpp...@li...
>> Cc:
>> Sent: Wednesday, March 13, 2013 10:39 AM
>> Subject: [Cppcms-users] Consumer producer problem
>>
>> Hi all.
>>
>> I am using a QProcess to run an external bash script. There I am
>> reading back stdout and stderr and
>> send these information back to the client via SSE.
>> Now I run into the problem that QProcess produces messages a lot
>> faster then they got send via SSE.
>> The result is that the used ring-buffer has overwritten elements and
>> not all "messages" got send.
>> At the moment I am using the sse::bounded_event_queue.
>>
>> Is there anything I need to take of when using condition_variable?
>
>> What is the best approach to get
>> this handled in a clean way?
>>
>> thanks
>
>> --
>> Christian Gmeiner, MSc
>>
>
> First of all how they are generated, synchronously, asynchronously?
> how many client do you have?
>

I have a JSON RPC server running:

        srv.applications_pool().mount(cppcms::applications_factory<TsswService>(),
cppcms::mount_point("/rpc(.*)", 1));

with a run
        bind("run", cppcms::rpc::json_method(&TsswService::run, this),
method_role);

Where run looks something like:

void TsswService::run(std::string key, std::string parameter)
{
    BOOSTER_DEBUG("TsswService") << __func__ << " key: '" << key << "'
parameter: '" << parameter << "'";

    // valid session needed
    if (!session().is_set("user"))
    {
        BOOSTER_DEBUG("TsswService") << __func__ << ": not logged in";
        return;
    }

   ... validate input ...


   ... start process ...

        Process *p = new Process();
        BOOSTER_DEBUG("TsswService") << __func__ << " cmd: '" <<
file.absoluteFilePath().toStdString()
                << "' parameter: '" << pList.join(" ").toStdString() << "'";
        int exit = p->exec(file.absoluteFilePath(), pList, &out, &err, true);
        delete p;

        BOOSTER_DEBUG("TsswService") << __func__ << out.toStdString();
        BOOSTER_DEBUG("TsswService") << __func__ << err.toStdString();
        BOOSTER_DEBUG("TsswService") << __func__ << exit;
}


>From the browser I am calling the run RPC with key "do_backup" which
start a new process on the
server (/opt/scripts/run_full_backup.sh). stdout and stderr are "send"
via SSE to the browser.

So I would say that the messages are generated synchronously - or I am wrong?

BTW: /opt/scripts/run_full_backup.sh check if its called multiple
times and prints on stderr
a warning.

>
> In general it is not a producer consumer problem
> but rather message broker problem.
>
> if you have 1 client and 1 source you don't really need an
> asynchronous SSE... you can do it easily synchronously as
> in any case process itself is synchronous.
>

In the end it looks like I have a one process <----> one browser session
connection.

>
> However if you have many clients and one source it is
> a different problem.
>
> What happens if somebody requests the date after the process
> started do you send it from the begging or from the last
> event? What happens on reconnect you need to start from the
> last point but how long do you want to keep the data?
>
> In general you can keep all events - all the output
>
> and send it from the beginning, in such case you can either
> make the ring buffer with a big ring or just alter
> the class
>
> http://sourceforge.net/p/cppcms/code/2238/tree/framework/trunk/contrib/server_side/sse/server_sent_events.h
>
>
> With the policy you need. You can derive the event_source
> class and provide the policy of keeping and throwing messages
> the way you need.
>

As far as I have seen the broadcast() method in the event_source class
works asynchronously. So I need
to change it to be synchronously.

--
Christian Gmeiner, MSc