cppcms-users — Disussions of users of CppCMS

Re: [Cppcms-users] SSE keep-alive

[Artyom B. <art...@ya...>]

Your code is incorrect, you must not use Mutexes...

Instead of this:

>     void push(std::string const &event,std::string const &data,bool 
> send=true)
>     {
>         message msg;
>         msg.event = event;
>         msg.data = data;
> 
>         mutex.lock();
>         messages_.push(msg);
>         mutex.unlock();
> 
>         if(send)
>             broadcast();
>     }


That I assume is called form an external thread you need to do something different


void thread_safe_push(std::string const &event,std::string const &data)
{      message msg;
      msg.event = event;
      msg.data = data;


      service().post([=] {  (lambda expression)

         // EXECUTED IN THE EVENT LOOP THREAD!!!
         messages_.push(msg);
         broadcast();

      }); 

}


or without C++11

void thread_safe_push(std::string const &event,std::string const &data)
{      message msg;
      msg.event = event;
      msg.data = data;


      service().post(boost::bind(&event_fifo::thread_unsef_push,this,msg));

}


void thread_unsefe_push(message const &msg)

{      messages_.push(msg);
      broadcast();

}


----- Original Message -----
> From: Christian Gmeiner <chr...@gm...>
> }
> 
> 
> My used SSE application looks like:
> 
> SSE::SSE(cppcms::service &srv) : cppcms::application(srv)
> {
>     stream_ = sse::event_fifo::create(srv.get_io_service());
>     stream_->enable_keep_alive(1);
> 
>     dispatcher().assign("/get",&SSE::get,this);
> }
> 
> void SSE::get()
> {
>     stream_->accept(release_context());
> }
> 
> void SSE::enqueue(std::string const &event, std::string const &data)
> {
>     stream_->push(event, data);
> }
> 
> 
> And my used event_fifo class:
> 
> class event_fifo : public event_source {
> protected:
>     event_fifo(booster::aio::io_service &srv) :
>         event_source(srv)
>     {
>     }
> public:
>     ///
>     /// Create a queue of maximal size \a size, such that user that
> connects too late
>     /// it would be able to receive at most \a size latest messages
>     ///
>     static booster::shared_ptr<event_fifo> create(
> booster::aio::io_service &srv)
>     {
>         booster::shared_ptr<event_fifo> p(new event_fifo(srv));
>         return p;
>     }
> 
>     virtual void accept(booster::shared_ptr<cppcms::http::context> ctx)
>     {
>         // remove old messages
>         mutex.lock();
>         while (messages_.empty() == false)
>         {
>             messages_.pop();
>         }
>         mutex.unlock();
> 
>         event_source::accept(ctx);
> 
>         // we need to send something that the EventSource on the client side
>         // gets notified that the stream is open.
>         push("welcome", "welcome");
>     }
> 
>     ///
>     /// put a message into the fifo
>     ///
>     /// If \a send is false the messages are not dispatched
>     /// immediately, you can dispatch them later by calling
> broadcast() or by calling push
>     /// event with send=true
>     ///
>     void push(std::string const &data,bool send=true)
>     {
>         push(std::string(),data,send);
>     }
> 
>     ///
>     /// put a message into the fifo
>     ///
>     /// If \a send is false the messages are not dispatched
>     /// immediately, you can dispatch them later by calling
> broadcast() or by calling push
>     /// event with send=true
>     ///
>     void push(std::string const &event,std::string const &data,bool 
> send=true)
>     {
>         message msg;
>         msg.event = event;
>         msg.data = data;
> 
>         mutex.lock();
>         messages_.push(msg);
>         mutex.unlock();
> 
>         if(send)
>             broadcast();
>     }
> 
> protected:
>     bool on_sent(event_stream &es)
>     {
>         size_t last_id = es.last_integer_id();
>         size_t id = last_id;
> 
>         mutex.lock();
>         while (messages_.empty() == false)
>         {
>             id++;
>             message &msg = messages_.front();
>             es.write(msg.data, id, msg.event);
>             messages_.pop();
>         }
>         mutex.unlock();
> 
>         return true;
>     }
> 
> private:
>     struct message {
>         std::string event;
>         std::string data;
>     };
>     std::queue<message> messages_;
>     booster::mutex mutex;
> };
> 
> } // namespace sse
> 
> Oh... it is planed to have one SSE stream per session.
> If you need it I can provide you a simple demo application.
> 
> greets
> --
> Christian Gmeiner, MSc
>



 
Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

Re: [Cppcms-users] SSE keep-alive

[Artyom B. <art...@ya...>]

I don't really understand why do you have mutex...

You may not write to the event stream from a thread that is not the event loop thread.

If you want to notify the asynchronous SSE on anything you need to use cppcms::service::post() 
with a callback that would be executed in the event loop.
 
Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/



>________________________________
> From: Christian Gmeiner <chr...@gm...>
>To: Artyom Beilis <art...@ya...>; cpp...@li... 
>Sent: Tuesday, April 2, 2013 2:00 PM
>Subject: Re: [Cppcms-users] SSE keep-alive
> 
>2013/4/2 Christian Gmeiner <chr...@gm...>:
>> 2013/4/1 Artyom Beilis <art...@ya...>:
>>>
>>>
>>> ----- Original Message -----
>>>> From: Christian Gmeiner <chr...@gm...>
>>>> To: cpp...@li...
>>>> Cc:
>>>> Sent: Monday, April 1, 2013 9:37 PM
>>>> Subject: [Cppcms-users] SSE keep-alive
>>>>
>>>> HI all,
>>>>
>>>> I am trying to understand the keep-alive mechanism used in the SSE
>>>> classes. I have the following problem:
>>>>
>>>> For test purposes I lowered the http timeout to 10 seconds, see
>>>> keep-live is set to 1 second and the session timeout is set to 20
>>>> seconds.
>>>> Now a client opens the sse stream /sse/get and gets a "ping" message.
>>>> Now in theory every second the see keep-alive worker should do its
>>>> work, but it
>>>> looks like long_pollers_ and streamers_ are empty. void
>>>> event_source::keep_alive(char const *comment) gets called every
>>>> seconds but as long_pollers_
>>>> and streamers are empty no keep alive is send.
>>>> There are two places where streamers_.insert is called: class
>>>> post_send and void
>>>> event_source::accept(booster::shared_ptr<cppcms::http::context> ctx).
>>>>
>>>> Maybe somebody can help me to under stand it!
>>>>
>>>> thanks
>>>> --
>>>> Christian Gmeiner, MSc
>>>>
>>>
>>>
>>>
>>> I don't really understand your setup. Does a user connected to the
>>> thread get the keep alive messages (empty comment messages like :keep-alive)
>>> or not?
>>>
>>
>> A user _NEVER_ gets the keep alive message.. thats why I am asking :)
>>
>> I have added some debug to void event_source::keep_alive(char const *comment):
>> http://dpaste.com/hold/1043987/
>>
>>
>> Here is an example log output:
>> http://dpaste.com/hold/1044005/
>>
>>
>> As you can see after then last data transfer via SSE the keep_alive
>> gets called, but does nothing.
>> After 5 seconds the connection run into a timeout.
>>
>>
>> My used config:
>>
>> {
>>         "service" : {
>>                 "api" : "http",
>>                 "ip" : "0.0.0.0",
>>                 "port" : 8080
>>         },
>>         "http" : {
>>                 "script" : "/tssw",
>>                 "timeout" : 5,
>>         },
>>         "session" : {
>>                 "expire" : "renew",
>>                 "timeout" : 15,
>>                 "location" : "server",
>>                 "gc" : 10,
>>                 "server": {
>>                         "storage":"memory"
>>                 }
>>         },
>>         "file_server" : {
>>                 "enable" : true,
>>                 "document_root" : "/opt/tssw/",
>>                 "listing" : false,
>>                 "alias" : [
>>                         { "url" : "/assets" , "path" : "/opt/tssw/" },
>>                         { "url" : "/downloads" , "path" : "/home/vis/" }
>>                 ],
>>         },
>>         "security" : {
>>                 "csrf" : {
>>                         "enable" : true
>>                 },
>>                 "multipart_form_data_limit" : 1048576,
>>                 "uploads_path" : "/home/vis/"
>>         },
>>         "localization" : {
>>                 "messages" : {
>>                         "paths" : [ "/opt/tssw/locale" ],
>>                         "domains" : [ "tssw" ]
>>                 },
>>                 "locales" : [ "en.UTF-8", "de.UTF-8", "zh.UTF-8" ]
>>         },
>>         "logging" : {
>>                 "level" : "debug",
>>                 "syslog" : {
>>                         "enable" : true,
>>                         "id" : "tssw",
>>                 },
>>         }
>> }
>>
>>
>> My used SSE application looks like:
>>
>> SSE::SSE(cppcms::service &srv) : cppcms::application(srv)
>> {
>>     stream_ = sse::event_fifo::create(srv.get_io_service());
>>     stream_->enable_keep_alive(1);
>>
>>     dispatcher().assign("/get",&SSE::get,this);
>> }
>>
>> void SSE::get()
>> {
>>     stream_->accept(release_context());
>> }
>>
>> void SSE::enqueue(std::string const &event, std::string const &data)
>> {
>>     stream_->push(event, data);
>> }
>>
>>
>> And my used event_fifo class:
>>
>> class event_fifo : public event_source {
>> protected:
>>     event_fifo(booster::aio::io_service &srv) :
>>         event_source(srv)
>>     {
>>     }
>> public:
>>     ///
>>     /// Create a queue of maximal size \a size, such that user that
>> connects too late
>>     /// it would be able to receive at most \a size latest messages
>>     ///
>>     static booster::shared_ptr<event_fifo> create(
>> booster::aio::io_service &srv)
>>     {
>>         booster::shared_ptr<event_fifo> p(new event_fifo(srv));
>>         return p;
>>     }
>>
>>     virtual void accept(booster::shared_ptr<cppcms::http::context> ctx)
>>     {
>>         // remove old messages
>>         mutex.lock();
>>         while (messages_.empty() == false)
>>         {
>>             messages_.pop();
>>         }
>>         mutex.unlock();
>>
>>         event_source::accept(ctx);
>>
>>         // we need to send something that the EventSource on the client side
>>         // gets notified that the stream is open.
>>         push("welcome", "welcome");
>>     }
>>
>>     ///
>>     /// put a message into the fifo
>>     ///
>>     /// If \a send is false the messages are not dispatched
>>     /// immediately, you can dispatch them later by calling
>> broadcast() or by calling push
>>     /// event with send=true
>>     ///
>>     void push(std::string const &data,bool send=true)
>>     {
>>         push(std::string(),data,send);
>>     }
>>
>>     ///
>>     /// put a message into the fifo
>>     ///
>>     /// If \a send is false the messages are not dispatched
>>     /// immediately, you can dispatch them later by calling
>> broadcast() or by calling push
>>     /// event with send=true
>>     ///
>>     void push(std::string const &event,std::string const &data,bool send=true)
>>     {
>>         message msg;
>>         msg.event = event;
>>         msg.data = data;
>>
>>         mutex.lock();
>>         messages_.push(msg);
>>         mutex.unlock();
>>
>>         if(send)
>>             broadcast();
>>     }
>>
>> protected:
>>     bool on_sent(event_stream &es)
>>     {
>>         size_t last_id = es.last_integer_id();
>>         size_t id = last_id;
>>
>>         mutex.lock();
>>         while (messages_.empty() == false)
>>         {
>>             id++;
>>             message &msg = messages_.front();
>>             es.write(msg.data, id, msg.event);
>>             messages_.pop();
>>         }
>>         mutex.unlock();
>>
>>         return true;
>>     }
>>
>> private:
>>     struct message {
>>         std::string event;
>>         std::string data;
>>     };
>>     std::queue<message> messages_;
>>     booster::mutex mutex;
>> };
>>
>> } // namespace sse
>>
>> Oh... it is planed to have one SSE stream per session.
>> If you need it I can provide you a simple demo application.
>>
>
>Finally I got keep_alive working by changing on_send
>
>    bool on_sent(event_stream &es)
>    {
>        bool something_send = false;
>        size_t last_id = es.last_integer_id();
>        size_t id = last_id;
>
>        mutex.lock();
>        while (messages_.empty() == false)
>        {
>            id++;
>            message &msg = messages_.front();
>            es.write(msg.data, id, msg.event);
>            messages_.pop();
>            something_send = true;
>        }
>        mutex.unlock();
>
>        return something_send;
>    }
>
>
>Now I am looking to prevent expiration of the user session.
>
>greets
>--
>Christian Gmeiner, MSc
>
>------------------------------------------------------------------------------
>Own the Future-Intel(R) Level Up Game Demo Contest 2013
>Rise to greatness in Intel's independent game demo contest. Compete 
>for recognition, cash, and the chance to get your game on Steam. 
>$5K grand prize plus 10 genre and skill prizes. Submit your demo 
>by 6/6/13. http://altfarm.mediaplex.com/ad/ck/12124-176961-30367-2
>_______________________________________________
>Cppcms-users mailing list
>Cpp...@li...
>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
>

Re: [Cppcms-users] SSE keep-alive

[Christian G. <chr...@gm...>]

2013/4/2 Christian Gmeiner <chr...@gm...>:
> 2013/4/1 Artyom Beilis <art...@ya...>:
>>
>>
>> ----- Original Message -----
>>> From: Christian Gmeiner <chr...@gm...>
>>> To: cpp...@li...
>>> Cc:
>>> Sent: Monday, April 1, 2013 9:37 PM
>>> Subject: [Cppcms-users] SSE keep-alive
>>>
>>> HI all,
>>>
>>> I am trying to understand the keep-alive mechanism used in the SSE
>>> classes. I have the following problem:
>>>
>>> For test purposes I lowered the http timeout to 10 seconds, see
>>> keep-live is set to 1 second and the session timeout is set to 20
>>> seconds.
>>> Now a client opens the sse stream /sse/get and gets a "ping" message.
>>> Now in theory every second the see keep-alive worker should do its
>>> work, but it
>>> looks like long_pollers_ and streamers_ are empty. void
>>> event_source::keep_alive(char const *comment) gets called every
>>> seconds but as long_pollers_
>>> and streamers are empty no keep alive is send.
>>> There are two places where streamers_.insert is called: class
>>> post_send and void
>>> event_source::accept(booster::shared_ptr<cppcms::http::context> ctx).
>>>
>>> Maybe somebody can help me to under stand it!
>>>
>>> thanks
>>> --
>>> Christian Gmeiner, MSc
>>>
>>
>>
>>
>> I don't really understand your setup. Does a user connected to the
>> thread get the keep alive messages (empty comment messages like :keep-alive)
>> or not?
>>
>
> A user _NEVER_ gets the keep alive message.. thats why I am asking :)
>
> I have added some debug to void event_source::keep_alive(char const *comment):
> http://dpaste.com/hold/1043987/
>
>
> Here is an example log output:
> http://dpaste.com/hold/1044005/
>
>
> As you can see after then last data transfer via SSE the keep_alive
> gets called, but does nothing.
> After 5 seconds the connection run into a timeout.
>
>
> My used config:
>
> {
>         "service" : {
>                 "api" : "http",
>                 "ip" : "0.0.0.0",
>                 "port" : 8080
>         },
>         "http" : {
>                 "script" : "/tssw",
>                 "timeout" : 5,
>         },
>         "session" : {
>                 "expire" : "renew",
>                 "timeout" : 15,
>                 "location" : "server",
>                 "gc" : 10,
>                 "server": {
>                         "storage":"memory"
>                 }
>         },
>         "file_server" : {
>                 "enable" : true,
>                 "document_root" : "/opt/tssw/",
>                 "listing" : false,
>                 "alias" : [
>                         { "url" : "/assets" , "path" : "/opt/tssw/" },
>                         { "url" : "/downloads" , "path" : "/home/vis/" }
>                 ],
>         },
>         "security" : {
>                 "csrf" : {
>                         "enable" : true
>                 },
>                 "multipart_form_data_limit" : 1048576,
>                 "uploads_path" : "/home/vis/"
>         },
>         "localization" : {
>                 "messages" : {
>                         "paths" : [ "/opt/tssw/locale" ],
>                         "domains" : [ "tssw" ]
>                 },
>                 "locales" : [ "en.UTF-8", "de.UTF-8", "zh.UTF-8" ]
>         },
>         "logging" : {
>                 "level" : "debug",
>                 "syslog" : {
>                         "enable" : true,
>                         "id" : "tssw",
>                 },
>         }
> }
>
>
> My used SSE application looks like:
>
> SSE::SSE(cppcms::service &srv) : cppcms::application(srv)
> {
>     stream_ = sse::event_fifo::create(srv.get_io_service());
>     stream_->enable_keep_alive(1);
>
>     dispatcher().assign("/get",&SSE::get,this);
> }
>
> void SSE::get()
> {
>     stream_->accept(release_context());
> }
>
> void SSE::enqueue(std::string const &event, std::string const &data)
> {
>     stream_->push(event, data);
> }
>
>
> And my used event_fifo class:
>
> class event_fifo : public event_source {
> protected:
>     event_fifo(booster::aio::io_service &srv) :
>         event_source(srv)
>     {
>     }
> public:
>     ///
>     /// Create a queue of maximal size \a size, such that user that
> connects too late
>     /// it would be able to receive at most \a size latest messages
>     ///
>     static booster::shared_ptr<event_fifo> create(
> booster::aio::io_service &srv)
>     {
>         booster::shared_ptr<event_fifo> p(new event_fifo(srv));
>         return p;
>     }
>
>     virtual void accept(booster::shared_ptr<cppcms::http::context> ctx)
>     {
>         // remove old messages
>         mutex.lock();
>         while (messages_.empty() == false)
>         {
>             messages_.pop();
>         }
>         mutex.unlock();
>
>         event_source::accept(ctx);
>
>         // we need to send something that the EventSource on the client side
>         // gets notified that the stream is open.
>         push("welcome", "welcome");
>     }
>
>     ///
>     /// put a message into the fifo
>     ///
>     /// If \a send is false the messages are not dispatched
>     /// immediately, you can dispatch them later by calling
> broadcast() or by calling push
>     /// event with send=true
>     ///
>     void push(std::string const &data,bool send=true)
>     {
>         push(std::string(),data,send);
>     }
>
>     ///
>     /// put a message into the fifo
>     ///
>     /// If \a send is false the messages are not dispatched
>     /// immediately, you can dispatch them later by calling
> broadcast() or by calling push
>     /// event with send=true
>     ///
>     void push(std::string const &event,std::string const &data,bool send=true)
>     {
>         message msg;
>         msg.event = event;
>         msg.data = data;
>
>         mutex.lock();
>         messages_.push(msg);
>         mutex.unlock();
>
>         if(send)
>             broadcast();
>     }
>
> protected:
>     bool on_sent(event_stream &es)
>     {
>         size_t last_id = es.last_integer_id();
>         size_t id = last_id;
>
>         mutex.lock();
>         while (messages_.empty() == false)
>         {
>             id++;
>             message &msg = messages_.front();
>             es.write(msg.data, id, msg.event);
>             messages_.pop();
>         }
>         mutex.unlock();
>
>         return true;
>     }
>
> private:
>     struct message {
>         std::string event;
>         std::string data;
>     };
>     std::queue<message> messages_;
>     booster::mutex mutex;
> };
>
> } // namespace sse
>
> Oh... it is planed to have one SSE stream per session.
> If you need it I can provide you a simple demo application.
>

Finally I got keep_alive working by changing on_send

    bool on_sent(event_stream &es)
    {
        bool something_send = false;
        size_t last_id = es.last_integer_id();
        size_t id = last_id;

        mutex.lock();
        while (messages_.empty() == false)
        {
            id++;
            message &msg = messages_.front();
            es.write(msg.data, id, msg.event);
            messages_.pop();
            something_send = true;
        }
        mutex.unlock();

        return something_send;
    }


Now I am looking to prevent expiration of the user session.

greets
--
Christian Gmeiner, MSc

Re: [Cppcms-users] SSE keep-alive

[Christian G. <chr...@gm...>]

2013/4/1 Artyom Beilis <art...@ya...>:
>
>
> ----- Original Message -----
>> From: Christian Gmeiner <chr...@gm...>
>> To: cpp...@li...
>> Cc:
>> Sent: Monday, April 1, 2013 9:37 PM
>> Subject: [Cppcms-users] SSE keep-alive
>>
>> HI all,
>>
>> I am trying to understand the keep-alive mechanism used in the SSE
>> classes. I have the following problem:
>>
>> For test purposes I lowered the http timeout to 10 seconds, see
>> keep-live is set to 1 second and the session timeout is set to 20
>> seconds.
>> Now a client opens the sse stream /sse/get and gets a "ping" message.
>> Now in theory every second the see keep-alive worker should do its
>> work, but it
>> looks like long_pollers_ and streamers_ are empty. void
>> event_source::keep_alive(char const *comment) gets called every
>> seconds but as long_pollers_
>> and streamers are empty no keep alive is send.
>> There are two places where streamers_.insert is called: class
>> post_send and void
>> event_source::accept(booster::shared_ptr<cppcms::http::context> ctx).
>>
>> Maybe somebody can help me to under stand it!
>>
>> thanks
>> --
>> Christian Gmeiner, MSc
>>
>
>
>
> I don't really understand your setup. Does a user connected to the
> thread get the keep alive messages (empty comment messages like :keep-alive)
> or not?
>

A user _NEVER_ gets the keep alive message.. thats why I am asking :)

I have added some debug to void event_source::keep_alive(char const *comment):
http://dpaste.com/hold/1043987/


Here is an example log output:
http://dpaste.com/hold/1044005/


As you can see after then last data transfer via SSE the keep_alive
gets called, but does nothing.
After 5 seconds the connection run into a timeout.


My used config:

{
        "service" : {
                "api" : "http",
                "ip" : "0.0.0.0",
                "port" : 8080
        },
        "http" : {
                "script" : "/tssw",
                "timeout" : 5,
        },
        "session" : {
                "expire" : "renew",
                "timeout" : 15,
                "location" : "server",
                "gc" : 10,
                "server": {
                        "storage":"memory"
                }
        },
        "file_server" : {
                "enable" : true,
                "document_root" : "/opt/tssw/",
                "listing" : false,
                "alias" : [
                        { "url" : "/assets" , "path" : "/opt/tssw/" },
                        { "url" : "/downloads" , "path" : "/home/vis/" }
                ],
        },
        "security" : {
                "csrf" : {
                        "enable" : true
                },
                "multipart_form_data_limit" : 1048576,
                "uploads_path" : "/home/vis/"
        },
        "localization" : {
                "messages" : {
                        "paths" : [ "/opt/tssw/locale" ],
                        "domains" : [ "tssw" ]
                },
                "locales" : [ "en.UTF-8", "de.UTF-8", "zh.UTF-8" ]
        },
        "logging" : {
                "level" : "debug",
                "syslog" : {
                        "enable" : true,
                        "id" : "tssw",
                },
        }
}


My used SSE application looks like:

SSE::SSE(cppcms::service &srv) : cppcms::application(srv)
{
    stream_ = sse::event_fifo::create(srv.get_io_service());
    stream_->enable_keep_alive(1);

    dispatcher().assign("/get",&SSE::get,this);
}

void SSE::get()
{
    stream_->accept(release_context());
}

void SSE::enqueue(std::string const &event, std::string const &data)
{
    stream_->push(event, data);
}


And my used event_fifo class:

class event_fifo : public event_source {
protected:
    event_fifo(booster::aio::io_service &srv) :
        event_source(srv)
    {
    }
public:
    ///
    /// Create a queue of maximal size \a size, such that user that
connects too late
    /// it would be able to receive at most \a size latest messages
    ///
    static booster::shared_ptr<event_fifo> create(
booster::aio::io_service &srv)
    {
        booster::shared_ptr<event_fifo> p(new event_fifo(srv));
        return p;
    }

    virtual void accept(booster::shared_ptr<cppcms::http::context> ctx)
    {
        // remove old messages
        mutex.lock();
        while (messages_.empty() == false)
        {
            messages_.pop();
        }
        mutex.unlock();

        event_source::accept(ctx);

        // we need to send something that the EventSource on the client side
        // gets notified that the stream is open.
        push("welcome", "welcome");
    }

    ///
    /// put a message into the fifo
    ///
    /// If \a send is false the messages are not dispatched
    /// immediately, you can dispatch them later by calling
broadcast() or by calling push
    /// event with send=true
    ///
    void push(std::string const &data,bool send=true)
    {
        push(std::string(),data,send);
    }

    ///
    /// put a message into the fifo
    ///
    /// If \a send is false the messages are not dispatched
    /// immediately, you can dispatch them later by calling
broadcast() or by calling push
    /// event with send=true
    ///
    void push(std::string const &event,std::string const &data,bool send=true)
    {
        message msg;
        msg.event = event;
        msg.data = data;

        mutex.lock();
        messages_.push(msg);
        mutex.unlock();

        if(send)
            broadcast();
    }

protected:
    bool on_sent(event_stream &es)
    {
        size_t last_id = es.last_integer_id();
        size_t id = last_id;

        mutex.lock();
        while (messages_.empty() == false)
        {
            id++;
            message &msg = messages_.front();
            es.write(msg.data, id, msg.event);
            messages_.pop();
        }
        mutex.unlock();

        return true;
    }

private:
    struct message {
        std::string event;
        std::string data;
    };
    std::queue<message> messages_;
    booster::mutex mutex;
};

} // namespace sse

Oh... it is planed to have one SSE stream per session.
If you need it I can provide you a simple demo application.

greets
--
Christian Gmeiner, MSc

Re: [Cppcms-users] SSE keep-alive

[Artyom B. <art...@ya...>]

----- Original Message -----
> From: Christian Gmeiner <chr...@gm...>
> To: cpp...@li...
> Cc: 
> Sent: Monday, April 1, 2013 9:37 PM
> Subject: [Cppcms-users] SSE keep-alive
> 
> HI all,
> 
> I am trying to understand the keep-alive mechanism used in the SSE
> classes. I have the following problem:
> 
> For test purposes I lowered the http timeout to 10 seconds, see
> keep-live is set to 1 second and the session timeout is set to 20
> seconds.
> Now a client opens the sse stream /sse/get and gets a "ping" message.
> Now in theory every second the see keep-alive worker should do its
> work, but it
> looks like long_pollers_ and streamers_ are empty. void
> event_source::keep_alive(char const *comment) gets called every
> seconds but as long_pollers_
> and streamers are empty no keep alive is send.
> There are two places where streamers_.insert is called: class
> post_send and void
> event_source::accept(booster::shared_ptr<cppcms::http::context> ctx).
> 
> Maybe somebody can help me to under stand it!
> 
> thanks
> --
> Christian Gmeiner, MSc
> 



I don't really understand your setup. Does a user connected to the
thread get the keep alive messages (empty comment messages like :keep-alive)
or not?

In general SSE has "comment" messages that are not dispatched but rather
used for Keep alive to let the browser know that the server had not gone
and the server to know that the client is alive (on TCP/IP level)
 
Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

Re: [Cppcms-users] multithread safety in async application

[Artyom B. <art...@ya...>]

>________________________________
> From: Markus Raab <us...@ma...>
>To: cpp...@li... 
>Sent: Monday, April 1, 2013 4:14 PM
>Subject: [Cppcms-users] multithread safety in async application
> 
>Hi Artyom!
>
>> Once you called async_flush_output you must wait for completion
>> handler to be executed before you send more data.
>
>Do you mean by "send more data" streaming to response().out() or do you mean 
>calling async_flush_output again?
>

You can not call async_flush_output(), but you can write to response().out()
the data would be buffered


>Is following sequence correct or did I misunderstood something?
>
>publish thread 1    |publish thread 2    |async cppcms thread
>mutex.lock()        |                    |
>response().out()<<  |                    |
>response().out()<<  |                    |
>async_flush_output()|                    |
>                    |                    |
>                    |                    |in on_complete()
>                    |                    |mutex.unlock()
>                    |mutex.lock()        | 
>                    |response().out()<<  |
>                    |response().out()<<  |
>                    |async_flush_output()|
>                    |                    |
>                    |                    |in on_peer_reset()
>                    |                    |mutex.unlock()
>stop sending        |stop sending        |
>
>Is there a better sequence (that the threads can work more in parallel)?
>
>So I need to have a mutex between:
>- before calling response().out()
>and:
>- on_complete() or on_peer_reset().
>


No, you misunderstood the stuff. You can't access cppcms::http::context (and all its
related objects like response) from the threads other than event loop thread.

The lock would not help and for example some events (like on_peer_reset) 
my be triggered by timers you have no control over.

What you need to do is to prepare a chunk and than pass it to the
asynchronous application thread (even loop) via post() callback.

What you propose is not safe.

>Do I have a guarantee that either on_peer_reset() (callback of 

>cppcms::http::context::async_flush_output) or on_complete() (callback of 
>cppcms::http::context::async_on_peer_reset) is called in any case?
>

Yes, also on_complete is always called with success or error. on_peer_reset
woks independently of sending sequence.

on_peer_reset is also not called on normal completion.









>> For example "state" sse service does exactly the same it sends
>> only latest data and if some was missed between async_flush_output
>> and completion handler it would not be sent, but rather latest
>> data should be.
>
>Thank you for the examples, they are really great!
>
>
>> About buffering,
>> 
>> when you work with async app. the write to response().out()
>> just collects the data to the buffer, the I/O is performed
>> uput async_flush_output request and than you can continue
>> once you receive the notification.
>
>Ok, good to know. See questions aboved about when exactly I am allowed to 
>continue.
>
>> So in general what you described is reasonable approach.
>
>Thanks,
>Markus Raab
>
>
>------------------------------------------------------------------------------
>Own the Future-Intel&reg; Level Up Game Demo Contest 2013
>Rise to greatness in Intel's independent game demo contest.
>Compete for recognition, cash, and the chance to get your game 
>on Steam. $5K grand prize plus 10 genre and skill prizes. 
>Submit your demo by 6/6/13. http://p.sf.net/sfu/intel_levelupd2d
>_______________________________________________
>Cppcms-users mailing list
>Cpp...@li...
>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
> 




Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/

[Cppcms-users] SSE keep-alive

[Christian G. <chr...@gm...>]

HI all,

I am trying to understand the keep-alive mechanism used in the SSE
classes. I have the following problem:

For test purposes I lowered the http timeout to 10 seconds, see
keep-live is set to 1 second and the session timeout is set to 20
seconds.
Now a client opens the sse stream /sse/get and gets a "ping" message.
Now in theory every second the see keep-alive worker should do its
work, but it
looks like long_pollers_ and streamers_ are empty. void
event_source::keep_alive(char const *comment) gets called every
seconds but as long_pollers_
and streamers are empty no keep alive is send.
There are two places where streamers_.insert is called: class
post_send and void
event_source::accept(booster::shared_ptr<cppcms::http::context> ctx).

Maybe somebody can help me to under stand it!

thanks
--
Christian Gmeiner, MSc

[Cppcms-users] multithread safety in async application

[Markus R. <us...@ma...>]

Hi Artyom!

> Once you called async_flush_output you must wait for completion
> handler to be executed before you send more data.

Do you mean by "send more data" streaming to response().out() or do you mean 
calling async_flush_output again?

Is following sequence correct or did I misunderstood something?

publish thread 1    |publish thread 2    |async cppcms thread
mutex.lock()        |                    |
response().out()<<  |                    |
response().out()<<  |                    |
async_flush_output()|                    |
                    |                    |
                    |                    |in on_complete()
                    |                    |mutex.unlock()
                    |mutex.lock()        | 
                    |response().out()<<  |
                    |response().out()<<  |
                    |async_flush_output()|
                    |                    |
                    |                    |in on_peer_reset()
                    |                    |mutex.unlock()
stop sending        |stop sending        |

Is there a better sequence (that the threads can work more in parallel)?

So I need to have a mutex between:
- before calling response().out()
and:
- on_complete() or on_peer_reset().

Do I have a guarantee that either on_peer_reset() (callback of 
cppcms::http::context::async_flush_output) or on_complete() (callback of 
cppcms::http::context::async_on_peer_reset) is called in any case?

> For example "state" sse service does exactly the same it sends
> only latest data and if some was missed between async_flush_output
> and completion handler it would not be sent, but rather latest
> data should be.

Thank you for the examples, they are really great!


> About buffering,
> 
> when you work with async app. the write to response().out()
> just collects the data to the buffer, the I/O is performed
> uput async_flush_output request and than you can continue
> once you receive the notification.

Ok, good to know. See questions aboved about when exactly I am allowed to 
continue.

> So in general what you described is reasonable approach.

Thanks,
Markus Raab

Re: [Cppcms-users] cppcms_tpml_cc diff

[Marcel H. <ke...@co...>]

On 01.04.2013 14:00, Marcel Hellwig wrote:
> Hi artyom,
>
> here is a diff file for the tpml_cc file, because on my system (arch)
> python 3 is standard and not 2.7 anymore.
>
> As you can see in [0] python 3 has not StringIO anymore, but moved it.
> So you either can explicitly use python2.7 like my diff does, or use
> python 3 rules.
>
>
> 1c1
> < #!/usr/bin/env python
> ---
>> #!/usr/bin/env python2.7
>
> Regards
> Marcel
>
>
>
> [0]
> http://docs.python.org/3.0/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit
Hmm, i see that you have a lot of python scripts, maybe you should do
this for every python file ;)

Regards

[Cppcms-users] cppcms_tpml_cc diff

[Marcel H. <ke...@co...>]

Hi artyom,

here is a diff file for the tpml_cc file, because on my system (arch)
python 3 is standard and not 2.7 anymore.

As you can see in [0] python 3 has not StringIO anymore, but moved it.
So you either can explicitly use python2.7 like my diff does, or use
python 3 rules.


1c1
< #!/usr/bin/env python
---
> #!/usr/bin/env python2.7


Regards
Marcel



[0]
http://docs.python.org/3.0/whatsnew/3.0.html#text-vs-data-instead-of-unicode-vs-8-bit

[Cppcms-users] timedout asynchronous socket

[Sergiu A. <ser...@ya...>]

Hello all,

I want to implement a timedout asynchronous socket read. My application extends json_rpc_server:

myapi::myapi(cppcms::service &srv) :cppcms::rpc::json_rpc_server(srv) ...


I have this class, "base_who", which has this method:

std::string base_who::timedout_request(cppcms::service &srv, std::string command){
    

    ... /* connect the socket, send some command etc */ ...


    booster::aio::deadline_timer timer(srv.get_io_service());
    timer.expires_from_now(booster::ptime::seconds(100));

    timer.async_wait(std::bind(&base_who::on_timeout,shared_from_this(),std::placeholders::_1));

    std::cout << "AFTER ASYNC WAIT" << std::endl;

    the_socket.async_read_some(
                        booster::aio::buffer(async_buffer_,sizeof(async_buffer_)),
                            std::bind(&base_who::on_read,shared_from_this(),
                                    std::placeholders::_1,
                                    std::placeholders::_2));

    ... /* etc  */ ...
}


The problem is that when I call timer.async_wait, something happens and I receive "Internal Service Error" somewhere. The line "AFTER ASYNC WAIT" is never reached.

I think this is because of this (from timer's async_wait method description):
            /// Wait asynchronously for the timer.
            ///
            /// If io_service is not assigned throws system::system_error, all other errors reported via \a h

So, I think the io_service is not correctly assigned.
The base_who class is used in a method in myapi and timedout_request is called from there:

void myapi::request(std::string command) {


    base_who myconnection(ip, port);
    myconnection.timedout_request(service(), std::string("some_command"));

    ... /* etc */ ...

 }

Do I use timer.async_wait correctly? I saw something similar in the json_rpc_chat example, but that uses <boost/bind.hpp>. I would like to not include it, use just "booster".

What could be the problem?
Thank you,

--
Fatol Sergiu Adrian
Team Leader & Senior PHP Developer
Arobs Transilvania Software

adi...@ar...
Skype: sergiu.adrian

Olimpia Business Center
98-100 Dorobantilor, 2nd Floor
400609 Cluj-Napoca Romania

Co-Founder -- Sosetaria.ro
http://sosetaria.ro

Re: [Cppcms-users] Shutdown issue

[Artyom B. <art...@ya...>]

Try to get the backtrace... It is hard to get what exactly happens there
 
Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/



>________________________________
> From: Szabolcs Rimanóczy <rim...@go...>
>To: Artyom Beilis <art...@ya...>; cpp...@li... 
>Sent: Thursday, March 14, 2013 7:56 PM
>Subject: Re: [Cppcms-users] Shutdown issue
> 
>
>It's a bit better, now, it doesn't hang in the background but displays an error on exit.
>This is what my application does:
>- does something, 
>- reboots
>- does something
>- exits
>
>
>When I set disable_global_exit_handling to false:
>  - on reboot: 
>    - the last log is Shutting down WebService... (just before shutdown)
>    - no log is output between  m_service->run(); and delete m_service;
>    - a log is output: "segfault at 0 ip 402744c0 sp bfaed3f0 error 4 in libcppcms.so.1.0.2[401f7000+211000]"
>  - on exit: 
>    - log is output between m_service->run(); and delete m_service; 
>    - the application never exits, stays in the background 
>
>
>When I set disable_global_exit_handling to true:
>  - on reboot: 
>    - the last log is Shutting down WebService... (just before shutdown)
>    - no log is output between m_service->run(); and delete m_service;
>    - a log is output: "segfault at 0 ip 402744c0 sp bfbb16c0 error 4 in libcppcms.so.1.0.2[401f7000+211000]"
>  - on exit: 
>    - the last log is Shutting down WebService... (just before shutdown)
>    - a log is output: "segfault at 0 ip 402744c0 sp bff07660 error 4 in libcppcms.so.1.0.2[401f7000+211000]"
>
>    - the application doesn't hang in the background
>    - no log is output between   m_service->run(); and delete m_service; 
>
>
>Any ideas what could be wrong?
>
>
>Sab
>
>
>2013/3/14 Artyom Beilis <art...@ya...>
>
>Few points:
>>
>>
>>a) Enable this option: http://cppcms.com/wikipp/en/page/cppcms_1x_config#service.disable_global_exit_handling to prevent from the CppCMS service to install signal handlers.
>>
>>
>>b) Put the log between
>>
>>
>>   m_service->run(); 
>>    << Log exited 
>>   delete m_service;
>>
>>
>>To see if service exits.
>>
>>
>>And probably put log after the delete to make sure that thread competes.
>>
>>
>>
>>Note the run() would not exit until all the worker threads had completed the job.
>>
>>
>>c) Why don't you put cppcms::service on the stack such that it would not leak in case of exception...
>>
>>you code is not exception safe.
>> 
>>Artyom Beilis
>>--------------
>>CppCMS - C++ Web Framework:   http://cppcms.com/
>>CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/
>>
>>
>>
>>>________________________________
>>> From: Sab <rim...@gm...>
>>>To: cpp...@li... 
>>>Sent: Thursday, March 14, 2013 11:03 AM
>>>Subject: [Cppcms-users] Shutdown issue
>>> 
>>>
>>>Hallo Artyom,
>>>
>>>I am running the cppcms::service in a separate thread. When the calling thread 
>>>restarts, I would also like to restart the cppcms service. On restart, I am 
>>>calling service.shutdown() from the main tread that executed it. I see the log 
>>>before the shutdown call but the log after the call is not printed -> I suspect 
>>>that the function hangs up because after the application exits the process still 
>>>keeps running in the background. This is my application’s main:
>>>
>>>(Sorry for the code formatting...)
>>>
>>>int main(int argc, char* argv[]) 
>>>{
>>>. . .
>>>do
>>>{
>>>. . .
>>>//start WebService
>>>WebService webService(connector);
>>>LOG_MAIN << "Starting the WebService thread";
>>>boost::thread webServiceThread(webService);
>>>. . .
>>>
>>>while ( system.IsRunning() && !shutdownSignalReceived )
>>>{
>>>. . .
>>>}
>>>
>>>//stop WebService
>>>LOG_MAIN << "Stopping
 WebService...";
>>>webService.ShutDown();
>>>webServiceThread.join();
>>>. . .
>>>} 
>>>while (runState == Common::Devices::Run::Reload);
>>>}
>>>
>>>------------------
>>>My WebService class:
>>>
>>>void WebService::operator()()
>>>{
>>>try
>>>{
>>>. . .
>>>m_service = new cppcms::service(serviceConfig);
>>>m_service->applications_pool().
>>>mount(cppcms::applications_factory<MainWebHandler>
>>>(m_connector));
>>>m_service->run();
>>>delete m_service;
>>>}
>>>catch(std::exception const &e)
>>>{
>>>LOG_MAIN_ERROR << "WebService exception: " << e.what();
>>>}
>>>}
>>>
>>>void WebService::ShutDown()
>>>{
>>>try
>>>{
>>>cout << "Shutting down WebService..." << endl;
>>>m_service->shutdown();
>>>cout << "Shutting down WebService finished." << endl;
>>>}
>>>catch(std::exception const &e)
>>>{
>>>LOG_MAIN_ERROR << "WebService exception: " << e.what();
>>>}
>>>}
>>>
>>>---------------
>>>I
 don't get any error or exception. As mentioned I see the log “Shutting down 
>>>WebService...” but I don’t see “Shutting down WebService finished.” that’s why I 
>>>think the service doesn’t shut down properly so the thread keeps running in the 
>>>background.
>>>
>>>Do you see anything wrong with my code?
>>>
>>>Sab
>>>
>>>
>>>
>>>------------------------------------------------------------------------------
>>>Everyone hates slow websites. So do we.
>>>Make your web apps faster with AppDynamics
>>>Download AppDynamics Lite for free today:
>>>http://p.sf.net/sfu/appdyn_d2d_mar
>>>_______________________________________________
>>>Cppcms-users mailing list
>>>Cpp...@li...
>>>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>>>
>>>
>>>
>>------------------------------------------------------------------------------
>>Everyone hates slow websites. So do we.
>>Make your web apps faster with AppDynamics
>>Download AppDynamics Lite for free today:
>>http://p.sf.net/sfu/appdyn_d2d_mar
>>_______________________________________________
>>Cppcms-users mailing list
>>Cpp...@li...
>>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>>
>>
>
>
>

Re: [Cppcms-users] Booster-PCRE linking

[Artyom B. <art...@ya...>]

Is it still relevant?

What usually you do is not to specifiy explicitly the PCRE path but rather let CMake find it,
 

        -DCMAKE_INCLUDE_PATH="../../pcre-8.32.target/pub/include" \
        -DCMAKE_LIBRARY_PATH="../../pcre-8.32.target/pub/lib:zlib-1.2.7.target/pub" ..

Without ZLIB=... PCRE_LIB= etc.

Note it is also to run make install in PCRE and ZLIB - install them to a single location with all correct files and libraries and THAN use rater
than pointing to source directories.

Also it seems that you are mixing shared and static libraries (../../pcre-8.32.target/pub/lib/libpcre.so")
   

Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/



>________________________________
> From: Sab <rim...@gm...>
>To: cpp...@li... 
>Sent: Wednesday, March 20, 2013 5:22 PM
>Subject: [Cppcms-users] Booster-PCRE linking
> 
>Hi Artyom,
>
>I'm cross compiling my application and having trouble with linking the 
>static libraries. CppCMS compiles fine but when I compile my application 
>that uses cppcms, I get undefined reference to 'pcre_exec', 'pcre_compile' 
>and 'pcre_fullinfo' needed by libbooster.a e.g. in function 
>'booster::regex::match'. 
>
>Here is how I compile CppCMS in my environment installer Makefile:
>
>Shared3rdParty/cppcms-1.0.2.target/pub/libcppcms.a: 
>    Shared3rdParty/zlib-1.2.7.target/pub/lib/libz.a
>    Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.a
>    cd Shared3rdParty/cppcms-1.0.2.target && mkdir -p -v build
>    cd Shared3rdParty/cppcms-1.0.2.target/build && cmake \
>        -DCMAKE_TOOLCHAIN_FILE="../../Product/gcc-${TOOLSUITE}.cmake" \
>        -DCMAKE_BUILD_TYPE=MinSizeRel \
>        -DDISABLE_ICU_LOCALE=ON \
>        -DDISABLE_POSIX_LOCALE=ON \
>        -DDISABLE_SHARED=ON \
>        -DDISABLE_GCRYPT=ON \
>        -DDISABLE_OPENSSL=ON \
>        -DDISABLE_OPENSSL=ON \
>        -DCMAKE_INSTALL_PREFIX="../pub" \
>        -DDISABLE_SCGI=ON \
>        -DDISABLE_HTTP=ON \
>        -DZLIB_INCLUDE_DIR="../../zlib-1.2.7.target/pub/include" \
>        -DZLIB="../../zlib-1.2.7.target/pub/lib/libz.a" \
>        -DPCRE_LIB="../../pcre-8.32.target/pub/lib/libpcre.so" \
>        -DCMAKE_INCLUDE_PATH="../../pcre-8.32.target/pub/include" \
>        -DCMAKE_LIBRARY_PATH="../../pcre-8.32.target/pub/lib" ..
>    cd Shared3rdParty/cppcms-1.0.2.target/build && make && make install
>
>Here is how I compile pcre in the same makefile:
>
>Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.a:
>    cd Shared3rdParty/pcre-8.32.target && ./configure --host=${TOOLSUITE} 
>--prefix=`pwd`/pub 
>    cd Shared3rdParty/pcre-8.32.target && make && make install
>    
>
>I do see the right path in CppCMS's CMakeCache.txt:
>
>//Path to a library.
>PCRE_LIB:FILEPATH=/work/eBox/Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.so
>
>and:
>
>//Dependencies for the target
>booster-static_LIB_DEPENDS:STATIC=general;
>/usr/arm-linux-gnueabi/lib/libpthread.so;
>general;
>/work/eBox/Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.so;general;
>/usr/arm-linux-gnueabi/lib/libdl.so;
>
>
>The pcre lib compiles fine and it is in the right path. 
>Cppcms uses the right path too. 
>Why isn't pcre get linked correctly to booster?
>
>Sab
>
>
>
>------------------------------------------------------------------------------
>Everyone hates slow websites. So do we.
>Make your web apps faster with AppDynamics
>Download AppDynamics Lite for free today:
>http://p.sf.net/sfu/appdyn_d2d_mar
>_______________________________________________
>Cppcms-users mailing list
>Cpp...@li...
>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
>

[Cppcms-users] Booster-PCRE linking

[Sab <rim...@gm...>]

Hi Artyom,

I'm cross compiling my application and having trouble with linking the 
static libraries. CppCMS compiles fine but when I compile my application 
that uses cppcms, I get undefined reference to 'pcre_exec', 'pcre_compile' 
and 'pcre_fullinfo' needed by libbooster.a e.g. in function 
'booster::regex::match'. 

Here is how I compile CppCMS in my environment installer Makefile:

Shared3rdParty/cppcms-1.0.2.target/pub/libcppcms.a: 
	Shared3rdParty/zlib-1.2.7.target/pub/lib/libz.a
	Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.a
	cd Shared3rdParty/cppcms-1.0.2.target && mkdir -p -v build
	cd Shared3rdParty/cppcms-1.0.2.target/build && cmake \
		-DCMAKE_TOOLCHAIN_FILE="../../Product/gcc-${TOOLSUITE}.cmake" \
		-DCMAKE_BUILD_TYPE=MinSizeRel \
		-DDISABLE_ICU_LOCALE=ON \
		-DDISABLE_POSIX_LOCALE=ON \
		-DDISABLE_SHARED=ON \
		-DDISABLE_GCRYPT=ON \
		-DDISABLE_OPENSSL=ON \
		-DDISABLE_OPENSSL=ON \
		-DCMAKE_INSTALL_PREFIX="../pub" \
		-DDISABLE_SCGI=ON \
		-DDISABLE_HTTP=ON \
		-DZLIB_INCLUDE_DIR="../../zlib-1.2.7.target/pub/include" \
		-DZLIB="../../zlib-1.2.7.target/pub/lib/libz.a" \
		-DPCRE_LIB="../../pcre-8.32.target/pub/lib/libpcre.so" \
		-DCMAKE_INCLUDE_PATH="../../pcre-8.32.target/pub/include" \
		-DCMAKE_LIBRARY_PATH="../../pcre-8.32.target/pub/lib" ..
	cd Shared3rdParty/cppcms-1.0.2.target/build && make && make install

Here is how I compile pcre in the same makefile:

Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.a:
	cd Shared3rdParty/pcre-8.32.target && ./configure --host=${TOOLSUITE} 
--prefix=`pwd`/pub 
	cd Shared3rdParty/pcre-8.32.target && make && make install
	

I do see the right path in CppCMS's CMakeCache.txt:

//Path to a library.
PCRE_LIB:FILEPATH=/work/eBox/Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.so

and:

//Dependencies for the target
booster-static_LIB_DEPENDS:STATIC=general;
/usr/arm-linux-gnueabi/lib/libpthread.so;
general;
/work/eBox/Shared3rdParty/pcre-8.32.target/pub/lib/libpcre.so;general;
/usr/arm-linux-gnueabi/lib/libdl.so;


The pcre lib compiles fine and it is in the right path. 
Cppcms uses the right path too. 
Why isn't pcre get linked correctly to booster?

Sab

Re: [Cppcms-users] bandwidth throtteling

[Artyom B. <art...@ya...>]

Once you called async_flush_output you must wait for completion

handler to be executed before you send more data.

For example "state" sse service does exactly the same it sends
only latest data and if some was missed between async_flush_output
and completion handler it would not be sent, but rather latest
data should be.

About buffering,

when you work with async app. the write to response().out()
just collects the data to the buffer, the I/O is performed
uput async_flush_output request and than you can continue
once you receive the notification.

So in general what you described is reasonable approach.

Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/



>________________________________
> From: Markus Raab <us...@ma...>
>To: cpp...@li... 
>Sent: Monday, March 18, 2013 12:54 PM
>Subject: [Cppcms-users] bandwidth throtteling
> 
>Hello!
>
>Is it possible to do some bandwidth throtteling for an asynchronous 
>application?
>
>The idea is to drop some data in a stream using SSE when the connection is 
>too slow to send everything.
>
>What happens if more data is sent with async_flush_output than the client 
>(connection) can handle. Which part of the system does the buffering in this 
>scenario?
>
>A simple approach for the bandwidth throtteling would be to get a number how 
>many async_flush_output are currently unprocessed (e.g. by increment when  
>async_flush_output is registered and decrement inside the callback) - and 
>based on that number - drop data. Is this an useful approach?
>
>best regards
>Markus
>
>
>------------------------------------------------------------------------------
>Everyone hates slow websites. So do we.
>Make your web apps faster with AppDynamics
>Download AppDynamics Lite for free today:
>http://p.sf.net/sfu/appdyn_d2d_mar
>_______________________________________________
>Cppcms-users mailing list
>Cpp...@li...
>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
> Se

[Cppcms-users] bandwidth throtteling

[Markus R. <us...@ma...>]

Hello!

Is it possible to do some bandwidth throtteling for an asynchronous 
application?

The idea is to drop some data in a stream using SSE when the connection is 
too slow to send everything.

What happens if more data is sent with async_flush_output than the client 
(connection) can handle. Which part of the system does the buffering in this 
scenario?

A simple approach for the bandwidth throtteling would be to get a number how 
many async_flush_output are currently unprocessed (e.g. by increment when  
async_flush_output is registered and decrement inside the callback) - and 
based on that number - drop data. Is this an useful approach?

best regards
Markus

Re: [Cppcms-users] Shutdown issue

[Artyom B. <art...@ya...>]

Few points:

a) Enable this option: http://cppcms.com/wikipp/en/page/cppcms_1x_config#service.disable_global_exit_handling to prevent from the CppCMS service to install signal handlers.

b) Put the log between

   m_service->run(); 
    << Log exited 
   delete m_service;

To see if service exits.

And probably put log after the delete to make sure that thread competes.


Note the run() would not exit until all the worker threads had completed the job.

c) Why don't you put cppcms::service on the stack such that it would not leak in case of exception...

you code is not exception safe.
 
Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/



>________________________________
> From: Sab <rim...@gm...>
>To: cpp...@li... 
>Sent: Thursday, March 14, 2013 11:03 AM
>Subject: [Cppcms-users] Shutdown issue
> 
>Hallo Artyom,
>
>I am running the cppcms::service in a separate thread. When the calling thread 
>restarts, I would also like to restart the cppcms service. On restart, I am 
>calling service.shutdown() from the main tread that executed it. I see the log 
>before the shutdown call but the log after the call is not printed -> I suspect 
>that the function hangs up because after the application exits the process still 
>keeps running in the background. This is my application’s main:
>
>(Sorry for the code formatting...)
>
>int main(int argc, char* argv[]) 
>{
>. . .
>do
>{
>. . .
>//start WebService
>WebService webService(connector);
>LOG_MAIN << "Starting the WebService thread";
>boost::thread webServiceThread(webService);
>. . .
>
>while ( system.IsRunning() && !shutdownSignalReceived )
>{
>. . .
>}
>
>//stop WebService
>LOG_MAIN << "Stopping WebService...";
>webService.ShutDown();
>webServiceThread.join();
>. . .
>} 
>while (runState == Common::Devices::Run::Reload);
>}
>
>------------------
>My WebService class:
>
>void WebService::operator()()
>{
>try
>{
>. . .
>m_service = new cppcms::service(serviceConfig);
>m_service->applications_pool().
>mount(cppcms::applications_factory<MainWebHandler>
>(m_connector));
>m_service->run();
>delete m_service;
>}
>catch(std::exception const &e)
>{
>LOG_MAIN_ERROR << "WebService exception: " << e.what();
>}
>}
>
>void WebService::ShutDown()
>{
>try
>{
>cout << "Shutting down WebService..." << endl;
>m_service->shutdown();
>cout << "Shutting down WebService finished." << endl;
>}
>catch(std::exception const &e)
>{
>LOG_MAIN_ERROR << "WebService exception: " << e.what();
>}
>}
>
>---------------
>I don't get any error or exception. As mentioned I see the log “Shutting down 
>WebService...” but I don’t see “Shutting down WebService finished.” that’s why I 
>think the service doesn’t shut down properly so the thread keeps running in the 
>background.
>
>Do you see anything wrong with my code?
>
>Sab
>
>
>
>------------------------------------------------------------------------------
>Everyone hates slow websites. So do we.
>Make your web apps faster with AppDynamics
>Download AppDynamics Lite for free today:
>http://p.sf.net/sfu/appdyn_d2d_mar
>_______________________________________________
>Cppcms-users mailing list
>Cpp...@li...
>https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
>

[Cppcms-users] Shutdown issue

[Sab <rim...@gm...>]

Hallo Artyom,

I am running the cppcms::service in a separate thread. When the calling thread 
restarts, I would also like to restart the cppcms service. On restart, I am 
calling service.shutdown() from the main tread that executed it. I see the log 
before the shutdown call but the log after the call is not printed -> I suspect 
that the function hangs up because after the application exits the process still 
keeps running in the background. This is my application’s main:

(Sorry for the code formatting...)

int main(int argc, char* argv[]) 
{
. . .
do
{
. . .
//start WebService
WebService webService(connector);
LOG_MAIN << "Starting the WebService thread";
boost::thread webServiceThread(webService);
. . .

while ( system.IsRunning() && !shutdownSignalReceived )
{
. . .
}

//stop WebService
LOG_MAIN << "Stopping WebService...";
webService.ShutDown();
webServiceThread.join();
. . .
} 
while (runState == Common::Devices::Run::Reload);
}

------------------
My WebService class:

void WebService::operator()()
{
try
{
. . .
m_service = new cppcms::service(serviceConfig);
m_service->applications_pool().
mount(cppcms::applications_factory<MainWebHandler>
(m_connector));
m_service->run();
delete m_service;
}
catch(std::exception const &e)
{
LOG_MAIN_ERROR << "WebService exception: " << e.what();
}
}

void WebService::ShutDown()
{
try
{
cout << "Shutting down WebService..." << endl;
m_service->shutdown();
cout << "Shutting down WebService finished." << endl;
}
catch(std::exception const &e)
{
LOG_MAIN_ERROR << "WebService exception: " << e.what();
}
}

---------------
I don't get any error or exception. As mentioned I see the log “Shutting down 
WebService...” but I don’t see “Shutting down WebService finished.” that’s why I 
think the service doesn’t shut down properly so the thread keeps running in the 
background.

Do you see anything wrong with my code?

Sab
 

Re: [Cppcms-users] Consumer producer problem

[Christian G. <chr...@gm...>]

--
Christian Gmeiner, MSc


2013/3/13 Artyom Beilis <art...@ya...>:
>>
>
>> [...]
>> In the end it looks like I have a one process <----> one browser session
>> connection.
>>
>
>> [...]
>> --
>> Christian Gmeiner, MSc
>
>
> So why not just use?
>
>    // disable gzip to allow transparent flush
>    response().io_mode(nogzip);
>    // something like that not remember exactly the content type and API
>    response().content_type("text/event-stream");
>
>
>    while(get_some_input_from_process()!=EOF) {
>
>      sse::write_event(response().out(),id,data);
>
>      response().out() << std::flush;
>    }
>

Are you talking about the response() of the JSON-RPC call? If yes then
I don't need
SSE.

thanks
--
Christian Gmeiner, MSc

Re: [Cppcms-users] One SSE stream per session

[Christian G. <chr...@gm...>]

2013/3/13 Artyom Beilis <art...@ya...>:
>
>
>>________________________________
>> From: Christian Gmeiner <chr...@gm...>
>>
>>Hi all.
>>
>>at the moment I am looking how to have one SSE stream per session. In
>>general I would put the SSE stream object into the session storage but
>>I am not sure how I could access it from other cppcms-apps.
>>
>
> I don't really understand what exactly you are doing as you can't
> store "sse" object in session storage as is - you can probably
> create its unique identification and store it in session object.
>

Maybe I could "store it" in kind of pointer address :)

>
>>Here is brief overview:
>>
>>I have an SSE application mounted at /sse/
>>
>>        booster::intrusive_ptr<SSE> sse = new SSE(srv);
>>        Configuration::instance()->sse = sse.get();
>>        srv.applications_pool().mount(sse, cppcms::mount_point("/sse(.*)", 1));
>>
>>It is possible for all other cppcms apps running to send data via SSE
>>to the browser.
>>
>>        Configuration::instance()->sse->enqueue("event", "data");
>>
>>
>
>>class SSE : public cppcms::application  {
>>public:
>>    SSE(cppcms::service &srv);
>>
>>    void post();
>>    void get();
>>    void redirect();
>>
>>    void enqueue(std::string const &event, std::string const &data);
>>
>>private:
>>    booster::shared_ptr<sse::bounded_event_queue> stream_;
>>};
>>
>
>
> Yes, however you should remember that SEE object is asynchronous one
> and runs in the event loop stream.
>
> That means that you can't just call sse->enqueue("event","data")
> from synchronous applications from the other thread
> but rather you need to "post" it to event loop's thread:
>
> Using C++11 lambdass:
>
>    std::string message = ...;
>    service().post([=]() {
>          see->equeue("event",message);
>    });
>

okay got it...

>>
>
>>
>>At the moment all clients are using the same SSE stream and this is a
>>no-go security wise.
>>So I want to extend my current approach to have multiple SSE streams -
>>for each session one.
>>As this "website" is used for embedded device configuration I think I
>>will not run into any
>>resource problems.
>>
>>Or is there a better way to deliver each session their own data-packages only?
>
>
> Use multiple sse stream object and for each one of them create unique id that
> would be stored in session and thus if user request sse with some id
> give it to him, this way you can even give multiple users same sse stream.
>

Sounds like a good idea. The only thing that comes to my mind is: where to store
the hashmap from key to sse object? Is there a concept for global
available data?
In theory a singleton class/object should to it.

thanks
--
Christian Gmeiner, MSc

Re: [Cppcms-users] URI selection?, double call??

[Juan C. B. <jcb...@gm...>]

Thanks mate!
Moving on to the dispatcher as I type this.
Cheers,
jcb!

On Wed, Mar 13, 2013 at 1:52 PM, Lee Elenbaas <lee...@gm...> wrote:
> The server listens on port 8080 - and will respond to any http request to
> that port
> the configuration provides the starting point for the URL dispatcher to find
> and call the correct code in your application in normal use - the hello
> world sample however does not use the dispatcher and instead bypass it by
> always returning the same response
>
> so you get this behavior - move on to work with app hierarchies and you will
> see much more use for that configuration
>
>
> On Wed, Mar 13, 2013 at 1:40 PM, Juan Carlos Borrás <jcb...@gm...>
> wrote:
>>
>> Hi all,
>> Two behaviours I don't understand in the hello_world example:
>>
>> 1) It looks like any URI under localhost:8080 will be responded upon
>> while config.js explicitly states that the application lives under
>> localhost:8080/hello. Bug, feature or PBKAC?
>>
>> 2) As a lame execution tracking I dump to std::cout Boost's
>> local_time() but I get two printed lines by server request (as if
>> my_hello_world:main() would be called twice). Again, bug, feature or
>> or PBKAC?
>>
>> Code replicating the effect at: https://github.com/jcborras/cppwap
>> Clone with git clone gi...@gi...:jcborras/cppwap.git (but you all know
>> that)
>>
>> Thanks in advance,
>> Cheers,
>> jcb!
>>
>>
>> ------------------------------------------------------------------------------
>> Everyone hates slow websites. So do we.
>> Make your web apps faster with AppDynamics
>> Download AppDynamics Lite for free today:
>> http://p.sf.net/sfu/appdyn_d2d_mar
>> _______________________________________________
>> Cppcms-users mailing list
>> Cpp...@li...
>> https://lists.sourceforge.net/lists/listinfo/cppcms-users
>
>
>
>
> --
> --
> lee
> Lee Elenbaas
> lee...@gm...
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_mar
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users
>



--
Cheers,
jcb!
_______________________
http://twitter.com/jcborras

Re: [Cppcms-users] URI selection?, double call??

[Lee E. <lee...@gm...>]

The server listens on port 8080 - and will respond to any http request to
that port
the configuration provides the starting point for the URL dispatcher to
find and call the correct code in your application in normal use - the
hello world sample however does not use the dispatcher and instead bypass
it by always returning the same response

so you get this behavior - move on to work with app hierarchies and you
will see much more use for that configuration


On Wed, Mar 13, 2013 at 1:40 PM, Juan Carlos Borrás <jcb...@gm...>wrote:

> Hi all,
> Two behaviours I don't understand in the hello_world example:
>
> 1) It looks like any URI under localhost:8080 will be responded upon
> while config.js explicitly states that the application lives under
> localhost:8080/hello. Bug, feature or PBKAC?
>
> 2) As a lame execution tracking I dump to std::cout Boost's
> local_time() but I get two printed lines by server request (as if
> my_hello_world:main() would be called twice). Again, bug, feature or
> or PBKAC?
>
> Code replicating the effect at: https://github.com/jcborras/cppwap
> Clone with git clone gi...@gi...:jcborras/cppwap.git (but you all know
> that)
>
> Thanks in advance,
> Cheers,
> jcb!
>
>
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_mar
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users
>



-- 
--
lee
Lee Elenbaas
lee...@gm...

[Cppcms-users] URI selection?, double call??

[Juan C. B. <jcb...@gm...>]

Hi all,
Two behaviours I don't understand in the hello_world example:

1) It looks like any URI under localhost:8080 will be responded upon
while config.js explicitly states that the application lives under
localhost:8080/hello. Bug, feature or PBKAC?

2) As a lame execution tracking I dump to std::cout Boost's
local_time() but I get two printed lines by server request (as if
my_hello_world:main() would be called twice). Again, bug, feature or
or PBKAC?

Code replicating the effect at: https://github.com/jcborras/cppwap
Clone with git clone gi...@gi...:jcborras/cppwap.git (but you all know that)

Thanks in advance,
Cheers,
jcb!

Re: [Cppcms-users] Consumer producer problem

[Artyom B. <art...@ya...>]

> 

> [...]
> In the end it looks like I have a one process <----> one browser session
> connection.
> 

> [...]
> --
> Christian Gmeiner, MSc


So why not just use?

   // disable gzip to allow transparent flush
   response().io_mode(nogzip);
   // something like that not remember exactly the content type and API
   response().content_type("text/event-stream");


   while(get_some_input_from_process()!=EOF) {

     sse::write_event(response().out(),id,data);

     response().out() << std::flush;
   }

 
Of course there is no way to reconnect and see what happens if you closed
the session...

Artyom Beilis
--------------
CppCMS - C++ Web Framework:   http://cppcms.com/
CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/


>> 
>>  However if you have many clients and one source it is
>>  a different problem.
>> 
>>  What happens if somebody requests the date after the process
>>  started do you send it from the begging or from the last
>>  event? What happens on reconnect you need to start from the
>>  last point but how long do you want to keep the data?
>> 
>>  In general you can keep all events - all the output
>> 
>>  and send it from the beginning, in such case you can either
>>  make the ring buffer with a big ring or just alter
>>  the class
>> 
>> 
> http://sourceforge.net/p/cppcms/code/2238/tree/framework/trunk/contrib/server_side/sse/server_sent_events.h
>> 
>> 
>>  With the policy you need. You can derive the event_source
>>  class and provide the policy of keeping and throwing messages
>>  the way you need.
>> 
> 
> As far as I have seen the broadcast() method in the event_source class
> works asynchronously. So I need
> to change it to be synchronously.
> 
> 
> ------------------------------------------------------------------------------
> Everyone hates slow websites. So do we.
> Make your web apps faster with AppDynamics
> Download AppDynamics Lite for free today:
> http://p.sf.net/sfu/appdyn_d2d_mar
> _______________________________________________
> Cppcms-users mailing list
> Cpp...@li...
> https://lists.sourceforge.net/lists/listinfo/cppcms-users
> 

Re: [Cppcms-users] Consumer producer problem

[Christian G. <chr...@gm...>]

2013/3/13 Artyom Beilis <art...@ya...>:
> ----- Original Message -----
>
>> From: Christian Gmeiner <chr...@gm...>
>> To: cpp...@li...
>> Cc:
>> Sent: Wednesday, March 13, 2013 10:39 AM
>> Subject: [Cppcms-users] Consumer producer problem
>>
>> Hi all.
>>
>> I am using a QProcess to run an external bash script. There I am
>> reading back stdout and stderr and
>> send these information back to the client via SSE.
>> Now I run into the problem that QProcess produces messages a lot
>> faster then they got send via SSE.
>> The result is that the used ring-buffer has overwritten elements and
>> not all "messages" got send.
>> At the moment I am using the sse::bounded_event_queue.
>>
>> Is there anything I need to take of when using condition_variable?
>
>> What is the best approach to get
>> this handled in a clean way?
>>
>> thanks
>
>> --
>> Christian Gmeiner, MSc
>>
>
> First of all how they are generated, synchronously, asynchronously?
> how many client do you have?
>

I have a JSON RPC server running:

        srv.applications_pool().mount(cppcms::applications_factory<TsswService>(),
cppcms::mount_point("/rpc(.*)", 1));

with a run
        bind("run", cppcms::rpc::json_method(&TsswService::run, this),
method_role);

Where run looks something like:

void TsswService::run(std::string key, std::string parameter)
{
    BOOSTER_DEBUG("TsswService") << __func__ << " key: '" << key << "'
parameter: '" << parameter << "'";

    // valid session needed
    if (!session().is_set("user"))
    {
        BOOSTER_DEBUG("TsswService") << __func__ << ": not logged in";
        return;
    }

   ... validate input ...


   ... start process ...

        Process *p = new Process();
        BOOSTER_DEBUG("TsswService") << __func__ << " cmd: '" <<
file.absoluteFilePath().toStdString()
                << "' parameter: '" << pList.join(" ").toStdString() << "'";
        int exit = p->exec(file.absoluteFilePath(), pList, &out, &err, true);
        delete p;

        BOOSTER_DEBUG("TsswService") << __func__ << out.toStdString();
        BOOSTER_DEBUG("TsswService") << __func__ << err.toStdString();
        BOOSTER_DEBUG("TsswService") << __func__ << exit;
}


>From the browser I am calling the run RPC with key "do_backup" which
start a new process on the
server (/opt/scripts/run_full_backup.sh). stdout and stderr are "send"
via SSE to the browser.

So I would say that the messages are generated synchronously - or I am wrong?

BTW: /opt/scripts/run_full_backup.sh check if its called multiple
times and prints on stderr
a warning.

>
> In general it is not a producer consumer problem
> but rather message broker problem.
>
> if you have 1 client and 1 source you don't really need an
> asynchronous SSE... you can do it easily synchronously as
> in any case process itself is synchronous.
>

In the end it looks like I have a one process <----> one browser session
connection.

>
> However if you have many clients and one source it is
> a different problem.
>
> What happens if somebody requests the date after the process
> started do you send it from the begging or from the last
> event? What happens on reconnect you need to start from the
> last point but how long do you want to keep the data?
>
> In general you can keep all events - all the output
>
> and send it from the beginning, in such case you can either
> make the ring buffer with a big ring or just alter
> the class
>
> http://sourceforge.net/p/cppcms/code/2238/tree/framework/trunk/contrib/server_side/sse/server_sent_events.h
>
>
> With the policy you need. You can derive the event_source
> class and provide the policy of keeping and throwing messages
> the way you need.
>

As far as I have seen the broadcast() method in the event_source class
works asynchronously. So I need
to change it to be synchronously.

--
Christian Gmeiner, MSc