Thread: [Cppcms-users] 100% cpu usage
Brought to you by:
artyom-beilis
Menu
▾
▴
cppcms-users
|
From: Marcel H. <ke...@co...> - 2013-07-26 22:18:19
Attachments:
signature.asc
perf.tar.bz2
|
Hi Artyom, Sometimes cppcms gets 100% cpu load. I don't know what triggers this behavoir and I don't know how to invegstigate that problem. Maybe you can tell me, what to do, to make a bug report for this. For now I made a perf record and attached it (about 10s of record). I hope you can open it :/ Regards |
|
From: Artyom B. <art...@ya...> - 2013-07-28 09:36:32
|
You need to be little bit more specific. - Try to get debugger trace when it gets 100% - Try to see strace output when it gets to the 100% Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/ >________________________________ > From: Marcel Hellwig <ke...@co...> >To: Artyom Beilis <art...@ya...> >Cc: cpp...@li... >Sent: Saturday, July 27, 2013 1:18 AM >Subject: 100% cpu usage > > >Hi Artyom, > >Sometimes cppcms gets 100% cpu load. I don't know what triggers this >behavoir and I don't know how to invegstigate that problem. >Maybe you can tell me, what to do, to make a bug report for this. >For now I made a perf record and attached it (about 10s of record). >I hope you can open it :/ > >Regards > > > |
|
From: Petr J. <ele...@ex...> - 2013-07-28 22:10:46
Attachments:
signature.asc
|
Is this happening in your async app? I experienced something similar, but I only just started writing an async app. Connection c(release_context()) set.insert(c); Connection class contains booster::shared_ptr<cppcms::http::context> as a member. I've opened up browser and went to the URL that runs the above code, got a white screen (which i suspect is expected). I'm using clang, and have c++11 enabled. After about 10 seconds, the CPU usage climbed to 100%. I haven't yet been really bothered by it yet (only just started designing the app, doing some tests). Will see when I get back to writing code. On 27/07/2013 8:18 AM, Marcel Hellwig wrote: > Hi Artyom, > > Sometimes cppcms gets 100% cpu load. I don't know what triggers this > behavoir and I don't know how to invegstigate that problem. > Maybe you can tell me, what to do, to make a bug report for this. > For now I made a perf record and attached it (about 10s of record). > I hope you can open it :/ > > Regards > > > > ------------------------------------------------------------------------------ > See everything from the browser to the database with AppDynamics > Get end-to-end visibility with application monitoring from AppDynamics > Isolate bottlenecks and diagnose root cause in seconds. > Start your free trial of AppDynamics Pro today! > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Cppcms-users mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cppcms-users > -- Please use PGP to encrypt your email to ensure our privacy is respected. |
|
From: Marcel H. <ke...@co...> - 2013-07-29 06:45:28
Attachments:
signature.asc
|
On 29.07.2013 00:10, Petr Janda wrote: > Is this happening in your async app? I experienced something similar, > but I only just started writing an async app. Yo. > I'm using clang, and have c++11 enabled. Both for me too, but only the app is c++11, not cppcms itself. > After about 10 seconds, the CPU usage climbed to 100%. Hmm. 10s is much too less and it does not happen every time. > I haven't yet been really bothered by it yet (only just started > designing the app, doing some tests). > > Will see when I get back to writing code. Would be great if you can trigger that problem and know what it causes. |
|
From: le ba <hma...@gm...> - 2013-07-30 16:32:50
|
Hi Marcel, Just some question, did you try with other compilator and did you have the same problem? I never had this problem before (with mingw 4.7.2) and it would be very interesting to know the reasons or the causes. Thanks 2013/7/29 Marcel Hellwig <ke...@co...> > On 29.07.2013 00:10, Petr Janda wrote: > > Is this happening in your async app? I experienced something similar, > > but I only just started writing an async app. > Yo. > > I'm using clang, and have c++11 enabled. > Both for me too, but only the app is c++11, not cppcms itself. > > After about 10 seconds, the CPU usage climbed to 100%. > Hmm. 10s is much too less and it does not happen every time. > > I haven't yet been really bothered by it yet (only just started > > designing the app, doing some tests). > > > > Will see when I get back to writing code. > Would be great if you can trigger that problem and know what it causes. > > > > > ------------------------------------------------------------------------------ > See everything from the browser to the database with AppDynamics > Get end-to-end visibility with application monitoring from AppDynamics > Isolate bottlenecks and diagnose root cause in seconds. > Start your free trial of AppDynamics Pro today! > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk > _______________________________________________ > Cppcms-users mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cppcms-users > > |
|
From: Marcel H. <ke...@co...> - 2013-07-31 07:48:15
Attachments:
signature.asc
|
On 30.07.2013 18:32, le ba wrote: > Hi Marcel, > > Just some question, did you try with other compilator and did you have > the same problem? No, I hadn't tried it yet. > I never had this problem before (with mingw 4.7.2) and it would be > very interesting to know the reasons or the causes. > > Thanks > > > > 2013/7/29 Marcel Hellwig <ke...@co... <mailto:ke...@co...>> > > On 29.07.2013 00:10, Petr Janda wrote: > > Is this happening in your async app? I experienced something > similar, > > but I only just started writing an async app. > Yo. > > I'm using clang, and have c++11 enabled. > Both for me too, but only the app is c++11, not cppcms itself. > > After about 10 seconds, the CPU usage climbed to 100%. > Hmm. 10s is much too less and it does not happen every time. > > I haven't yet been really bothered by it yet (only just started > > designing the app, doing some tests). > > > > Will see when I get back to writing code. > Would be great if you can trigger that problem and know what it > causes. > > > > ------------------------------------------------------------------------------ > See everything from the browser to the database with AppDynamics > Get end-to-end visibility with application monitoring from AppDynamics > Isolate bottlenecks and diagnose root cause in seconds. > Start your free trial of AppDynamics Pro today! > http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk > _______________________________________________ > Cppcms-users mailing list > Cpp...@li... > <mailto:Cpp...@li...> > https://lists.sourceforge.net/lists/listinfo/cppcms-users > > > > > ------------------------------------------------------------------------------ > Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk > > > _______________________________________________ > Cppcms-users mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cppcms-users |
|
From: Artyom B. <art...@ya...> - 2013-07-31 07:55:21
|
Guys To try to figure out what happens I really need a test case that reproduces the problem to see what happens, or somebody would join the process that uses 100% CPU with a debugger and bring some stack trace and see what is the loop that happens there Without it I can't tell what is the problem. It also may be incorrect use of API. I had already seen some problems that were reported that actually was API abuse. It is especially gentle in case of multi-threading and asynchronous programming. Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/ >________________________________ > From: Marcel Hellwig <ke...@co...> >To: cpp...@li... >Sent: Wednesday, July 31, 2013 10:48 AM >Subject: Re: [Cppcms-users] 100% cpu usage > > > >On 30.07.2013 18:32, le ba wrote: > >Hi Marcel, >> >> >>Just some question, did you try with other compilator and did you have the same problem? No, I hadn't tried it yet. > >I never had this problem before (with mingw 4.7.2) and it would be very interesting to know the reasons or the causes. >> >> >>Thanks >> >> >> >> >> >>2013/7/29 Marcel Hellwig <ke...@co...> >> >>On 29.07.2013 00:10, Petr Janda wrote: >>>> Is this happening in your async app? I experienced something similar, >>>> but I only just started writing an async app. >>> Yo. >>> >>>> I'm using clang, and have c++11 enabled. >>> Both for me too, but only the app is c++11, not cppcms itself. >>> >>>> After about 10 seconds, the CPU usage climbed to 100%. >>> Hmm. 10s is much too less and it does not happen every time. >>> >>>> I haven't yet been really bothered by it yet (only just started >>>> designing the app, doing some tests). >>>> >>>> Will see when I get back to writing code. >>> Would be great if you can trigger that problem and know what it causes. >>> >>> >>> >>>------------------------------------------------------------------------------ >>>See everything from the browser to the database with AppDynamics >>>Get end-to-end visibility with application monitoring from AppDynamics >>>Isolate bottlenecks and diagnose root cause in seconds. >>>Start your free trial of AppDynamics Pro today! >>>http://pubads.g.doubleclick.net/gampad/clk?id=48808831&iu=/4140/ostg.clktrk >>>_______________________________________________ >>>Cppcms-users mailing list >>>Cpp...@li... >>>https://lists.sourceforge.net/lists/listinfo/cppcms-users >>> >>> >> >> >> >>------------------------------------------------------------------------------ Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk >> >> >>_______________________________________________ Cppcms-users mailing list Cpp...@li... https://lists.sourceforge.net/lists/listinfo/cppcms-users > >------------------------------------------------------------------------------ >Get your SQL database under version control now! >Version control is standard for application code, but databases havent >caught up. So what steps can you take to put your SQL databases under >version control? Why should you start doing it? Read more to find out. >http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk >_______________________________________________ >Cppcms-users mailing list >Cpp...@li... >https://lists.sourceforge.net/lists/listinfo/cppcms-users > > > |
|
From: Marcel H. <ke...@co...> - 2013-07-31 18:31:37
Attachments:
signature.asc
|
On 31.07.2013 09:55, Artyom Beilis wrote: > Guys > > To try to figure out what happens I really need a test case that > reproduces the problem > to see what happens, or somebody would join the process that uses 100% > CPU with a debugger > and bring some stack trace and see what is the loop that happens there The source code is here: http://cookiesoft.de/git/?p=spaceio.git I still don't have a strace, but I will provide one as soon as I get time. I know what happens when cppcms gets 100%. There are 19 threads stuck at thread_pool.cpp in line 114 and always do polling, so they don't wait, the always poll. I think there is some kind of racecondition, but I don't know what kind :/ Hope you will fix it. |
|
From: Artyom B. <art...@ya...> - 2013-07-31 18:56:26
|
1. how do you trigger the loop? 2. I don't understand why are you using asynchronous application for the WebAPI. You don't do anything asynchronous. More than that calling blocking SQL from event loop is BAD idea. also is there any way to make it with less dependencies so I can build it Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/ >________________________________ > From: Marcel Hellwig <ke...@co...> >To: cpp...@li... >Sent: Wednesday, July 31, 2013 9:31 PM >Subject: Re: [Cppcms-users] 100% cpu usage > > > >On 31.07.2013 09:55, Artyom Beilis wrote: > >Guys >> >> >>To try to figure out what happens I really need a test case that reproduces the problem >>to see what happens, or somebody would join the process that uses 100% CPU with a debugger >>and bring some stack trace and see what is the loop that happens there The source code is here: >http://cookiesoft.de/git/?p=spaceio.git > >I still don't have a strace, but I will provide one as soon as I get time. >I know what happens when cppcms gets 100%. >There are 19 threads stuck at thread_pool.cpp in line 114 and always do polling, so they don't wait, the always poll. I think there is some kind of racecondition, but I don't know what kind :/ > >Hope you will fix it. > >------------------------------------------------------------------------------ >Get your SQL database under version control now! >Version control is standard for application code, but databases havent >caught up. So what steps can you take to put your SQL databases under >version control? Why should you start doing it? Read more to find out. >http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk >_______________________________________________ >Cppcms-users mailing list >Cpp...@li... >https://lists.sourceforge.net/lists/listinfo/cppcms-users > > > |
|
From: Marcel H. <ke...@co...> - 2013-07-31 19:16:40
Attachments:
signature.asc
|
On 31.07.2013 20:56, Artyom Beilis wrote: > 1. how do you trigger the loop? which loop? the thread_pool loop? I don't know ;) > 2. I don't understand why are you using asynchronous application for > the WebAPI. You don't do anything asynchronous. * the json_rpc_chat is also async * you wrote that an async handler is able to handle more requests at one than sync one can and it will be designed that the webapi will provide the whole logic for the game, so you can also do rpc request from desktop applications. * async is cool > More than that calling blocking SQL from event loop is BAD idea. Hmmm, how to fix it? > > also is there any way to make it with less dependencies so I can build it Hmmm, there aren't many dependencies :/ just cppcms/cppdb/cryptopp and a self made logger framework logre. logre can also be found at the git page i provided. I also have a logre-less version for you. http://cookiesoft.de/sp.tar.bz2 |
|
From: Petr J. <ele...@ex...> - 2013-07-31 21:55:40
Attachments:
signature.asc
|
I had a quick look over your code, and I also don't understand why you are using async, when all your methods operating synchronously, and block. If you are blocking in an async app, you're blocking the thread pool. Any heavy operation must be processed in the thread-pool using thread_pool::post() method. Petr On 1/08/2013 5:16 AM, Marcel Hellwig wrote: > On 31.07.2013 20:56, Artyom Beilis wrote: >> 1. how do you trigger the loop? > which loop? the thread_pool loop? I don't know ;) >> 2. I don't understand why are you using asynchronous application for >> the WebAPI. You don't do anything asynchronous. > * the json_rpc_chat is also async > * you wrote that an async handler is able to handle more requests at > one than sync one can and it will be designed that the webapi will > provide the whole logic for the game, so you can also do rpc request > from desktop applications. > * async is cool >> More than that calling blocking SQL from event loop is BAD idea. > Hmmm, how to fix it? >> >> also is there any way to make it with less dependencies so I can build it > Hmmm, there aren't many dependencies :/ just cppcms/cppdb/cryptopp and a > self made logger framework logre. > logre can also be found at the git page i provided. > > I also have a logre-less version for you. > http://cookiesoft.de/sp.tar.bz2 > > > > > > > ------------------------------------------------------------------------------ > Get your SQL database under version control now! > Version control is standard for application code, but databases havent > caught up. So what steps can you take to put your SQL databases under > version control? Why should you start doing it? Read more to find out. > http://pubads.g.doubleclick.net/gampad/clk?id=49501711&iu=/4140/ostg.clktrk > > > > _______________________________________________ > Cppcms-users mailing list > Cpp...@li... > https://lists.sourceforge.net/lists/listinfo/cppcms-users > -- Please use PGP to encrypt your email to ensure our privacy is respected. |
|
From: <ele...@ex...> - 2013-08-02 06:05:12
|
Marcel,
Here is an example how you could design your asynchronous application
void nonblocking_function1()
{
// release_context() is the key
booster::shared_ptr<cppcms::http::context> ctx = release_context();
connections_.insert(ctx);
}
void nonblocking_function2()
{
srv_.thread_pool().post([=]() {
//do some heavy work like sql select
cppdb::result r = ....;
for(auto conn : connections_) {
// write out response to clients directly or add to a // queue
}
})
}
Depending on what you need to do, this might not work for you.
Artyom,
Is cppcms::http::response and cppcms::http::context thread safe?
I'm trying to decide, in the design of my app, whether I should be writing
to response().out() from the thread pool or whether I should be queuing
messages and sending them out from a single thread.
Cheers,
Petr
|
|
From: Artyom B. <art...@ya...> - 2013-08-02 07:31:01
|
> >Artyom, > >Is cppcms::http::response and cppcms::http::context thread safe? >I'm trying to decide, in the design of my app, whether I should be writing >to response().out() from the thread pool or whether I should be queuing >messages and sending them out from a single thread. > >Cheers, >Petr > No, THEY ARE NOT. http://cppcms.com/wikipp/en/page/thread_safety In async app you need to use response from the event loop and in synch app from the thread executing the request. If you want to do something in thread pool post a job to the thread pool, from the pool post a job to event loop with the results and **than** use the response(). Artyom Beilis -------------- CppCMS - C++ Web Framework: http://cppcms.com/ CppDB - C++ SQL Connectivity: http://cppcms.com/sql/cppdb/ |
|
From: Marcel H. <ke...@co...> - 2013-07-31 22:01:53
Attachments:
signature.asc
|
On 31.07.2013 23:55, Petr Janda wrote: > I had a quick look over your code, and I also don't understand why you > are using async, when all your methods operating synchronously, and block. Until now... I said this is stub only until now. I could split them into sync and async operations and provide two different classes for them. This code is totaly under development and still in early pre-alpha ;) > If you are blocking in an async app, you're blocking the thread pool. > Any heavy operation must be processed in the thread-pool using > thread_pool::post() method. So this means, one thread for all async apps, but multiple workers for the thread pool? How is the async app connected to the thread pool? Why does it block it? That makes no sense to me :/ But yes, I will regard your answers and will revise my code |
|
From: Petr J. <ele...@ex...> - 2013-07-31 23:26:32
Attachments:
signature.asc
|
> Until now... I said this is stub only until now. I could split them into > sync and async operations and provide two different classes for them. > This code is totaly under development and still in early pre-alpha ;) Generally, an async app is designed to handle multiple connections, and that's it. It's used when you need to push data to multiple clients at the same time. ie. stock market price update. > So this means, one thread for all async apps, but multiple workers for > the thread pool? > How is the async app connected to the thread pool? Why does it block it? > That makes no sense to me :/ I'm going to summarize the differences between synchronous, asynchronous, event loop, and thread pool here. Synchronous operation = request in -> response out Asynchronous operation = connection in -> non-blocking wait -> response out when data is ready The key is non-blocking wait. You can have thousands of connections in non-blocking wait, all waiting for a single information/data to be processed. When the data is ready, it's sent out at the same time to all, or some clients. Event loop is the central dispatch point of cppcms. When a connection comes in, it goes to the event loop first. The even loop is really just a loop. If you are blocking in the event loop, you are blocking the whole application. If the application is synchronous (using the application_factory class), it is immediately dispatched from the event loop to the thread pool, therefore it doesn't block. (thread_pool::post() is a non-blocking operation, and it returns immediately) If the application is asynchronous, it's using booster::intrusive_ptr class. This application is running inside the event loop, and therefore must NOT block. Communication between the event loop and the thread pool is facilitated using thread_pool::post() and service::post() methods. They take a functor as an argument. Petr |
