Thread: [Sqlrelay-discussion] Busy listeners counter patch
Brought to you by:
mused
|
From: Renat S. <sr...@st...> - 2010-05-18 11:52:59
Attachments:
13_busy_listeners_counter.patch
|
Hello David and others! I've discovered that busy listeners counter (semaphore #10) is not decreased when auth error occures. Look at the sqlrlistener::clientSession. Counter is going to be decremented in handOffClient, but if authstatus!=0 then counter gets corrupted (is never decremented). As a result listener start forking on every query, which significally decrease the speed and raise system resources consumption. -- Renat Sabitov e-mail: sr...@st... Stack Soft jid: sr...@ja... |
|
From: Cal H. <ca...@fb...> - 2010-05-18 15:25:09
|
Renat, Are your patches against the 0.41 release, or the CVS code? I'd like to give a few of these a try. Thanks, --Cal On Tue, May 18, 2010 at 6:52 AM, Renat Sabitov <sr...@st...> wrote: > Hello David and others! > > I've discovered that busy listeners counter (semaphore #10) is not > decreased when auth error occures. > > Look at the sqlrlistener::clientSession. Counter is going to be decremented > in handOffClient, but if authstatus!=0 then counter gets corrupted (is never > decremented). > > As a result listener start forking on every query, which significally > decrease the speed and raise system resources consumption. > > -- > Renat Sabitov e-mail: sr...@st... > Stack Soft jid: sr...@ja... > > > ------------------------------------------------------------------------------ > > > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > |
|
From: Cal H. <ca...@fb...> - 2010-05-18 19:30:24
|
Oh ok, I picked just the status_more_info patch and applied that, I had compilation errors with trying to access a private member in rudiments. I'm also working up a patch of my own, to do dynamically allocated cursors. I tried to compile the CVS version of relay, but realized it needed a new function in the CVS version of rudiments, which wouldn't compile. I gave up on that venture. A few dusty corners of my web application consume a lot of statement handles, so as a result I needed to crank up my cursors to 1320 in the config file. Each server side connection consumed around 50 - 100MB of memory. Multiply that by around 200 concurrent connections on a machine with 16GB of memory results in bad behavior. By dropping the initial cursors array to 10, each connection only consumes around 14MB of memory. I have an initial test running now, using malloc() and realloc() to resize the sqlrcursor_svr array and it seems to be operating correctly. Before I go any futher though, has anyone already implemented this? Or do you guys foresee any problems with what I'm trying to do? Thanks, --Cal On Tue, May 18, 2010 at 2:19 PM, Renat Sabitov <sr...@st...> wrote: > 18.05.2010 19:24, Cal Heldenbrand пишет: > > Are your patches against the 0.41 release, or the CVS code? I'd like to > give a few of these a try. > > They are against the 0.41, but some of them depends on others so you > can't just apply the latest to the original 0.41 code. > > I have posted all patches to this list, but it might be better to get them > from this location: > > http://www.srr.pp.ru/www/sqlrelay/ > > Be aware that 05_exit_on_broken_socket.patch requires patch to rudiments ( > poll.FULL.patch.gz<http://www.srr.pp.ru/www/sqlrelay/rudiments-0.32/poll.FULL.patch.gz> > ) > > > > > ------------------------------------------------------------------------------ > > > _______________________________________________ > Sqlrelay-discussion mailing list > Sql...@li... > https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion > > |
|
From: Cal H. <ca...@fb...> - 2010-05-18 21:04:24
|
On Tue, May 18, 2010 at 3:40 PM, Renat Sabitov <sr...@st...> wrote:
> 18.05.2010 23:29, Cal Heldenbrand пишет:
> > Oh ok, I picked just the status_more_info patch and applied that, I
> > had compilation errors with trying to access a private member in
> > rudiments.
>
> Hm. Neither this patch nor others require private members in rudiments
> (I use 0.32 version). I'll try to apply my patches against the clean
> sqlrelay-0.41 tomorrow and test it for compiling.
>
I can't seem to find the error in my console history... it had to do with
this line:
for (int i=0; i< SEM_COUNT; i++) {
sem[i] = conn->semset->getValue(i);
}
Saying that "semset" is a private data member of that object. I've already
rolled back my source code, but I can try it later after I'm done with the
cursor stuff.
> > Before I go any futher though, has anyone already implemented this?
> > Or do you guys foresee any problems with what I'm trying to do?
> No, I suppose nobody have tried to do this. As for me, in our
> application we use no more than 2 cursors in a moment.
>
Yeah, we usually have at most 5 statements allocated, but it's difficult to
force programmers to close stuff manually in a garbage collected
environment. As a result, we have some pages that go crazy and allocate
hundreds, sometimes thousands of cursor objects. The number of lines of
high level web code is so large, that it's just easier to handle this
management in lower layers.
I'm almost done with the patch, just a few things to clean up and I'll
attach it to an email to the list.
Thanks,
--Cal
> ------------------------------------------------------------------------------
>
> _______________________________________________
> Sqlrelay-discussion mailing list
> Sql...@li...
> https://lists.sourceforge.net/lists/listinfo/sqlrelay-discussion
>
|
|
From: Renat S. <sr...@st...> - 2010-05-19 05:17:35
|
19.05.2010 01:03, Cal Heldenbrand пишет:
> I can't seem to find the error in my console history... it had to do
> with this line:
>
> for (int i=0; i< SEM_COUNT; i++) {
> sem[i] = conn->semset->getValue(i);
> }
>
> Saying that "semset" is a private data member of that object. I've
> already rolled back my source code, but I can try it later after I'm
> done with the cursor stuff.
Ok, now I see. semset became public since 03_statistics_mutex.patch, and
not in rudiments but in sqlrelay-0.41/src/connection/sqlrconnection.h
I needed semaphores in public to control access to statistics shm.
--
Renat
|
|
From: Renat S. <sr...@st...> - 2010-05-18 19:17:18
|
18.05.2010 19:24, Cal Heldenbrand ?????: > Are your patches against the 0.41 release, or the CVS code? I'd like > to give a few of these a try. > They are against the 0.41, but some of them depends on others so you can't just apply the latest to the original 0.41 code. I have posted all patches to this list, but it might be better to get them from this location: http://www.srr.pp.ru/www/sqlrelay/ Be aware that 05_exit_on_broken_socket.patch requires patch to rudiments (poll.FULL.patch.gz <http://www.srr.pp.ru/www/sqlrelay/rudiments-0.32/poll.FULL.patch.gz>) |
|
From: Renat S. <sr...@st...> - 2010-05-18 20:37:45
|
18.05.2010 23:29, Cal Heldenbrand пишет: > Oh ok, I picked just the status_more_info patch and applied that, I > had compilation errors with trying to access a private member in > rudiments. Hm. Neither this patch nor others require private members in rudiments (I use 0.32 version). I'll try to apply my patches against the clean sqlrelay-0.41 tomorrow and test it for compiling. > Before I go any futher though, has anyone already implemented this? > Or do you guys foresee any problems with what I'm trying to do? No, I suppose nobody have tried to do this. As for me, in our application we use no more than 2 cursors in a moment. |