libdbi-users — General libdbi discussion and support

Re: [libdbi-users] libdbi+mysql speed issues

[Markus H. <mar...@mh...>]

Olivier Doucet <web...@aj...> was heard to say:

> Hello everyone,
>
> I'm following a quite old topic about libdbi speed issues.
> I was able to track the cause of these issues : The major problem is
> how libdbi goes from one row to another.
>
> RRDTool (the tool that used libdbi and that I was inspecting) is using
> dbi_result_next_row() function (as stated in libdbi documentation
> btw).
>
> This function moves from one row to another with function
> dbi_result_seek_row(), incrementing currentRow index each time. This
> gives a call to dbd_mysql.c::dbd_goto_row() that uses
> mysql_data_seek() each time...
>
> That's why for a query result of 34k rows (yes it happens. No it is
> not a problem in the query itself), we have tens of thousands of call
> to this function (which is very low), and this is definitely not
> needed, because as we use fetch_row(), we automatically move from one
> row to another. Seeking is just a useless task (as internal driver
> does not know where we are, and needs to start from row 0 and seek to
> the given row - where we already were).
>
> I'm absolutely not a libdbi user, and I don't know what could be done
> outside libdbi to not use dbi_result_next_row() and use directly
> RESULT->onn->driver->functions->fetch_row() directly. Is it possible ?
>
> And/or patching dbi_result.c :
> just check RESULT->currowidx near line 102 before calling doing
> goto_row() function and call it only if we are not on the good row. Am
> I right ?
>

Hi,

your analysis is pretty much correct. If you look at the comments in  
dbd_mysql.c::dbd_goto_row(), the original author of the mysql driver  
was well aware of the limitations of his implementation. The reason is  
that other database APIs, e.g. PostgreSQL, allow to fetch rows from a  
result set by index, whereas the MySQL API assumes that you step  
through the rows sequentially. The original design of libdbi appears  
to somewhat favor PostgreSQL in this respect.

Anyway, without having thought about the issue in too much detail, one  
possible solution comes to mind. We could modify the driver function  
dbd_goto_row() by passing both the wanted row index rowidx and the  
current row index currowidx(which libdbi keeps track of anyway). This  
would allow drivers to decide whether they have to actually seek the  
position. pgsql doesn't have to anyway, and mysql doesn't have to if  
rowidx = currowidx+1. This API change would not mandate changes to  
existing drivers as they may ignore the additional parameter and keep  
working as before, but it may offer options to speed up queries in  
some drivers.

regards,
Markus



-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] libdbi+mysql speed issues

[Olivier D. <web...@aj...>]

Hello everyone,

I'm following a quite old topic about libdbi speed issues.
I was able to track the cause of these issues : The major problem is
how libdbi goes from one row to another.

RRDTool (the tool that used libdbi and that I was inspecting) is using
dbi_result_next_row() function (as stated in libdbi documentation
btw).

This function moves from one row to another with function
dbi_result_seek_row(), incrementing currentRow index each time. This
gives a call to dbd_mysql.c::dbd_goto_row() that uses
mysql_data_seek() each time...

That's why for a query result of 34k rows (yes it happens. No it is
not a problem in the query itself), we have tens of thousands of call
to this function (which is very low), and this is definitely not
needed, because as we use fetch_row(), we automatically move from one
row to another. Seeking is just a useless task (as internal driver
does not know where we are, and needs to start from row 0 and seek to
the given row - where we already were).

I'm absolutely not a libdbi user, and I don't know what could be done
outside libdbi to not use dbi_result_next_row() and use directly
RESULT->onn->driver->functions->fetch_row() directly. Is it possible ?

And/or patching dbi_result.c :
just check RESULT->currowidx near line 102 before calling doing
goto_row() function and call it only if we are not on the good row. Am
I right ?

Olivier

[libdbi-users] Can't connect to MS SQLServer,

[<mar...@mh...>]

Daniel Hilst writes:
 > I'm trying to setup collectd to monitor a MS SQLServer, collectd uses 
 > libdbi, witch uses freetds driver.. I can connectd with tsql, but 
 > test_dbi can't.. Here is its output:
 > 
 > http://pastebin.com/6kfzfPjw
 > 

Hi,

I'm not familiar with the freetds driver, and I do not have a
SQLServer or Sybase installation to fiddle with. Can you check the
server logs to find out why the driver was unable to connect?
 > 
 > Also I have a question, what should I put on version?. I tried 7.0 which 
 > is the tds version used with freetds, but without successs.
 > 

The driver source suggests that the version should use the format
X.XX.XX. If you're using version 7.x, only the first character is
checked though, so "7.0" should have worked here as well.


regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

[libdbi-users] Can't connect to MS SQLServer,

[Daniel H. <dan...@gm...>]

I'm trying to setup collectd to monitor a MS SQLServer, collectd uses 
libdbi, witch uses freetds driver.. I can connectd with tsql, but 
test_dbi can't.. Here is its output:

http://pastebin.com/6kfzfPjw


Also I have a question, what should I put on version?. I tried 7.0 which 
is the tds version used with freetds, but without successs.


Best regards,
Hilst
-- 
Follow the white rabbit!

Re: [libdbi-users] libdbi transaction support

[<mar...@mh...>]

mar...@mh... writes:
 > I've stolen some time from myself to provide a first shot at
 > transaction and savepoint support, see the current cvs revisions of
 > libdbi and libdbi-drivers. The code is entirely untested except that
 > the drivers which I use myself compile and don't crash upon loading. I
 > didn't get round to adding the documentation and the tests, but feel
 > free to test the current code yourself. Usage should be pretty obvious
 > if you look at the diffs. I'm sure some rough edges remain, but
 > then... it's a start.
 > 

FYI I've also updated the docs in cvs to reflect the latest changes.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] libdbi transaction support

[<mar...@mh...>]

Rainer Gerhards writes:
 > > This should be easy to retrofit. I'lls see if I find
 > > some time but feel free to beat me at it.
 > 
 > I am quite busy myself at the moment, but I could try and see if I
 > could craft something along that path...
 > 

Hi all,

I've stolen some time from myself to provide a first shot at
transaction and savepoint support, see the current cvs revisions of
libdbi and libdbi-drivers. The code is entirely untested except that
the drivers which I use myself compile and don't crash upon loading. I
didn't get round to adding the documentation and the tests, but feel
free to test the current code yourself. Usage should be pretty obvious
if you look at the diffs. I'm sure some rough edges remain, but
then... it's a start.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] libdbi transaction support

[Toby T. <to...@te...>]

On 20/11/12 11:50 AM, Markus Hoenicka wrote:
> Mike Rylander<mry...@gm...>  was heard to say:
>
>> I think I'd add three calls to the proposed API, though, for support
>> of savepoints.  They're part of the SQL standard, and supported by
>> several SQL RDBMS' including Postgres (the db my project uses, and we
>> use libdbi to connect to PG).  The relevant PG documentation is
>> available at http://www.postgresql.org/docs/9.1/static/sql-savepoint.html
>> .  I have wrappers for savepoints in my implementation, and they are
>> very handy for complex DB interactions.
>>
>
> Sounds like a no-brainer if it is as simple as you say. I don't keep
> the SQL standard underneath my pillow, but is it safe to assume that
> database engines are responsible to deal with any pending savepoints
> if a transaction is committed (i.e. no extra work for libdbi)?
>
> This brings up another question. Applications should of course check
> the "transaction_supported" and "savepoints_supported" driver
> capabilities and act responsibly. But if a database engine does not
> support savepoints, or transactions altogether, should libdbi just go
> ahead when asked, or should it throw an error instead? I recall that
> MySQL "supported" transactions in MyISAM tables using no-ops.

That is correct. The transactional engine is InnoDB, and it's the 
recommended one if you care about your data and its integrity.

I would suggest that libdbi should not try to be clever here - it's 
basically infeasible to try and determine whether a sequence of 
statements is transactionally unsafe in some way. For one thing, storage 
engines can be mixed in one MySQL schema, and a sequence of statements 
can involve them in arbitrary combinations (which is very foolish, but 
still). A warning about MyISAM might be placed in the manual, however.

I don't know anything about savepoints so can't comment on that part.

--Toby

>
> regards,
> Markus
>

Re: [libdbi-users] libdbi transaction support

[Markus H. <mar...@mh...>]

Mike Rylander <mry...@gm...> was heard to say:

> I think I'd add three calls to the proposed API, though, for support
> of savepoints.  They're part of the SQL standard, and supported by
> several SQL RDBMS' including Postgres (the db my project uses, and we
> use libdbi to connect to PG).  The relevant PG documentation is
> available at http://www.postgresql.org/docs/9.1/static/sql-savepoint.html
> .  I have wrappers for savepoints in my implementation, and they are
> very handy for complex DB interactions.
>

Sounds like a no-brainer if it is as simple as you say. I don't keep  
the SQL standard underneath my pillow, but is it safe to assume that  
database engines are responsible to deal with any pending savepoints  
if a transaction is committed (i.e. no extra work for libdbi)?

This brings up another question. Applications should of course check  
the "transaction_supported" and "savepoints_supported" driver  
capabilities and act responsibly. But if a database engine does not  
support savepoints, or transactions altogether, should libdbi just go  
ahead when asked, or should it throw an error instead? I recall that  
MySQL "supported" transactions in MyISAM tables using no-ops.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] libdbi transaction support

[Mike R. <mry...@gm...>]

On Tue, Nov 20, 2012 at 7:59 AM, Rainer Gerhards <rge...@gm...> wrote:
> On Tue, Nov 20, 2012 at 1:02 PM, Markus Hoenicka
> <mar...@mh...> wrote:
>> In fact, we already have a (not well documented) infrastructure for things
>> like this in place. Every driver keeps a list of boolean "capabilities"
>> which is merely a key-value list of whatever we think is useful or
>> necessary. To date, only the "safe_dlclose" capability seems to be used, see
>> e.g. dbd_initialize in libdbi-drivers/drivers/mysql/dbd_mysql.c. There is an
>> internal function _dbd_register_driver_cap() which can be used by the
>> drivers to announce a capability. And there is a public function
>> dbi_driver_cap_get() which allows applications to query the list of
>> capabilities at runtime. So all we'd have to do is modify the drivers and
>> have them add a "transaction_support" capability set to 1 for those who
>> support transactions.
>
> ok, that makes sense and is excellent to have!
>
>>> For TX support, I'd expect calls for
>>>
>>> - begin transaction
>>> - commit
>>> - rollback
>>>
>>> that's it, so all in all 4 calls. Would you agree to that?
>>

First, I'm glad to so movement on this! The code in the project I'm
involved with handles transactions on its own already using something
along the lines of the wrapper pattern Markus mentioned, so it won't
make a big difference to me immediately, but having a sanctioned API
is almost always better, IMO.

I think I'd add three calls to the proposed API, though, for support
of savepoints.  They're part of the SQL standard, and supported by
several SQL RDBMS' including Postgres (the db my project uses, and we
use libdbi to connect to PG).  The relevant PG documentation is
available at http://www.postgresql.org/docs/9.1/static/sql-savepoint.html
.  I have wrappers for savepoints in my implementation, and they are
very handy for complex DB interactions.

I can't offer many properly shaped tuits toward the development effort
right now, but supporting savepoints should amount to essentially
adding cut-and-paste duplicates of begin, commit and rollback.  Would
it be objectionable to ask for savepoint support to piggy-back on
transaction support?

-- 
Mike Rylander
 | Director of Research and Development
 | Equinox Software, Inc. / Your Library's Guide to Open Source
 | phone:  1-877-OPEN-ILS (673-6457)
 | email:  mi...@es...
 | web:  http://www.esilibrary.com

Re: [libdbi-users] libdbi transaction support

[Rainer G. <rge...@gm...>]

On Tue, Nov 20, 2012 at 1:02 PM, Markus Hoenicka
<mar...@mh...> wrote:
> In fact, we already have a (not well documented) infrastructure for things
> like this in place. Every driver keeps a list of boolean "capabilities"
> which is merely a key-value list of whatever we think is useful or
> necessary. To date, only the "safe_dlclose" capability seems to be used, see
> e.g. dbd_initialize in libdbi-drivers/drivers/mysql/dbd_mysql.c. There is an
> internal function _dbd_register_driver_cap() which can be used by the
> drivers to announce a capability. And there is a public function
> dbi_driver_cap_get() which allows applications to query the list of
> capabilities at runtime. So all we'd have to do is modify the drivers and
> have them add a "transaction_support" capability set to 1 for those who
> support transactions.

ok, that makes sense and is excellent to have!

>> For TX support, I'd expect calls for
>>
>> - begin transaction
>> - commit
>> - rollback
>>
>> that's it, so all in all 4 calls. Would you agree to that?
>
>
> yes, except that we don't need the drvrTXSupport() function as we can use
> dbi_driver_cap_get() instead.

I'll see if I can dig a bit into the code within the next couple of days.

Thanks!
Rainer

Re: [libdbi-users] libdbi transaction support

[Markus H. <mar...@mh...>]

Rainer Gerhards <rge...@gm...> was heard to say:

> That's what I would expect. However, I think it would be useful to
> have a call like "drvrTXSupport()" returning 1 if the driver supports
> transaction and 0 otherwise. That way, an app could either fall back
> to non-transcation mode OR tell the user that the database system is
> not suitable for that kind of application. That would probably also
> provide a smooth migration path, were only the primary TX interface
> needs to be implemented and drivers can be upgraded as time permits
> (assuming that some drivers are maintained by external entities).

In fact, we already have a (not well documented) infrastructure for  
things like this in place. Every driver keeps a list of boolean  
"capabilities" which is merely a key-value list of whatever we think  
is useful or necessary. To date, only the "safe_dlclose" capability  
seems to be used, see e.g. dbd_initialize in  
libdbi-drivers/drivers/mysql/dbd_mysql.c. There is an internal  
function _dbd_register_driver_cap() which can be used by the drivers  
to announce a capability. And there is a public function  
dbi_driver_cap_get() which allows applications to query the list of  
capabilities at runtime. So all we'd have to do is modify the drivers  
and have them add a "transaction_support" capability set to 1 for  
those who support transactions.

>
> For TX support, I'd expect calls for
>
> - begin transaction
> - commit
> - rollback
>
> that's it, so all in all 4 calls. Would you agree to that?

yes, except that we don't need the drvrTXSupport() function as we can  
use dbi_driver_cap_get() instead.

>
>> This should be easy to retrofit. I'lls see if I find
>> some time but feel free to beat me at it.
>
> I am quite busy myself at the moment, but I could try and see if I
> could craft something along that path...
>
> Rainer

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] libdbi transaction support

[Rainer G. <rge...@gm...>]

On Tue, Nov 20, 2012 at 11:51 AM, Markus Hoenicka
<mar...@mh...> wrote:
> not exceptionally dumb, but rather touching a sensitive issue. I've
> used simple wrappers in one of my applications that could easily be
> moved into libdbi. I think a transaction interface has been missing
> from libdbi because some database engines that were supported in the
> days of yore did not have transaction support.

Ah! At least this explains.

> I think most database
> engines which are supported or have experimental support these days
> support transactions by sending queries along the lines of "begin" or
> "begin work" etc.

That's what I would expect. However, I think it would be useful to
have a call like "drvrTXSupport()" returning 1 if the driver supports
transaction and 0 otherwise. That way, an app could either fall back
to non-transcation mode OR tell the user that the database system is
not suitable for that kind of application. That would probably also
provide a smooth migration path, were only the primary TX interface
needs to be implemented and drivers can be upgraded as time permits
(assuming that some drivers are maintained by external entities).

For TX support, I'd expect calls for

- begin transaction
- commit
- rollback

that's it, so all in all 4 calls. Would you agree to that?

> This should be easy to retrofit. I'lls see if I find
> some time but feel free to beat me at it.

I am quite busy myself at the moment, but I could try and see if I
could craft something along that path...

Rainer

Re: [libdbi-users] libdbi transaction support

[Markus H. <mar...@mh...>]

Rainer Gerhards <rge...@gm...> was heard to say:

> Hi all,
>
> sorry if this is an exceptionally dumb question, but: does libdbi
> support transactions (and, if so, via which functions)? I tried to
> find the transaction support functions, but did not come up with
> anything. Also a (granted, quick) look at the source did not tell me
> anything.
>
> Any help is deeply appreciated,
> Rainer
>

Hi,

not exceptionally dumb, but rather touching a sensitive issue. I've  
used simple wrappers in one of my applications that could easily be  
moved into libdbi. I think a transaction interface has been missing  
from libdbi because some database engines that were supported in the  
days of yore did not have transaction support. I think most database  
engines which are supported or have experimental support these days  
support transactions by sending queries along the lines of "begin" or  
"begin work" etc. This should be easy to retrofit. I'lls see if I find  
some time but feel free to beat me at it.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

[libdbi-users] libdbi transaction support

[Rainer G. <rge...@gm...>]

Hi all,

sorry if this is an exceptionally dumb question, but: does libdbi
support transactions (and, if so, via which functions)? I tried to
find the transaction support functions, but did not come up with
anything. Also a (granted, quick) look at the source did not tell me
anything.

Any help is deeply appreciated,
Rainer

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[<mar...@mh...>]

Shakthi Kannan writes:
 > Which is the current cvs revision, and which file is it? I am not able
 > to see any commit logs at:
 > 
 >   http://sourceforge.net/mailarchive/forum.php?forum=libdbi-cvs
 > 

Hi,

I'm afraid that the cvs log at sourceforge has been broken for a
while. I always get back error messages related to the mailing lists
when I check in changes. In any case, you can review the changes
through the web cvs frontend:

http://libdbi.cvs.sourceforge.net/viewvc/libdbi/libdbi/src/

The leak was fixed in dbd_helper.c which is now at 1.44.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Shakthi K. <sha...@gm...>]

Hi Markus:

--- On Tue, Aug 9, 2011 at 4:51 PM,  <mar...@mh...> wrote:
| I've reproduced the leak that you detected in the current cvs
| sources. According to my analysis the drivers pass error messages
| through dbd_geterror() as allocated
| strings. _dbd_internal_error_handler() created another copy with
| strdup() which isn't necessary. The latter copy was freed, the
| original copy was left in place. I've fixed this leak in the current
| cvs revision.
\--

Which is the current cvs revision, and which file is it? I am not able
to see any commit logs at:

  http://sourceforge.net/mailarchive/forum.php?forum=libdbi-cvs

Thanks for your prompt response.

SK

-- 
Shakthi Kannan
http://www.shakthimaan.com

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[<mar...@mh...>]

Shakthi Kannan writes:
 > The API has been changed in the latest source. So, I updated the
 > following code snippet, and I still see the memory leak:

Hi Shakthi,

I've reproduced the leak that you detected in the current cvs
sources. According to my analysis the drivers pass error messages
through dbd_geterror() as allocated
strings. _dbd_internal_error_handler() created another copy with
strdup() which isn't necessary. The latter copy was freed, the
original copy was left in place. I've fixed this leak in the current
cvs revision.

This is what I get now on my box when I run your test program:

==3717== HEAP SUMMARY:
==3717==     in use at exit: 161,359 bytes in 5,251 blocks
==3717==   total heap usage: 20,903 allocs, 15,652 frees, 16,739,701 bytes allocated
==3717== 
==3717== LEAK SUMMARY:
==3717==    definitely lost: 0 bytes in 0 blocks
==3717==    indirectly lost: 0 bytes in 0 blocks
==3717==      possibly lost: 0 bytes in 0 blocks
==3717==    still reachable: 157,263 bytes in 5,250 blocks
==3717==         suppressed: 4,096 bytes in 1 blocks
==3717== Reachable blocks (those to which a pointer was found) are not shown.
==3717== To see them, rerun with: --leak-check=full --show-reachable=yes
==3717== 
==3717== For counts of detected and suppressed errors, rerun with: -v
==3717== Use --track-origins=yes to see where uninitialised values come from
==3717== ERROR SUMMARY: 289 errors from 1 contexts (suppressed: 48 from 8)

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Shakthi K. <sha...@gm...>]

Hi,

Replying to my own post:

--- On Tue, Aug 9, 2011 at 11:30 AM, Shakthi Kannan
<sha...@gm...> wrote:
|    else {
|      dbi_conn_close(conn);
|    }
\--

With libdbi-0.8.3, libdbi-drivers-0.8.3 adding a sleep(1) after
dbi_conn_close(conn) doesn't cause any leak.

SK

-- 
Shakthi Kannan
http://www.shakthimaan.com

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Shakthi K. <sha...@gm...>]

Hi Markus,

--- On Thu, Aug 4, 2011 at 5:44 PM, Markus Hoenicka
<mar...@mh...> wrote:
| I had almost forgotten that bug. That's indeed the most likely reason that
| your test app leaks memory. Unfortunately you're not using an old version as
| 0.8.3 is the latest official release. The bug is fixed in the cvs version
| which, by all means, ist tested, stable, and reliable. I just won't have the
| time to finish a new release until September or so. So if that leak really
| hurts you, you'd have to build libdbi and libdbi-drivers from cvs.
\--

I checked out the latest cvs sources (August 9, 2011). For libdbi, I did:

  $ autogen.sh
  $ ./configure --disable-docs
  $ make
  $ sudo make install

For libdbi-drivers, I used:

  $ autogen.sh
  $ ./configure --with-mysql --with-mysql-libdir=/usr/lib/mysql
--with-mysql-incdir=/usr/include/mysql
--with-dbi-libdir=/usr/local/lib --disable-docs
  $ export LD_LIBRARY_PATH=/usr/local/lib

The API has been changed in the latest source. So, I updated the
following code snippet, and I still see the memory leak:

=== CODE ===

#include <stdio.h>
#include <dbi/dbi.h>

int main()
{
  dbi_conn conn;
  dbi_inst instance;

  dbi_initialize_r(NULL, &instance);

  while (1) {

    conn = dbi_conn_new_r("mysql", instance);

    dbi_conn_set_option(conn, "host", "localhost");
    dbi_conn_set_option(conn, "username", "foo");
    dbi_conn_set_option(conn, "password", "bar");
    dbi_conn_set_option(conn, "dbname", "test");
    dbi_conn_set_option(conn, "encoding", "UTF-8");

    if (dbi_conn_connect(conn) < 0) {
      printf ("Could not connect\n");
    }
    else {
      dbi_conn_close(conn);
    }
  }

  dbi_shutdown_r(instance);
  return 0;

}

=== END ===

Is there anything that I am missing? Appreciate any inputs in this regard,

SK

-- 
Shakthi Kannan
http://www.shakthimaan.com

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Markus H. <mar...@mh...>]

Shakthi Kannan <sha...@gm...> was heard to say:

> I came across this memory leak post:
>
>   http://comments.gmane.org/gmane.comp.db.libdbi.devel/309
>
> Is this fixed in the sources and am I using old version of the software?
>
> libdbi-devel-0.8.3-4.fc14.i686
> libdbi-0.8.3-4.fc14.i686
> libdbi-drivers-0.8.3-6.fc14.i686
> libdbi-dbd-mysql-0.8.3-6.fc14.i686
>

Hi,

I had almost forgotten that bug. That's indeed the most likely reason  
that your test app leaks memory. Unfortunately you're not using an old  
version as 0.8.3 is the latest official release. The bug is fixed in  
the cvs version which, by all means, ist tested, stable, and reliable.  
I just won't have the time to finish a new release until September or  
so. So if that leak really hurts you, you'd have to build libdbi and  
libdbi-drivers from cvs.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Shakthi K. <sha...@gm...>]

Hi Markus,

I came across this memory leak post:

  http://comments.gmane.org/gmane.comp.db.libdbi.devel/309

Is this fixed in the sources and am I using old version of the software?

libdbi-devel-0.8.3-4.fc14.i686
libdbi-0.8.3-4.fc14.i686
libdbi-drivers-0.8.3-6.fc14.i686
libdbi-dbd-mysql-0.8.3-6.fc14.i686

SK

-- 
Shakthi Kannan
http://www.shakthimaan.com

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Shakthi K. <sha...@gm...>]

Hi Markus:

--- On Wed, Aug 3, 2011 at 12:41 PM, Markus Hoenicka
<mar...@mh...> wrote:
| In any case, I'd suggest to modify your code. dbi_initialize() is
| supposed to be run once per process
\--

I have changed this. If I try to comment the invocation call to
dbi_conn_connect(conn), then I don't see any increase in dynamic
memory allocation. I check with 'pmap -x <pid>' and observe the
mapping entry of "[ anon ]" in its output.

Thanks for the quick reply!

SK

-- 
Shakthi Kannan
http://www.shakthimaan.com

Re: [libdbi-users] Memory leak when using dbi_initialize in loop

[Markus H. <mar...@mh...>]

Shakthi Kannan <sha...@gm...> was heard to say:

> int main() {
>   dbi_conn conn;
>
>   while (1) {
>     dbi_initialize(NULL);
>
[...]
>
>     dbi_shutdown();
>   }
>
>   return 0;
> }

Hi,

I'll test your code on my dev box as soon as time permits. libdbi is  
not supposed to lose memory under any circumstances. I'll have to see  
the valgrind output as sometimes the leak is not within libdbi itself,  
but within a driver or the client library (if the driver forgets to  
free memory allocated by the library).

In any case, I'd suggest to modify your code. dbi_initialize() is  
supposed to be run once per process, so the following is more  
appropriate:

int main() {
   dbi_conn conn;

   dbi_initialize(NULL);
   while (1) {

[...]

   }
   dbi_shutdown();

   return 0;
}

If it is indeed libdbi that leaks memory, it won't do so in every  
cycle of your loop.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

[libdbi-users] Memory leak when using dbi_initialize in loop

[Shakthi K. <sha...@gm...>]

Hi,

Using libdbi on Fedora 14:

libdbi-dbd-sqlite-0.8.3-6.fc14.i686
libdbi-devel-0.8.3-4.fc14.i686
libdbi-0.8.3-4.fc14.i686
libdbi-drivers-0.8.3-6.fc14.i686
libdbi-dbd-mysql-0.8.3-6.fc14.i686

The following code continues to allocate memory, but, doesn't
deallocate it. Can you please let me know what is wrong?

=== CODE ===

#include <stdio.h>
#include <dbi/dbi.h>

int main() {
  dbi_conn conn;

  while (1) {
    dbi_initialize(NULL);

    conn = dbi_conn_new("mysql");

    dbi_conn_set_option(conn, "host", "localhost");
    dbi_conn_set_option(conn, "username", "foo");
    dbi_conn_set_option(conn, "password", "bar");
    dbi_conn_set_option(conn, "dbname", "test");
    dbi_conn_set_option(conn, "encoding", "UTF-8");

    if (dbi_conn_connect(conn) < 0) {
      printf("Could not connect. Please check the option settings\n");
    }
    else {
      /* Do something, and delay */
      dbi_conn_close(conn);
    }

    dbi_shutdown();
  }

  return 0;
}

=== END ===

I am trying to do a task after the database connect, give a delay and
loop around, but the above flow is leaking memory. Appreciate any
inputs in this regard!

SK

-- 
Shakthi Kannan
http://www.shakthimaan.com

Re: [libdbi-users] MingW and libdbi

[A.sashikanth <a.s...@gm...>]

Markus Hoenicka <markus.hoenicka <at> mhoenicka.de> writes:

> 
> Duncan McQueen writes:
>  > In file included from c:/mysql-5.1.20-beta-win32/include/mysql/mysql.h:67,
>  >                  from dbd_mysql.c:50:
>  > c:/mysql-5.1.20-beta-win32/include/mysql_com.h:191: error: expected
>  > specifier-qualifier-list before 'SOCKET'
>  > c:/mysql-5.1.20-beta-win32/include/mysql_com.h:366: error: expected
>  > ')' before 's'
>  > dbd_mysql.c: In function 'dbd_get_socket':
>  > dbd_mysql.c:227: error: 'NET' has no member named 'fd'
>  > make[3]: *** [dbd_mysql.lo] Error 1
>  > make[2]: *** [all-recursive] Error 1
>  > make[1]: *** [all-recursive] Error 1
>  > make: *** [all] Error 2
>  > 
>  > Anyone make sense of this?
>  > 
> 
> Not really. Usually these errors indicate incompletely ported system
> header files which lack some stuff that mysql depends on. I guess
> you'll get more interesting answers on a MinGW list.
> 
> regards,
> Markus
> 



hi markus 
event i  am stuck with the same problem...
if u have found any possible solution pls let me know
hope a positive response from your


Regards
A.SASHIKANTH
a.s...@gm...