libdbi-devel — Discussion about development of the core library and DBD interface

[libdbi-devel] [PATCH] src/dbi_main.c: Include <strings.h> for `strcasecmp'.

[Florian F. <sou...@no...>]

From: Florian Forster <oc...@ve...>

---
 src/dbi_main.c |    1 +
 1 files changed, 1 insertions(+), 0 deletions(-)

diff --git a/src/dbi_main.c b/src/dbi_main.c
index 1eb6819..1afbccb 100644
--- a/src/dbi_main.c
+++ b/src/dbi_main.c
@@ -32,6 +32,7 @@
 #include <stdlib.h>
 #include <stdarg.h>
 #include <string.h>
+#include <strings.h>
 #include <sys/types.h>
 #include <stddef.h>
 
-- 
1.5.6.5

[libdbi-devel] [PATCH] src/dbd_helper.c: Remove an unused variable.

[Florian F. <sou...@no...>]

From: Florian Forster <oc...@ve...>

---
 src/dbd_helper.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/src/dbd_helper.c b/src/dbd_helper.c
index 73ec904..09b3764 100644
--- a/src/dbd_helper.c
+++ b/src/dbd_helper.c
@@ -303,7 +303,6 @@ time_t _dbd_parse_datetime(const char *raw, unsigned int attribs) {
 	int _tz_dir = 0;
 	int _tz_hours = 0;
 	int _tz_mins = 0;
-	int _hour_len = 2;
 
 	int check_time = 1;
 
-- 
1.5.6.5

[libdbi-devel] [PATCH] src/dbi_main.c: Don't use `RTLD_DEFAULT' if it is not defined.

[Florian F. <sou...@no...>]

From: Florian Forster <oc...@ve...>

---
 src/dbi_main.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/src/dbi_main.c b/src/dbi_main.c
index e103dfc..9b2d80a 100644
--- a/src/dbi_main.c
+++ b/src/dbi_main.c
@@ -1309,10 +1309,14 @@ static dbi_driver_t *_get_driver(const char *filename, dbi_inst_t *inst) {
 		if (DLSYM_HANDLE) { /* most OSes */
 		  symhandle = dlhandle;
 		}
+#ifdef RTLD_DEFAULT
 		else { /* the BSDs */
 /* 		  symhandle = RTLD_NEXT; */
 		  symhandle = RTLD_DEFAULT;
 		}
+#else
+		else { return (NULL); }
+#endif
 
 		while (custom_functions_list && custom_functions_list[idx] != NULL) {
 			custom = malloc(sizeof(dbi_custom_function_t));
-- 
1.5.6.5

Re: [libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Artyom writes:
 >      b. Sqlite2 backend (and Sqlite3 in 8.2 and above)

I've replaced the readdir() and strtok() calls in sqlite and sqlite3
with the reentrant versions. I didn't find any issues in the other
drivers so far.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] Thread Safety question

[Florian F. <oc...@ve...>]

Hi Markus,

On Thu, Nov 27, 2008 at 10:57:57PM +0100, Markus Hoenicka wrote:
> Florian Forster writes:
>  > I've just tried to compile libdbi with
>  >   ./configure --disable-docs \
>  >     CFLAGS="-Wall -Werror" \
>  >     CPPFLAGS="-D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=500"
>  > an ran into problems with [v]asprintf: Since they're no in any
>  > standard, Mac OS X (and therefore probably FreeBSD as well) will
>  > deactivate the prototypes. I'll follow up on this..
> 
> Just for the record: on FreeBSD 6.1 there are problems with snprintf,
> strcasecmp, and (v)asprintf using these flags.

Yeah, I ultimately ran into problems under Mac OS X, too. The
(v)asprintf problem is, that the configure script checks if the C
library exports such a symbol but not if the header files declare it.

`strcasecmp' is easy: Just include <strings.h> (notice the second `s'!).

I didn't have any problems with snprintf, but I ran into a signedness
problem: Somewhere a `const unsigned char *' gets casted to a
`const char *' which produces a warning.

Another problem that prevents me from using `-Wall -Werror' is that some
functions are declared ``deprected'' but used internally. I think the
solution to that would be to put a `#ifndef LIBDBI_API_DEPRECATED' block
around the definition in include/dbi/dbi.h. That way, src/dbi_main.c can
define
 #define LIBDBI_API_DEPRECATED /**/
before including the header, effectively deactivating the warnings for
that file only.

I have some changed code for most of the above problems, but since work
more *against* CVS than I work *with* CVS creating patches suddenly
becomes a challenging task ;) I'll send the patches tomorrow. Does
anybody have a Git import of the repository public somewhere?

Regards,
-octo
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/

Re: [libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Florian Forster writes:
 > I've just tried to compile libdbi with
 >   ./configure --disable-docs \
 >     CFLAGS="-Wall -Werror" \
 >     CPPFLAGS="-D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=500"
 > an ran into problems with [v]asprintf: Since they're no in any standard,
 > Mac OS X (and therefore probably FreeBSD as well) will deactivate the
 > prototypes. I'll follow up on this..
 > 

Just for the record: on FreeBSD 6.1 there are problems with snprintf,
strcasecmp, and (v)asprintf using these flags.

 > Uh, I wouldn't use `-1' as an error indicator of type `size_t': `size_t'
 > is declared unsigned on many platforms, comparing it with `-1' will
 > cause warnings on many systems. Functions returning a memory size or a
 > negative error condition should use `ssize_t' (see read(2) for example).
 > 

Good point. Another case of careless code copying (tm). I've changed
the function to return 0 in case of an error, as a buffer size of 0 is
not going to do us any good.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] Thread Safety question

[Florian F. <oc...@ve...>]

Hi Markus,

On Thu, Nov 27, 2008 at 01:01:48AM +0100, Markus Hoenicka wrote:
>  > So if you compile with
>  >  CPPFLAGS="-D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=500 -D_REENTRANT -D_THREAD_SAFE"
>  > you should get the XS-interface on all noted platforms, as far as I
>  > know.
> 
> Would you please provide some pointers to these switches? Are they
> gcc-specific in any way? I'd like to add something like this to
> configure.in.

Sure, _POSIX_C_SOURCE and _XOPEN_SOURCE are specified by POSIX/SUS
directly:
<https://www.opengroup.org/onlinepubs/000095399/functions/xsh_chap02_02.html#tag_02_02_01_01>

Don't know if _REENTRANT is specified officially anywhere, but it's used
by Solaris.
<http://docsun.cites.uiuc.edu/sun_docs/C/solaris_9/SUNWdev/MTP/p44.html#COMPILE-4>

Since I haven't seen a source for _REENTRANT and apparently Solaris
doesn't need it if _POSIX_C_SOURCE is defined, it's probably reasonable
to leave that out until the need arises.

_THREAD_SAFE is, according to GNU's feature.h, a synonym for _REENTRANT,
so you can probably leave that out, too.

I've just tried to compile libdbi with
  ./configure --disable-docs \
    CFLAGS="-Wall -Werror" \
    CPPFLAGS="-D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=500"
an ran into problems with [v]asprintf: Since they're no in any standard,
Mac OS X (and therefore probably FreeBSD as well) will deactivate the
prototypes. I'll follow up on this..

> - I've used a more portable way to calculate the buffer size required
>   for readdir_r, see
>   http://womble.decadentplace.org.uk/readdir_r-advisory.html

Uh, I wouldn't use `-1' as an error indicator of type `size_t': `size_t'
is declared unsigned on many platforms, comparing it with `-1' will
cause warnings on many systems. Functions returning a memory size or a
negative error condition should use `ssize_t' (see read(2) for example).

> - I've fixed an obvious typo in the timegm.c patch (gmtime_r
>   vs. gmtime)

D'oh! How did that compile on my machine?

Regards,
-octo
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/

Re: [libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Florian Forster writes:
 > since all XSI compliant systems are required to provide thread- and
 > reentrant-safe functions, those functions are fortunately available on
 > the vast majority of systems.
 > 
 > So if you compile with
 >  CPPFLAGS="-D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=500 -D_REENTRANT -D_THREAD_SAFE"
 > you should get the XS-interface on all noted platforms, as far as I
 > know.
 > 
 > Of course, there are the occasional oddities here and there, as with all
 > ``modern'' interfaces. For example, under Solaris, if you don't specify
 > the above flags, you need to at least specify
 >  CPPFLAGS="-D_POSIX_PTHREAD_SEMANTICS"
 > to get the standard-conforming versions of some functions, such as
 > `strtok_r'.
 > 

Would you please provide some pointers to these switches? Are they
gcc-specific in any way? I'd like to add something like this to
configure.in.


 > Please consider the attached patch. It's against what I considered to be
 > the current CVS revision, but I haven't CVS for quite some time, so
 > please forgive me if I'm a bit off ;)
 > 

Thanks for the patch, this saved me quite a bit of my time. I've
applied the patch with the following minor edits:

- dbi_initialize_r must not return the value of status (which is set
to the value of ERRNO in case of a failure). These are positive
integers and cannot be distinguished by the caller from a successful
execution loading that many drivers. num_loaded better reflects the
success of the function call as it is zero if readdir_r fails,
indicating that no drivers were loaded.

- I've used a more portable way to calculate the buffer size required
  for readdir_r, see
  http://womble.decadentplace.org.uk/readdir_r-advisory.html

- I've fixed an obvious typo in the timegm.c patch (gmtime_r
  vs. gmtime)


 > I did not remove that ugly `strtok == strtok_r' hack in
 > src/dbi_result.c, but I added a warning to it. Is that really necessary?
 > 

Apparently yes. I've found no hints on the web that MINGW now has
strtok_r.

I've checked in the above changes. I'd like to get some feedback from
non-FreeBSD systems.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Rainer Gerhards writes:
 > An important thing (which you would consider to be reasonable) is that
 > no multiple threads access the very same connection object. This causes
 > at least problems with MySQL, but I would expect also with other
 > database engines. It may be worth adding that to a piece of doc.
 > 

Thanks a lot for this hint. I'll try and find a good place for this
advice in the docs.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] Thread Safety question

[Artyom <art...@ya...>]

> > > For the record I had seen in sqlite (not sqlite3)
> backend usage of
> > > strtok. So it should be thread unsafe.
> 
> How does that make it thread-safe?

So it should be thread **un**safe.

-------------------------------------------
So... for the summary:

At this point:

On platforms where GNU extendtion timegm() exists the system is:
  1. Has not thread safe parts in;
     a. Loading/Unloading modules: dbi_initialize/dbi_shutdown
     b. Sqlite2 backend (and Sqlite3 in 8.2 and above)
  2. The rest is thread safe.

On MINGW it is not safe due to lack of strtok
(is it??????, may be it worth to add check as for timegm?)

At any platform this can easily be fixed to thread safe if
gmtime replaced with gmtime_r (can be checked using 
the patch attached above.

Am I right???

Artyom

----------------------------------------
--- configure.in-bck    2008-11-26 07:57:37.913799169 +0200
+++ configure.in        2008-11-26 07:59:58.815035118 +0200
@@ -135,6 +135,7 @@
 AC_REPLACE_FUNCS(atoll timegm)
 AC_CHECK_FUNCS(vasprintf)
 AC_REPLACE_FUNCS(asprintf)
+AC_CHECK_FUNCS(gmtime_r)

 dnl ==============================
 dnl Checks for header files
-----------------------------------------
--- timegm.c-bck        2008-11-26 07:57:53.373496023 +0200
+++ timegm.c    2008-11-26 08:01:43.612977664 +0200
@@ -30,7 +30,7 @@

 time_t timegm(struct tm *tm) {
   time_t temp_ltime;
-  struct tm *temp_gm;
+  struct tm temp_gm;

   if (!tm) {
     temp_ltime = 0;
@@ -39,9 +39,13 @@
     temp_ltime = mktime(tm);
   }

-  temp_gm = gmtime(&temp_ltime);
+  #ifdef HAVE_TIMEGM_R
+    gmtime_r(&temp_ltime,&temp_gm);
+  #else
+    temp_gm = *gmtime(&temp_ltime);
+  #endif

-  return (time_t)(temp_ltime + (temp_ltime - mktime(temp_gm)));
+  return (time_t)(temp_ltime + (temp_ltime - mktime(&temp_gm)));
 }

 #endif /* HAVE_TIMEGM */
-------------------------------------------


      

Re: [libdbi-devel] Thread Safety question

[Florian F. <oc...@ve...>]

Hi,

On Tue, Nov 25, 2008 at 11:57:07AM +0100, Markus Hoenicka wrote:
> > gmtime_r, readdir_r exists on both Linux and Cygwin platforms and
> > according to man mages exists in FreeBSD as well:
> 
> Yes, I've mentioned those as I do have access to them and positively
> know they support the reentrant versions. I'm more concerned about the
> systems that I don't run myself, like OSX, the other *BSDs, Solaris
> and so on.

since all XSI compliant systems are required to provide thread- and
reentrant-safe functions, those functions are fortunately available on
the vast majority of systems.

So if you compile with
 CPPFLAGS="-D_POSIX_C_SOURCE=200112L -D_XOPEN_SOURCE=500 -D_REENTRANT -D_THREAD_SAFE"
you should get the XS-interface on all noted platforms, as far as I
know.

Of course, there are the occasional oddities here and there, as with all
``modern'' interfaces. For example, under Solaris, if you don't specify
the above flags, you need to at least specify
 CPPFLAGS="-D_POSIX_PTHREAD_SEMANTICS"
to get the standard-conforming versions of some functions, such as
`strtok_r'.

> > However readdir_r is less important because it is quite reasonable
> > to load shared objects from single thread.
> >
> 
> Once again, it is reasonable, but we must not rely on users doing only
> things that we assume as reasonable.

Agreed. I use libdbi for a plugin to collectd[0]. Each plugin is called
from a thread-pool, so using functions with static memory is an absolute
no-go. Especially since it's not that big of a deal, really. In my
experience, switching from `strerror' to `strerror_r' is the biggest
deal, because it's often used in lots of places.

> > Is there other points where non reenterable functions are used?
> 
> To the best of my knowledge, not in libdbi.

I didn't find any more thread-unsafe functions with a really quick
check:
  octo@leeloo:~/libdbi $ egrep -riI '\<(asctime|ctermid|ctime|getgrgid|getgrnam|getlogin|getpwnam|getpwuid|gmtime|localtime|rand|readdir|srand|strerror|strtok|tmpnam|ttyname)\>' drivers/
  octo@leeloo:~/libdbi $
Dunno if that's all thread-unsafe functions, but it's better than
nothing ;)

> This is a good excuse to make libdbi thread-safe :-)

Please consider the attached patch. It's against what I considered to be
the current CVS revision, but I haven't CVS for quite some time, so
please forgive me if I'm a bit off ;)

I did not remove that ugly `strtok == strtok_r' hack in
src/dbi_result.c, but I added a warning to it. Is that really necessary?

> > For the record I had seen in sqlite (not sqlite3) backend usage of
> > strtok. So it should be thread unsafe.

How does that make it thread-safe?

Regards,
-octo

[0] <http://collectd.org/>
-- 
Florian octo Forster
Hacker in training
GnuPG: 0x91523C3D
http://verplant.org/

Re: [libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Quoting Andreas Ericsson <ae...@op...>:

> Not necessarily. You can advertise certain interfaces as thread-safe
> while making no such statement of other interfaces. For example, it
> doesn't make sense to have certain init functions thread-safe, because
> they might be designed to run once and once only during the apps'
> lifetime.
>

But this is again one thing that we can't rely upon. We've implemented  
"instances" (will be available in 1.0) just to avoid that one part of  
a program shuts down libdbi while another is still using it. This was  
requested for a program which loads modules at runtime. Some of these  
modules are linked against libdbi, and it is not known at compile time  
when and which modules are loaded or unloaded.

>
> In all likelihood, those platforms will be the ones with decent thread-
> support to begin with, so it should be a reasonably safe bet that apps
> on such platforms will either not be multi-threaded anyway, or that it's
> relatively straightforward to use reentrant functions in libdbi there.
>

Agreed. But it won't hurt if everyone checks for the drivers and  
platforms he uses. After all, it is basically just readdir(),  
gmtime(), and strtok() which are prone to cause problems.

regards,
Markus


-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] Thread Safety question

[Andreas E. <ae...@op...>]

Markus Hoenicka wrote:
> Quoting Artyom <art...@ya...>:
> 
>> I'm mostly interested in accessing different dbi_conn objects
>> from same thread in safe way.
>>
>> So, it is quite reasonable to call dbi_initialize from single thread
>> and call dbi_shutdown from single thread.
>>
> 
> This is perfectly reasonable. However, if we advertise libdbi as  
> thread-safe, we must also accommodate programmers who use one instance  
> per thread, for whatever reason.
> 

Not necessarily. You can advertise certain interfaces as thread-safe
while making no such statement of other interfaces. For example, it
doesn't make sense to have certain init functions thread-safe, because
they might be designed to run once and once only during the apps'
lifetime.

> 
>> It would be great, at least it is possible to know what
>> backends are not safe.
>>
>> For the record I had seen in sqlite (not sqlite3) backend
>> usage of strtok. So it should be thread unsafe.
>>
>> What happens with others?
>>
> 
> I'll start reviewing the drivers as soon as time permits. We may end  
> up labelling individual drivers as thread-safe if we can't get all of  
> them fixed. We may also end up claiming thread-safety only for a  
> subset of platforms (i.e. those which implement reentrant versions of  
> functions which we cannot do without).
> 

In all likelihood, those platforms will be the ones with decent thread-
support to begin with, so it should be a reasonably safe bet that apps
on such platforms will either not be multi-threaded anyway, or that it's
relatively straightforward to use reentrant functions in libdbi there.

-- 
Andreas Ericsson                   and...@op...
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: [libdbi-devel] Thread Safety question

[Rainer G. <rge...@hq...>]

An important thing (which you would consider to be reasonable) is that
no multiple threads access the very same connection object. This causes
at least problems with MySQL, but I would expect also with other
database engines. It may be worth adding that to a piece of doc.

Rainer

> -----Original Message-----
> From: Markus Hoenicka [mailto:mar...@mh...]
> Sent: Tuesday, November 25, 2008 11:57 AM
> To: art...@ya...
> Cc: lib...@li...
> Subject: Re: [libdbi-devel] Thread Safety question
> 
> Quoting Artyom <art...@ya...>:
> 
> > I'm mostly interested in accessing different dbi_conn objects
> > from same thread in safe way.
> >
> > So, it is quite reasonable to call dbi_initialize from single thread
> > and call dbi_shutdown from single thread.
> >
> 
> This is perfectly reasonable. However, if we advertise libdbi as
> thread-safe, we must also accommodate programmers who use one instance
> per thread, for whatever reason.
> 
> > gmtime_r, readdir_r exists on both Linux and Cygwin platforms and
> > according to man mages exists in FreeBSD as well:
> >
> 
> Yes, I've mentioned those as I do have access to them and positively
> know they support the reentrant versions. I'm more concerned about the
> systems that I don't run myself, like OSX, the other *BSDs, Solaris
> and so on.
> 
> > However readdir_r is less important because it is quite reasonable
> > to load shared objects from single thread.
> >
> 
> Once again, it is reasonable, but we must not rely on users doing only
> things that we assume as reasonable.
> 
> > Also, I had seen that gmtime used from timegm only if it
> > is not implemented. (Undef cygwin/Linux it is implemented
> > as GNU extension). Thus, gmttime shoudn't be called.
> >
> 
> It is called whenever a platform does not implement timegm. We've had
> at least one such platform, otherwise there wouldn't be a workaround.
> 
> There is another function in dbi_result.c which uses gmtime:
> dbi_result_get_as_string_copy_idx(). This needs to be fixed too.
> 
> 
> > Is there other points where non reenterable functions are used?
> 
> To the best of my knowledge, not in libdbi.
> 
> >
> > I started to use the library for CppCMS project that naively
> > runs in multiple threads. I assumed that the library is thread
> > safe, however if I'm wrong it may be quite problematic for me.
> >
> 
> This is a good excuse to make libdbi thread-safe :-)
> 
> > It would be great, at least it is possible to know what
> > backends are not safe.
> >
> > For the record I had seen in sqlite (not sqlite3) backend
> > usage of strtok. So it should be thread unsafe.
> >
> > What happens with others?
> >
> 
> I'll start reviewing the drivers as soon as time permits. We may end
> up labelling individual drivers as thread-safe if we can't get all of
> them fixed. We may also end up claiming thread-safety only for a
> subset of platforms (i.e. those which implement reentrant versions of
> functions which we cannot do without).
> 
> regards,
> Markus
> 
> --
> Markus Hoenicka
> mar...@ca...
> (Spam-protected email: replace the quadrupeds with "mhoenicka")
> http://www.mhoenicka.de
> 
> 
>
-----------------------------------------------------------------------
> --
> This SF.Net email is sponsored by the Moblin Your Move Developer's
> challenge
> Build the coolest Linux based applications with Moblin SDK & win great
> prizes
> Grand prize is a trip for two to an Open Source event anywhere in the
> world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> libdbi-devel mailing list
> lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libdbi-devel

Re: [libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Quoting Artyom <art...@ya...>:

> I'm mostly interested in accessing different dbi_conn objects
> from same thread in safe way.
>
> So, it is quite reasonable to call dbi_initialize from single thread
> and call dbi_shutdown from single thread.
>

This is perfectly reasonable. However, if we advertise libdbi as  
thread-safe, we must also accommodate programmers who use one instance  
per thread, for whatever reason.

> gmtime_r, readdir_r exists on both Linux and Cygwin platforms and
> according to man mages exists in FreeBSD as well:
>

Yes, I've mentioned those as I do have access to them and positively  
know they support the reentrant versions. I'm more concerned about the  
systems that I don't run myself, like OSX, the other *BSDs, Solaris  
and so on.

> However readdir_r is less important because it is quite reasonable
> to load shared objects from single thread.
>

Once again, it is reasonable, but we must not rely on users doing only  
things that we assume as reasonable.

> Also, I had seen that gmtime used from timegm only if it
> is not implemented. (Undef cygwin/Linux it is implemented
> as GNU extension). Thus, gmttime shoudn't be called.
>

It is called whenever a platform does not implement timegm. We've had  
at least one such platform, otherwise there wouldn't be a workaround.

There is another function in dbi_result.c which uses gmtime:  
dbi_result_get_as_string_copy_idx(). This needs to be fixed too.


> Is there other points where non reenterable functions are used?

To the best of my knowledge, not in libdbi.

>
> I started to use the library for CppCMS project that naively
> runs in multiple threads. I assumed that the library is thread
> safe, however if I'm wrong it may be quite problematic for me.
>

This is a good excuse to make libdbi thread-safe :-)

> It would be great, at least it is possible to know what
> backends are not safe.
>
> For the record I had seen in sqlite (not sqlite3) backend
> usage of strtok. So it should be thread unsafe.
>
> What happens with others?
>

I'll start reviewing the drivers as soon as time permits. We may end  
up labelling individual drivers as thread-safe if we can't get all of  
them fixed. We may also end up claiming thread-safety only for a  
subset of platforms (i.e. those which implement reentrant versions of  
functions which we cannot do without).

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] Thread Safety question

[Artyom <art...@ya...>]

Hello,

> there yet. Some of the recent changes (available in CVS
> only) include
> "instances" which allow threads to use the
> library independently of
> each other. This feature was critical for applications that
> load and
> unload modules which in turn are linked against libdbi.

I'm mostly interested in accessing different dbi_conn objects
from same thread in safe way.

So, it is quite reasonable to call dbi_initialize from single thread
and call dbi_shutdown from single thread.

> However, I
> recall that libdbi uses gmtime for time conversions and
> readdir to
> locate available drivers. I don't know whether every
> supported OS has
> the reentrant forms gmtime_r and readdir_r (FreeBSD, Linux,
> and Cygwin
> do). If they do, this would be easy to fix.

gmtime_r, readdir_r exists on both Linux and Cygwin platforms and
according to man mages exists in FreeBSD as well:

http://www.gsp.com/cgi-bin/man.cgi?section=3&topic=gmtime_r
http://www.gsp.com/cgi-bin/man.cgi?section=3&topic=readdir_r

However readdir_r is less important because it is quite reasonable
to load shared objects from single thread.

Also, I had seen that gmtime used from timegm only if it
is not implemented. (Undef cygwin/Linux it is implemented
as GNU extension). Thus, gmttime shoudn't be called.

Is there other points where non reenterable functions are used?

I started to use the library for CppCMS project that naively
runs in multiple threads. I assumed that the library is thread
safe, however if I'm wrong it may be quite problematic for me.

> I'd be glad if everyone could
> have a look at
> "their" driver and client library and let the
> list know whether or not
> it is thread-safe.
>

It would be great, at least it is possible to know what
backends are not safe.

For the record I had seen in sqlite (not sqlite3) backend
usage of strtok. So it should be thread unsafe.

What happens with others?

Thanks,
  Artyom


      

Re: [libdbi-devel] Announcement: C++ Wrapper for libdbi

[Artyom <art...@ya...>]

Thanks

Artyom


--- On Tue, 11/25/08, Markus Hoenicka <mar...@mh...> wrote:

> From: Markus Hoenicka <mar...@mh...>
> Subject: [libdbi-devel] Announcement: C++ Wrapper for libdbi
> To: art...@ya...
> Cc: lib...@li...
> Date: Tuesday, November 25, 2008, 1:30 AM
> Hi Artyom,
> 
> thanks for sharing the information. I'll make sure to
> add a link to
> the libdbi homepage.
> 
> regards,
> Markus
> 
> Artyom writes:
>  > Hello, 
>  > 
>  > I have written a C++ Wrapper for libdbi called dbixx
> as a part of CppCMS 
>  > Project. It gives C++ friendly access to libdbi. I
> think,
>  > many libdbi users may be interested in it.
>  > 
>  > Documentation Wiki:
>  > http://art-blog.no-ip.info/wikipp/en/page/ref_dbixx
>  > Download:
>  >
> http://sourceforge.net/project/showfiles.php?group_id=209965
>  > 
>  > Best Regards,
>  >    Artyom
>  > 
>  > 
>  >       
>  > 
>  >
> -------------------------------------------------------------------------
>  > This SF.Net email is sponsored by the Moblin Your
> Move Developer's challenge
>  > Build the coolest Linux based applications with
> Moblin SDK & win great prizes
>  > Grand prize is a trip for two to an Open Source event
> anywhere in the world
>  >
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
>  > _______________________________________________
>  > libdbi-devel mailing list
>  > lib...@li...
>  >
> https://lists.sourceforge.net/lists/listinfo/libdbi-devel
> 
> -- 
> Markus Hoenicka
> mar...@ca...
> (Spam-protected email: replace the quadrupeds with
> "mhoenicka")
> http://www.mhoenicka.de


      

[libdbi-devel] Thread Safety question

[Markus H. <mar...@mh...>]

Hi Artyom,

we have made some progress towards this goal, but I'm afraid we're not
there yet. Some of the recent changes (available in CVS only) include
"instances" which allow threads to use the library independently of
each other. This feature was critical for applications that load and
unload modules which in turn are linked against libdbi. However, I
recall that libdbi uses gmtime for time conversions and readdir to
locate available drivers. I don't know whether every supported OS has
the reentrant forms gmtime_r and readdir_r (FreeBSD, Linux, and Cygwin
do). If they do, this would be easy to fix.

I don't know all drivers in sufficient detail to say whether or not
they are thread-safe. We'd have to do a similar code audit to fix
whatever is necessary. I'd be glad if everyone could have a look at
"their" driver and client library and let the list know whether or not
it is thread-safe.

regards,
Markus

Artyom writes:
 > Hello,
 > 
 > I want to know if libdbi is thread safe, meaning:
 > 
 > 1. Safety for accessing the library form different threads 
 >    with different connection objects.
 > 2. I do not need safety on using same connection from different
 >    threads.
 > 
 > Usually this means:
 > 
 > 1. No thread unsafe functions like strtok 
 > 2. No static data that can be accessed from different threads -- full
 >    reenterability
 > 
 > Is this library thread save? Is there specific backends that are not
 > safe? (For example I had seen that sqlite3 backed had removed in 8.3
 > usage of strtoc)
 > 
 > Thank you,
 >   Artyom
 > 
 > 
 >       
 > 
 > -------------------------------------------------------------------------
 > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
 > Build the coolest Linux based applications with Moblin SDK & win great prizes
 > Grand prize is a trip for two to an Open Source event anywhere in the world
 > http://moblin-contest.org/redirect.php?banner_id=100&url=/
 > _______________________________________________
 > libdbi-devel mailing list
 > lib...@li...
 > https://lists.sourceforge.net/lists/listinfo/libdbi-devel

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[libdbi-devel] Announcement: C++ Wrapper for libdbi

[Markus H. <mar...@mh...>]

Hi Artyom,

thanks for sharing the information. I'll make sure to add a link to
the libdbi homepage.

regards,
Markus

Artyom writes:
 > Hello, 
 > 
 > I have written a C++ Wrapper for libdbi called dbixx as a part of CppCMS 
 > Project. It gives C++ friendly access to libdbi. I think,
 > many libdbi users may be interested in it.
 > 
 > Documentation Wiki:
 > http://art-blog.no-ip.info/wikipp/en/page/ref_dbixx
 > Download:
 > http://sourceforge.net/project/showfiles.php?group_id=209965
 > 
 > Best Regards,
 >    Artyom
 > 
 > 
 >       
 > 
 > -------------------------------------------------------------------------
 > This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
 > Build the coolest Linux based applications with Moblin SDK & win great prizes
 > Grand prize is a trip for two to an Open Source event anywhere in the world
 > http://moblin-contest.org/redirect.php?banner_id=100&url=/
 > _______________________________________________
 > libdbi-devel mailing list
 > lib...@li...
 > https://lists.sourceforge.net/lists/listinfo/libdbi-devel

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[libdbi-devel] Thread Safety question

[Artyom <art...@ya...>]

Hello,

I want to know if libdbi is thread safe, meaning:

1. Safety for accessing the library form different threads 
   with different connection objects.
2. I do not need safety on using same connection from different
   threads.

Usually this means:

1. No thread unsafe functions like strtok 
2. No static data that can be accessed from different threads -- full
   reenterability

Is this library thread save? Is there specific backends that are not
safe? (For example I had seen that sqlite3 backed had removed in 8.3
usage of strtoc)

Thank you,
  Artyom


      

[libdbi-devel] Announcement: C++ Wrapper for libdbi

[Artyom <art...@ya...>]

Hello, 

I have written a C++ Wrapper for libdbi called dbixx as a part of CppCMS 
Project. It gives C++ friendly access to libdbi. I think,
many libdbi users may be interested in it.

Documentation Wiki:
http://art-blog.no-ip.info/wikipp/en/page/ref_dbixx
Download:
http://sourceforge.net/project/showfiles.php?group_id=209965

Best Regards,
   Artyom


      

Re: [libdbi-devel] [Libdbi-drivers-devel] libdbi oracle ocilib

[Markus H. <mar...@mh...>]

vincent rogier writes:
 > For the cleanup function, it's not an oligation but would be nice ! At the
 > moment i hold a counter to call the initialization and cleanup functions in
 > the libdbi connect and disconnect functions.
 > 

As per your request I've added a dbd_finalize() function to all
drivers which is called from libdbi just before the driver is
dlclose()'d. This usually happens when an instance (new interface) or
the library (old interface) is shut down. The dbd_finalize() function
is the proper place to do any cleanup in the client libraries. I've
noted that the ingres driver has a FIXME note saying it would require
such a shutdown. I'd appreciate if the ingres maintainer could add the
required code to do so.

The current cvs revision of libdbi-drivers (1.0-pre2) which has these
changes requires the current cvs revision of libdbi (also
1.0-pre2). Let me know if there are any problems.

regards,
Markus

-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

Re: [libdbi-devel] dbcheck build error?

[J. H. F. <jo...@gm...>]

Sorry, wrong list.

Re: [libdbi-devel] dbcheck build error?

[Markus H. <mar...@mh...>]

Hi,

is that libdbi-related in any way (except that bacula apparently uses  
libdbi)? dbcheck.c appears to be a bacula source file.

regards,
Markus

Quoting João Henrique Freitas <jo...@gm...>:

> Hello,
>
> ==Entering directory /.../regress/build/src/tools
> make[2]: Entering directory `/.../regress/build/src/tools'
> Compiling bsmtp.c
> Compiling dbcheck.c
> Compiling drivetype.c
> dbcheck.c: In function 'bool check_idx(const char*)':
>
> dbcheck.c:1346: error: 'connst' was not declared in this scope
>
>
> http://regress.bacula.org/viewBuildError.php?buildid=608
> http://regress.bacula.org/viewBuildError.php?buildid=633
>
>
> Thanks
> --
> -----------------------------------------------------------
> João Henrique Freitas - joaohf_at_gmail.com
> Campinas-SP-Brasil
> BSD051283
> LPI 1
> http://www.joaohfreitas.eti.br
> -------------------------------------------------------------------------
> This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
> Build the coolest Linux based applications with Moblin SDK & win great prizes
> Grand prize is a trip for two to an Open Source event anywhere in the world
> http://moblin-contest.org/redirect.php?banner_id=100&url=/
> _______________________________________________
> libdbi-devel mailing list
> lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libdbi-devel
>



-- 
Markus Hoenicka
mar...@ca...
(Spam-protected email: replace the quadrupeds with "mhoenicka")
http://www.mhoenicka.de

[libdbi-devel] dbcheck build error?

[J. H. F. <jo...@gm...>]

Hello,

==Entering directory /.../regress/build/src/tools
make[2]: Entering directory `/.../regress/build/src/tools'
Compiling bsmtp.c
Compiling dbcheck.c
Compiling drivetype.c
dbcheck.c: In function 'bool check_idx(const char*)':

dbcheck.c:1346: error: 'connst' was not declared in this scope


http://regress.bacula.org/viewBuildError.php?buildid=608
http://regress.bacula.org/viewBuildError.php?buildid=633


Thanks
-- 
-----------------------------------------------------------
João Henrique Freitas - joaohf_at_gmail.com
Campinas-SP-Brasil
BSD051283
LPI 1
http://www.joaohfreitas.eti.br