libdbi-users

[libdbi-users] New API?

[Vikram N. A. <noe...@gm...>]

I just built libdbi and libdbi-drivers from svn. When recompiling my 
application, I got a warning saying, dbi_initialize, dbi_conn_new and 
dbi_shutdown has been deprecated.

I think the docs online are out of data:
http://libdbi.sourceforge.net/docs/programmers-guide/reference-core.html

What are those functions being replaced with?


Vikram.

[libdbi-users] New API?

[Markus H. <mar...@mh...>]

Vikram Noel Ambrose writes:
 > I just built libdbi and libdbi-drivers from svn. When recompiling my 
 > application, I got a warning saying, dbi_initialize, dbi_conn_new and 
 > dbi_shutdown has been deprecated.
 > 
 > I think the docs online are out of data:
 > http://libdbi.sourceforge.net/docs/programmers-guide/reference-core.html
 > 
 > What are those functions being replaced with?

The docs reflect the situation as of the latest official release. This
should probably be spelled out somewhere to make things clear. If you
fiddle with the current svn version, please build the programmers
guide from the sources.

The current svn version implements a new feature called "instances"
which required us to change the API slightly. In a nutshell, a libdbi
instance allows you to initialize libdbi, load and unload drivers, and
shut down the library independently from other parts of your program
which also use libdbi. The problem came up with some piece of software
which was designed to load modules at runtime. Some of these modules
used libdbi. In order to unload such a module properly, you'd have to
shut down the library. But that would have made libdbi unavailable to
the remaining modules.

Therefore the new interface uses instance handles to manage the
library. A single process can open as many instances as it needs, and
initialize and shut them down individually. The new initializer
requires a pointer to a dbi_inst structure:

int dbi_initialize_r(const char *driverdir, dbi_inst *pInst);

You'll need this handle later to create connections and to shut down
the instance:

dbi_conn dbi_conn_new_r(const char *name, dbi_inst Inst);
void dbi_shutdown_r(dbi_inst Inst);

The older versions (dbi_initialize, dbi_conn_new and dbi_shutdown) are
now implemented on top of these functions, using a single static instance
handle.

HTH
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] New API?

[Vikram N. A. <noe...@gm...>]

Markus Hoenicka wrote:
> The older versions (dbi_initialize, dbi_conn_new and dbi_shutdown) are
> now implemented on top of these functions, using a single static instance
> handle.
>   

Then why should those functions be deprecated? I'm sure the majority of 
users do not have such peculiar requirements. I vote for them staying.


Vikram.

Re: [libdbi-users] New API?

[Toby T. <to...@te...>]

On 22-Jan-10, at 9:19 PM, Vikram Noel Ambrose wrote:

> Markus Hoenicka wrote:
>> The older versions (dbi_initialize, dbi_conn_new and dbi_shutdown)  
>> are
>> now implemented on top of these functions, using a single static  
>> instance
>> handle.
>>
>
> Then why should those functions be deprecated? I'm sure the  
> majority of
> users do not have such peculiar requirements. I vote for them staying.

AFAIK the old functions are also not safe for threaded applications.

I think the deprecation is progressive because all in all it is  
better if users always specify their instance explicitly. Hidden  
global data has proven to be a liability in many such instances  
(strtok(), errno, ...)

--Toby

>
>
> Vikram.
>
> ---------------------------------------------------------------------- 
> --------
> Throughout its 18-year history, RSA Conference consistently  
> attracts the
> world's best and brightest in the field, creating opportunities for  
> Conference
> attendees to learn about information security's most important  
> issues through
> interactions with peers, luminaries and emerging and established  
> companies.
> http://p.sf.net/sfu/rsaconf-dev2dev
> _______________________________________________
> libdbi-users mailing list
> lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libdbi-users

Re: [libdbi-users] New API?

[Markus H. <mar...@mh...>]

Toby Thain <to...@te...> was heard to say:

> I think the deprecation is progressive because all in all it is
> better if users always specify their instance explicitly. Hidden
> global data has proven to be a liability in many such instances
> (strtok(), errno, ...)

This is certainly the rationale behind the decision to deprecate the  
old versions. Also, most applications will only have to define an  
additional structure and pass an additional argument in three function  
calls. Sanity doesn't always come that cheap.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

[libdbi-users] Timeout / Reconnect

[Vikram A. <noe...@gm...>]

I make use of the mysql driver and I find that when I come back to my PC 
the next day, the first query I issue always results in an error. I need 
to reconnect to the server for it to work again.

I've setup a callback to do the reconnect, but the original query is lost.

Is there a way of having the mysql driver, reconnect and then complete 
the query without erroring out, in the case of driver/server timeout? 
This way I do not lose my query to a timeout or need to have some sort 
of global command buffer that the callback would use to re-execute the 
failed query.


Vikram.

Re: [libdbi-users] Timeout / Reconnect

[Markus H. <mar...@mh...>]

Vikram Ambrose <noe...@gm...> was heard to say:

> Is there a way of having the mysql driver, reconnect and then complete
> the query without erroring out, in the case of driver/server timeout?
> This way I do not lose my query to a timeout or need to have some sort
> of global command buffer that the callback would use to re-execute the
> failed query.

Hi,

does the MySQL API provide enough information to tell from a failure  
of mysql_query that the connection has stalled? I've found a log entry  
in cvs which added a timeout option to the driver, so with that in  
place and a useful error message from libmysqlclient I'd say it's  
doable. We'd just have to loop over a limited number of retries,  
trying to reconnect each time before resending the query.

regards,
Markus


-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] Timeout / Reconnect

[Vikram A. <noe...@gm...>]

On 07/01/2010 03:12 AM, Markus Hoenicka wrote:
> Vikram Ambrose <noe...@gm...> was heard to say:
>
>> Is there a way of having the mysql driver, reconnect and then complete
>> the query without erroring out, in the case of driver/server timeout?
>> This way I do not lose my query to a timeout or need to have some sort
>> of global command buffer that the callback would use to re-execute the
>> failed query.
>
> Hi,
>
> does the MySQL API provide enough information to tell from a failure 
> of mysql_query that the connection has stalled? I've found a log entry 
> in cvs which added a timeout option to the driver, so with that in 
> place and a useful error message from libmysqlclient I'd say it's 
> doable. We'd just have to loop over a limited number of retries, 
> trying to reconnect each time before resending the query.
>

There is indeed enough information to find this scenario. Here is a very 
simple callback I use at the moment:

void DB_error_callback(dbi_conn conn, void *udata){
     int rv;
     const char *errormsg = NULL;

     rv = dbi_conn_error(DB_conn,&errormsg);
     if(strstr(errormsg,"gone away")){
         rv = dbi_conn_connect(DB_conn);
         if(rv){
             printf("ERROR: DB_error_callback: Tried to reconnect - 
failed\n");
         }else {
             printf("ERROR: DB_error_callback: Reconnected\n");
         }
     }
}

Re: [libdbi-users] Timeout / Reconnect

[Markus H. <mar...@mh...>]

Vikram Ambrose writes:
 > There is indeed enough information to find this scenario. Here is a very 
 > simple callback I use at the moment:
 > 
 > void DB_error_callback(dbi_conn conn, void *udata){
 >      int rv;
 >      const char *errormsg = NULL;
 > 
 >      rv = dbi_conn_error(DB_conn,&errormsg);
 >      if(strstr(errormsg,"gone away")){
 >          rv = dbi_conn_connect(DB_conn);
 >          if(rv){
 >              printf("ERROR: DB_error_callback: Tried to reconnect - 
 > failed\n");
 >          }else {
 >              printf("ERROR: DB_error_callback: Reconnected\n");
 >          }
 >      }
 > }
 > 

Hi,

I just found some time to fiddle with this. As it seems, your solution
is pretty much focused on MySQL. However, I assume that the server
going away is a problem of all database engines, except maybe the
embedded ones. Adding some code to the MySQL driver to fix your
original problem thus might be the wrong solution. Don't you
experience the same problems with other DB engines? In that case, we'd
have to fix libdbi, rather than a driver.

regards,
Markus

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] Timeout / Reconnect

[Vikram A. <noe...@gm...>]

On 07/15/2010 04:11 PM, Markus Hoenicka wrote:
>   I assume that the server going away is a problem of all database engines, except maybe the
> embedded ones. Adding some code to the MySQL driver to fix your
> original problem thus might be the wrong solution. Don't you
> experience the same problems with other DB engines? In that case, we'd
> have to fix libdbi, rather than a driver.
>
>    

I'm not sure Markus. I'm almost exclusively been using Mysql for a while 
now. In either case, I doubt a gone-awayreconnect/retransmit fix would 
be too intrusive to libdbi or the driver.

[libdbi-users] sqlite and binary

[Vikram A. <noe...@gm...>]

I can't seem to get BLOBs to work with SQLite.

Here is some test code.
http://en.pastebin.ca/2026862

Output on my machine is;

dbi_conn_connect: rv = 0:
data = ERROR

What am I doing wrong?



Vikram.

[libdbi-users] sqlite and binary

[<mar...@mh...>]

Vikram Ambrose writes:
 > I can't seem to get BLOBs to work with SQLite.
 > 
 > Here is some test code.
 > http://en.pastebin.ca/2026862
 > 
 > Output on my machine is;
 > 
 > dbi_conn_connect: rv = 0:
 > data = ERROR
 > 
 > What am I doing wrong?

Probably nothing, at least not in your own code. Your test code prints
the following on my box:

$ ./sqlite_binary_test
dbi_conn_connect: rv = 0: 
dbi_conn_query: 
data = close your eyes, this is binary

All I changed is the driver. I've used sqlite3 as I currently don't
have sqlite installed. However, as far as I can see there is no
difference between those drivers in terms of blob handling. Could you
test sqlite3 as well just to see whether it is a problem of your box
or indeed a problem of the driver.

Did you run make check in libdbi-drivers after building and installing
the drivers? The test program includes checks for blob handling, so if
there is a problem in the driver it should report errors. Runs fine here.

Also, could you send a few specs of your OS and libdbi/libdbi-drivers
versions?

regards,
Markus

P.S. your code contains a strlen() call on binary data. I assume you
are aware that this wouldn't work with arbitrary binary data.

-- 
Markus Hoenicka
http://www.mhoenicka.de
AQ score 38

Re: [libdbi-users] sqlite and binary

[Vikram A. <noe...@gm...>]

On 12/23/2010 02:35 PM, mar...@mh... wrote:
> Vikram Ambrose writes:
>   >  I can't seem to get BLOBs to work with SQLite.
>   >
>   >  Here is some test code.
>   >  http://en.pastebin.ca/2026862
>   >
>   >  Output on my machine is;
>   >
>   >  dbi_conn_connect: rv = 0:
>   >  data = ERROR
>   >
>   >  What am I doing wrong?
>
> Could you test sqlite3 as well just to see whether it is a problem of your box
> or indeed a problem of the driver.
>

Interestingly enough the test code works fine under sqlite3. Looks like 
a bug in the sqlite(2) driver so far.

> Did you run make check in libdbi-drivers after building and installing
> the drivers? The test program includes checks for blob handling, so if
> there is a problem in the driver it should report errors. Runs fine here.
>

make check gives me an error. Missing dependency probably?

test_dbi.c:33: fatal error: cgreen/cgreen.h: No such file or directory


> Also, could you send a few specs of your OS and libdbi/libdbi-drivers
> versions?


Using CVS versions of both. Check'd out yesterday.

The strlen() was used because I knew what the input data was.

On a side note, in general, I often use BLOB instead of VARCHAR/STRING 
for clear text strings as its much easier to deal with character 
encoding/locale at the application level than messing around with it at 
the DB level.

I suppose I'll just switchover to sqlite3. Doesn't make a difference to 
me really. Just needed a database that could be used for a test run 
without the user requiring administrative privileges.

thanks,


Vikram.