libdbi-users — General libdbi discussion and support

[libdbi-users] memory issues

[David P. <da...@ne...>]

Hi all,

After wasting almost a full day debugging weird seemingly random bugs on
project I'm using libdbi in, I found a few memory-related bugs in libdbi.
I think these were messing up my project's stack, causing such odd results
(the segfaults were occurring all over the place, not just in libdbi).
Anyway, electric-fence has served me well, and I'm just about done fixing
everything. Just wanted to let everyone know in case they were pulling
their hair out all day debugging like me :)

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

[libdbi-users] RE: Just for RDBMS's?

[Anthony W. Y. <Ant...@EC...>]

Okay, a few hints about multi-value :-) which to my mind is a far better
setup than SQL ...

Pick was originally a complete environment (the database WAS the computer, a
bit like I understand the AS/400 was). It was also coded in such a way that
the entire filestore/database was accessed as virtual memory (which made for
blindingly fast access). As a result, many Pick systems ran on unbelievably
small hardware requirements. For example, think of an NT/MS-SQL system on a
pc. Now replace the software with Oracle9i/linux - it'll run a lot faster,
won't it :-) Now replace the software again with D3/linux - you'll get the
same speed increase AGAIN (okay, you'll have to rewrite all your application
code, but ...)

And there are a hell of a lot of crappily designed mv systems out there, but
that's because they were designed, written and maintained by USERS - can you
imagine a SQL DP shop happily letting users write large chunks of their own
systems? I'm not saying I agree with it, but there are a lot of systems out
there like that that work.

Anyway, enough of the advocacy. To look at stuff about multi-value, go to
www.maverick-dbms.org which is the site for the Open Source clone we're
writing. This is part of the multi-value ring which will take you to other
sites like jBASE, Ardent/Informix/IBM/insert-name-of-choice-here... There
are three major vendors and several minor ones all selling variants on the
same basic theme. So picking up your database and moving from one vendor's
mv platform to another vendor's is probably simpler than moving from one SQL
engine to another, actually!

The best way to describe mv is probably the "new" concept in SQL of tables
within tables. Let's assume we have a country file (table in SQL parlance).
This contains records (rows in SQL parlance) and fields (columns in SQL
parlance). The two crucial advantages mv has is that within my country file
I can define two fields as "state code" and "state name". So for any given
record I can store a list of codes and names in these two fields - the
"cell" defined by country record and state code actually contains a table of
state codes, and similarly with state name. These are related because mv
contains a concept of order - by "associating" these two fields, you are
saying that the first entry in each field form a row in their own right, etc
etc. I believe this concept in MS-speak is called a "chaptered row-set". And
in XML it's a repeating attribute.

I've heard it said that programmers with mv experience make far better XML
programmers because they understand this concept of repeating attributes,
and that conventional SQL programmers (almost) never use tables within
tables because they cannot understand it or see the point. Myself - while I
believe it is easily misused, I wouldn't be without it - it's like C - a
language for experts widely abused by novices ... :-(


NB - x-post - I am not subscribed to the libdbi list so I won't see any
responses unless I'm cc'd - and if the list rejects non-subscribers this
message won't get onto it.

-----Original Message-----
From: David Parker [mailto:da...@ne...]
Sent: 08 August 2001 03:00
To: Anthony W. Youngman
Subject: Re: Just for RDBMS's?


Hi Wol,

> As per the subject, will it work on non-relational DBs such as object or
multi-value etc?

Although relational DBs are what I originally had in mind when writing
libdbi,
I don't think it has to be limited to just that. I'm not exactly sure what
you
mean by object and multi-value databases, all I have experience with are
mSQL/MySQL/PostgreSQL/MS-SQL/Access(shudder). Do you have any URLs I can
check
out to learn more about those other DB types, or even example code or
documentation showing how they work? Also, it would be great if you could
email everything to lib...@li..., maybe some good
discussion can get started with other people who have more diverse DB
experience.

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------




This transmission is intended for the named recipient only. It may contain
private and confidential information. If this has come to you in error you
must not act on anything disclosed in it, nor must you copy it, modify it,
disseminate it in any way, or show it to anyone. Please e-mail the sender to
inform us of the transmission error or telephone ECA International
immediately and delete the e-mail from your information system.
Telephone numbers for ECA International offices are: London +44 (0)20 7351
5000, Hong Kong + 852 (0)2 121 2388 and New York +1 (0)212 582 2333.

Re: [libdbi-users] num_fields, field_names and field_types

[David P. <da...@ne...>]

Hey Dave,

> Firstly, thanks to David Parker and Mark Tobenkin for libdbi - a very useful
> project.

;)

> My first problem is (unless I've missed something obvious) that the API
> doesn't give access to the numfields, field_names and
> field_types/field_attribs fields within a result structure.  This
information
> is required when the SQL statement being executed isn't known in advance.

During an early rewrite of the API, these functions got lost. They should
magically reappear in the next version. I was also wondering if there are any
other "accessor"-like functions that need to be added -- I was thinking that
something like a GUI database query construction program or something
similarly dynamic might need access to other information from DBI.

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

Re: [libdbi-users] libdbi vs ODBC

[David P. <da...@ne...>]

Hi Jacob,

> As a user of both PostreSQL and MySQL I am really thrilled to see any
> attempt to unite these two worlds.  However, I have not understood what
> the conceptual difference between ODBC and libdbi is.  So:
>
> How does libdvi differ from ODBC?  Which should I choose for abstract
> database interfacing?

I believe ODBC's general goal is the same as libdbi: to create an abstract DB
API. Other than that, I actually am not sure of the differences. I'm pretty
sure I looked for an ODBC library in C for linux before I started libdbi, but
either didn't find anything or didn't like what I found. The last time I used
ODBC was for a one-day project in Windows about 5(?) years ago, so the details
are hazy. I'll look into this, it would go well into a FAQ, but does anyone
else who is familiar with ODBC have any comments?

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

Re: [libdbi-users] Re: libdbi and a few things

[David P. <da...@ne...>]

> On Tue, Aug 07, 2001, David Parker wrote:
> > Hi Tim,
> ahem, thats "Tom"

Whoops :) Gotta get some sleep.

> heh, good point.  I was thinking of a win32 version of the lib, but I know
> nothing of win32 or mssql for that matter........thats what I get for
> writing emails when Im boozed :-)
>
> but, Im sure we will need to write a plugin or two.....

Yesterday someone wrote to me who got libdbi compiled on Win NT using cygwin,
so maybe the booze wasn't so far off. Also, I did a quick google search and
found this, it might just work:

http://www.freetds.org/faq.html#11

Cheers,
Divad :)

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

[libdbi-users] Re: libdbi and a few things

[Tom v. S. <tv...@et...>]

Hey David,


On Tue, Aug 07, 2001, David Parker wrote:
> Hi Tim,

ahem, thats "Tom"

:-)

> > - We plan to support as many RDBMS's as possible, so we will be writing
> >   some more plugins for you, first one will likely be MS-SQL...
> 
> Excellent! How can that be done... surely MS doesn't supply a linux client
> library, do they?

heh, good point.  I was thinking of a win32 version of the lib, but I know
nothing of win32 or mssql for that matter........thats what I get for
writing emails when Im boozed :-)

but, Im sure we will need to write a plugin or two.....

> > - We also need a PHP API, so we may look into building a PHP module out of
> >   your lib.  This could warrent a project of its own, Im really not sure.
> > Let me kow what you think.
> 
> It would probably require more discussion, but it would most likely have to be
> an independent or side project (although I've never worked with PHP's
> internals so I'm not really sure what it would entail)

it would require a great deal of discussion....and I have never done it
before either, but I have learned quite a bit since I started this project,
so I figure I might as well learn how to write PHP modules in C :-)
 
> > - We have a guy who had plans to basically build a C++ version of what you
> >   have....he registered the SF project a few days ago.  He is a big fan of
> > OO so I suspect that he will still want to do this now that we know about
> > your project, but I'll try to put him in touch with you so maybe you can
> > coordinate efforts or exchange useful info with him...
> 
> IIRC, there are some other C++ projects that try to do the same thing as
> libdbi. I haven't investigated any of them, but it would probably be good to
> check them out before starting a new one from scratch :)

Thanks for the tip.  From a quick FM search I just found a DBI PHP
class which I never knew existed......so Ill check around some more

-Tom
  ^
  :-)


-- 
}+-----------------------------------------+{
-|  Tom von Schwerdtner  <tv...@et...>  |-
-|              www.etria.org              |-
}+-----------------------------------------+{

Re: [libdbi-users] Seg Faulting.

[David P. <da...@ne...>]

Hi Tim,

> I did that and it still seg faults.

I think Mark already got back to you, but for the sake of the mailing list
archive...

The problem is probably that the database connection can't be established for
whatever reason, but the mysql plugin still goes ahead as if the connection is
valid. This will be fixed in the next release (probably in a few days, a week
at most). If you have gdb installed and you compile both libdbi and the test
program with debugging enabled ("make debug" with libdbi, "gcc -g" with the
program using libdbi), it should be possible to check the mysql error message
(if you don't want to wait).

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

[libdbi-users] Re: libdbi and a few things

[David P. <da...@ne...>]

Hi Tim,

(I forwarded this to the libdbi mailing lists to help prevent duplicated
efforts... hopefully you don't mind)

>   I'm working on a project called the Standard Mp3 Database Project.  The
> overall aim is to provide a standard database schema and a db-transparent
> API/lib to "bring together" all the current mp3/database apps out there as
> well as to encourage developers of new apps to provide database support
> by making it alot easier for them....blah blah.  Point being, it looks like
> your project could cut out alot of the work for us.

Wow, with such an abundance of mp3 database projects, that's a pretty
ambitious project :) Actually, the reason Mark and I started writing libdbi in
the first place was for our own MP3 database project. After a year of
production usage (in a radio station), it's being rewritten from scratch, and
should be done RSN(tm). If you'd like, contact me privately and I can send you
our DB schema in case you want to get ideas.

> - We plan to support as many RDBMS's as possible, so we will be writing
>   some more plugins for you, first one will likely be MS-SQL...

Excellent! How can that be done... surely MS doesn't supply a linux client
library, do they?

> - We also need a PHP API, so we may look into building a PHP module out of
>   your lib.  This could warrent a project of its own, Im really not sure.
> Let me kow what you think.

It would probably require more discussion, but it would most likely have to be
an independent or side project (although I've never worked with PHP's
internals so I'm not really sure what it would entail)

> - We have a guy who had plans to basically build a C++ version of what you
>   have....he registered the SF project a few days ago.  He is a big fan of
> OO so I suspect that he will still want to do this now that we know about
> your project, but I'll try to put him in touch with you so maybe you can
> coordinate efforts or exchange useful info with him...

IIRC, there are some other C++ projects that try to do the same thing as
libdbi. I haven't investigated any of them, but it would probably be good to
check them out before starting a new one from scratch :)

> BTW: your main download link is https://

Thanks, fixed.

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

Re: [libdbi-users] num_fields, field_names and field_types

[Brad P. <br...@li...>]

On Tuesday 07 August 2001 13:47, Dave Chapman wrote:
> Hello,
>
> Firstly, thanks to David Parker and Mark Tobenkin for libdbi - a very
> useful project.

I agree that its useful and much cleaner than ODBC.  One thing I noticed is 
that there is no support for date/time/timestamp fields which I think is 
required.

Any plans to add this?

PS: I would also like to write a C++ API above the C one when it gets a 
little further along.  A simple wrapper should do the trick and make working 
with it easier for us C++ types!

-- 
Brad Pepers
br...@li...

[libdbi-users] num_fields, field_names and field_types

[Dave C. <da...@dc...>]

Hello,

Firstly, thanks to David Parker and Mark Tobenkin for libdbi - a very useful 
project.

My first problem is (unless I've missed something obvious) that the API 
doesn't give access to the numfields, field_names and 
field_types/field_attribs fields within a result structure.  This information 
is required when the SQL statement being executed isn't known in advance.

Obviously, I can access these variables from my programs by casting the 
dbi_result variable to a dbi_result_t*, but this isn't pretty.

Was this a deliberate design decision, or do you have plans to add this 
functionality in a later version of the library?  If it's the latter, then 
I'm happy to contribute.

Thanks,

Dave.

Re: [libdbi-users] Seg Faulting.

[Tim G. <ti...@of...>]

> I don't think this is anywhere in the docs, so here: You'll need to link your
> programs agains BOTH libdbi and libdl - "gcc -ldl -ldbi foo.c".
> 
> In the docs, the example source is a bit wacked. First,
> "dbi_driver_open" should be "dbi_driver_new". Also, the "database"
> setting should be "dbname", and there is no "efficient-query" option
> as of now (although setting it won't hurt anyone). libdbi-users
> mailing list lib...@li...

I did that and it still seg faults.

[libdbi-users] libdbi vs ODBC

[Jakob L. <jl...@it...>]

Hi,

As a user of both PostreSQL and MySQL I am really thrilled to see any
attempt to unite these two worlds.  However, I have not understood what
the conceptual difference between ODBC and libdbi is.  So:

How does libdvi differ from ODBC?  Which should I choose for abstract
database interfacing?

Best regards,

- Jakob Lichtenberg

-- 
Jakob Lichtenberg - jl...@it... - http://www.itu.dk/people/jl

Re: [libdbi-users] Seg Faulting.

[David P. <da...@ne...>]

Hi Tim,

> I only just downloaded and installed libdbi, i took the exmaple program
> from the PDF and compiled, it seg faulted, i added some error handling
> and it still seg faults, can anyone offer some suggestions ?

Unfortunately, I forgot to update the example source code with the new version
of libdbi. Quoting from the download page:

===
NOTES: (stuff that'll be fixed in the next release)

I don't think this is anywhere in the docs, so here: You'll need to link your
programs agains BOTH libdbi and libdl - "gcc -ldl -ldbi foo.c".

In the docs, the example source is a bit wacked. First, "dbi_driver_open"
should be "dbi_driver_new". Also, the "database" setting should be "dbname",
and there is no "efficient-query" option as of now (although setting it won't
hurt anyone).
===

Those few changes should fix the segfaulting problem. Sorry about that!

David

-------
da...@ne...
www.neongoat.com
PGP Key ID/Fingerprint:
0xF90FFFE5 / F362 51F7 6D51 85EB AF68 75B9 D29B 1AFC F90F FFE5
-------

[libdbi-users] Seg Faulting.

[Tim G. <ti...@of...>]

I only just downloaded and installed libdbi, i took the exmaple program
from the PDF and compiled, it seg faulted, i added some error handling
and it still seg faults, can anyone offer some suggestions ?

Compile line is 

gcc test.c -o test -ldl -ldbi

Code is

NOTE: the printf("1\n"); works and then it seg faults, all the testing i
have done points to the connect.

Tim

#include <stdio.h>
#include <dlfcn.h>
#include <dbi/dbi.h>

int main() {
    dbi_driver driver;
    dbi_result result;

    double threshold = 4.333333;
    unsigned int idnumber;
    const char *fullname;

    if (dbi_initialize(NULL) == -1)
      {
        printf(" dbi_initialize Failed... Aborting\n");
        exit(0);
      }
    if ((driver = dbi_driver_open("mysql")) == NULL)
      {
        printf(" dbi_driver_open Failed... Aborting\n");
        exit(0);
      }

    dbi_driver_set_option(driver, "host", "localhost");
    dbi_driver_set_option(driver, "username", "username");
    dbi_driver_set_option(driver, "password", "password");
    dbi_driver_set_option(driver, "database", "database");
    dbi_driver_set_option_numeric(driver, "efficient-queries", 1);

printf("1\n");

    if (dbi_driver_connect(driver) == -1)
      {
printf("2a\n");
        printf(" Driver Connection Failed....Aborting\n");
        exit(0);
      }
printf("2b\n");
    result = dbi_driver_query(driver, "SELECT id, name FROM coders"
                                      "WHERE hours_of_sleep > %0.2f", threshold);


    while (dbi_result_next_row(result)) {
        idnumber = dbi_result_get_long(result, "id");
        fullname = dbi_result_get_string(result, "name");
        printf("%i. %s\n", idnumber, fullname);
    }

    dbi_result_free(result);
    dbi_driver_close(driver);
    dbi_shutdown();

    return 0;
}