Re: [opendbx] Future plans again

[Norbert S. <no...@li...>]

Hi Thibault

> One or more connections? Can you point a backend that does this?
> (pgsql seems to keep only one alive)

Every native database library supports more than one connection. What is
not supported are several simultaneous queries resp. result sets per
connection.

> To allocate on stack and leave the mallocs to the client library, I
> would see something like: struct odbx_conn { /* First comes the
> common storage. */ int initialized; int backend_type;
>
> /* Then comes the storage that is specific to each backend. */ void
> *generic; }; I realized (sigh) that there is no other choice but to
> have this void* pointer. At initialization for pgsql it would give
> generic = (pgconn*)PQconnectdbParams(...). The malloc is left to the
> client library. In the client program it would look like: struct
> odbx_conn conn; odbx_init(&conn, ...);

OK, but you don't gain much because this is only one malloc you will
save. One more is needed for the auxiliary data and I think several are
done by the native client libraries. The reason I opted against this was
the choices you have to care about when you a) get an already
initialized structure and b) get a structure that only seem to be
initialized due to random data.

> Good point. After careful thought however I still have two points to
> suggest: _ The idea to merge all backends was more a usability
> objection: it seems excessive to have to manually install each
> package, that will weigh a few ko. A 500ko package is acceptable.
> Technically, I believe (hope?) it is still easily feasible, by
> dynamically loading the client libraries for example (with dlopen and
> such functions). _ To avoid exposing the functions pointers in
> odbx_t, you could keep them is static internal arrays. You would then
> store a backend_type variable in odbx_t, that would be used to index
> the arrays of function pointers.

Well you have to know the structure of odbx_t but it's not exposed
anywhere. Nevertheless, the more details are hidden, the better for
avoiding mistakes.

>> The prepared statement API is on the todo list but I haven't had
>> time to implement this up to now.
>
> Actually, it is not only on prepared statements. I know that sending
> the values separately can be tricky for the motto of odbx. Indeed,
> the client will either have to provide one statement for each backend
> (INSERT ... VALUES ($1,$2,$3) for pgsql), or a preprocessing will
> have to be added. I still argue in favor of sending the values only
> separately, since it is secure by design: the simplest solution is
> the most secure. I will not insist on manual escaping being insecure
> by design ^^.

Many native database libraries support prepared statements. It's
relatively simple:
- Send the SQL with markers to the server and get back a handle
- bind your values to this handle
- Send only the data to the server (one or several times)
- Get the binary values from the result set

Contrary to e.g. DBI I would favour a single function that copies the
data to a memory area provided by the client. This can be used with all
types of data and is faster and safer than allocating memory that must
be freed by the client.

> I know this would seem teasing, but can you point a backend as
> example? I would like to imagine a proper design but have still lack
> of knowledge about the variety of implementations.

mssql, odbc, oracle, sybase ... they all use several functions for
initialization before binding to the database.

> I was not clear enough on this question. I consider that: _
> unbinding/binding to a different database is expected to be uncommon,
> most of the time users will want to connect to a single database. _
> reinit a whole connection is also uncommon, because in case of error
> odbx could try to reconnect internally. The two cases where you
> reinit would be if odbx returned error and you waited a few minutes
> to reconnect, or if you want to change the host.

It's rather common. If you get a fatal error you have to rebind and
it saves much time when using the named databases if you don't have to
reinitialize the whole data structures again.

> OK. Will you use positive error codes to return values?

No, as this would interfere with the positive return values for e.g.
odbx_result().

> One more thing, maybe the result handle should not be separated from
> the odbx_t handle in function calls. This is not a technical
> discussion, but users might think that they can free the connection
> and still use the result handle (Is it actually possible??).

The initial design decision was to allow having multiple result sets per 
connection even if most databases doesn't support this. But I didn't 
want to limit the client libraries that would be able to so even if none 
of the currently implemented ones can do so :-)


Norbert