Re: [opendbx] Future plans again

[Thibault R. <tr...@kt...>]

> The reason why the memory for odbx_t is returned via odbx_t** is because
> it has to be allocated dynamically anyway (there can be one or more
> connections) and it's not returned by the function to be able to return
> an error code in case anything goes wrong. I'm not sure how you would
> like to do this on the stack.

One or more connections? Can you point a backend that does this? (pgsql seems to keep only one alive)
I completely agree for the return code.
To allocate on stack and leave the mallocs to the client library, I would see something like:
struct odbx_conn {
    /* First comes the common storage. */
    int initialized;
    int backend_type;

    /* Then comes the storage that is specific to each backend. */
    void *generic;
};
I realized (sigh) that there is no other choice but to have this void* pointer.
At initialization for pgsql it would give generic = (pgconn*)PQconnectdbParams(...). The malloc is left to the client library.
In the client program it would look like:
struct odbx_conn conn;
odbx_init(&conn, ...);

> These points are all based on your assumption that everybody only needs
> a single .so file containing all "backends". Unfortunately, this isn't
> true. The reason for separating the core library from the backends is
> that they can be installed separately. Think about the Linux
> distributions building .rpm or .deb files: There are dependencies
> between the "backends" and the native database library. If you install
> the OpenDBX library containing all backends as single library, the
> distributions must install all native database libraries as well to
> satisfy the dependencies. This is nothing the user or the packager would
> like to to. The SINGLELIB option is mainly for embedded systems where
> the requirements are exactly known and the memory/space is limited.

Good point. After careful thought however I still have two points to suggest:
_ The idea to merge all backends was more a usability objection: it seems excessive to have to manually install each package, that will weigh a few ko. A 500ko package is acceptable.
Technically, I believe (hope?) it is still easily feasible, by dynamically loading the client libraries for example (with dlopen and such functions).
_ To avoid exposing the functions pointers in odbx_t, you could keep them is static internal arrays. You would then store a backend_type variable in odbx_t, that would be used to index the arrays of function pointers.

> The prepared statement API is on the todo list but I
> haven't had time to implement this up to now.

Actually, it is not only on prepared statements. I know that sending the values separately can be tricky for the motto of odbx.
Indeed, the client will either have to provide one statement for each backend (INSERT ... VALUES ($1,$2,$3) for pgsql), or a preprocessing will have to be added.
I still argue in favor of sending the values only separately, since it is secure by design: the simplest solution is the most secure.
I will not insist on manual escaping being insecure by design ^^.

> Different native database libraries have different requirements. What
> you say is true for PostgreSQL, but not for all other DBMS libraries.
> Furthermore, separating init() from bind() allows us to specify options
> in a generic way and they are still available for rebinding.

I know this would seem teasing, but can you point a backend as example? I would like to imagine a proper design but have still lack of knowledge about the variety of implementations.
I was not clear enough on this question. I consider that:
_ unbinding/binding to a different database is expected to be uncommon, most of the time users will want to connect to a single database.
_ reinit a whole connection is also uncommon, because in case of error odbx could try to reconnect internally. The two cases where you reinit would be if odbx returned error and you waited a few minutes to reconnect, or if you want to change the host.

> If one should ever become obsolete, it has to stay until the next major
> release because otherwise we would break the ABI.

OK. Will you use positive error codes to return values?

One more thing, maybe the result handle should not be separated from the odbx_t handle in function calls. This is not a technical discussion, but users might think that they can free the connection and still use the result handle (Is it actually possible??).

Thank you for your quick answers!

Regards,
Thibault