Re: [Firebird-devel] library soname for fbclient and fbembed

[Damyan I. <da...@mo...>]

Hi, Alex,

Thank you for reading so far. Here's another long mail you may leave for
the weekend (I hope not though) :)

-=| Alex Peshkov,  7.08.2007 12:05 |=-
> Suppose we have application originally linked with libfbclient.so.2.1 having 
> soname libfbclient.so.2.0. Me thinks it's desirable to have it working 
> correctly with libfbclient.so.2.0? This may not always be so. I.e. building 
> with version N, and using with N+M is OK - should be OK if no new errors :-), 
> but building with N and using with N-M may be bad even in subreleases.

This is the way I see it too.

> For example (it's about windows, but something similiar may be applied to 
> linux too in the future). Version 2.1 has trusted authentication capability 
> with server 2.1 (or higher), running on windows server. Application certainly 
> may relay on it, even not providing login box at all. Be sure, it will not 
> work with old client.

Windows traditionally keeps backwards compatibility. When this is not
possible, the API is *extended* with new functions (xxx2, xxx_ex), but
old functions are kept. POSIX' approach is quite different.

> I always thought that package managers can solve this problem with some better 
> logic than simple comparison of 2 text strings like soname works.

They do, actually :)

For Debian[0], the package build process uses some additional
information, provided by package maintainer[1] in order to figure out
the necessary dependencies. This result is like this:

You build package X using package libfbclient2 (version 2.1.0,
containing library with soname libfbclient.so.2), The resulting package
X specifies "Depends: libfbclient2 (>= 2.1.0)". This way, the package
manager will not allow package X to be installed, unless a version
later-or-equal than 2.1.0 of package libfbclient2 is installed,
guaranteeing that all functionality of the library is here.

The same holds when X is built using libfbclient2, version 2.0.1 (same
soname). X would then use "Depends: libfbclient (>= 2.0.1)". And the
resulting package will continue to be installable when the libfbclient2
package is upgraded to version 2.1.0.

Note that package manager handles only the installability of a package.
In order for the application contained within the package to continue to
work, it is strictly necessary that the new library version is 100%
backwards-compatible. It may add new functionality/API, but the
functionality and API of the old version must be preserved exactly
(minus bugfixes, of course).

Soname changes kick in when the requirement for 100% backward
compatibility is not satisfied (example libfbembed 2.1 has stricter
requirement on SQL). Such changes come with soname change, which is
reflected by binary distributions by package name change, keeping the
old package for the other packages that still depend on it and providing
a new one for newly built applications. Another useful application of
this is that I can have gbak/isql/etc linked with old and new libfbembed
installed at the same time, allowing me to migrate databases between
versions.

> The same 
> applies to fb3 architecture - people will be able to have fb4 with it's 
> libfirebird.so.4.x installed, but it will be able to call old engine for old 

FB3+ is very different beast, see my comments below.

> databases. On my mind, use of libfirebird.so with version even in point 
> release smaller then required by program is bad - it can contain bugs fixed 
> only on the next point release, but present in old one and preventing program 
> from normal operation.

Do I understand correctly that you prefer changing soname on each minor
release even when backward compatibility is kept? As I hope I made clear
above, this is not necessary for Debian. It is my belief this is not
necessary for other distributions too. Additionally, unnecessarily
changing soname creates additional burden to maintainers of packages
that depend on the library (QT, PHP, python).

> I believe we can invent some hack for 2.1, but much more interesting now (me 
> thinks) is preparing the 'ideal' requiremnets from the project to so naming, 
> is not it so? And deal with 2.1 in that same manner.

It depends of which problems you want to solve. I believe that all
binary distribution are fine if soname is changed only when new library
version is not 100% backwards compatible.

Now, for FB3. From my blurred understanding, any approach we take with
2.1 would not be 1:1 applicable to FB3. FB3 is much different. There
will be one central library (libfirebird[2]) that will be used to "pipe"
access to the relevant engine, authentication provider etc. Applications
would link with libfirebird only and ask for provider, auth etc.

(this and below are my guesses about the future FB3 architecture, but
still may have valid points)

Each such provider then lives in its own shared library, which is used
only by libfirebird. Perhaps it would be then even possible to make
older libfirebird talk to newer providers, as long as the interface
between them is compatible. I think that soname is irrelevant for
provider libraries, since they are loaded using dlopen() and not by the
dynamic linker.

So instead of one library, we have several. Typically, an application
needs the "switchboard" plus some db provider plus some authentication
provider.

The fact that applications link only with the "switchboard" limits the
usability of soname for keeping track of compatibility information. It
is still useful, but only for the communication between the application
and the switchboard, which, in my understanding, would be quite limited.
Since the switchboard interface won't be that complicated, it will very
seldom change without keeping backwards compatibility.

The task about keeping the compatibility with regards to database
interaction is then responsibility of relevant providers. If an
application requests 'engine7' provider, it will get that if it is
available and get error at API-level otherwise. The important thing is
that the interface to 'engine7' and its semantics (behaviour) should not
change. In other words, we have "provider name" instead of "soname" and
the concept of "change only if not back-compatible" can be kept. That
could be extended if application is able to ask for specific versions of
the 'engine7' provider, but I still think that managing backward
compatibility within the provider is the only sane thing to do.

Note that this all is more-or-less about embedded-style of applications,
which are minority. Other, "normal" applications can't be guaranteed to
talk to certain version of the server unless they check that by
themselves. And there is nothing soname or package maintainer can do
about it.


Footnotes:
[0] I believe other binary distributions are similar in this regard

[1] Package maintainer is a real person (or a group of persons)
responsible for maintaining the packages. Not to be confused with
"package manager", which is a set of distribution-specific tools for
managing packages -- dpkg/apt for Debian, rpm/yum/apt for Red Hat etc.

[2] the name here is arbitrary. It only must be different from any other
name used before (libfbclient etc)
-- 
dam            JabberID: da...@ja...