#4 Please version bump libschily

[martin.vaeth]

Since schily-tools-2017.10.10, the ABI of libschily.so.1.0 has changed, so that it can no longer be replaced by the library with the same name from e.g. cdrtools-3.02_alpha09: When doing so it leads to errors like:

symbol lookup error: /usr/lib64/libshedit.so.1.0: undefined symbol: fsgetlen

Since cdrtools is an official gentoo package and schily-tools is a package from an overlay, there is no easy way to get rid of this filename collision. Therefore, I would appreciate if libschily from schily-tools could eventually be version bumped after this update from 2017.10.10 to e.g. libschily.so.1.1 or (2.0).
(Of course, I could patch that in the schily-tools ebuild, but IMHO introducing a divergence in library names/versions is not a good idea.)

(There is also a collision in libmdigest.so.1.0, but as far as I can see, these libraries are ABI-compatbile when installed from cdrtools-3.02_alpha09 or schily-tools-2018.05.02, so this is probably fine.)

[Jörg Schilling]

Your problem is not an incompatible change in libschily but rather using an old version of a library that has been enhanced in a compatible way.

The function fsgetlen() has been added December 30 2017 to libschily. If you did use a recent version of libschily, you did not have the problem.

BTW: libfind got an incompatible change on April 12 2018 since the size of a structure changed, all functions related to the new structure size got new version numbers in the map file and libfind did get a new major version number. So you need to keep the old libfind file in case you like to run older binaries that use the old version of the lib.

This is not needed with libschily since no change in behavior has been made. Old binaries will continue to work with the new grown libschily.

As a hint: the ELF tool "pvs" can be used to check library version information.

pvs -d libschily.so.1

prints all libschily versions. If you observe less output, this refers to an older version of the library.

pvs -ds libschily.so.1

prints all symbol names related to the library versions in addition.

If there are lines that are missing in a newer version of a library compared to an older version, the newer version of the library is incompatible to the older one and thus needs a new major version number in the filename. If the newer version of the library only adds entries and the maintainer did not mess up with the map file, the newer version of the library is compatible to the older one.

[martin.vaeth]

I do not care how you call it:

Unfortunately, neither the unix file system nor the runtime linker understand the terminology "old version of libschily-1.0" or "new version of libschily-1.0": They simply understand in both cases "/usr/lib/libschily-so.1.0" (unless one plays dirty tricks with wrappers for LDPATH).
So in particular, the assertion "the newer version of libschily-1.0 is compatible with with the older version of libschily-1.0 but not vice versa" becomes in practice the assertion
"/usr/lib/libschily-1.0 is compatible with /usr/lib/libschily-1.0 but not vice versa".
The main purpose of version numbers is to communicate the distinction of "old" version and "new" version to the linker and file system, isn't it?

Fact is that it has become almost impossible to maintain an additional "schily-tools" package for any distribution which provides a package for cdrtools. Indeed, any sane package manager prevents file collisions (exactly for the reason that one does not want to let the install order decide about the actually installed file). As long as the files in both packages could be exchanged without problems, this was no difficulty: The packaging of schily-tools could simply prevent the installation of libschily-so.1.0. Now that libschily-so.1.0 != libschsily-so.1.0, this is not possible.
Now it is necessary to patch a different package (cdrtools, usually provided by the distribution) in order to create the package schily-tools.

If you refuse to adapt the version number to the actual content, another solution might be to update libschily-1.0 in cdrtools: Then at least schily-tools can be installed in parallel to cdrtools, once the distribution has bumped to the updated version of cdrtools.

[Jörg Schilling]

Changing the filename for shared libraries rapidely is a method that has been given up long ago (> 25 years before). This is what map files have been introduced for > 25 years ago.

The correct way to handle the problem is to use "make" and let it decide whether there are source files that are more up to date than the compiled library.

libschily has the state of an enhanced libc and of a portability enabler that is used by many projects. If you have several sources for such a library and if you - as in this case - create multiple instances of libschily where at least one instance has been derived from a less frequently updated source, your problem is the way you do packaging.

You need one unique source for libschily that is always up to date. If you do not create only one single binary package with all content from schilytools, you need to create a package that is specific to an up to date libschily and make other packages depend on this package. Since such packages usually have timestamps in their ID, you make packages that need a newer version of libschily depend on that newer version if libschily. Since the name of the library is kept stable as long as there os no breakage in backwards compatibilit, it is safe to update the library in such a case.

Following your proposal would cause confusion sice new version numbers would be introduced even though there is still backwards compatibility.

If there will ever be an incompatible change in libschily, the (major) version number that is part of the filename will be incremented.

BTW: the run time linker is expected to warn you that libschily-1.9 is not available.

The problem seems to be that Linux does not use a recent run time linker but code that has been derived from the SunOS-4.0 run time linker.

Here is the expected output from a run time linker that understands library versioning via map files:

ld.so.1: bsh: fatal: libschily.so.1: version 'SCHILY_1.9' not found (required by file /opt/schily/bin/bsh)

and this is what you get from ldd:

ldd OBJ/i386-sunos5-cc/bsh
libxtermcap.so.1 => /opt/schily/lib/libxtermcap.so.1
libschily.so.1 => /opt/schily/lib/libschily.so.1
libschily.so.1 (SCHILY_1.9) => (version not found)
...

[martin.vaeth]

Most libraries use the minor version(s) to indicate backward compatible changes and provide symlinks with major versions only (sometimes even without that) to which consumers of the library actually link. Some random example:

lrwxrwxrwx 1 libfdk-aac.so.1 -> libfdk-aac.so.1.0.1
-rwxr-xr-x 1 libfdk-aac.so.1.0.1
lrwxrwxrwx 1 libfdk-aac.so -> libfdk-aac.so.1.0.1
$ ldd /usr/bin/ghb  | grep fdk
libfdk-aac.so.1 => /usr/lib64/libfdk-aac.so.1

The correct way to handle the problem is to use "make"

I would never let run "make" (incl. "make install") as root without a container or otherwise isolated sandbox. The risk of some bug in some project is too large, and you can never be sure to have all installed files registered correctly (e.g. for deinstallation/upgrade/checksum verification etc). Gentoo (and I suppose most other distributions) have their own installation tool which thus provides a unique point of possible failure (instead of having such a point of possible failure in every single project).
So for installation, you cannot rely on any installation date. Moreover, the installation date has no meaning, since the user might have re-installed cdrtools very recently.

libschily has the state of an enhanced libc and of a portability enabler that is used by many projects.

In the official gentoo repository it is used by exactly one project, namely cdrtools.
If you would stop bundling different versions of libschily into different tarballs (and e.g. provide only one independent libschily source tarball) that problem might go away automatically.

your problem is the way you do packaging.

No, my problem is the way how cdrtools is packaged, namely that it contains (the old version of) libschily. This is what I cannot change since this is done by gentoo. (If I would file a bug, asking them to change the packaging to be compatible with my overlay, they would of course immediately close this bug as INVALID). I suppose that one would run into the same problem with other distributions as well if they provide cdrtools but nothing else from schily-tools.

OTOH if you would stop bundling an old libschily with the cdrtools tarball and provide an independent libschily package then probably most distributions will sooner or later update to the latest versions of both (whether they provide them as 1 or 2 packages internally plays no role).

BTW: the run time linker is expected to warn you that libschily-1.9 is not available.

For the user it plays no role whether the package fails immediately or later on.

[Jörg Schilling]

Using more than one number (the major version number) in the filename has been done with the old SunOS-4.0 dynamic linker from 1988 that was based on the a.out executable format.

With changing the dymic linker for ELF support, only one version number has been left in the filename.

If you are not able to find a cooperative solution for the cdrtools packagem it is probably the best idea if you always install the shared libschily from schilytools.

Since the publishing frequency is for schilytools is alway higher than the one for cdrtools, you will always get a more recent version this way. If there will ever be a major version upgrade due to an incompatible change, schilytools will install a new file (keeping the old one) instead of overwriting the current libschily.

[martin.vaeth]

With changing the dymic linker for ELF support, only one version number has been left in the filename.

Not sure what you want to say here: All libraries do it wrong when they indicate backward compatbile changes in minor versions and only libschily does it right by not bumping and relying on override magically always happening in the right order?

If you are not able to find a cooperative solution for the cdrtools packagem it is probably the best idea if you always install the shared libschily from schilytools.

If I am unable to find a cooperative solution, I am not able to provide schily-tools in an overlay.
The package manager simply refuses to install a package with colliding files.
And this is absolutely sane behaviour of the package manager:

publishing frequency is for schilytools

Installation on user's system has almost nothing to do with publishing frequency, since there are so many reasons to reinstall a package. (Switch to different compile, CFLAGS (e.g. recently -fPIE or more recently flags related to spectre), ABI (not API) change of a basic library, some hardware change, etc.) In no circumstance such a reinstall must break other things. (Not to speak about the checksum and timestamp of installed files which are both stored by the package manager for each package for safety reasons.)

My current "solution" is to override the gentoo package cdrtools in the overlay, but this "solution" is actually none I intend to continue very long, since it is a lot of work for me and confusing and problematic for the users: It means that I have to maintain the full cdrtools package (duplicating gentoo's effort) with nonstandard USE-flags which users have to manually select, and they have to manually select "my" cdrtools and completely rely on me for such a basic package. Probably not too many users will do this (and I am actually not interested in maintaining cdrtools for many users; it costs time for me which gentoo devs spend anyway).

[Jörg Schilling]

Your problem is the same as if any other two entities did try to publish e.g. a "vi" package with different content. If the package manager does not let you overwrite binaries, you need to uninstall the other package first.

BTW: I do not see any difference in cdrtools other than the version and if you e.g. install the latest schilytools, you e.g. get a fixed mkisofs ;-)

[martin.vaeth]

I was not aware that the version of cdrtools inside of schily-tools was meant to be used as a recommended full replament for cdrtools. I suppose the same will then also hold for e.g. star.

In this case, the problem is easy to solve: I let the schily-tools ebuild block the cdrtools and star packages and make it the provider for the cdrtools binaries (if the user activates the corresponding USE-flags that he wants that). For cdrtools I just add a dummy ebuild (which forces installation of schily-tools with the corresponding USE-flags and evades blocking by schily-tools) to satisfy dependencies.

This is now all already experimentally implemented in the mv overlay.

[Jörg Schilling]

OK, I just published a new version that includes an updated section "SOURCE DOWNLOAD" in the man pages and this now includes schilytools as an official reference.