#130 extraneous bundled libs!

[Nathan Phillip Brink]

http://blog.flameeyes.eu/2009/01/02/bundling-libraries-for-despair-and-insecurity
https://bugs.gentoo.org/251464

freeimage bundles and quite fixedly compiles against a number of very common libraries, such as zlib, tiff, libpng, jpeg, etc. Any reasonable buidl environment will already contain copies of these libraries and, most importantly, will exported them as shared objects. As most of these libs have quite stable APIs and ABIs, there are some reasons to take advantage of these system-installed libs.

1. Utilizing shared libraries slightly reduces the total amount of memory that must be allocated on modern OSes
2. System versions of shared libraries can be updated when security vulnerabilities are found in those libs. If every program which uses zlib would link to the shared, system-provided zlib, there would be far fewer exploits of old versions of zlib found in old versions of libraries that distribute zlib.
3. This encourages other software packages to bundle FreeImage, thereby indirectly bundling 2 levels of library dependencies and encouraging propagation of the two above problems.

The build system should provide a way to compile against system-installed versions of all of the individual raster format libraries that freeimage ``supports''. The Makefile.gnu, which I examined, does not provide a means for this.

[Hervé Drolon]

Hi,

This topic was already discussed several times in the forums. To sum up, it is a design choice.

About point 2:
Each FreeImage version is developped, tested and designed to run with a particular set of third party libraries, each library being configured with a particular set of options. Each new FreeImage release takes into account the changes log published by each third party library : this way, the FreeImage code and it's underlying libraries are always up-to-date regarding bugs and security issues.
FreeImage provides a very simple API to handle various kinds of bitmaps. It's provided as a single library (.dll or .so). The fact that it uses a number of third party libraries is convenient for us, the library developers, but shouldn't concern the users. We patch the libraries to our needs, to make optimal use of them.

About points 1 and 3.
What you says is not true.
I advise you to read this article by Ulrich Drepper (lead maintainer of GNU glibc) on the subject of good DSO design :
http://people.redhat.com/drepper/dsohowto.pdf
See also this page on the way to reconciliate both Linux and Microsoft worlds:
http://gcc.gnu.org/wiki/Visibility

Thus, the way FreeImage is packaged now will not change in the future.

Hervé Drolon
FreeImage Project Manager

[Nathan Phillip Brink]

Point #1 is supported by the linked PDF on page 3 in the left column.

And, besides, it's against Gentoo and Debian policy to bundle libraries ( http://www.debian.org/doc/debian-policy/ch-source.html#s-embeddedfiles ). This means that, if libfreeimage is to be included on Gentoo, it will have to be heavily patched and modified so that it doesn't use bundled libraries.

Concerning security vulnerabilities: distributions can easily allocate resources and time to the system-installed copy of, e.g., zlib. They can push another version to be consumed by their users sometimes within hours of zlib's authors fixing the problem. Distributions cannot, however, look for every package which bundles zlib and give the same priority to bumping out a new version. If you are interested in targeting GNU/Linux as a platform, it would make sense to take advantage of the conventions in place for GNU/Linux OSes when using raster libraries.

Thanks for providing the information that you know about visibility issues such as symbol collisions.

Concerning patching the underlying libraries: is it not better to work with upstream to improve the APIs of these libraries for all of their users rather than just for libfreeimage's use? I cannot imagine that every library you use had to be modified very much to enable your IO abstractions to work.

If I am able to provide patches to allow using system libraries, do they have a chance of finding acceptance?

[Samuli Suominen]

Here's some fun reading from glibc developer, http://people.redhat.com/drepper/no_static_linking.html

The argument "it's by design" is one of the worst I've heard

[Hervé Drolon]

This article just suggest that FreeImage shouldn't be linked statically and I agree with that ...

[Dan Kegel]

I just got bitten by this on OSX 10.9. The symptom was a runtime abort in a local unit test of a library that uses FreeImage and a bunch of other shared libraries. The cause was having a static libfreeimage.a and failing to have a libfreeimage.dylib. So it was self-inflicted -- apps were linking to the static libfreeimage.a, which made it a lot easier for them to accidentally pick up system libraries instead of the bundled ones.

Having .a's of things usually isn't quite that dangerous, but with libfreeimage, it's fatal.

For completeness, here's what the problem looked like:

$ ./libFooTests
...
Wrong JPEG library version: library is 70, caller expects 90
...

This was surprising. libfreeimage does seem to contain a nicely bundled libjpeg:
$ otool -L /opt/foo/lib/libfreeimage.3.dylib
/opt/foo/lib/libfreeimage.3.dylib:
/opt/foo/lib/libfreeimage-3.17.0.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libc++.1.dylib (compatibility version 1.0.0, current version 120.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1197.1.1)
$ strings /opt/foo/lib/libfreeimage.3.dylib | grep Wrong.JPEG
Wrong JPEG library version: library is %d, caller expects %d

Binary editing /opt/foo/lib/libfreeimage.3.dylib to change the message did not change the output.

The unit test, though, runtime links to the "system" libjpeg:
$ otool -L libFooTests | grep libj
/opt/foo/lib/libjpeg.7.dylib (compatibility version 8.0.0, current version 8.0.0)

Binary editing /opt/foo/lib/libjpeg.7.dylib to change the message did change the output.

Adding printfs shows that the error is output during a call to FreeImage_LoadFromMemory().

So it seems the copy of libjpeg embedded in FreeImage is insufficiently isolated, and gets "replaced" by the runtime linker by the "system" version.

Removing the static libfreeimage.a, and getting apps to link to the shared version as was intended, resolved the problem.