#534 SemaphoreInfo assertion failed when dynamic library's ID changed in macOS

[Vinyl Darkscratch]

This has been a frustrating issue for my program for a long time, and I finally traced the issue down. I'm using CMake's BundleUtilities to bundle GraphicsMagick into my app bundle, but every time that I use it, my application will crash with the following failed assertion:

Assertion failed: (semaphoreinfo != (SemaphoreInfo ) NULL), function LockSemaphoreInfo, file magick/semaphore.c, line 601.

I wouldn't get this issue when I manually copied the .dylib files, and even when I changed all of the linker paths to link my bundle's libraries, such as @executable_path/../Libraries/libwhatever.0.dylib. It's when I change the library's ID to the bundle's library path, which is a part of the BundleUtilities process. I'm running InitializeMagick(ExecutablePath); (where ExecutablePath == "lightpad.app/Contents/MacOS/lightpad") at the very beginning of my program.

Something much deeper within GraphicsMagick that's beyond my knowledge of its code must be going on. This has been something that has stumped me for weeks, and delayed deployment of the mentionned application, Lightpad. If it helps, I have version 1.3.26, which I've installed using Homebrew. I'm more than happy to provide any additional information needed for this!

[Bob Friesenhahn]

Since you are invoking InitializeMagick(), the most likely cause of problems like this could be if there are multiple GraphicsMagick shared libraries/headers and the wrong ones are being used when compiling, linking, or at run-time. There is also the possibility of mixing header files from two different GraphicsMagick builds. CMake might not be making it obvious what it is doing.

At the shell prompt do:

export MAGICK_DEBUG=configure

and run your program from that shell prompt. You should see a trace of everything which is done to find GraphicsMagick files. It should then be obvious if it is looking for files in places other than you anticipated. If it instantaneously crashes, then you have a problem other than file locations.

Make sure that you configured GraphicsMagick with the '--disable-installed' option so that it searches based on the location of the invoking program or GraphicsMagick shared library rather than using hard-coded paths. In this case, the GraphicsMagick files need to be co-located with the using application in a particular way. Since OS X is a Unix type system, I think that the files would be co-located in sub-directories similar to how they would be installed under an installation 'prefix'.

[Vinyl Darkscratch]

Since I only have one version of GraphicsMagick installed on my machine (and I just updated to 1.3.27, completely uninstalling 1.3.26), we can eliminate that as a result. I just went ahead and attempted to use the --disable-installed argument (as well as set the MAGICK_DEBUG environment variable) to see if that would fix anything. No luck, unfortunately. I had a slightly different result at first, where it aborted due to being unable to find all the .mgk files, but I manually copied those files over and re-ran. Then it would give me the same assertion.

Something I notice is that it seems to crash right after loading png.la and calling RegisterPNGImage then UnregisterPNGImage. I'm loading PNGs into my project at the time when I start to utilize GraphicsMagick (which is the exact time of crash), maybe that has something to do with it? I have copied GraphicsMagick/lib/GraphicsMagick over into lightpad.app/Contents/lib/GraphicsMagick without modifying any of the files, so the library will find them. The full log can be found here, if it helps!

On the subject of the hard-coded paths, I personally feel that they should be avoided as much as possible. If you're able to determine the path of the library and where it's required files are, why not simply use that all the time? (Of course, this may just be me -- I'm one of those that leans towards the simplest of answers. :P)

[Bob Friesenhahn]

On Tue, 19 Dec 2017, Vinyl Darkscratch wrote:

Something I notice is that it seems to crash right after loading
png.la and calling RegisterPNGImage then UnregisterPNGImage.

That does look suspicious.

Your debug log appears so show that files (e.g. log.mgk) are not at
expected places.

Does only PNG fail and other formats appear to work?

What does

grep SETJMP_IS_THREAD_SAFE magick/magick_config.h

in the build directory show?

Did you do 'make check' after doing a compile to make sure that the
test suite passes?

The best way to get the GraphicsMagick files into a form suitable for
using is to do

mkdir /path/to/tmp/installdir
make DESTDIR=/path/to/tmp/installdir install

and then you should have installed files under
'/path/to/tmp/installdir'. The installed files will have a different
layout than in the build directory. For the "installed" case you
would want to replicate starting at 'installdir' and you could put
your own binary in 'installdir/bin'.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/

[Vinyl Darkscratch]

For log.mgk, I actually don't see this file anywhere on my system. Perhaps it wasn't included in my particular install?

A quick grep of magick/magick_config.h showed no results for SETJMP_IS_THREAD_SAFE (Homebrew builds the formulas on their own servers and deploys pre-built "bottles", so I ran this in the install directory instead).

I also looked at the formula, however, and it seems that there's no call to make check in the test stage. Adding that and submitting a PR to get that fixed...

One of the issues that I'm seeing here is that I wish to utilize Homebrew for dependency management, as does my CI of choice, Travis CI. What it could be, is that the GraphicsMagick Homebrew formula has some issues that need to be resolved. I'll attempt to compile this directly from the source download, and report any changes I find, as well as update the Homebrew formula.

[Bob Friesenhahn]

On Mon, 1 Jan 2018, Vinyl Darkscratch wrote:

For log.mgk, I actually don't see this file anywhere on my system.
Perhaps it wasn't included in my particular install?

This file is delivered with GraphicsMagick source code in the 'config'
subdirectory. It is always installed. It is the first file which is
attempted to be read and without the file, there are baked-in
defaults.

A quick grep of magick/magick_config.h showed no results for
SETJMP_IS_THREAD_SAFE (Homebrew builds the formulas on their own
servers and deploys pre-built "bottles", so I ran this in the
install directory instead).

This definition was added for 1.3.27 but you seem to be using 1.3.26.

I also looked at the formula, however, and it seems that there's no
call to make check in the test stage. Adding that and submitting
a PR to get that fixed...

One of the issues that I'm seeing here is that I wish to utilize
Homebrew for dependency management, as does my CI of choice, Travis
CI. What it could be, is that the
GraphicsMagick Homebrew formula has some issues that need to be
resolved. I'll attempt to compile this directly from the source
download, and report any changes I find, as well as update the
Homebrew formula.

I am not sure what this CMake and Homebrew build formula is doing.
Perhaps it is applying patches or using a completely different build
methodology. You say that it is built on a 'server' so perhaps the
binary product is not compatible with the final execution environment.

If you can post the full output of 'gm convert -version' then it is
possible that something useful may be revealed.

Bob

[Vinyl Darkscratch]

I see log.mgk in the source code now, it's strange that it wouldn't be included in the install directory... What's even weirder, the Homebrew formula isn't applying any patches to the code.

This definition was added for 1.3.27 but you seem to be using 1.3.26.

Actually, I've upgraded to 1.3.27 now. I was under the assumption that some of the problems I'm facing may be related to having the older version, plus hey, bug fixes and enhancements are always helpful! My grep was on the version for 1.3.27.

I posted the results of gm version in my last response, though it was an edit, so it didn't show in the email. I moved it into this response instead. The command displays the following results:

GraphicsMagick 1.3.27  Q16 http://www.GraphicsMagick.org/
Copyright (C) 2002-2017 GraphicsMagick Group.
Additional copyrights and licenses apply to this software.
See http://www.GraphicsMagick.org/www/Copyright.html for details.

Feature Support:
  Native Thread Safe       yes
  Large Files (> 32 bit)   yes
  Large Memory (> 32 bit)  yes
  BZIP                     yes
  DPS                      no
  FlashPix                 no
  FreeType                 yes
  Ghostscript (Library)    no
  JBIG                     no
  JPEG-2000                no
  JPEG                     yes
  Little CMS               no
  Loadable Modules         yes
  OpenMP                   no
  PNG                      yes
  TIFF                     yes
  TRIO                     no
  UMEM                     no
  WebP                     no
  WMF                      no
  X11                      no
  XML                      yes
  ZLIB                     yes

Host type: x86_64-apple-darwin17.3.0

Configured using the command:
  ./configure  '--prefix=/usr/local/Cellar/graphicsmagick/1.3.27' '--disable-dependency-tracking' '--enable-shared' '--disable-static' '--with-modules' '--without-lzma' '--disable-openmp' '--with-quantum-depth=16' '--disable-installed' '--without-gslib' '--with-gs-font-dir=/usr/local/share/ghostscript/fonts' '--without-x' '--without-lcms2' 'CC=clang' 'CXX=clang++'

Final Build Parameters:
  CC       = clang
  CFLAGS   = -g -O2 -Wall -D_THREAD_SAFE
  CPPFLAGS = -I/usr/local/opt/freetype/include/freetype2 -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/include/libxml2
  CXX      = clang++
  CXXFLAGS = -D_THREAD_SAFE
  LDFLAGS  = -L/usr/local/opt/freetype/lib -L/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.13.sdk/usr/lib
  LIBS     = -lfreetype -lbz2 -lz -lltdl -lm -lpthread

The --prefix=/usr/local/Cellar/graphicsmagick/1.3.27 looks suspicious -- perhaps if we remove it, we may obtain better results?

[Bob Friesenhahn]

On Tue, 2 Jan 2018, Vinyl Darkscratch wrote:

I see log.mgk in the source code now, it's strange that it
wouldn't be included in the install directory... What's even
weirder, the Homebrew
formula
isn't applying any patches to the code.

I do see that there are no patches. I also see that the script ends
with "make install" and there is just one trivial sanity test of its
own design rather than using the built-in test suite.

There must be some other script/definition which bundles up the result
of "make install" so that it can actually be installed.

Bob

[Bob Friesenhahn]

Any more information on this issue? A stack backtrace of a core dump using a debugger would be useful to identify where the problem is occuring. A response to my prior questions is appreciated. At the moment I have nothing to go on so I don't know how to help.

[Vinyl Darkscratch]

First off, thank you for all of your help on this matter! Second, I apologize for not responding, I have just returned from a week-long vacation visiting relatives, and decided that it would be better to take a break from the project during that time so I can clear my mind.

That...and it seems I've run into some issues with CMake linking a certain library, though it's one I'm wanting to get rid of anyways so perhaps it's a sign from my computer saying "hey, remove this sooner than later!"

I will post my findings based upon your suggestions shortly!

[Vinyl Darkscratch]

A little bit of information that may help: when --disable-installed is included, the program crashes once I attempt to utilize GraphicsMagick in any way. Also, removing the --prefix options causes some issues with the install step, since it's attempting to install to the typical folders, rather than remain in Brew's "Cellar" (which symlinks files into /usr/local as needed), so that must stay.

I still need to try compiling and installing it without using Homebrew. That's been big on my to-do list for quite a while...

[Bob Friesenhahn]

Unless further useful information such as a debugger backtrace is provided, I will be closing this issue soon since I have no way to reproduce it.

[Bob Friesenhahn]

Requested information was not provided by reporter.

[Vinyl Darkscratch]

Finally got around to this, whew! So, I've eliminated it being an issue with Homebrew -- I tried to compile and install it from the source, and got the same issues during bundling. I tried with a different series of arguments, however, and had gotten satisfactory results. So, it's something to do with the specific arguments passed in, it looks like.

To compare them, here's the arguments (as reported by 'gm version') between the failing version and the working one:

Failing: ./configure '--prefix=/usr/local/Cellar/graphicsmagick/1.3.28_1' '--disable-dependency-tracking' '--enable-shared' '--disable-static' '--with-modules' '--without-lzma' '--disable-openmp' '--with-quantum-depth=16' '--without-gslib' '--with-gs-font-dir=/usr/local/share/ghostscript/fonts' '--with-webp=no' '--without-x' '--without-lcms2' '--without-wmf' 'CC=clang' 'CXX=clang++'

Working: ./configure '--enable-shared' '--with-quantum-depth=16' '--prefix=/Users/vinyldarkscratch/Developer/Nightwave/lightpad/graphicsmagick/' '--without-x' '--with-png'

Further research into the matter is required, which I will, of course, be conducting.