#624 GM Causes JVM Instability

[gudenau]

When calling InitializeMagick inside a JVM via JNI it causes the JVM to become unstable and crash shorly after or spin and pin a CPU thread to 100%.

I have had the JVM crash inside:
java.lang.String.isEmpty
java.util.HashSet.iterator
java.lang.String.length
java.util.zip.ZipUtils.CENNAM
java.lang.StringLatin1.indexOf
and more.

Any idea what might be causing this?

Relivent JNI:

JNIEXPORT void JNICALL Java_net_gudenau_magicktest_MagickTest_InitializeMagick
(JNIEnv* env, jclass klass, jlong path){
    UNUSED(env);
    UNUSED(klass);
    InitializeMagick((const char*)path);
}

(Note, this happens with nullptr and a native string)

I am using OpenJDK 13.0.1 on Arch Linux with Graphics Magick 7.0.9-22.

[Bob Friesenhahn]

On Tue, 18 Feb 2020, gudenau wrote:

I am using OpenJDK 13.0.1 on Arch Linux with Graphics Magick 7.0.9-22.

Are you sure that you are using GraphicsMagick? 7.0.9-22 sounds like
an ImageMagick release number.

The most recent GraphicsMagick release is 1.3.34.

In GraphicsMagick, the InitializeMagick function does set up signal
handlers. This might conflict with Java, which likely sets up its own
signal handlers. I am not really sure what to do about this other
than introduce a new form of InitializeMagick() which provides more
control over what gets initialized.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[gudenau]

Well, I ran gm version. There is always a chance that IM might be doing something screwy. Is there any way to check what signal handlers are in use?

[Bob Friesenhahn]

On Tue, 18 Feb 2020, gudenau wrote:

Well, I ran gm version. There is always a chance that IM might be
doing something screwy. Is there any way to check what signal
handlers are in use?

It sounds like you are using the library rather than the gm utility.
Are you sure that you are using the GraphicsMagick library rather than
ImageMagick?

When calling InitializeMagick inside a JVM via JNI it causes the JVM to become unstable and crash shorly after or spin and pin a CPU thread to 100%.

I have had the JVM crash inside:
java.lang.String.isEmpty
java.util.HashSet.iterator
java.lang.String.length
java.util.zip.ZipUtils.CENNAM
java.lang.StringLatin1.indexOf
and more.

Any idea what might be causing this?

My first guess would be a locale (internationalized character sets)
issue. Calling InitializeMagick may be adjusting the locale settings.

I am using OpenJDK 13.0.1 on Arch Linux with Graphics Magick 7.0.9-22.

Once again, there is an ImageMagick 7.0.9-22 but there is no
GraphicsMagick 7.0.9-22. There almost 18 years of separation between
the software packages!

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[gudenau]

The header in /usr/lib/GraphicsMagick/magick/version.h seems to say 1.3.34.

#define MagickPackageName "GraphicsMagick"
#define MagickCopyright  "Copyright (C) 2002-2019 GraphicsMagick Group.\nAdditional copyrights and licenses apply to this software.\nSee http://www.GraphicsMagick.org/www/Copyright.html for>
#define MagickLibVersion  0x232002
#define MagickLibVersionText  "1.3.34"
#define MagickLibVersionNumber 23,20,2
#define MagickChangeDate   "20191224"
#define MagickReleaseDate  "2019-12-24"

So weird that gm version reports such a diffrent value than the header. Probably from a diffrent package that has the bin or something.

[Bob Friesenhahn]

On Tue, 18 Feb 2020, gudenau wrote:

The header in /usr/lib/GraphicsMagick/magick/version.h seems to say 1.3.34.

```C

define MagickPackageName "GraphicsMagick"

define MagickCopyright "Copyright (C) 2002-2019 GraphicsMagick Group.\nAdditional copyrights and licenses apply to this software.\nSee http://www.GraphicsMagick.org/www/Copyright.html for>

define MagickLibVersion 0x232002

define MagickLibVersionText "1.3.34"

define MagickLibVersionNumber 23,20,2

define MagickChangeDate "20191224"

define MagickReleaseDate "2019-12-24"

```

So weird that gm version reports such a diffrent value than the
header. Probably from a diffrent package that has the bin or
something.

This suggests that 'gm' is not really GraphicsMagick. The information
should be exactly the same.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[Bob Friesenhahn]

To be clear, the 'gm' utility contains practically no code.
Everything pertaining to GraphicsMagick (including the reported
version) is in the library so if the gm utility does not report the
expected version, then it seems like there must be a problem.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[gudenau]

I've manually compiled GM from the source archive and it's still doing the same thing, but gm version is reporting GraphicsMagick 1.3.34 2019-12-24 Q8 http://www.GraphicsMagick.org/ now.

[Bob Friesenhahn]

On Tue, 18 Feb 2020, gudenau wrote:

I've manually compiled GM from the source archive and it's still
doing the same thing, but gm version is reporting GraphicsMagick 1.3.34 2019-12-24 Q8 http://www.GraphicsMagick.org/ now.

That seems a bit better.

I would try steps like disabling OpenMP support to see if that helps.

Unless things have changed, my memory of Java JNI was that it can only
support single-threaded invocation. This means there would be a
problem if more than one Java thread called through JNI at once.

If Java crashes merely by calling InitializeMagick() then something
else is going on.

Assuming that Java is really using the library that you build, you
could try editing magick/magick.c to comment the line

InitializeMagickSignalHandlers(); / Signal handlers /

I can't think of anything else which might impact Java.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[gudenau]

That change does seem to make this work.

Why does GM use signal handlers?

I suppose making some sort of InitializeMagickEx(InitInfo* initInfo) might be in order?

[Bob Friesenhahn]

On Wed, 19 Feb 2020, gudenau wrote:

That change does seem to make this work.

Why does GM use signal handlers?

Signals are how the operating system tells the program to quit,
whether it be due to CONTROL-C at the keyboard, or that the program
did something wrong.

Depending on the signal, GraphicsMagick will attempt to do things like
remove temporary files so that the disk does not fill up.

I suppose making some sort of InitializeMagickEx(InitInfo* initInfo) might be in order?

Maybe, if the option flags can be successfully determined so the
design is future safe.

One thing which was added recently is that it uses an alternate signal
stack if the feature is available. This is around lines
624-626 and was added in changeset 15457.

I see some documentation regarding how Java deals with signals at
https://docs.oracle.com/javase/9/troubleshoot/handle-signals-and-exceptions.htm#JSTGD345

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[gudenau]

You could create a stuct that has a version member in it and when you check for a flag you could check against it's version. If it's too low you know to use the default value.

[Bob Friesenhahn]

I see that DestroyMagick() and PanicDestroyMagick() are already
publically exposed for use.

The DestroyMagick() function should be used for a normal clean-up of
GraphicsMagick such as for an orderly shutdown which releases all
acquired resources.

The PanicDestroyMagick() function should be used for the case where
something has gone severely wrong but there is a desire to clean up
things which don't go away when the program goes away such as
temporary files.

The new InitializeMagickEx() function could look like

MagickPassFail InitializeMagickEx(const char path,
unsigned int options,
ExceptionInfo exception);

The options flags could support a flag which says to not provide
signal handlers.

There would now be a return status so you can know if the function has
failed, and an ExceptionInfo report so you can know why.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[Bob Friesenhahn]

On Thu, 20 Feb 2020, gudenau wrote:

You could create a stuct that has a version member in it and when
you check for a flag you could check against it's version. If it's
too low you know to use the default value.

The problem with a struct is that it establishes a binary interface,
which must then be preserved. Since this is C rather than C++,
there is no way to automatically initialize a version member.

A simple integer flags value seems better.

Functions would be provided that external code could call to do the
emergency clean-up that the current signal handlers provide.

I don't know how Java deals with signals and if threads may still
continue executing while it is handling a signal. This is a risk
factor.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[Bob Friesenhahn]

As of Mercurial changeset 16251:f86618d5e836 there is now an
InitializeMagickEx() function:

MagickPassFail
InitializeMagickEx(const char path, unsigned int options,
ExceptionInfo exception);

as well as this definition

define MAGICK_OPT_NO_SIGNAL_HANDER 0x0001

You can call the function like:

ExceptionInfo exception;
GetExceptionInfo(&exception);
(void) InitializeMagickEx(path,MAGICK_OPT_NO_SIGNAL_HANDER,&exception);
if (exception.severity != UndefinedException)
CatchException(&exception);
DestroyExceptionInfo(&exception);

And now GraphicsMagick should be initialized but without registering
signal handlers.

Please give this a try an verify that it avoids the Java VM crash.

Bob

Bob Friesenhahn
bfriesen@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt

[gudenau]

Alright, I will have to give it a shot in a few.

Been kinda busy, sorry.

[gudenau]

Seems to work!

[Bob Friesenhahn]

Resolved by providing new initailization interface which provides more control.