#5 More flexible and consistent loadLibrary behaviour

Unstable (example)
open
nobody
None
5
2014-03-22
2014-01-28
Ximin Luo
No

Please apply the following patch; it makes the loadLibrary behaviour consistent on all platforms. The following behaviour changes apply:

  1. We try to load "jnotify" then fallback "jnotify-${os.arch}".
  2. regarding the comment feature request #4, modern JVMs all seem to use x86/x86_64 on windows and i386/amd64 on linux[1], so you can just named them jnotify-x86.dll and libjnotify-i386.so respectively. mac os libs have cross-platform libs so the attempt to load simply "jnotify" will succeed.
  3. in the next release you will need to rename jnotify_64bit.dll but arguably this is for the best since the name is non-standard. also rename "64-bit Linux/libjnotify.so" to simply libjnotify-amd64.so.
  4. don't print an error message, instead throw an exception. this is more flexible - it can be caught by the user application, as in [2], if printing is really needed. if it's not caught, the JVM prints a stack trace by default (unless something else blocks this, such as ant-junit's crappy defaults).

[1] www.java-gaming.org/index.php/topic,14110
[2] https://github.com/infinity0/briar-prototype/commit/5fd64fb1dc5e76c5691b171ba5812924493b42d0#diff-570bb91cd91ad7d8f1be25d6cbfc7b5bR24

1 Attachments

Discussion

  • Omry Yadan

    Omry Yadan - 2014-01-29

    Good change, except for the build.xml change that snuck in .

     
    Last edit: Omry Yadan 2014-01-29
  • Ximin Luo

    Ximin Luo - 2014-01-29

    Yeah, I only did that because I couldn't get it to work with the debuglevel, and it wasn't essential for our purposes. Feel free to drop that part, it could just be a problem with my setup.

     
  • Matthew Donoughe

    I think we should probe in the reverse order (try to load jnotify-amd64 before jnotify) because in most cases we should find the specific library, and if we don't then it doesn't exist and we aren't trying to load something that does exist but is the wrong architecture first. In all cases except Mac OS we should ultimately be loading jnotify-${os.arch} anyway.

     
  • Omry Yadan

    Omry Yadan - 2014-03-16

    Here is a plan:
    make a table with all supported OS and JRE combinations. make sure whatever we come up with supports all known configurations.
    there is no goal to make things 'consistent' across operating systems: if there is some opportunity for reuse we can take it, but that's not a goal or anything.
    I can help testing on Windows 7 64bit.

     
  • Matthew Donoughe

    See Util.java 1.2. I've attempted to normalize the architecture because different compatible(?) JVMs can report different values on the same architecture.

     
  • Matthew Donoughe

    I also fixed the build issue encountered with the new JDK.

     
  • Ximin Luo

    Ximin Luo - 2014-03-17

    I think it's worse to normalise the architecture actually - it adds extra complexity and lines to the code for not much gain. As I said, "modern JVMs all seem to use x86/x86_64 on windows and i386/amd64 on linux".

    Take writing a distribution-specific build script, for example. If you use ${os.arch} without "normalising", the build script author can also use it without normalising. But now, you must explicitly document how you're "normalising" os.arch (it becomes part of your public interface), and the distribution will need to copy this non-standard logic as well.

     
  • Matthew Donoughe

    Would you really want to use os.arch in a build script? The architecture of the build machine is not necessarily the same as the architecture of the target machine. I think it's more likely somebody would have a list of files for this architecture and a list of files for that architecture or just include all the libjnotify-*.so files.

     
  • Ximin Luo

    Ximin Luo - 2014-03-21

    Hi, I tried posting this a few days ago, it seems my message got lost.

    The main point is to simplify the public interface of how you're loading libraries. "We will load x-${os.arch}" is a simple interface, "we will load x for i386, y for amd64" is a complex interface that does not extend consistently to architectures you haven't thought about, etc.

    Also, as responsible software developers, you ought to document this interface somewhere. I'm sure you'll agree that "we will load x-${os.arch}" is much simpler to document.

    It is by far the most common case that you will build for your own platform. But in this case, a simple build script would build simply "${prefix}jnotify${suffix}", not an architecture-specific prefix. This is why I chose to look for this file first. If there is no arch-suffix, it will very likely be for the current architecture.

    Actually, you guys shouldn't be trying too hard to solve this problem, it is not the developer's problem to solve. Instead, it is the distributor's problem to solve, and they will know how to solve it best. As a Debian maintainer, we have infrastructure like multiarch that handles this, and it would make my life easier if you would just not have the arch-suffix. OTOH, as a binary distributor (what I am doing for the Briar project) that wants to explicitly support multiple platforms, it makes my life easier if you guys have a simple public interface on how to load libraries - namely, ${os.arch}.

    So, I would suggest to follow my original guideline of falling back only to ${os.arch}, when the non-arch-suffix version is not found. And stick with ${os.arch}.

     
  • Matthew Donoughe

    You can still use libjnotify.so without a suffix, but I think it's better to try loading with the suffix first because it's much more likely to be the file we're looking for if it exists.

    The reason I don't like os.arch is that it is inconsistent across operating systems and JVM implementations (at least old ones) because the standard seems to specify that the property will exist but not what its value is. I think some of the inconsistencies actually came from using the operating system's equivalent of os.arch to get stuff like i386 i686, when really things are more complicated and we want to load the i386 version on i686 because they are compatible. Then other JVMs will report x86 because it's more useful for library loading, but it's actually different from what the operating system says.

    What I don't want to do is release a new version that allows easier use in binary distributions, but just doesn't work for some people who had been using jnotify previously unless they rename libraries, because this will likely be a problem for end users. I don't have any information on who is actually running this code and what JVMs they are using.

    Maybe we could use os.arch, and ship the x86 libraries without suffixes. Previously only Windows JNotify could handle loading on x86 and x86_64 without modification, and on Windows I think it would be highly unlikely to encounter multiple representations of x86_64. On Linux, the x86 version would still be loaded for any representation of x86, but also if you are unfortunate enough to have a different representation of x86_64 from what openjdk uses, which is fairly unlikely, I think, because JVMs where more mature before x86_64 was in use. Leaving the x86 assemblies unsuffixed greatly reduces the chances we'll cause issues with library loading on systems that did not previously have issues with library loading. Trying to load the suffixed library first reduces the chance we will direct the JVM to load something that is incompatible, which seems like it could be slightly risky but is probably okay.

     
  • Ximin Luo

    Ximin Luo - 2014-03-22

    I suppose I forgot to mention one item of advice - please please please, do not ship the libraries without arch-suffixes in your own binary distribution. All my experience packaging and distribution says this will cause very great headaches. I am not going to use the binary 0.94 jar in my own products as-is, I will be renaming all of them to have an arch-suffix.

    "What I don't want to do is release a new version that allows easier use in binary distributions, but just doesn't work for some people who had been using jnotify previously..." - this is not going to be a problem, no-one is going to upgrade the .jar but not upgrade the binary libraries.

    "Leaving the x86 assemblies unsuffixed greatly reduces the chances we'll cause issues with library loading on systems that did not previously have issues with library loading." - there should be no issues, if people consistently upgrade as I mentioned earlier. Actually, 64-bit JVMs are very widespread these days on GNU/Linux, and you won't be able to load a 32-bit jnotify.so on these, so default-to-x86 will cause much more problems than you think you're solving.

    General packaging advice: the non-arch-suffix forms should only be used when the platform has support for this, such as a .deb version of jnotify, or multi-arch binaries in Mac OS X. In such an instance, the platform will deal with the fact that the non-arch-suffix either is known to be for the current platform (.deb), or supports all platforms (Mac OS X).

    ${os.arch} being inconsistent across operating systems is not a big deal, it might just look weird when you create a multi-platform binary distribution for jnotify. All my research points to ${os.arch} being consistent within a single OS, so everything will still work, and it makes life easier for other distributors of jnotify (e.g. me, for both known-arch .debs and an end-user multi-platform product, Briar).

    Regarding the point about i386 vs i686, the only time I see this happening is on GNU/Linux with GCJ. GNU/Linux with Sun/OpenJDK will report "i386" for os.arch on all x86 platforms including i686. I do not think this is worth making your public interface much more complex over. You can put in a symlink from i686 to i386 if it worries you, though. But, sane Linux distributors, such as Debian, will instead ignore your binary jnotify.jars and compile their own per-arch jnotify.jars with jnotify.so in each jar for the specific architecture. So you should take my advice and not worry about it.

     
  • Ximin Luo

    Ximin Luo - 2014-03-22

    "Maybe we could use os.arch, and ship the x86 libraries without suffixes." - OK, I suppose this is fine for your own binary distribution of jnotify.jar. So you'd have libjnotify.so and libjnotify-amd64.so. I agree it's unlikely anyone will have something other than "amd64" on that platform.

     
  • Ximin Luo

    Ximin Luo - 2014-03-22

    "But, sane Linux distributors, such as Debian, will instead ignore your binary jnotify.jars and compile their own per-arch jnotify.jars with jnotify.so in each jar for the specific architecture." - I was confused here, let me correct myself. Debian will not compile separate .jars, but it will compile separate libaries for each arch, all called /usr/lib/jni/libjnotify.so, but distribute these in arch-specific .debs. (And the non-working stuff I said about symlinks in jars was based on the same conceptual confusion.)

    I think your suggestion of "Maybe we could use os.arch, and ship the x86 libraries without suffixes" is acceptable and I'll probably be fine using that in Briar. I still think it's unnecessary, but at least it doesn't make your library-loading public interface more complex, so I'll stop overanalysing here and let you get on with it. :)

     
  • Matthew Donoughe

    Sourceforge keeps deciding that some of your comments require moderation and I don't understand why.

    I've looked into the GCC source, and apparently GCJ has been normalizing their os.arch values since 2005. It's probably safe to drop support GCJ from nine years ago, in which case we don't need the normalization or the unprefixed x86 libraries.

     

Get latest updates about Open Source Projects, Conferences and News.

Sign up for the SourceForge newsletter:





No, thanks