Architecture independent build of the native part
Status: Beta
Brought to you by:
bobby_newmark
Menu
▾
▴
#4 Architecture independent build of the native part
The Makefile currently offers just the following two options:
DEFAULTCONF=Release_x86-64bit
DEFAULTCONF=Release_x86-32bit
But we have tested, that the native part can be built without problems
on other architectures too.
We currently build the native part on x86(_64), s390(x), sparc(64) and ppc(64).
Please, change the targets to generic ones:
DEFAULTCONF=Release_64bit
DEFAULTCONF=Release_32bit
Things are as they are because:
1) There is no support for architectures other than x86 32/64 bit, simply because I don't have access to them and therefore cannot run the test suite there
2) Because everything is built through NetBeans, and that's how the IDE manages the Makefiles for me. I don't want to interfere with the way the IDE builds those libraries. For the individual projects that exist for each OS, see [1]
Things you can do:
1) build it yourself and don't care about the way I build stuff (but this way this won't make it into the codebase)
2) submit a patch that adds the target architecture you need to the NetBeans projects in question (I'm assuming you're talking about the Linux or BSD native projects).
Cheers,
Uwe
[1] http://jpathwatch.svn.sourceforge.net/viewvc/jpathwatch/trunk/jpathwatch/jpathwatch-native/prj/
Hello Uwe.
Yes, I'm talking about Linux, since I'm the jpathwatch maintainer for Fedora. I introduced this package in Fedora approximately 2 weeks ago, because it was needed as a dependency of other packages which are platform independent.
I understand Your worries. Would it be at least possible to equip the source tarball with something like a very short BUILD HOWTO containing instructions for building? There could be some note mentioning, that the x86 targets can be used also for other platforms (but without any guarantees).
I already have the packages built ... so ... I could say : "I don't care anymore" ... but as this information could help other distribution maintainers with jpathwatch packaging, it would be really nice to document the build procedure better.
Thank You and have a nice day.
Regards,
Jaromir.
Hi Jaromir,
Thanks for your input. I do agree that there should be build instructions; I've actually just added them:
http://jpathwatch.svn.sourceforge.net/viewvc/jpathwatch/trunk/jpathwatch/BUILD.txt?revision=133&view=markup
The problem with the targets is that I can't really say anything first hand about how to do what you did, since I never build it that way (the IDE does it all for me, on all supported platforms).
But let me suggest this:
Could you read the new BUILD.txt file that's in SVN and let me know if its clear for you? You could then also add the missing architectures to the jpathwatch-native-linux project, and see if this works for you. If that's all fine I'll add your patch to the codebase.
I suppose this should make your life easier, as you won't have to merge your changes in every time you pull a new version from SVN.
What do you think?
-Uwe
Hi Uwe.
NetBeans IDE is not needed for the build at all since the source directory contains ANT compatible build.xml file for the Java part and working Makefile for the native part.
Generally all distribution build systems are fully automated and cannot run anything graphical like NetBeans IDE.
It seems that NetBeans produce all files needed for building with build systems and since You build the whole product with NetBeans, it's probably difficult for You to do any manual changes in the files needed for the build and maintain such changes. I just had no idea about Your way of building the product.
At the moment I don't use any patches for the Makefile. I use the x86 targets in the RPM spec file instead.
The following is a BUILD section of the RPM spec file I use for the jpathwatch package. %{name} is defined in the head section as jpathwatch, %{site_ver} as 0-94
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
# native part
cd %{name}-native/prj/linux/%{name}-native-linux
%ifarch x86_64 s390x sparc64 ppc64
make DEFAULTCONF=Release_x86-64bit %{?_smp_mflags} CXXFLAGS="%{optflags}"
%else
make DEFAULTCONF=Release_x86-32bit %{?_smp_mflags} CXXFLAGS="%{optflags}"
%endif
# java part
cd ../../../..
ant -Dplatforms.JDK_1.5.home=%{_jvmdir}/java jar
# javadoc target exists but doesn't work - generating
find %{name}-java/src -name '*.java' | xargs javadoc -classpath dist:%{name}-%{site_ver}.jar -d doc
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
So this part of the spec file could be rewritten to a second HOWTO suitable for building with build systems
The following flags are Fedora specific and should be removed.
%{?_smp_mflags} CXXFLAGS="%{optflags}"
========================================================
Next issue - obtaining the source code from SVN.
As I've already written. The build systems are often automated and some of them (for example Arch Linux and derived ones using Pacman) can download the source archive automatically during the package build.
But they use just simple wget call expecting a directly downloadable source archive via HTTP protocol.
Therefore it would be VERY nice, if You could create a source archive (probably ZIP, since it is for multiple platforms) with each new jpathwatch release and put it in the Download section (without changing anything in the HTTP path ... just the version number change is expected with each release).
Hi Jaromir,
Wouldn't additional Release_s390 and Debug_s390 configurations in the NB project do what you need? Because all of this should propagate through to the Makefile.
Also, feel free to add to BUILD.txt if you want to leave notes behind for other distro maintainers.
As for the tarball, I suggest you look at this:
http://jpathwatch.svn.sourceforge.net/viewvc/jpathwatch/tags/0-94/
It contains a link 'Download GNU tarball',
http://jpathwatch.svn.sourceforge.net/viewvc/jpathwatch/tags/0-94/?view=tar
which I suppose is what you'd need, is it?
-Uwe
Hi Uwe.
Additional configurations would make the spec file longer and more complicated, but yes, it's also acceptable. I would have to install NetBeans and test that. But as it's a long time ago when I was doing anything in NetBeans, I'll rather reserve a sufficient time window for that (definitely not this week).
I know about the 'Download GNU tarball' option. It's ok for manual downloading, but if You do that with the wget tool, the filename then always looks like this : "index.html?view=tar" and as You can see, there's no package name and version in such filename and that would replace older / different source archives with the same name in the source cache. That's why it always leads to fallback to manual downloading ... It's questionable if this is a wget's feature or a bug.
Regards,
Jaromir
[quote]Additional configurations would make the spec file longer and more
complicated, but yes, it's also acceptable (...) [/quote]
I just noticed there is another issue as well with the way you currently build things: the native libraries you build will NOT WORK on an s390 system. jpathwatch explicitely selects the native libraries from it's JAR file that it loads. The idea is that the JAR file contains native libs for all supported platforms, and the one matching the current OS/architecture combination will be loaded at runtime.
What you're currently doing might build fine, but will not work at runtime, because when running on s390, jpathwatch will not find an entry in its native library list for this architecture. So all that people will get is the polling fallback (which is a very limited pure Java implementation).
So if you need support for s390, there will be no way around implementing 'official' support for it. Since this is Linux on s390, this is quite trivial (half a days work, if at all).
I'm happy to help you with that.
[/quote]I know about the 'Download GNU tarball' option. It's ok for manual
downloading, but if You do that with the wget tool, the filename then
always looks like this : "index.html?view=tar" (...) [/quote]
I see that this sucks. But couldn't you do something like this?:
[code]wget http://sourceforge...../0-94/...?view=tar
mv index.html?view=tar jpathwatch-0.94.tar.gz[/code]
However, I've also seen wget use redirects with shonky URLs like this one and then eventually it will select the right filename. I guess that would be a feature request for source forge. You're not the only one with this problem; I've had this request previously as well. The issue is that everything has to be done manually, which is quite a PITA.
-Uwe
Related
Code: code
Hi Uwe.
Sorry for my late response.
I was checking the JAR file produced on the s390 arch and You're right. There's a directory called "GNU-Linux-x86" inside the JAR file. So ... even if the library is build for s390, it won't be found.
I would really appreciate Your help with that since I'm a bit overloaded these days.
Please, let me know if You're ok with that. I can at least immediately resend the altered sources to my colleague, who's responsible for the s390 archs, for testing.
Once again, Thanks!
Regards,
Jaromir.
Hi Jaromir,
I'm happy to help, here's what we'll do:
* I will add an S390 config to the Linux project.
* You will run a little java program to get the system properties we need on s390 (see below)
* I will make the code changes so that the s390 native lib gets loaded
* You pull the latest sources (the trunk in SVN), compile everything and test
Once we're happy with the changes, I'll either make a new release or back-port to 0.94 (but a 0.95 is overdue anyways, so I might as well do that)
Here's the source of a java program that prints the necessary system properties:
--- begin of sysinfos.java ---
class sysinfos{
public static void main(String[] args){
String osname = System.getProperty("os.name");
String osversion = System.getProperty("os.version");
String osarch = System.getProperty("os.arch");
System.out.println("os.name = " + osname);
System.out.println("os.version = " + osversion);
System.out.println("os.arch = " + osarch);
}
}
--- end of sysinfos.java ---
compile it:
> javac sysinfos.java
run it:
> java sysinfos
On my box, this produces this output:
> java sysinfos
os.name = Linux
os.version = 2.6.34.8-0.2-desktop
os.arch = amd64
Please run it on the s390 box and post the output here.
Cheers,
Uwe
Hi Uwe.
Sorry for the delay.
I had no access to the s390 devel system.
Here's the output from s390x ...
os.name = Linux
os.version = 2.6.38.6-26.rc1.fc15.s390x
os.arch = s390x
I'll try to attach the s390 output asap.
But I believe it'll be similar.
Regards,
Jaromir.
Ok .... like I expected ....
Output from the s390 mock environment ...
os.name = Linux
os.version = 2.6.38.6-26.rc1.fc15.s390x
os.arch = s390
as the mock environment is just an advanced chroot, the os.version still has that "x" suffix ... I expect it wouldn't be there on a real s390 .... we unfortunately have just the "x" versions and the old s390 is just emulated ... the same applies to our s390 build system.
Hi Uwe.
It's been a long time since we discussed the other archs support.
Do You have any news here?
Thanks,
Jaromir.
Hi Jaromir,
Geez, I didn't realize that it's been such a long time. I've finally managed to release jpathwatch 0.95 (as of tonight :). However, I've also been looking at your s390 problem again, and although I can't officially support s390 (since I don't have access to the platform), I added s390(x) configurations to the jpathwatch's linux native library project.
I also included support in the NativeLibLoader so that on s390, jpathwatch will *attempt* to load a library for s390. Have a look at the NativeLibLoader.java source file, and the binary 0.95 distribution for comparison to see where the libs need to go; in principle it should all build out of the box for you (but obviously I couldn't test it, so you might have to play around a little).
Also note that 0.95 release in principle contains support to load libraries for s390 and s390x, so in theory adding the native libs to jpatchwatch-0-95.jar in the right spot should also just work.
Hope this isn't too late (well, it is late, I admit), and let me know how you're going.
Cheers,
Uwe