Menu
▾
▴
#587 Make life easier for Linux distributions such as Debian
Hi, I'm trying to update Jmol in Debian and your non-standard development workflow makes it very hard for us to do this. No other software does what you do, and other Linux distributions such as Fedora, Gentoo and Arch also have the same problems as us - we're in contact with the Gentoo Jmol maintainer actually, and he has basically given up on "doing it properly".
Would you be able to release source packages of Jmol, JSmol and JSpecView together as one source tarball? I notice there are already circular dependencies between all three packages, so none of us Linux distributions would be able to package it as separate source packages anyways. What I mean by "source package" is a tarball that only contains source files, and no "built" binary files. Instead, there would be one build file that goes through all of the steps currently listed in build.README.txt and applies them all at once. For example:
Jmol-14.6.4_2016.10.21-src.tar.gz:
Jmol/
JSmol/
JSpecView/
JSpecViewLib/
etc.
If you wish, you can keep your dependency jars in jars/ etc, we are used to removing them and replacing them with system versions. But the rest of the package is basically not possible to package given our limited volunteer resources.
Oh, one further question: where is netscape.jar from?
I'm also getting stuck trying to run "ant -f build_13_tojs.xml" - it tries to copy some files from JSmol/bin/..., but the bin/ directory doesn't exist. Where can I get or generate these files?
Ximin Luo,
Yes, I'm aware of the complexity of this business. I assure you that I would do it in a simpler manner if I could. There are reasons that JSpecView and Jmol and JSmol are all separate projects. I think if you look, though, you will see that all of the JSpecView code is in the full.tar.gz distribution file. So that should be all you need for that.
As for JSmol, that is an Eclipse J2S project. As such, it cannot be compiled outside of Eclipse. I would love to be able to do that, but I know of no way to do it. In any case, I wouldn't imagine that you have interest in building JSmol -- it's the HTML5 framework and JavaScript version of Jmol and JSpecView. It is not to be compiled into Java for any purpose other than to create JavaScript. That's all in JSmol.zip, in the binary distribution zip or tar file. JavaScript is source code itself, so I can't imagine there is any Debian issue there. Just use it.
Right?
Bob
ps -- By the way, build_13_tojs.xml is for the trunk; you are talking about the release branch. The ant tasks for that begin with build_03. You cannot run any of those tasks outside of Eclipse, I think.
You might want them to be separate projects, but the current circular development workflow means that they are not actually separate projects. Keeping them in separate directories within the same project is what you're actually doing, and this is common and good practise. But from a software engineering point of view, they are not separate projects even if they are stored under separate sourceforge "projects" - do you ever release JSmol with independent version numbers from Jmol?
It is possible to do things in a simpler way - (1) don't mix up the original source code into an unbuildable form in a release tarball that needs to be reviewed and deduplicated manually - just keep the original tree structure that you have during your development workflow, and (2) don't build binaries in separate steps with potential source code edits in between them; just build everything in one go.
This is beneficial for you and other developers too - instead of having to manually click lots of different buttons in Eclipse, and having to document complex relationships between which parts of the code you should edit, which parts you shouldn't, and which parts you should edit before others - instead of all of this, you can just run one simple command.
From another angle: having JSpecView source code inside the full.tar.gz does not count as "open source". The definition of "source code" is "preferred form of modification"; you must have heard of this before somewhere. From reading your developer documentation, it's clear that you have JSpecView and JSpecViewLib outside of the Jmol directory, and there are some manual steps you run (which can be automated) to update the binary parts. The source code dump inside full.tar.gz is a build result, not the source of that result. It's reference material, and not source code. We cannot easily use it to reproduce the full.tar.gz. The same goes for autogenerated Javascript code - it's not the preferred form of modification.
In Debian, we certainly do have an interest in rebuilding JSmol, as do all other Linux distributions. This is the only way to actually adhere to open source engineering principles, to make sure that what is claimed to be "source code" is actually the source of the binaries being distributed. This includes build scripts to automate this process.
I've tried to automate some of the instructions in your build.README.txt and I can actually automate most of it. The build01 / build02 steps do work outside of Eclipse, run directly in ANT.
However I'm stuck on the bin/java stuff that I mentioned. I'm guessing that these are generated by java2script? Upon doing some further research, I found this:
https://github.com/zhourenjian/java2script/blob/master/incubator/net.sf.j2s.ui.cmdline/docs/j2s-cmdline-api/j2s-cmdline-api.xml
which is a command-line interface for j2s. In theory you should be able to use this to automate away the last manual step (building JSmol inside Eclipse) via something like the following command line.
$ eclipse -nosplash -application net.sf.j2s.ui.cmdlineApi -cmd newProject -path Jmol -projectName Jmol -sourceFolders src -outputFolders js -cmd build -projectName Jmol
Would you be able to test this, figure out what the correct invocation is, and document it somewhere so that I can add it into my build automation script?
JSmol is a different project specifically because it is not a Java project.
It is a J2S project. So that source code cannot be integrated into Jmol, or
at least as far as I know. Likewise the other way around.
I think what you are struggling with is that we have two projects that are
linked. It's not circular. Components of both the Jmol code and the
JspecView code are used in the final "Jmol release". It so happens that
that code derives from two different projects. That's all. I suppose I
could store all the code for JSpecView in the Jmol src, and create
JSpecView from that, but that's not fundamentally any different from what
we are doing now. The JSpecView source code is in the tar file. You asked
for that, and that is what we have. What's to change?
On Sat, Oct 22, 2016 at 10:45 AM, Ximin Luo infinity0x@users.sf.net wrote:
Any "source code" in a tar file is not "the preferred source of
modification" -- doesn't matter where it comes from. It turns out that in
this case the jspecview code comes from a different SVN location. That's
all. Or, you can use the code in the Jmol project. It's the same code.
Again, you cannot build JSmol that way. It isn't a Java project.
A few more thoughts on this. I know this is important, and I want to work with you to make it work for you. Note that there is a Maven port for Jmol. Spencer Bliven is managing this.
There are two completely separate issues here. (1) Jmol/JSpecView, (2) JSmol.
(1) Jmol/JSpecView
These two projects share code. It is not a circular dependence. But there is a sequence of steps that must be followed. The sequence of operations for constructing the working applications is as follows, currently. These operations are done in Eclipse.
a) Check out JSpecView dev2 JSpecView and JSpecViewLib as separate projects.
b) Check out Jmol trunk or the latest branch (currently 14.6) as the "Jmol" project..
c) Run build.xml in the JSpecViewLib project.
d) Run build.xml in the JSpecView project. (This completes the creation of JSpecView.jar.)
e) Run build.xml in the Jmol project.
Step (c) imports a limited set of Java code from Jmol and compiles the OS-independent classes of JSpecView. Step (d) imports a limited set of Java code from Jmol and creates the JSpecView Java application and applet and copies JSpecView.jar to the Jmol project. Step (e) builds Jmol.jar, JmolData.jar, JmolApplet.jar, JmolAppletSigned.jar, and alll the modular Jmol applet files.
(2) JSmol
The creation of working JSmol applications is complicated by several factors:
With that in mind, here is the preferred form of modification for JSmol, as it stands today:
Before you start, be sure you are running an acceptable version of Eclipse, currently 3.7, 3.8, 4.2, 4.3, or 4.4.
a) Check out the JSmol project.
b) If you have not done so already, install the appropriate J2S core plug in into Eclipse. These can be found as JSmol/j2s/eclipse-plugin/j2s.core-x.x-v??????.zip. If you have to do this, you probably need to restart Eclipse.
c) Do all of the above for Jmol/JSpecView.
d) Run build_11_fromjmol.xml.
e) Run build_12_fromjmol.xml.
f) Refresh the JSmol project.
g) Build the JSmol project
h) Run build_13_tojs.xml
The JSmol/site directory is created and can be used for testing.
Step (d) simply copies source code from the Jmol project into the JSmol project.
Step (e) simply copies source code from the JSpecView into the JSmol project.
Step (f) is required for unknown reasons. If it is not done, Step (g) will not work properly.
Step (g) is a J2S build. It creates the .js files in JSmol/bin directory as well as the usual Java class files of a Java project. Only the .js files generated by the J2S transpiler are used. The class files, though created, are never used. Their importance is that if there are errors in the Java compilation, they will appear at this stage; the J2S transpiler process will not occur unless Java compilation is error-free. If, after refreshing the JSmol/bin directory, no .js files are seen, check that the .j2s project file includes the line;
j2s.compiler.status=enable
and that there were no compilation problems of type ERROR.
Step (h) is the main modification. It carries out a large number of tasks, including working around certain J2S transpiler bugs, creating the JSmol/site directory, transferring files into it, and creating JSmol.zip in the Jmol/appeltweb directory.
If the desire is to create release files for Jmol, then after doing all this, Jmol's build.xml must be run once more, as it packages all of the Jmol Java code along with JSmol.zip for distribution.
So, that's where we stand now. I wish it were fully automated, but it is not. I sincerely welcome any contribution that would streamline this operation and, especially, make it work from the command line.
Last edit: Bob Hanson 2016-10-23
Hey, thanks for the detailed reply! That's very useful.
(1) I'm fairly sure Jmol/JSpecView does have a circular dependency because JSpecView/build.xml and JSpecViewLib/build.xml both contain jmol.path and last time I tried, the build failed if Jmol wasn't present. The other part of the circle is where Jmol itself needs JSpecView.jar to build.
Likely, JSpecView only needs a subset of Jmol's files in order to compile against it, but this is nevertheless a circular dependency. It's possible to avoid this circular dependency, by factoring out the API files of Jmol (that JSpecView needs) into a separate project. A precursor to this would be to split these files out into a separate jar within the Jmol project, and have JSpecView use this jar instead of the whole Jmol project. This remains a project-to-project circular dependency but jar-to-jar it would no longer be a circular dependency.
I can agree that JSmol does not have a circular dependency with the others though, I was a bit imprecise with my wording above.
(2) Thanks, this helps a lot. When I next have some time, I will see if I can use this to figure out how to call J2S's cmdlineApi to do the equivalent thing.
The fact that J2S is only available for Eclipse until 4.4 is not a problem (yet) for Debian since we are still stuck on 3.8 sadly. This could turn out to be quite a nasty can of worms in the future, but for now we can ignore it.
--
It's not "fundamentally" any different, but the critical point here is the development-time cost it takes to do this.
In the full.tar.gz, there are no build scripts for JSpecView. So we can't do an automated build in Debian using full.tar.gz without tons of extra patching. First, we need to grab the correct SVN revision of JSpecView/build.xml and JSpecViewLib/build.xml. This is not technically hard but it's needlessly time-consuming and can't be automated. We would have to expend this cost for every release of Jmol. Every other Linux distribution would also have to expend this cost. This is a waste - you are the one that prepared full.tar.gz, so it's much easier for you to figure out which SVN revision of those files correspond to which releases of full.tar.gz, and directly include these in a source release.
This is not enough however: if you simply added these build files in full.tar.gz, we would still have to write scripts to extract JSpecView.jar and massage the source tree into a structure that is expected by the JSpecView build files. Again, this is not technically hard, it's just time-consuming and the cost multiplies for every new Jmol release (if you change the layout, we have to update our scripts) and as well as every new distributions that wants to package Jmol.
(Another issue is you have JSpecView.jar in two places, ./ and ./jars. This makes it confusing to package, and we again have to spend time figuring out which one to use, whether it's safe to delete one of them in case both are referenced from build files, etc etc.)
Similar points apply for jsmol.zip, but I didn't get far enough to make specific comments on that yet.
TL;DR: "it's possible" to work with full.tar.gz (after we figure out the JSmol automation), but the time cost is too much. That's why the title of this report is "make life easier". To give some background: I'm packaging Jmol because it's a dependency of SageMath. SageMath has 150+ dependencies. If all of these projects took as much time as Jmol to write automated build scripts for, it would be physically impossible for us to do this. (We're already finding it quite of an effort to do it, we're just about coping.)
Generally, FOSS convention is that the upstream project provides a tarball that contains development sources (including build scripts) in the form that the developer normally makes modifications in. To be precise, by "form" this means build scripts, as well as the exact layout of the directories and files that the build scripts expect them to be laid out in. Yes, the tarball itself is not strictly "the preferred form" but it's trivial to convert it into that - which is definitely not the case for the current full.tar.gz.
From your second post, I think you roughly understood what I just said, but I thought I'd point out the more specific details on why, just to make this point clear, and to describe more exactly what we would like Jmol to do.
(sorry if this post was longer than necessary)
Ah, now I have the context. Thank you. SageMath uses the Jmol applet, not the Jmol application. The Jmol applet does not use JSpecView -- only Jmol.jar depends upon that, and you are interested only in JmolApplet.jar. You can compile without JSpecView. Just ignore it completely. Just skip build_02_fromjsv. It is for creating the JavaScript version of the JSpecView applet; it doesn't have anything to do with the JavaScript version of the Jmol applet, which is what SageMath uses.
So with that behind us (possibly?), the real concern you have is with JSmol and the creation of the JavaScript code used in SageMath. Then the real issue is just the running of a J2S project in Eclipse and finding a way to automate that. It would be fantastic if you can figure out how to do the entire operation using the command line.
Please work with Zhou Renjian [https://sourceforge.net/u/zhourenjian/profile/] on that, as he has responsibility for that part of the project. It's really out of my skill set.
Bob
Last edit: Bob Hanson 2016-10-23
Ah, not quite. I'm packaging Jmol because I started off packaging SageMath. But that's just the history of what I was personally doing, and I mentioned SageMath to give you sense of the scale of what we're trying to automate. However there are other people in Debian interested in having a "full" version of Jmol, including the JSpecView part, unrelated to SageMath; so it wouldn't suffice (for Debian as a whole) if I only packaged the parts that are relevant to SageMath.
I've contacted Zhou Renjian already on Github, so yes I'll see if we can work together to figure out the J2S automation issue. Do you know if he is still active, and does he work with you on Jmol directly? That would certainly make things easier, I expect.
Outside of that, it would still be good to get a source tarball that contains "everything" (as I described earlier), for future packaging purposes.
If it's too much effort to create such a tarball yourself, it would also be OK if you documented, in a machine-readable format, the SVN repos and revisions that we would have to checkout, in order to automatically generate such a tarball ourselves. (edit: to clarify, this would be for each and every release)
Last edit: Ximin Luo 2016-10-23
Sorry -- I've lost the thread of the conversation. Tell me again how a
tarball would help you, and how you would use it.
On Sun, Oct 23, 2016 at 12:56 PM, Ximin Luo infinity0x@users.sf.net wrote:
--
Robert M. Hanson
Larson-Anderson Professor of Chemistry
St. Olaf College
Northfield, MN
http://www.stolaf.edu/people/hansonr
If nature does not answer first what we want,
it is better to take what answer we get.
-- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900
Related
Bugs: #587
A source code tarball that contains:
would help us to write scripts to automate the production of Jmol binaries. This is what Debian and other Linux distributions need.
The current full.tar.gz release doesn't allow us to do this easily. In particular, we have to manually figure out which SVN revisions of JSpecView / JSmol to checkout, to get their build scripts (build.xml) from. We also have to maintain extra scripts to extract and rename things into the "original developer directory-tree structure", which could change between different releases of Jmol. It would be much easier (for automation) to start directly from the SVN checkouts.
However, we still need to know which SVN revisions correspond to a given Jmol release. Hence why my alternative suggestion was a machine-readable file that documents this information. You'd commit ("promise") to keep this file in a stable location and format, across multiple releases - and we could write our automation scripts to run against this file.
But you said you were only interested in the html5 applet. How are you going to create that?
Robert M. Hanson
St. Olaf College Chemistry
from my Windows phone
-----Original Message-----
From: "Ximin Luo" infinity0x@users.sf.net
Sent: 10/23/2016 2:00 PM
To: "[jmol:bugs] " 587@bugs.jmol.p.re.sf.net
Subject: [jmol:bugs] Re: #587 Make life easier for Linux distributions such asDebian
A source code tarball that contains:
the original development tree of Jmol (plus side projects), with the files and directories laid out exactly as how they exist on developers' machines (including build scripts)
would help us to write scripts to automate the production of Jmol binaries. This is what Debian and other Linux distributions need.
The current full.tar.gz release doesn't allow us to do this easily. In particular, we have to manually figure out which SVN revisions of JSpecView / JSmol to checkout, to get their build scripts (build.xml) from. We also have to maintain extra scripts to extract and rename things into the "original developer directory-tree structure", which could change between different releases of Jmol. It would be much easier (for automation) to start directly from the SVN checkouts.
However, we still need to know which SVN revisions correspond to a given Jmol release. Hence why my alternative suggestion was a machine-readable file that documents this information. You'd commit ("promise") to keep this file in a stable location and format, across multiple releases - and we could write our automation scripts to run against this file.
[bugs:#587] Make life easier for Linux distributions such as Debian
Status: open
Group: v10
Created: Sat Oct 22, 2016 12:38 AM UTC by Ximin Luo
Last Updated: Sun Oct 23, 2016 05:56 PM UTC
Owner: nobody
Hi, I'm trying to update Jmol in Debian and your non-standard development workflow makes it very hard for us to do this. No other software does what you do, and other Linux distributions such as Fedora, Gentoo and Arch also have the same problems as us - we're in contact with the Gentoo Jmol maintainer actually, and he has basically given up on "doing it properly".
Would you be able to release source packages of Jmol, JSmol and JSpecView together as one source tarball? I notice there are already circular dependencies between all three packages, so none of us Linux distributions would be able to package it as separate source packages anyways. What I mean by "source package" is a tarball that only contains source files, and no "built" binary files. Instead, there would be one build file that goes through all of the steps currently listed in build.README.txt and applies them all at once. For example:
Jmol-14.6.4_2016.10.21-src.tar.gz:
Jmol/
JSmol/
JSpecView/
JSpecViewLib/
etc.
If you wish, you can keep your dependency jars in jars/ etc, we are used to removing them and replacing them with system versions. But the rest of the package is basically not possible to package given our limited volunteer resources.
Oh, one further question: where is netscape.jar from?
Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/jmol/bugs/587/
To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/
Related
Bugs: #587
I am not only interested in the HTML5 applet. As I explained, if I am to package this for Debian, I must package all components of it: Jmol, JSmol, and JSpecView.
I will figure out a way to automate the JSmol part. I am happy to explore this myself; the details you supplied in part (2) above will be useful for this.
But I do also need the other information I mentioned, namely (for each release of Jmol), either (a) a source tarball that follows the structure that I described, or less preferably (b) a machine-readable file that I can use to automatically recreate such a tarball. This is something you can start doing now, independently of my attempts to automate the JSmol part. It will help a lot, since it allows me to construct 80% of the Debian packaging build scripts. The JSmol automation will be the final 20%, but you don't need to worry about that.
I'm sorry, I'm not seeing what is being asked here yet. If you contribute
the ANT task that creates this tarball, I will see what i can do to
implement it in Eclipse.
Bob, I've tried to explain what I need very clearly in the text above. I'm not sure which part you don't understand. Can you at least explain your thought process on how you're understanding the text that I already wrote?
Option (a) would look something like:
Jmol-14.6.4_2016.10.23-src.tar.gz:
Jmol/
JSmol/
JSpecView/
JSpecViewLib/
Option (b) would look something like
Jmol-14.6.4_2016.10.23-src.list:
svn://svn.code.sf.net/p/jmol/code/trunk r21227
svn://svn.code.sf.net/p/jspecview/svn/ rXXXX
svn://svn.code.sf.net/p/jsmol/code/trunk rYYYY
with rXXXX and rYYYY replaced by the actual revision numbers that was used to produce release 14.6.4_2016.10.23.
What is not understandable about what I just wrote?
If I have this right, your goal is to recreate JSmol.zip from source. This
is what SageMath uses. They do not use the Jmol Java applet or application.
Correct?
It seems to me a hugely more difficult process for you and also totally
unnecessary to have to recreate all the developer build scripts with
multiple projects. Why not just let me create a single set of source
directories and supply one build script that you (or anyone else) can run
to create the part of JSmol.zip that you need?
src/org/jmol/
src/jsjava
srcjs/...
java/
jsmoljs/
Build.xml
Q: Do you need all the source files for the language translations? Or can
you use POT files?
You know this includes a major part of the Java language in JavaScript,
right? What's your plan for that?
As I have stated already three times - no, I am not only interested in recreating JSmol.zip from source. I am interested in creating all parts of Jmol from source.
We need all the source files for everything, including language translations, etc. That is why it is easier to start from SVN. This is not a problem, we do it all the time.
The problem is that Jmol does not make standardised source tarballs in the form that I described (option (a) above). This is what I would like you to help with, but you keep changing the topic.
Yes, I have seen that J2S recreates a lot of the Java language in JavaScript. I will use the J2S cmdlineApi to handle this. I can do this myself, you don't need to think about it. What I'm asking you to think about, is option (a) or option (b) as I described in my previous post.
I would be happy to explain to you the details of how our automation works, once I complete it. At first it will be in Makefile format, but you could translate this into build.xml format if you want to run it in Eclipse yourself.
I can't explain to you the exact details of it right now, because I haven't done it yet. In many cases, I do packaging by trial-and-error, so it's hard for me to explain the fine-level details of it because I haven't finished it yet. So, sorry if you find my description of the overall process insufficient. But my intuition and reading up on J2S so far indicates to me that it will work, and I haven't failed once doing this sort of thing yet.
In fact, let's pause this conversation for now; I don't think we are getting anywhere. I will come back with a concrete example of (b) by reverse-engineering the correct SVN revisions myself for the first version of our package. Then, hopefully you can see what I am talking about. After that, it would be good if you take over the maintenance and update of the small file I am talking about, since it's easier for you to forward-engineer this for each new Jmol release, and harder for me to reverse-engineer it. How does that sound?
OK, so you mean, actually, you are not just doing this for SageMath.
Since I seem to be too dense to understand your endeavor, and since we
already have a Maven build for Jmol, I simply need you to create the
tarball yourself once, send it to me, and then I will see what I can do to
recreate it.
Thanks in advance for your contibution.
OK, I managed to figure out the J2S step. All you have to do is build the
cmdlineApiJ2S plugin, install it, then run this sequence of commands (in Makefile syntax):This can be done without opening up the Eclipse GUI, but you do need both Jmol and JSmol checked out from SVN (so you have their
.projectfiles). The "(f) Refresh the JSmol project" step seems to be unnecessary here; I have lots of .js files in JSmol/bin and build_13 succeeds.I guess the reason why it's necessary in your case, is you run build_11 and build_12 inside Eclipse, but this invalidates Eclipse's own internal cache about what the outside filesystem looks like. That's usually what "refresh" means, to re-populate a cache. Since I don't have Eclipse open in my automation script, there is no invalid cache; the
eclipsestep builds its cache from scratch and after it finishes, it's no longer needed.I'll complete the rest of the automation and give you a source tarball example, as requested. This will likely take me about a week or so more. But the steps are clear now, I think there are no "unknown" steps left to fill in.
(edit: necessary -> unnecessary)
Last edit: Ximin Luo 2016-10-24
Two questions:
I'm having difficulty regenerating
JSmol/j2s/java. I thought I could do it by running J2S on thenet.sf.j2s.java.coreproject from java2script's git repo, but there are some discrepancies. For example, you haveSequenceInputStream, but the other project doesn't. How are you regenerating these files?Is
JSmol/lib/jsmeessential? We probably have to omit this from Debian, since I don't see a feasible way to regenerate this:Would it be so bad to omit it? It doesn't seem to be used for any critical functionality; the rest of JSmol seems to work fine without it.
Last edit: Ximin Luo 2016-10-25
The J2S transpiler does not depend at all upon any specific Java language
source or already transpiled Java language source. It does not create,
read, or process .class files. It is solely a transpiler. So I'm not sure
what issue you are talking about there. All it does is to create .js files
in the bin/ directory directly from the .java files in the src/ directory.
So it would not matter at all if I had one specific Java class or not.
src/java is generated by build_01 -- take a look there for reference.
Specially optimized Java language source files are in srcjs/java along with
some special pre-transpiled .js Java classes that I created myself. These
are moved into src/java by build_01. After J2S transpiling, which creates
.js versions of those in bin/java, build_03 first copies the original J2S
project java files from {project}/j2s/java into site/jsmol/j2s/java and
then adds the newly created bin/java .js files over those. Additional
transpiled Java language classes and methods are provided by
jsmoljs/JSmolJavaExt.js.
It appears to me that SequenceInputStream.js should be in srcjs/java, not
j2s/java. You are right that it is not in the original J2S distribution.
There could be others. Early on I was not making this distinction. I see
there are about 30 such files.
BTW, I just checked in changes to the build tasks in JSmol that does a
better job of isolating references to other projects -- build_01 only
references Jmol; build_02 only references JSpecView; build_03 does not
reference either, except at the very end, where it deposits its final
jsmol.zip product in a Jmol subdirectory for distribution and release,
which is all done in the Jmol project using its build.xml and
tools/jmol-release.xml.
Note that you will need to use an up-to-date j2s.core in order to generate
the JavaScript files with a minimum of bugs. What version of j2s.core are
you using? This needs to be one of the ones in j2s/eclipse_plugin.
My issue is not a technical one, but a policy one. I am using J2S from this commit from 2015-04-12 for Eclipse 3.8 which I compiled myself, which is newer than j2s.core-3.8-v20150228.zip from JSmol/j2s/eclipse_plugins. So everything should be fine from a technical viewpoint.
However, most of the files under JSmol/j2s/java look like they are generated from another source Java file, so they are not the "preferred form of modification".
This means that in Debian we cannot (by policy) use them directly; we must generate them from the real sources. Furthermore, we are not supposed include generated files in "source tarballs" (such as the one I will be creating for you as an example).
If SequenceInputStream.js (and the 30 other files) are not actually needed for JSmol to work, then great - we can just drop them. However, if it is needed, then (by policy) I must regenerate them. Do you remember how you created them in the first place?
In other words, for every file underneath JSmol/j2s/java, I'd like to know:
Manually-written javascript is "source code", autogenerated javascript is not "source code". I can only include the former directly, I cannot include the latter and I must generate those - which requires me to know how to generate those.
Hi, I have completed the initial version of the script to autogenerate a source tarball of Jmol, in the form that Debian and other Linux distributions require.
You can see the script here:
https://github.com/infinity0/jmol/blob/master/debian/make-orig-tarball.sh
You can download an example tarball here:
https://github.com/infinity0/jmol/archive/854b863180b81c9f041166c5af87961bc68b3666.tar.gz
The resulting tarball contains a Makefile which lets you do a completely automated full rebuild of Jmol, JSmol and JSpecView, without ever opening up Eclipse, or even using the mouse.
Note that it is yet incomplete, there are 2 outstanding issues:
But everything else is ready, and "the general gist" is there ready to be reviewed.
Please take some time to study the script and understand what it does. Let me know if you think you will be able to reproduce the equivalent tarball, for future releases of Jmol. This is absolutely essential in order to keep Jmol maintained in the long run.
It has been out-of-date in Debian for 3 years because nobody took the effort to do what I just did. But I do not have time myself, to repeatedly do this for all future releases of Jmol - I have 150 other packages to take care of.
For example, in JSmol SVN revision r928 you checked in a load of jars into SVN. This is something that must explicitly be kept out of the tarball that we and other Linux distributions require. I do not have the time to monitor your SVN repositories like this, and keep make-orig-tarball.sh up-to-date.
All those other 150 projects are able to provide these source tarballs without much effort, on their part and on our part. If you follow the advice I give in the shell script, you will also be able to do this, without significantly affecting your development workflow.
Please let me know if you'll be able to do this.
(edit: clarify that the Makefile rebuilds JSmol and JSpecView, as well as Jmol itself)
Last edit: Ximin Luo 2016-10-26
Hello Bob, were you able to take a look at the script that I provided above? Do you "get" what it's trying to do?
I wrote it as a UNIX shell script because it takes much less time to do that. However, I'd more than happy to re-write it as a build.xml ANT script - but only if you can agree to spend the effort to keep it maintained in the long run! ANT build scripts do take quite a lot of time and effort to write, but there are ways to make this take less time. For example, by following more standard software development practises and avoiding checking binary build result output into SVN control - then the script could be greatly reduced.
Without these things, it would be too hard for Linux distributions to include Jmol/JSmol in the long run. I have already spent more than 10x the time on Jmol/JSmol (not exaggerating) that I have spent on other projects of a similar size. I won't be able to do this going forward in the long run.
That would be a real shame, since it's obviously a very advanced and sophisticated piece of software - but some of the more basic engineering around it is lacking, which makes it hard to distribute in a way that is able to concretely give users the abilities and freedoms that Free Software is supposed to promise.
I will file a separate issue for tracing the "real source code" of the J2S library portions of JSmol, as well as regenerating it via the command-line. This is another critical issue for Free and Open Source software distributions, separate from the source-tarball packaging that I've been discussing in the rest of this post. (edit: now filed as #589)
Last edit: Ximin Luo 2016-11-03