Menu ▾ ▴

#589 Document the original source of JSmol/j2s/java and JSmol/srcjs

v10
open
nobody
None
5
2016-11-03
2016-11-03
Ximin Luo
No

Hi,

(Filing this here because the JSmol project has "create ticket" disabled.)

As mentioned earlier on #587 we'd like to know the original source of the files in JSmol/j2s/java/** and JSmol/srcjs.

This is needed to pass Debian policy audits, and also it's generally good engineering practise that allows us to be sure that users have the software freedoms that FOSS software should have - here, specifically, to be able to study the code, and to make modifications to it themselves if they wish, and distribute those as well.

The Java files look like they are taken from some version of a JDK. It's probably possible for me to argue this past the Debian review processes - but it would greatly help me do this, if you could tell me which versions of the JDK you took these files from. (You could document it in SVN for others to read as well; that would be ideal.)

The JavaScript files look like they were auto-generated, and some of them look like they were manually hand-optimised later. By default, this does not count as "source code" by the common definition ("preferred form of modification"). For me to be able to argue this past the Debian policy reviewers, I need to know how they were originally generated.

If you do actually hand-modify them as part of your normal development process, please document and describe this workflow (ideally in SVN) - it would help me to potentially argue that some of these files are "source code". However, I can see that this definitely won't be enough for all of those files.

By default, we have to treat autogenerated code as "not source code" to be safe - e.g. we don't know if it breaks someone's copyright, or if it contains security backdoors, or whatever. I'm not accusing you of these things, it's just that we have to be strict in our auditing processes to make sure that we don't get screwed in the future.

Now, onto the details of the specific files:

(A) Some files are trivial/tiny or laid out relatively nicely, e.g.:

  • j2s/java/io/DataInput.js // one line of code
  • srcjs/java/util/AbstractList.js // potentially reviewable and modifiable by a human

Files of this type could fit within a loose stretched definition of "source code". However, we'd still want to know how the original version was generated.

(B) Other files do not fit the characteristics of (A), e.g.:

  • j2s/java/lang/Class.js
  • j2s/java/lang/ClassLoader.js

There is no way we can reasonably argue that these files are "source code", and we definitely need to know how this was generated.

(C) Some files seem to have been manually-edited after being automatically generated:

$ git grep -l '// *BH' -- srcjs j2s/java
srcjs/com/jcraft/jzlib/InflaterInputStream.java
srcjs/java/io/BufferedInputStream.java
srcjs/java/lang/Enum.js
srcjs/java/util/AbstractList.js
srcjs/java/util/ArrayList.js
srcjs/java/util/Arrays.js
srcjs/java/util/Collections.js
srcjs/java/util/HashMap.js
srcjs/java/util/Hashtable.js
srcjs/java/util/zip/ZipOutputStream.java

If these also come under (A) then it might be reasonable to argue that these are the "preferred form of modification".

Finally, I'm sorry if this seems like stupid amounts of bureaucracy and paperwork - I don't like having to do this either. But it is necessary to be able to properly know that what we're using is actually free open source software. If we don't know this, and what we're using is only "maybe free software" then we're not really sticking to any principles and we might as well go use proprietary software instead.

Related

Bugs: #589

Discussion

  • Bob Hanson

    Bob Hanson - 2016-11-04

    On Thu, Nov 3, 2016 at 2:39 PM, Ximin Luo infinity0x@users.sf.net wrote:

    The Java files look like they are taken from some version of a JDK. It's
    probably possible for me to argue this past the Debian review processes -
    but it would greatly help me do this, if you could tell me which versions
    of the JDK you took these files from. (You could document it in SVN for
    others to read as well; that would be ideal.)

    It's probably not possible to go back and determine this. I'm sorry about
    that. 6.14 or 6.27 for the ones I created; I don't know about Zhou Renjian.

    The JavaScript files look like they were auto-generated, and some of them
    look like they were manually hand-optimised later. By default, this does
    not count as "source code" by the common definition ("preferred form of
    modification"). For me to be able to argue this past the Debian policy
    reviewers, I need to know how they were originally generated.

    I do not know about the ones in j2s/java, although I think some of those
    are mine. The vast majority are from the j2s project.

    If you do actually hand-modify them as part of your normal development
    process, please document and describe this workflow (ideally in SVN) - it
    would help me to potentially argue that some of these files are "source
    code". However, I can see that this definitely won't be enough for all
    of those files.

    Hand modified means just that. They were hand modified. What more can I say?

    By default, we have to treat autogenerated code as "not source code" to be
    safe - e.g. we don't know if it breaks someone's copyright, or if it
    contains security backdoors, or whatever. I'm not accusing you of these
    things, it's just that we have to be strict in our auditing processes to
    make sure that we don't get screwed in the future.

    I understand. It could be JSmol in particular will not fit your model and
    cannot be introduced into Debian. I suggest considering that before going
    any further.

    Now, onto the details of the specific files:

    (A) Some files are trivial/tiny or laid out relatively nicely, e.g.:

    • j2s/java/io/DataInput.js // one line of code
    • srcjs/java/util/AbstractList.js // potentially reviewable and
      modifiable by a human

    Files of this type could fit within a loose stretched definition of
    "source code". However, we'd still want to know how the original version
    was generated.

    These were pulled from a bin/ directory after running the J2S compiler
    within Eclipse. AbstractList.js was further modified for JavaScript by
    hand. Java source unknown.

    (B) Other files do not fit the characteristics of (A), e.g.:

    • j2s/java/lang/Class.js
    • j2s/java/lang/ClassLoader.js

    These two are not used in JSmol.

    There is no way we can reasonably argue that these files are "source
    code", and we definitely need to know how this was generated.

    (C) Some files seem to have been manually-edited after being automatically
    generated:

    $ git grep -l '// *BH' -- srcjs j2s/java
    srcjs/com/jcraft/jzlib/InflaterInputStream.java
    srcjs/java/io/BufferedInputStream.java
    srcjs/java/lang/Enum.js
    srcjs/java/util/AbstractList.js
    srcjs/java/util/ArrayList.js
    srcjs/java/util/Arrays.js
    srcjs/java/util/Collections.js
    srcjs/java/util/HashMap.js
    srcjs/java/util/Hashtable.js
    srcjs/java/util/zip/ZipOutputStream.java

    If these also come under (A) then it might be reasonable to argue that
    these are the "preferred form of modification".

    Hand-modified.

    Finally, I'm sorry if this seems like stupid amounts of bureaucracy and
    paperwork - I don't like having to do this either. But it is necessary to
    be able to properly know that what we're using is actually free open source
    software. If we don't know this, and what we're using is only "maybe free
    software" then we're not really sticking to any principles and we might as
    well go use proprietary software instead.

    I understand. And, more importantly, you need to find out if it is even
    possible to get these into Debian. I hope my comments help you make that
    decision.

     

    • Ximin Luo

      Ximin Luo - 2016-11-06

      I understand. It could be JSmol in particular will not fit your model and cannot be introduced into Debian. I suggest considering that before going any further.

      It's on the borderline, and I need to learn more about it to answer this question with more certainty.

      But I want to try hard to make it work. It's used by a significant number of other scientific software, so I think it would be good to have it. Necessary, even - whenever something depends on Jmol and we want to package that for Debian, then we have to package Jmol first.

      It's probably not possible to go back and determine this. I'm sorry about that. 6.14 or 6.27 for the ones I created; I don't know about Zhou Renjian.

      Understood, thanks, this is still useful - it gives me something to start from. e.g. I can try to diff JSmol against those JDK sources.

      Hand modified means just that. They were hand modified. What more can I say?

      [..]

      I do not know about the ones in j2s/java, although I think some of those are mine. The vast majority are from the j2s project.

      [..]

      These were pulled from a bin/ directory after running the J2S compiler within Eclipse. AbstractList.js was further modified for JavaScript by hand. Java source unknown.

      Do you remember which bin/ directory - was it net.sf.j2s.java.core/bin? If we can identify this, then I could potentially reproduce this myself.

      In fact, could you go into more detail on how j2s/java and srcjs were created in the first place? If you don't remember the exact details, the general gist of it would be useful too - it gives me a starting point to think of solutions.

      As for the hand modifications, I guess they were motivated by bugs that you could see in JSmol. I'm wondering if it would be feasible (for me) to delete j2s/java and srcjs and recreate them by running the newest J2S on the newer Java 7/8/9 SDKs. Java are generally pretty good at keeping backwards compatibility, this has a decent chance of actually working.

      These two are not used in JSmol.

      Do you know which files are used by JSmol? If these aren't used, then I can simply drop them from the Debian package and then we don't need to argue about if it's "source code" or not.

       

      • Bob Hanson

        Bob Hanson - 2016-11-08

        On Sun, Nov 6, 2016 at 6:49 AM, Ximin Luo infinity0x@users.sf.net wrote:

        Do you remember which bin/ directory - was it net.sf.j2s.java.core/bin?
        If we can identify this, then I could potentially reproduce this myself.

        It was the bin directory in my Eclipse project -- the Java files were
        added to the project, transpiled, and removed. The .js files were then
        edited to work in JavaScript.

        In fact, could you go into more detail on how j2s/java and srcjs were
        created in the first place?

        j2s/java: My understanding is that the standard Java files were place in a
        project, the j2s transpiler was run so as to create the JS equivalent of
        "class" files, and then those js files were included. It's possible that if
        you go to the j2s site or talk with Zhou Renjian, you may find your answer.

        srcjs: These should be "the source of the .js files" - that is, the
        originals used to generate those.

        As for the hand modifications, I guess they were motivated by bugs that you
        could see in JSmol. I'm wondering if it would be feasible (for me) to
        delete j2s/java and srcjs and recreate them by running the newest J2S on
        the newer Java 7/8/9 SDKs. Java are generally pretty good at keeping
        backwards compatibility, this has a decent chance of actually working.
        That certainly will not work. Oh, that it were that simple! I don't have
        time here to go into the reason "raw" j2s-transpiled Java does not work.
        It's a starting point, but plenty of nuances in Java2Script require
        substantial changes to the Java, either on the Java side or on the
        JavaScript side. Looking back, some of this did result from bugs in the J2S
        runtime library, but much of it has to do with inefficiencies in JavaScript
        when you do that. And there are native library calls that must be created
        from scratch in JavaScript, for instance.

        (B) Other files do not fit the characteristics of (A), e.g.:

        • j2s/java/lang/Class.js
        • j2s/java/lang/ClassLoader.js

        These two are not used in JSmol.

        Do you know which files are used by JSmol?

        Hmm. Interesting question! These two are particularly known not to be used
        in JSmol because their function is largely replaced by the core J2S object,
        window.Clazz. Beyond that, if you look in the jsmoljs/JSmolJavaExt.js file
        you will find that several rewritten classes in java.lang, namely, Boolean,
        Byte, Short, Integer, Float, Double, Long, and Exception.

        If these aren't used, then I can simply drop them from the Debian package
        and then we don't need to argue about if it's "source code" or not.

        Generally I do not recommend micro-analyzing the package that way.

         

        • Bob Hanson

          Bob Hanson - 2016-11-08

          With respect to where the j2s/java sources are from, they can be found in
          https://github.com/zhourenjian/java2script.git under
          trunk/sources/net.sf.j2s.java.core . As you can see here:

          https://github.com/zhourenjian/java2script/tree/master/sources/net.sf.j2s.java.core/src/java/util

          in the comments, many of these are from Sun Java 1.4, some are from Apache
          Harmony 1.5.

          On Mon, Nov 7, 2016 at 7:05 PM, Bob Hanson hansonr@users.sf.net wrote:

          On Sun, Nov 6, 2016 at 6:49 AM, Ximin Luo infinity0x@users.sf.net wrote:

          Do you remember which bin/ directory - was it net.sf.j2s.java.core/bin?
          If we can identify this, then I could potentially reproduce this myself.

          It was the bin directory in my Eclipse project -- the Java files were
          added to the project, transpiled, and removed. The .js files were then
          edited to work in JavaScript.

          In fact, could you go into more detail on how j2s/java and srcjs were
          created in the first place?

          j2s/java: My understanding is that the standard Java files were place in a
          project, the j2s transpiler was run so as to create the JS equivalent of
          "class" files, and then those js files were included. It's possible that if
          you go to the j2s site or talk with Zhou Renjian, you may find your answer.

          srcjs: These should be "the source of the .js files" - that is, the
          originals used to generate those.

          As for the hand modifications, I guess they were motivated by bugs that you
          could see in JSmol. I'm wondering if it would be feasible (for me) to
          delete j2s/java and srcjs and recreate them by running the newest J2S on
          the newer Java 7/8/9 SDKs. Java are generally pretty good at keeping
          backwards compatibility, this has a decent chance of actually working.
          That certainly will not work. Oh, that it were that simple! I don't have
          time here to go into the reason "raw" j2s-transpiled Java does not work.
          It's a starting point, but plenty of nuances in Java2Script require
          substantial changes to the Java, either on the Java side or on the
          JavaScript side. Looking back, some of this did result from bugs in the J2S
          runtime library, but much of it has to do with inefficiencies in JavaScript
          when you do that. And there are native library calls that must be created
          from scratch in JavaScript, for instance.

          (B) Other files do not fit the characteristics of (A), e.g.:

          • j2s/java/lang/Class.js
          • j2s/java/lang/ClassLoader.js

          These two are not used in JSmol.

          Do you know which files are used by JSmol?

          Hmm. Interesting question! These two are particularly known not to be used
          in JSmol because their function is largely replaced by the core J2S object,
          window.Clazz. Beyond that, if you look in the jsmoljs/JSmolJavaExt.js file
          you will find that several rewritten classes in java.lang, namely, Boolean,
          Byte, Short, Integer, Float, Double, Long, and Exception.

          If these aren't used, then I can simply drop them from the Debian package
          and then we don't need to argue about if it's "source code" or not.

          Generally I do not recommend micro-analyzing the package that way.

          Status: open
          Group: v10
          Created: Thu Nov 03, 2016 07:39 PM UTC by Ximin Luo
          Last Updated: Thu Nov 03, 2016 07:39 PM UTC
          Owner: nobody

          Hi,

          (Filing this here because the JSmol project has "create ticket" disabled.)

          As mentioned earlier on #587 we'd like to know the original source of the
          files in JSmol/j2s/java/** and JSmol/srcjs.

          This is needed to pass Debian policy audits, and also it's generally good
          engineering practise that allows us to be sure that users have the software
          freedoms that FOSS software should have - here, specifically, to be able to
          study the code, and to make modifications to it themselves if they wish,
          and distribute those as well.

          The Java files look like they are taken from some version of a JDK. It's
          probably possible for me to argue this past the Debian review processes -
          but it would greatly help me do this, if you could tell me which versions
          of the JDK you took these files from. (You could document it in SVN for
          others to read as well; that would be ideal.)

          The JavaScript files look like they were auto-generated, and some of them
          look like they were manually hand-optimised later. By default, this does
          not count as "source code" by the common definition ("preferred form of
          modification"). For me to be able to argue this past the Debian policy
          reviewers, I need to know how they were originally generated.

          If you do actually hand-modify them as part of your normal development
          process, please document and describe this workflow (ideally in SVN) - it
          would help me to potentially argue that some of these files are "source
          code". However, I can see that this definitely won't be enough for all
          of those files.

          By default, we have to treat autogenerated code as "not source code" to be
          safe - e.g. we don't know if it breaks someone's copyright, or if it
          contains security backdoors, or whatever. I'm not accusing you of these
          things, it's just that we have to be strict in our auditing processes to
          make sure that we don't get screwed in the future.

          Now, onto the details of the specific files:

          (A) Some files are trivial/tiny or laid out relatively nicely, e.g.:

          • j2s/java/io/DataInput.js // one line of code
          • srcjs/java/util/AbstractList.js // potentially reviewable and
            modifiable by a human

          Files of this type could fit within a loose stretched definition of
          "source code". However, we'd still want to know how the original version
          was generated.

          (B) Other files do not fit the characteristics of (A), e.g.:

          • j2s/java/lang/Class.js
          • j2s/java/lang/ClassLoader.js

          There is no way we can reasonably argue that these files are "source
          code", and we definitely need to know how this was generated.

          (C) Some files seem to have been manually-edited after being automatically
          generated:

          $ git grep -l '// *BH' -- srcjs j2s/java
          srcjs/com/jcraft/jzlib/InflaterInputStream.java
          srcjs/java/io/BufferedInputStream.java
          srcjs/java/lang/Enum.js
          srcjs/java/util/AbstractList.js
          srcjs/java/util/ArrayList.js
          srcjs/java/util/Arrays.js
          srcjs/java/util/Collections.js
          srcjs/java/util/HashMap.js
          srcjs/java/util/Hashtable.js
          srcjs/java/util/zip/ZipOutputStream.java

          If these also come under (A) then it might be reasonable to argue that
          these are the "preferred form of modification".

          Finally, I'm sorry if this seems like stupid amounts of bureaucracy and
          paperwork - I don't like having to do this either. But it is necessary to
          be able to properly know that what we're using is actually free open source
          software. If we don't know this, and what we're using is only "maybe free
          software" then we're not really sticking to any principles and we might as
          well go use proprietary software instead.

          Sent from sourceforge.net because you indicated interest in
          https://sourceforge.net/p/jmol/bugs/589/

          To unsubscribe from further messages, please visit
          https://sourceforge.net/auth/subscriptions/

          --
          Robert M. Hanson
          Larson-Anderson Professor of Chemistry
          St. Olaf College
          Northfield, MN
          http://www.stolaf.edu/people/hansonr

          If nature does not answer first what we want,
          it is better to take what answer we get.

          -- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900

           

          Related

          Bugs: #589

          alternate

Log in to post a comment.

MongoDB Logo MongoDB