Menu

j2mod 2.0 - bye bye RxTx

Steve
2016-03-16
2016-03-28
  • Steve

    Steve - 2016-03-16

    We have a pretty desperate need to upgrade from the dreaded RxTx library and at the same time, move from jamod to j2mod. I've selected jSerialComm as my weapon of choice to replace RxTx.

    I'm in the process of porting j2mod to this library and at the same time I am upgrading everything to JDK 1.7.
    During this process, I've been addressing all the inconsistencies in synchronisation and the general oddness inherited from jamod. I've fixed a lot of the javadoc errors and I've also converted the library to maven.
    The process of moving to JDK1.7 means removing redundant code, using generics etc etc.

    I've been guided by FindBugs and the rather lovely IntelliJ inspectors and I think I'm at the point where I can start getting back to the day job of actually trying to use the library.
    If anyone else is interested and could lend a hand, particularly if you have a ready made test bed, I would be very grateful for the help.

    We're all git here but I'll sort out a branch on SVN at the point I've actually got it working and we can take it from there.
    Anyone interested/can help?

     
    • Josh Hansen

      Josh Hansen - 2016-03-17

      Hi Steve,

      On 3/16/2016 3:14 PM, Steve wrote:

      I'm in the process of porting j2mod to this library and at the same time I am upgrading everything to JDK 1.7.
      During this process, I've been addressing all the inconsistencies in synchronisation and the general oddness inherited from jamod. I've fixed a lot of the javadoc errors and I've also converted the library to maven.
      The process of moving to JDK1.7 means removing redundant code, using generics etc etc.

      I've been guided by FindBugs and the rather lovely IntelliJ inspectors and I think I'm at the point where I can start getting back to the day job of actually trying to use the library.
      If anyone else is interested and could lend a hand, particularly if you have a ready made test bed, I would be very grateful for the help.

      We're all git here but I'll sort out a branch on SVN at the point I've actually got it working and we can take it from there.
      Anyone interested/can help?

      We have some stuff that might be useful:
      JUnit tests for several functions
      SLF4J logging (we use logback)
      * Maven integration

      This was several years ago and included a lot of changes. Also, we're
      stuck on JDK 6, so generics are great but anything JDK 7 or newer is
      going to problematic for us.

      A question for the embedded users: will SL4J logging will be a problem?
      I suppose I can provide some builds and let people try them out, then
      we can figure out how (and what) to merge from there.

      I personally use Monotone, but I'll start converting what I have over to
      Git. I'll try to get something posted on GitHub in the next couple weeks.

      Josh

       
  • Julie Haugh

    Julie Haugh - 2016-03-17

    I'm 100% in support of what you're doing. I think the worst thing for me has been I don't hear about all the issues other people are having, so I'm not sure what needs to be changed.

    I had planned on moving j2mod to GitHub, because it seems to be where all the open source focus is these days. What I'd suggest is we stick with SourceForge for the time being and once j2mod 2.0 is fully baked, I'll put the code in a GitHub repo.

    How that sound?

    And of course, thank you so much for all of your hard work to make j2mod even more awesome.

     
    • Josh Hansen

      Josh Hansen - 2016-03-17

      Hi Julie,

      On 3/16/2016 7:40 PM, Julie Haugh wrote:

      I'm 100% in support of what you're doing. I think the worst thing for me has been I don't hear about all the issues other people are having, so I'm not sure what needs to be changed.

      I had planned on moving j2mod to GitHub, because it seems to be where all the open source focus is these days. What I'd suggest is we stick with SourceForge for the time being and once j2mod 2.0 is fully baked, I'll put the code in a GitHub repo.

      Nice to hear about moving j2mod to GitHub! Would it be possible to have
      "Move to GitHub" be a feature that is part of the "fully baked" 2.0?

      The main reason is that Git, being distributed, supports commits and
      branching without actually having commit access to your repo. In other
      words, I can branch, finish a feature, commit, branch, finish another
      feature, etc., and, if anyone else wants it, make it available for
      review, merge it back, etc. in bite-size pieces. Subversion is really
      painful in this regard; hijacks everywhere, and all work would go into a
      single commit assuming I even get commit permission. Yikes!

      BTW -- Thanks for all your work on j2mod!

      Josh

       
      • Josh Hansen

        Josh Hansen - 2016-03-17

        Hi Julie,

        Actually, having said all that, whatever gets 2.0 released is best. As
        long as the source is available, anyone who needs it can put it on GitHub.

        When do you think 2.0 will be published?

        Josh

        On 3/17/2016 4:48 PM, Josh Hansen wrote:

        Hi Julie,

        On 3/16/2016 7:40 PM, Julie Haugh wrote:

        I'm 100% in support of what you're doing. I think the worst thing for me has been I don't hear about all the issues other people are having, so I'm not sure what needs to be changed.

        I had planned on moving j2mod to GitHub, because it seems to be where all the open source focus is these days. What I'd suggest is we stick with SourceForge for the time being and once j2mod 2.0 is fully baked, I'll put the code in a GitHub repo.

        Nice to hear about moving j2mod to GitHub! Would it be possible to have
        "Move to GitHub" be a feature that is part of the "fully baked" 2.0?

        The main reason is that Git, being distributed, supports commits and
        branching without actually having commit access to your repo. In other
        words, I can branch, finish a feature, commit, branch, finish another
        feature, etc., and, if anyone else wants it, make it available for
        review, merge it back, etc. in bite-size pieces. Subversion is really
        painful in this regard; hijacks everywhere, and all work would go into a
        single commit assuming I even get commit permission. Yikes!

        BTW -- Thanks for all your work on j2mod!

        Josh


        j2mod 2.0 - bye bye RxTx


        Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/j2mod/discussion/general/

        To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

         
  • Steve

    Steve - 2016-03-17

    That sounds good to me, no problem working with SourceForge until I get a working version up and running. Could you add me to the probject and I'll create a branch to work on.

    Great work Julie, always difficult to keep the faith when there's only one person working on it so your enthusiasm is much appreciated

     
  • Julie Haugh

    Julie Haugh - 2016-03-18

    Steve -

    I've gone ahead and added you as a developer.

    Josh -

    You didn't mention if you were willing to work on this before it's move to GitHub. If you do, please let me know.

    One thing I've not mentioned yet is that once this up on GitHub there will be a C library to follow.

     
    • Josh Hansen

      Josh Hansen - 2016-03-18

      Hi Julie,

      On 3/17/2016 10:57 PM, Julie Haugh wrote:

      Josh -

      You didn't mention if you were willing to work on this before it's move to GitHub. If you do, please let me know.

      I'm up for working on it before the move to GitHub. If you are willing
      to add me as a developer.

      Can you also give me permission to create branches and tags, assuming
      you are ok with a "feature" branch process (create a branch, implement,
      then merge)?

      One thing I've not mentioned yet is that once this up on GitHub there will be a C library to follow.

      I don't think that will affect us. Which brings up a good point; our
      unit tests only test the TCP/IP communications, since that's all we've
      needed (so far).

      Josh

       
      • Julie Haugh

        Julie Haugh - 2016-03-19

        I've gone ahead and added you as a developer as well.

        I'm pretty familiar with Git and SVN workflows. I've been hoping to get other people involved with this project over the years and you two are a very welcome sight.

        As regards the unit tests only working with TCP/IP, I used to manufacturer Modbus slave devices that spoke Modbus/RTU, and I've done Modbus/RTU testing on Linux using pseudo-TTY devices. This might be an area where I can get some things done while Steve does the heavy-lifting of getting the new serial code in place.

        One of the things that hiding around here somewhere is an Arduino sketch I've tested up to 115.2KBaud. I don't know how much experience y'all have with Arduinos, but they are cheap and easy to find. My Modbus code tends to support as many function codes as possible, so testing can be very thorough in that regard.

         
  • Steve

    Steve - 2016-03-20

    Update on progress

    Hi Julie/Josh after a bit of thinking and given that I want to get some other people involved here at 4energy, I've taken the liberty of creating j2mod on GitHub.

    I've copied the code as is into the master branch and created a branch to work on for the v2 transition https://github.com/steveohara/j2mod/tree/version-2-migration

    I've made a very large number of changes, mostly driven by static code analysis but I've also added some features;

    • added the log4j framework,
    • moved all System.out calls to logger calls,
    • made all logger calls use constant strings for performance
    • removed redundant code,
    • re-formatted and re-ordered the code,
    • brought the source up to JDK1.6,
    • re-factored the directory structure to fit with Maven,
    • added a Maven build, setup the Maven build ready for submission to Maven Central,
    • reset all the code headers to use the LGPL declaration,
    • changed the license to LGPL
    • replaced RxTxComm with jSerialComm (untested)
    • added junit as a dependency

    There's still a lot of synchronisation problems being shown up in the static analysis but I will need to tackle these later along with the other warnings.
    At this point, I haven't made any structural changes so now is the time to get as many unit tests added as possible.
    Josh, can you jump in and add these or send me them and I'll add them in?
    Julie/Josh, can you get a GitHub account so that I can share the repo with you?

    I hope nobody thinks I'm being too presumptious, I'm very appreciative of all the work everyone has done on j2mod and I've continued with all the attributions from jamod. If anyone is unhappy, let me know.

     
    • Josh Hansen

      Josh Hansen - 2016-03-21

      Hi Steve,

      That all sounds great except for the license change to LGPL. Anything
      short of a complete re-write is a derivative work of the original
      copyright, and thus subject to it. A re-license will require permission
      from all contributors up to this point.
      https://en.wikipedia.org/wiki/Clean_room_design

      So, it needs to stay Apache 2.0, unless/until you can get permission
      from all contributors.

      Personally, I think LGPL is a nice "best of both worlds" license, so I
      don't have an objection to the license itself. I also haven't made any
      contributions yet, so you don't need permission from me. :)

      For Github, my account is 'jhansen2015'.

      Josh

      On 3/20/2016 4:53 PM, Steve wrote:

      Update on progress

      Hi Julie/Josh after a bit of thinking and given that I want to get some other people involved here at 4energy, I've taken the liberty of creating j2mod on GitHub.

      I've copied the code as is into the master branch and created a branch to work on for the v2 transition https://github.com/steveohara/j2mod/tree/version-2-migration

      I've made a very large number of changes, mostly driven by static code analysis but I've also added some features;

      • added the log4j framework,
      • moved all System.out calls to logger calls,
      • made all logger calls use constant strings for performance
      • removed redundant code,
      • re-formatted and re-ordered the code,
      • brought the source up to JDK1.6,
      • re-factored the directory structure to fit with Maven,
      • added a Maven build, setup the Maven build ready for submission to Maven Central,
      • reset all the code headers to use the LGPL declaration,
      • changed the license to LGPL
      • replaced RxTxComm with jSerialComm (untested)
      • added junit as a dependency

      There's still a lot of synchronisation problems being shown up in the static analysis but I will need to tackle these later along with the other warnings.
      At this point, I haven't made any structural changes so now is the time to get as many unit tests added as possible.
      Josh, can you jump in and add these or send me them and I'll add them in?
      Julie/Josh, can you get a GitHub account so that I can share the repo with you?

      I hope nobody thinks I'm being too presumptious, I'm very appreciative of all the work everyone has done on j2mod and I've continued with all the attributions from jamod. If anyone is unhappy, let me know.


      j2mod 2.0 - bye bye RxTx


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/j2mod/discussion/general/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       
    • Josh Hansen

      Josh Hansen - 2016-03-21

      Hi Steve,

      Regarding logging, do you have any objections to switching to SLF4J?
      It's a standardized API which provides bindings for various logging
      packages, including Log4J, Logback, JUL, and commons logging. We use
      Logback here.

      http://www.slf4j.org/
      http://www.slf4j.org/manual.html

      Besides the plugin logging library aspect, it supports "placeholders"
      which allows logging to be very fast by not requiring toString()
      invocations on objects which will never actually be logged. See the
      "Typical usage pattern" on the manual page linked above.

      Here is a link to Logback's home page.
      http://logback.qos.ch/

      Josh

      Joshua Hansen
      (541) 760-7685
      ZAPS Technologies, Inc.
      Detect. Respond.
      www.zapstechnologies.com

      This e-mail may contain confidential or privileged information. It is
      intended only for the use of the recipient named above. If you are not
      the intended recipient, any use or disclosure of this transmission is
      strictly prohibited. If you have received this in error, please
      immediately notify us and delete this e-mail.

      On 3/20/2016 4:53 PM, Steve wrote:

      Update on progress

      Hi Julie/Josh after a bit of thinking and given that I want to get some other people involved here at 4energy, I've taken the liberty of creating j2mod on GitHub.

      I've copied the code as is into the master branch and created a branch to work on for the v2 transition https://github.com/steveohara/j2mod/tree/version-2-migration

      I've made a very large number of changes, mostly driven by static code analysis but I've also added some features;

      • added the log4j framework,
      • moved all System.out calls to logger calls,
      • made all logger calls use constant strings for performance
      • removed redundant code,
      • re-formatted and re-ordered the code,
      • brought the source up to JDK1.6,
      • re-factored the directory structure to fit with Maven,
      • added a Maven build, setup the Maven build ready for submission to Maven Central,
      • reset all the code headers to use the LGPL declaration,
      • changed the license to LGPL
      • replaced RxTxComm with jSerialComm (untested)
      • added junit as a dependency

      There's still a lot of synchronisation problems being shown up in the static analysis but I will need to tackle these later along with the other warnings.
      At this point, I haven't made any structural changes so now is the time to get as many unit tests added as possible.
      Josh, can you jump in and add these or send me them and I'll add them in?
      Julie/Josh, can you get a GitHub account so that I can share the repo with you?

      I hope nobody thinks I'm being too presumptious, I'm very appreciative of all the work everyone has done on j2mod and I've continued with all the attributions from jamod. If anyone is unhappy, let me know.


      j2mod 2.0 - bye bye RxTx


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/j2mod/discussion/general/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       
    • Josh Hansen

      Josh Hansen - 2016-03-21

      Hi Steve,

      Nice to see the "JDK1.6" in there. Was it problematic to downgrade from
      your 1.7 changes?

      Josh

      On 3/20/2016 4:53 PM, Steve wrote:

      Update on progress

      Hi Julie/Josh after a bit of thinking and given that I want to get some other people involved here at 4energy, I've taken the liberty of creating j2mod on GitHub.

      I've copied the code as is into the master branch and created a branch to work on for the v2 transition https://github.com/steveohara/j2mod/tree/version-2-migration

      I've made a very large number of changes, mostly driven by static code analysis but I've also added some features;

      • added the log4j framework,
      • moved all System.out calls to logger calls,
      • made all logger calls use constant strings for performance
      • removed redundant code,
      • re-formatted and re-ordered the code,
      • brought the source up to JDK1.6,
      • re-factored the directory structure to fit with Maven,
      • added a Maven build, setup the Maven build ready for submission to Maven Central,
      • reset all the code headers to use the LGPL declaration,
      • changed the license to LGPL
      • replaced RxTxComm with jSerialComm (untested)
      • added junit as a dependency

      There's still a lot of synchronisation problems being shown up in the static analysis but I will need to tackle these later along with the other warnings.
      At this point, I haven't made any structural changes so now is the time to get as many unit tests added as possible.
      Josh, can you jump in and add these or send me them and I'll add them in?
      Julie/Josh, can you get a GitHub account so that I can share the repo with you?

      I hope nobody thinks I'm being too presumptious, I'm very appreciative of all the work everyone has done on j2mod and I've continued with all the attributions from jamod. If anyone is unhappy, let me know.


      j2mod 2.0 - bye bye RxTx


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/j2mod/discussion/general/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       
  • Julie Haugh

    Julie Haugh - 2016-03-21

    Steve -

    I'd created a personal GitHub account a few weeks back to do this. My ID is 'jfhaugh' and my email address is 'julie78787@gmail.com'.

    I'm very happy to have everyone interested in helping. I tried getting help a number of years ago, but my guess is that RxTxComm wasn't driving enough people batty. I'd hoped to make another attempt at getting people interested by publishing more Modbus and slave construction info, and then you came along.

    My only request, because I've been working on this for so many years, is that the project be transferred to "ghgande" (greenHouse Gas and Electric - my company), which is the organization I'd created to do this.

     
  • Steve

    Steve - 2016-03-21

    Hi Julie

    I'm happy to transfer the repo - you will need to get a pull request regime in place or allow work on unprotected branches so that we can get the library up to v2 asap.
    You will also need to get the ticket raised with Sonatype to get the Maven Central submission process rolling. Do you have a CI server or are you planning to do the releases manually?

    Transferring the repo is a simple process, submitting to Maven Central isn't quite as easy.

    For the time being and to get the whole thing up, perhaps it would be more expedient to keep it where it is and then transfer it to ghande when we get a working package? I don't need any attribution for 4energy beyond being a contributor, my only motivation is to get the library up and running and tested quickly.

     
  • Steve

    Steve - 2016-03-25

    Progress Update:

    You know how you someties start a job and think, "this won't take too long..."?
    This library is like opeing up a Pandoras box, nearly every single class in the library has had some sort of issue with it: synchronisation, thread-safety, poor/non-existent error handling blah blah.
    The UDP implementation nearly pushed me over the edge - that stuff was in a barely usable state and when I found the Facade pattern stuff I thought about committing hari kari!
    Julie, hats off to you, living with this codebase must have been a torture beyond words.

    Anyway, I've got to the point where the number of bugs reported by FindBugs is at least approachable and I've got 60 unit tests working so it is worth having a go with it locally.

    The changes in this version are as follows;

    • Serial connections now use JSerialComm (tested with RTU only)
    • BIN Serial transport has been dropped (proprietary and quite frankly, I'd had enough when I got to that class)
    • Logging uses a facade of log4j (sorry Josh) that uses placeholders as per String.format (thanks 4energy). When the dust has settled on a production release, I'll take a look at moving it to slf4j
    • junit tests added for all major function codes, TCP/UDP transports Slave and Master
    • There is a suite of unit tests that use the external modpoll utility to independently verify the TCP slave implementation - the executables are included in the test resources and loaded dynamically but it only works on Windows and Linux
    • The build is prepared for submission to Maven Central
    • A lot of javadoc updates to try and improve the quality of the documentation
    • Fixes: so many, that I briefly considered scrapping the library and starting again...
    • The library now properly functions as a Slave (threads are shutdown properly)
    • Generally better handling of failure conditions - needs much more work

    Tasks to do:

    • Get the library released to Maven Central
    • Add more performance related unit tests
    • Add unit tests that probe the more exotic function codes (Julie - this seems to be your area of expertise)
    • Add more error handling unit tests
     

    Last edit: Steve 2016-03-25
    • Julie Haugh

      Julie Haugh - 2016-03-25

      Steve -

      Yes! It was a very maddening experience! The code is also fairly brittle, making it very challenging to fix just one thing. Your effort is probably the most attention the code base has gotten in the past 2 years. It is very much appreciated.

      I have tried adding the more exotic functions to j2mod because my applications required functions which were never present. When I started looking at creating higher functionality sensing devices, as well as IoT parts, the need for the lesser used functions became greater.

      I have several very near-term pieces of work I have to wrap up before I can truly dive in and see what youv'e done. I also want to start moving towards uploading what I'm going to tentatively call "c2mod" and "go2mod" to GitHub.

       
    • Josh Hansen

      Josh Hansen - 2016-03-25

      Hi Steve,

      Nice work!

      Q. What's your take on the license? Will you change it back to Apache 2.0?

      Regarding SLF4J, I can do some of that. I think Log4j 2 now supports
      SLF4J as well.

      Regarding unit tests, I have a number of unit tests that are use a very
      simple network client I wrote just for testing. It should remove a
      dependency on modpoll.

      Need to know about the license though.

      Josh

      On 3/25/2016 6:51 AM, Steve wrote:

      Progress Update:

      You know how you someties start a job and thing, "this won't take too long..."?
      This library is like opeing up Pandorras box, nearly every single class in the library has had some sort of issue with it: synchronisation, thread-safety, no error handling blah blah.
      The UDP implementation nearly pushed me over the edge - that stuff was in a barely usable state and when I found the Facade pattern stuff I nearly committed hari kari!
      Julie, hats off to you, living with this codebase must have been a torture beyond words.

      Anyway, I've got to the point where the number of bugs reported by FindBugs is at least approachable and I've got 60 unit tests working so it is worth having a go with it locally.

      The changes in this version are as follows;

      • Serial connections now use JSerialComm (tested with RTU only)
      • BIN Serial transport has been dropped (proprietary and quite frankly, I'd had enough when I got to that class)
      • Logging uses a facade of log4j (sorry Josh) that uses placeholders as per String.format (thanks 4energy). When the dust has settled on a production release, I'll take a look at moving it to slf4j
      • junit tests added for all major
      • There is a suite of unit tests that use the external modpoll utility to independently verify the TCP slave implementation - this only works on Windows and Linux
      • The build is prepared for submission to Maven Central
      • A lot of javadoc updates to try and improve the quality of the documentation
      • Fixes: so many, that I briefly considered scrapping the library and starting again...
      • The library now properly functions as a Slave (threads are shutdown properly)
      • Generally better handling of failure conditions - needs much more work

      Tasks to do:
      Get the library released to Maven Central
      Add more performance related unit tests
      Add unit tests that probe the more exotic function codes (Julie - this seems to be your area of expertise)
      Add more error handling unit tests


      j2mod 2.0 - bye bye RxTx


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/j2mod/discussion/general/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       
  • Julie Haugh

    Julie Haugh - 2016-03-26

    Steve -

    We need to get the ALv2 versus LGPL issue resolved. The Free Software Foundation accepts the Apache License as a free software license, but the Apache Software Foundation has issues with the GPL. Ignoring the legalities of changing a license without the consent of all the contributors, I think this pretty much means we need to stick with the Apache license.

     
  • Steve

    Steve - 2016-03-26

    I have reverted the license back to Apache 2.0

    Modpoll is neccesary as a way of inependently testing the j2mod TCP Slave. Once verified, the j2mod Slave is used in all other tests locally so no outside resources are needed. The test suite degrades properly if you are not running the tests on a platform that supports modpoll.

    The library is in the master branch now, ready for release to Maven Central once I get their permission. Please check it out and take a look, run the unit tests etc.
    You can get a copy of the RC1 jar here for the time being: https://www.dropbox.com/s/yt3f020qe6xcaov/j2mod-2.0-rc1.zip?dl=0

    Josh: What is your GitHub username so that I can add you as a collaborator?

     
    • Josh Hansen

      Josh Hansen - 2016-03-28

      Hi Steve,

      Great to hear about the Apache 2.0 license.

      My Github username is jhansen2015. I'm already a watcher on the project.

      Josh

      On 3/26/2016 6:33 AM, Steve wrote:

      I have reverted the license back to Apache 2.0

      Modpoll is neccesary as a way of inependently testing the j2mod TCP Slave. Once verified, the j2mod Slave is used in all other tests locally so no outside resources are needed. The test suite degrades properly if you are not running the tests on a platform that supports modpoll.

      The library is in the master branch now, ready for release to Maven Central once I get their permission. Please check it out and take a look, run the unit tests etc.
      You can get a copy of the RC1 jar here for the time being: https://www.dropbox.com/s/yt3f020qe6xcaov/j2mod-2.0-rc1.zip?dl=0

      Josh: What is your GitHub username so that I can add you as a collaborator?


      j2mod 2.0 - bye bye RxTx


      Sent from sourceforge.net because you indicated interest in https://sourceforge.net/p/j2mod/discussion/general/

      To unsubscribe from further messages, please visit https://sourceforge.net/auth/subscriptions/

       
  • Julie Haugh

    Julie Haugh - 2016-03-26

    Steve -

    Thanks. We should be able to remove the modpoll dependency once c2mod has been packaged and validated.

     

Log in to post a comment.

Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.