Learn how easy it is to sync an existing GitHub or Google Code repo to a SourceForge project! See Demo

Close

stafproc ending abnormally after using STAX

Help
jon lueng
2013-04-24
2013-06-12
  • jon lueng
    jon lueng
    2013-04-24

    When I found this issue is after config the UTF-8 codepage(stafcodepageoverride=utf-8). if I use STAX service or STAFEvent service, stafproc can not be shutdown normally. Its message:
    "STAFProc ending normally
    STAFProc ending abnomally: terminate() called"

    if I dont use STAX, it's ok. and if I use STAX but dont change the STAF codepage, it's also OK.
    my staf.cfg is shown below:
    # Turn on tracing of internal errors and deprecated options
    trace enable tracepoints "error deprecated"

    # Enable TCP/IP connections
    interface ssl library STAFTCP option Secure=Yes option Port=6550
    interface tcp library STAFTCP option Secure=No  option Port=6500

    # Set default local trust
    trust machine local://local level 5

    # Add default service loader
    serviceloader library STAFDSLS

    SERVICE STAX LIBRARY JSTAF EXECUTE {STAF/Config/STAFRoot}/services/stax/STAX.jar OPTION J2=-Xmx384m
    SERVICE EVENT LIBRARY JSTAF EXECUTE {STAF/Config/STAFRoot}/services/stax/STAFEvent.jar
    SERVICE EMAIL LIBRARY JSTAF EXECUTE {STAF/Config/STAFRoot}/services/email/STAFEmail.jar PARMS MAILSERVER 192.168.1.27

    #trust drun server
    TRUST LEVEL 5 MACHINE YBOXRUNNER-PC30
    TRUST LEVEL 5 MACHINE 10.10.87.151
    TRUST LEVEL 5 MACHINE DBASIC-VM225
    TRUST LEVEL 5 MACHINE 10.14.39.10

    #enable log trace
    trace SET DESTINATION TO FILE {STAF/Config/STAFRoot}/staf_debug.log

    Could anyone help me to resolve this? Thanks.

     
  • Sharon Lucas
    Sharon Lucas
    2013-04-24

    This is issue is probably due to a STAX job running a STAF service request (probably a PROCESS START request via the <process> element) that is returns a result that does not contain data that can be converted to the UTF-8 codepage.  I'm not sure why you are overriding the STAF codepage to UTF-8 in the first place.

    What version of STAF are you using on your STAX service machine and what operating system?  Please provide the output of:   STAF local MISC LIST PROPERTIES

    If you are not running the latest STAF version, please upgrade to the latest version, 3.4.13, to see if this fixes the problem. This is especially important if you are running STAF for Windows 64-bit, Version 3.4.12 or earlier as 3.4.13 contains the following fix (as documented in the STAF History at http://staf.sourceforge.net/history.php):


    Version 3.4.12.1: 01/17/2013

      - Fixed a problem on Windows 64-bit systems where STAFProc.exe (or STAF.exe) could crash due to a codepage version issue when non-English characters are used in a STAF service request or returned in a result.  Updated the STAF V3.4.12 download for Windows AMD64 (aka x64) and Windows IA64 to contain this fix (Bugs #2076450, #3490877, #3046761, #1728621)

     
  • jon lueng
    jon lueng
    2013-04-25

    I used the newest STAF(3.4.13) and stax(3.5.5) on win7.

    version     : 3.4.13
    platform    : win32
    architecture: 32-bit
    installer   : IA
    file        : STAF3413-setup-win32.exe
    osname      : Windows
    osversion   : *
    osarch      : x86

    I dont run a stax job. I just put the stax files in c:\staf\services, then start and shutdown stafproc, the problem occurs.

    I'm overriding the STAF codepage to UTF-8 because I want to write stax xml in utf-8 codec, so i dont need to write the terrible unicode(\uxxxx) for Non-Ascii characters.

    Thanks in advance.

     
  • Sharon Lucas
    Sharon Lucas
    2013-04-25

    On a Windows 32-bit system using STAF V3.4.13 and STAX V3.5.5, I don't have any problems starting STAFProc with STAX and the Event services registered in the STAF.cfg file when I have set STAFCODEPAGEOVERRIDE=UTF-8.  I am also able to run STAX jobs.  I can also shutdown STAF using "STAF local SHUTDOWN SHUTDOWN" with no errors shown in the STAFProc output.

    Do you have any errors logged in your STAX JVM log?  The STAX JVM log file is C:\STAF\data\STAF\lang\java\jvm\STAFJVM1\JVMLog.1 if STAF was installed to C:\STAF.

    Do you have any issues running STAX jobs?

    Is the only problem you see the "STAFProc ending abnomally: terminate() called" message when submitting a "STAF local SHUTDOWN SHUTDOWN" request (which I don't believe that itself prevents you from doing anything).

    Note that I don't think setting the STAF codepage to UTF-8 will allow you to write a STAX xml file using non-ASCII characters.

     
  • jon lueng
    jon lueng
    2013-04-26

    I dont know why you cannot reproduce this issue. it occurs in every machines of mine. Is it possible that I use Non-English os? And I viewed the JVM log, there is no error, and i also upgrade to jre 1.7 to have a trial, but it keeps the same:
    ******************************************************************************
    *** 20130425-14:17:12 - Start of Log for JVMName: STAFJVM1
    *** JVM Executable: java
    *** JVM Options   : none
    *** JVM Version   : java version "1.6.0_29"
    Java(TM) SE Runtime Environment (build 1.6.0_29-b11)
    Java HotSpot(TM) Client VM (build 20.4-b02, mixed mode, sharing)
    *** JVM PID       : 4572
    ******************************************************************************
    *sys-package-mgr*: processing modified jar, 'C:\STAF\services\stax\STAX.jar'

    Registered Extensions for STAX Version 3.5.5:

    ^C
    ******************************************************************************
    *** 20130426-10:23:44 - Start of Log for JVMName: STAFJVM1
    *** JVM Executable: java
    *** JVM Options   : none
    *** JVM Version   : java version "1.7.0_21"
    Java(TM) SE Runtime Environment (build 1.7.0_21-b11)
    Java HotSpot(TM) Client VM (build 23.21-b01, mixed mode, sharing)
    *** JVM PID       : 4284
    ******************************************************************************
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\resources.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\rt.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\jsse.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\jce.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\charsets.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\jfr.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\access-bridge.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\dnsns.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\jaccess.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\localedata.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\sunec.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\sunjce_provider.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\sunmscapi.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\sunpkcs11.jar'
    *sys-package-mgr*: processing new jar, 'C:\Program Files\Java\jre7\lib\ext\zipfs.jar'

    Registered Extensions for STAX Version 3.5.5:

    And you're right, actually the problem annoy me isn't this. After I changed codepage to utf-8, i could write non-ASCII characters in xml indeed, by making the xml's encoding be utf-8, too. And the job can be submitted normally. But for some certain non-ASCII characters, it has problem. for example, I use <process> to make the stax machine run a python script as follows:
    <process>
      <location>'local'</location>
      <command>"python"</command>
      <parms>'qta_server_driver.py get_latest_package_path -p %s -v %s'%(product_name,main_version)</parms>
      <returnstdout />
      <returnstderr />
    </process>

    i catch the value of "product_name" and "version" in the python script for doing something. It works well in most condition, but for some non-ASCII characters, it will get a corrupted value. for example, for the character "盾", its utf-8 is 'xe7\x9b\xbe', but what i get in python script will be 'xe7\x9b' - it miss the last byte.

    Actually this problem has make me upset for a few months, I asked the shutdown problem is also because I suspect these two issues have a connection.

     
  • Sharon Lucas
    Sharon Lucas
    2013-04-26

    Yes, you're probably encountering this issue because you're using a Non-English operating system and there is non-English data in a request/result that cannot be converted to the UTF-8 codepage.  This is why you really don't want to override the STAF codepage and use UTF-8.

    It would be great if there was a better way other than using unicode('\uxxxx) to specify non-ASCII characaters in a stax xml file.  We have an open feature request, # 750309 "Codepage Conversion support for STAX files" at https://sourceforge.net/tracker/?func=detail&aid=750309&group_id=33142&atid=407384, for this.

     
  • jon lueng
    jon lueng
    2013-05-17

    Hi sharon,

      I have tried using unicode like u'\uxxxx', but it doesn't resolve the problem, so I don't think it is because I specify non-ASCII characters directly, but IS AN ISSUE WHEN USING UTF-8 CODEPAGE IN NON-ENGLISH OS. If I don't change the codepage, the problem won't occur.

      Why I have to change to utf-8 is because our python script is utf-8 codec, and We call it from STAX using <process> and send serverl strings in just like command line.(then resolve it in python script by "getopt" module) If i change STAF codepage to utf-8, I can use these strings directly, without decoding from my local codec and encoding to UTF-8.

     
  • jon lueng
    jon lueng
    2013-05-17

    Corrected: "I can use these strings directly IN THE PYTHON SCRIPT, without decoding from my local codec and encoding to UTF-8."

    to make my clarification have no confusion. :-)