Menu

STAF/STAX Events and Handlers (sorry, noob Q)

2005-11-04
2013-06-12
  • Nelson Kidd

    Nelson Kidd - 2005-11-04

    Hi:

    I have a C++ service that is succesfully using the EVENT service to report its current state. Is there any way for a STAX job to "react" to these events?

    I apologize if the question is documented somplace. I've spent the better part of a day experimenting and reading the staxug and stafug files, with little progress. So far, I've only made limited progress using the command-line. (i.e. <i>staf local event list eventids</i> and <i>staf local queue get wait</i>.) I've tried using <i>staf local queue peek<i> and other variants, but I keep getting RC=29.

     
    • Charles Rankin

      Charles Rankin - 2005-11-04

      There's no good way for a STAX job to react to these events without an intermediary.  The main issue is that the STAX job uses it's STAF Queue for lots of things, and having the job try to get data off the queue interferes with this processing.  We have a feature to provide access to the STAF queue from a STAX job, but it's not at the top of the list at the moment.

      So, given that, there are a couple of things you could try, all involving some form of intermediary.  You could have the intermediary listen for events and post event semaphores, which the job could be waiting on.  Alternatively, the job could have some "held" blocks which the intermediary could "release" when certain events come in.

      You could have the job use the EventManager as the intermediary, or you could just write a small program to act as the intermediary.

      Techincally, there is another way.  It is possible to access Java facilities from within a STAX <script> element.  Using this capability, you can create a separate (Java) STAFHandle object, and have it register for the events and wait on them in a <script> element.  This will work, but you need to be careful with it, as it can deadlock STAX if you aren't careful.  Anytime one of your jobs is in a <script> waiting on that STAFHandle to get data off the queue, one of the STAX Threads is being used.  If you have too many jobs/threads waiting on STAF queues, STAX will run out of STAX Threads.  You can largely get around this by not doing an infinate wait on the QUEUE GET command, instead provide a timeout, like QUEUE GET WAIT 1000 (which would wait for a message for one second).  This causes a polling loop, which is generally not a good thing, but it will prevent any deadlocking in STAX.

       
      • Nelson Kidd

        Nelson Kidd - 2005-11-14

        Thank you for the response.

        1] Is QUEUE GET reliable? From a command-line environment, I'm only receiving event messages if the message arrives when the QUEUE GET command is in a wait state. In a networked setup (machine-a and machine-b), if machine-a generates events (with the same type/subtype but different propertyMap field) in a quick manner, then machine-b will only report the "last message in the sequence" when blocked during QUEUE GET WAIT. Am I doing something incorrectly? What's the best way of formulating my events?

        2] Are there any samples of using those types of intermediaries? I grasp the concept of what you're saying, but I'm having trouble piecing things together into something that works. Ideally, the solution will allow me to parse out the data in the propertyMap field. However, if using propertyMaps is a bad idea, then I can probably change things so that all relevant information is encapsulated in the TYPE and SUBTYPE.

        3] If I use the EventManager as an intermediary, is it simply a matter of using EVENTMANAGER REGISTER to create a binding between an TYPE+SUBTYPE and another STAF request? Am I also correctly assuming that using this method does not allow me to parse out the propertyMap fields within an event message?

        4] The "small program" that you described - is that simply a custom STAF service, or is it something else?

        5] Is it normal for STAF to induce a delay of about 2 seconds over the network?

         
      • Nelson Kidd

        Nelson Kidd - 2006-01-31

        Do you have a snippet for using a Java STAFHandle in a <script> element, or point me to a doc that describes the concept? I searched staxug, and I only saw examples for using STAFUtil.wrapData. I specifically don't understand the following:

        [1] What do I use for the import line? Is it "import com.ibm.staf.* from STAFHandle"?

        [2] How do I instantiate the Java STAFHandle in <script> Specifically..

        [2a] What is the syntax for calling a Java constructor in <script>?

        [2b] When instantiating the STAFHandle, should I use the constructor that takes a staticHandleNumber, or is it sufficient to create the handle with a string since I'll be using the same STAFHandle for both "EVENT REGISTER" and "QUEUE GET" calls.

        [3] Assuming I need to use the Java STAFHandle to call "EVENT REGISTER", should I use BYNAME or BYHANDLE?... or is this simply a matter of using a value that corresponds to what was used in the constructor?

        [4] Am I correct in assuming that the return value from STAFHandle.submit2() will result in a dictionary within the <script> element?

        [5] Am I correct that TYPE and SUBTYPE args in "EVENT REGISTER" correspond to the same args in "EVENT GENERATE" and those have no relationship to the TYPE AND SUBTYPE args in "QUEUE GET"?

        [6] When I use the STAFHandle to call "QUEUE GET", is it sufficient to use "TYPE STAF/Service/Event", or do I need different arguments to ensure that "QUEUE GET" triggers when other things call "EVENT GENERATE"? I have several STAF services on different machines that will submit a "GENERATE EVENT" request to the machine running STAX.

        [7] Is the lifetime of the Java STAFHandle confined to the <script> element that declares it, or is the lifetime good for the duration of the <function> element? e.g. Do I need to do all of my event handling within a single <script> element, or can I use a series of <script> and other STAX elements in a single STAX function?

         
    • Sharon Lucas

      Sharon Lucas - 2005-11-15

      1) Yes, the QUEUE GET request is very reliable.  Many of our services use it.  For example:  PROCESS service to get process completion messages, STAF DemoController, STAX service, STAX Monitor, Cron service, EventManager service. etc.  You can look at the source for any of th

      I'm not really sure what you're asking.  The QUEUE service's GET request allows you to retrieve and remove a single element from the queue.  So, you may want to have a loop that continuously submits a GET WAIT request to the QUEUE service.

      2) Here's are a couple of snippets from the staf/services/eventmanager/EventManagerService.java program that uses the QUEUE service's GET WAIT request to retrieve messages from the event service on its queue and to access data like the event properties:

          public void run()
          {
              STAFResult queueGetResult;

              for (;;)
              {
                  queueGetResult = fHandle.submit2("local", "QUEUE", "GET WAIT");
                     
                  if (queueGetResult.rc != 0)
                       continue;  // Skip

                  try
                  {
                      STAFQueueMessage queueMessage = new STAFQueueMessage(
                          queueGetResult.result);

                      String queueType = queueMessage.type;

                      if (queueType.equalsIgnoreCase("STAF/Service/Event"))
                      {
                          if (!(queueMessage.message instanceof Map))
                          {
                              // Ignore this message
                              continue;
                          }

                          // Provide both the unmarshalled message and the
                          // marshalled message string because it needs to
                          // unmarshall it using Python
                          handleEventServiceMsg(queueMessage, queueGetResult.result);
                      }
                   }
                   ...
                  catch (Exception e)
                  {
                      // Log an error and continue processing messages on the queue
                  }
              }
          }

          private void handleEventServiceMsg(STAFQueueMessage queueMessage,          String marshalledQueuedMsg)
          {
              // A STAF/Service/Event message is a map containing keys:
              // eventServiceName, eventID, machine, handleName, handle, timestamp,
              // type, subtype, and propertyMap (whose values could contain
              // marshalled data).

              // Assign the data in the message to Java variables

              Map messageMap = (Map)queueMessage.message;

              String eventService = (String)messageMap.get("eventServiceName");
              String eventID = (String)messageMap.get("eventID");
              String generatingMachine = (String)messageMap.get("machine");
              String generatingProcess = (String)messageMap.get("handleName");
              String generatingHandle = (String)messageMap.get("handle");
              String eventTimestamp = (String)messageMap.get("timestamp");
              String eventType = (String)messageMap.get("type");
              String eventSubtype = (String)messageMap.get("subtype");
              Map eventProperties = (Map)messageMap.get("propertyMap");

              String eventInfoMsg = "Event service message information:" +
                  "\n  eventservice: " + eventService +
                  "\n  eventid: " + eventID +
                  "\n  type: " + eventType +
                  "\n  subtype: " + eventSubtype +
                  "\n  generatingmachine: " + generatingMachine +
                  "\n  generatingprocess: " + generatingProcess +
                  "\n  generatinghandle: " + generatingHandle +
                  "\n  eventtimestamp: " + eventTimestamp +
                  "\n  properties: " + eventProperties;
            
              // Process all EventManager registrations whose type/subtype
              // match that in the event message
              ...
          }

      3) Yes.  The EventManager service does make the event properties available to you.  Please see the EventManager Service User's Guide, REGISTER section, sub-section "Event Generation".  It says:

      When an event is generated with a matching type/subtype, the following Python variables will be available when the PREPARE, PYTHONMACHINE, PYTHONSERVICE, and PYTHONREQUEST options are evaluated as python strings:

      eventservice
      eventid
      generatingmachine
      generatingprocess
      generatinghandle
      eventtimestamp
      eventtype
      eventsubtype

      In addition, each PROPERTY option name=value pair for the generated event will be set as Python variables.

      Also, a Python dictionary named "eventinfo" will contain all of the above name/value pairs.

      Note, the EventManager service provides a GUI to simplify the EventManager Registration.  I highly recommend that you use this GUI to create and submit your REGISTER request to the EventManager service since you'll probably be using the PREPARE option with Python code to access the event properties.  Again, see the EventManager Service User's Guide, section "Registration GUI" for more info.

      4) It could be, or it could be a Java program etc.  I think you'll probably just want to use the EventManager service instead as an intermediary.

      5) It depends on the network, the request, etc.  I answered this in more detail under your posting titled "Overhead latency for networked STAF requests" on the STAF Help forum.

       
    • Nelson Kidd

      Nelson Kidd - 2005-11-15

      Thank you very much for your help on this. I'll work on digesting all of this. I'll try not to ask more rudimentary questions.

       
    • Nelson Kidd

      Nelson Kidd - 2006-01-31

      Ugh - I misinterpreted the "HTML tags" warning. Here's the same msg that's properly XML-escaped.

      Do you have a snippet for using a Java STAFHandle in a &lt;script&gt; element, or point me to a doc that describes the concept? I searched staxug, and I only saw examples for using STAFUtil.wrapData. I specifically don't understand the following:

      [1] What do I use for the import line? Is it &quot;import com.ibm.staf.* from STAFHandle&quot;?

      [2] How do I instantiate the Java STAFHandle in &lt;script&gt; Specifically..

      [2a] What is the syntax for calling a Java constructor in &lt;script&gt;?

      [2b] When instantiating the STAFHandle, should I use the constructor that takes a staticHandleNumber, or is it sufficient to create the handle with a string since I'll be using the same STAFHandle for both &quot;EVENT REGISTER&quot; and &quot;QUEUE GET&quot; calls.

      [3] Assuming I need to use the Java STAFHandle to call &quot;EVENT REGISTER&quot;, should I use BYNAME or BYHANDLE?... or is this simply a matter of using a value that corresponds to what was used in the constructor?

      [4] Am I correct in assuming that the return value from STAFHandle.submit2() will result in a dictionary within the &lt;script&gt; element?

      [5] Am I correct that TYPE and SUBTYPE args in &quot;EVENT REGISTER&quot; correspond to the same args in &quot;EVENT GENERATE&quot; and those have no relationship to the TYPE AND SUBTYPE args in &quot;QUEUE GET&quot;?

      [6] When I use the STAFHandle to call &quot;QUEUE GET&quot;, is it sufficient to use &quot;TYPE STAF/Service/Event&quot;, or do I need different arguments to ensure that &quot;QUEUE GET&quot; triggers when other things call &quot;EVENT GENERATE&quot;? I have several STAF services on different machines that will submit a &quot;GENERATE EVENT&quot; request to the machine running STAX.

      [7] Is the lifetime of the Java STAFHandle confined to the &lt;script&gt; element that declares it, or is the lifetime good for the duration of the &lt;function&gt; element? e.g. Do I need to do all of my event handling within a single &lt;script&gt; element, or can I use a series of &lt;script&gt; and other STAX elements in a single STAX function?

       
    • David Michael Bender

      Here is a sample STAX job that demonstrates how to call into the Java STAF APIs:

      <?xml version="1.0" encoding="UTF-8" standalone="no"?>
      <!DOCTYPE stax SYSTEM "stax.dtd">

      <stax>

        <defaultcall function="main"/>

        <function name="main">

          <sequence>

            <script>
              from com.ibm.staf import STAFHandle
             
              try:
                handle = STAFHandle("MyTest")
              except STAFException, e:
                print "Error registering with STAF, RC: %d" % e.rc
                sys.exit(e.rc)

              result = handle.submit2("local", "ping", "ping")
              print "RC=%s, Result=%s" % (result.rc, result.result)
            </script>
           
            <script>
              result = handle.submit2("local", "var", "resolve string {STAF/Config/OS/Name}")
              print "RC=%s, Result=%s" % (result.rc, result.result)
            </script>

          </sequence>

        </function>

      </stax>

      Sample output from running this STAX job is:

      RC=0, Result=PONG
      RC=0, Result=WinXP

      (note that the print statement output will be in the JVM log, C:/STAF/data/STAF/lang/java/jvm/STAFJVM1/JVMLog.1 if you used the default setting).

      [1] from com.ibm.staf import STAFHandle

      [2] handle = STAFHandle("MyTest")

      [2a] It is essentially the same as in Java, but you omit the class in the object definition (so, instead of STAFHandle handle = STAFHandle("MyTest") you have the syntax shown in [2].

      [2b] It is sufficient to create the handle with a string as shown abouve.

      [3] You should be able to use BYNAME (but since that is the default, you don't need to specify either option).

      [4] The return value from STAFHandle.submit2() will be a STAFResult object, containing the "rc" and "result" variables.  The sample STAX job above shows how to access them.

      [5] Yes, the TYPE and SUBTYPE options in "EVENT REGISTER" correspond to the same options in "EVENT GENERATE".  The TYPE option for QUEUE GET has no relationship to the TYPE option for EVENT REGISTER/GENERATE.  There is no SUBTYPE option for QUEUE GET.

      [6] It is sufficient to use "TYPE STAF/Service/Event" when calling call "QUEUE GET".  

      [7] As you can see in the sample STAX job above, you can use a series of <script> and other STAX elements in a single STAX function.  In addition, if the scope of your STAX function is defined as global (not local) then you can use the STAFHandle in multiple STAX functions.

      You can get more information about the Java STAF APIs in the STAF Java User's Guide (http://staf.sourceforge.net/current/STAFJava.htm).

       
      • Nelson Kidd

        Nelson Kidd - 2006-02-01

        Is there an easy way to convert the marshalled data to a Python dictionary within the 'script' element?

        The brute force method seems to involve calling STAFMarshallingContext.unmarshall and then writing code that is specific to the format of the EVENT message.

        I'm not opposed to searching some Java source and copying it into the 'script', but I suspect that there might be an easier way since STAX does it automagically.

         
        • Sharon Lucas

          Sharon Lucas - 2006-02-01

          Use the Python unmarshall method (instead of the Java unmarshall method) to convert a string containing marshalled data into a Python object.  For example:

          <script>
            STAFMarshalling.unmarshall(myMarshalledData)
          </script>

          See the "STAX Python Interfaces" section of the STAX V3.1.1 User's Guide for more information.

           
    • Nelson Kidd

      Nelson Kidd - 2006-01-31

      As always, your assistance is very much appreciated. Thank you!

       
    • Nelson Kidd

      Nelson Kidd - 2006-02-02

      In a scenario involving getting EVENT messages from a QUEUE, should I recursively call STAFMarshalling.unmarshall() or should I specify a marshalling context in my call to STAFMarshalling.unmarshall()?

      Currently, I do something like the following and I run into an error. I suspect that when the embedded stuff gets unmarshalled, it reduces it to a string.

      retdata = STAFMarshalling.unmarshall(result.result)
      event_msg = retadata['Message'] # this line causes an error

       
      • Sharon Lucas

        Sharon Lucas - 2006-02-02

        I think you have a typo.  There is no key named 'Message'.  It's 'message'.

        event_msg = retadata['message']

        If this isn't the problem, please provide more information.
        - What is the exact error you get
        - What does result.result contain?

         
    • Nelson Kidd

      Nelson Kidd - 2006-02-02

      It turns out the problem is both with that particular typo and the fact I wasn't treating the return type as a marshallingcontext. The code below seems to work in a manner that I need.

      mc = STAFMarshalling.unmarshall(result.result)
      retdata = mc.getRootObject()
      event_msg = retdata['message']

       

Log in to post a comment.