I m having an issue unregistering a handle via a perl program. The program has two handles, numbers 162 and 163 in the example below. Handle number 162 has as STAFLog object constructed with this handle. While handle 163 was used to run a specific program (via staf PROCESS service)
Handle 163 unregisters without an issue, but when unregistering handle 162 a return code from unRegister() is an rc = 5. The below example shows the handles that exist AFTER deleting handle 163, and prior to deleting handle 162
I validated that the correct handles objects were being used to unregister. Can you explain the anomoly here (aka: why handle 162 is not being unregistered/deleted and an rc = 5 is being returned by unRegister()).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We are using Active Perl 5.8, with the STAF 5.8 Perl object. Below is the code logic, while a test run with stdout is at the end of this message.
my $log = $hash->{samLog};
my $ref = $hash->{jobList};
#
# Release all the staf handles
#
# Note: STAF handles are persistent after program exit on Linux,
# but not the case not on Win32 (handles are deleted upon exit)
#
We will investigate this further (I was able to recreate using a test Perl file).
Is there a reason that you need multiple STAF handles in your Perl program? Typically an application that registers with STAF would only register one time and use that handle until the application completes (at which time it unregisters its single handle).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
The reason that multiple handles are being used is because multiple jobs are being started, and are started with "notify onend", such that the result is put on the queue. If the same handle is being used to start the jobs, then there could be multiple entries on the queue, and no way (as far as I am aware) to distiquish which queue entry is for which process that was started. If there is another way, please inform me of the technique.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I'll certainly install and use the next version of STAF when it becomes available. In the mean time, I'm still interested in learning how I should have utilized handles. Please see the note below, that is also my last message thread:
The reason that multiple handles are being used is because multiple jobs are being started, and are started with "notify onend", such that the result is put on the queue. If the same handle is being used to start the jobs, then there could be multiple entries on the queue, and no way (as far as I am aware) to distiquish which queue entry is for which process that was started. If there is another way, please inform me of the technique.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
When you start a process asynchronously, you will get back a handle number. This combined with the target system identifier uniquely identifies this handle/process across all STAF systems.
When STAF on the target system detects that the process has completed, it will queue you a message if you specified NOTIFY ONEND. It seems that you are already doing this part. Now, part of this message will contain the system identifier (from the system sending the message, which is the system that you started the process on) and another part will contain the handle number of the process. Thus, with these two pieces of data, you can determine which specific process has just completed.
Section 8.12.3 of the STAF User's Guide describes the basic structure of a message delivered to a queue. It is
The <Machine> is what you need to identify the system that you started the process on.
The <Message> for a process end notification is documented in section 8.11.6 of the STAF User's Guide. It is (note, the current User's Guide fails to document the KEY string)
The <Handle> is the handle that you got back from the PROCESS START request.
Note, if you have trouble matching up system identifiers (perhaps because you are using an IP address, while STAF sends you a host name), then you can provide a KEY on the PROCESS START which will be sent to you as part of the process end notification. With this, you can generate your own unique keys to identify each process, and you don't have to do the system/handle matching.
With the above, you can easily use a single handle to handle processes getting started on any number of systems. Post again if you have more questions.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Thank you for the last bit of information regarding how the queue message is built.
The next question is how to delete a specific message on the Queue.
Ex:
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
If I want to delete the 2nd message - or even all the messages associated with this handle (again via the Perl API). The delete returns OK, but he message remains on the Queue:
$ staf local queue delete handle 169
Response
--------
0
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
I also attempted to delete via the start handle:
$ staf local queue delete handle 172
Response
--------
0
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
Can you please explain the descrepency.
Thanks!
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Every time you run the STAF executable (when you run "staf local ....") that instance of the STAF client is going to be assigned a new handle number, and will have an associated queue. So, when you run "staf local queue ...." you are actually dealing with the handle that has been created for that STAF client and its associated queue.
The HANDLE option for the Queue service is used differently for LIST and DELETE commands:
--------------------------------------------------------
LIST [HANDLE <Handle>]
HANDLE specifies the handle of the process for which you want the queue contents. The default is the handle of the submitting process.
HANDLE specifies that you want to delete messages originating from a process with the given handle.
--------------------------------------------------------
Note that the HANDLE specified for the DELETE command is used to specify a handle that *originated* the queued message(s), but you are only deleting messages in the queue for the current STAF client (and that's why the command is returning 0, since the STAF client's queue did not contain any messages).
So, in this case, the QUEUE DELETE needs to be requested from the program that registered with STAF and was assigned handle 169.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Still having the delete message issue. The following shows all the handles that are open, with handle 286 being the program handle that later created handle 287, which is the handle to which the message was queued.
As can be seen below, I attempted to delete the message via handle 286, then attempted to delete with handle 287. What am I not understanding?
Keep in mind that every time you call into the STAF executable (such as executing "staf m-dt queue delete handle 287"), a new, unique handle is created. So you are requesting the "queue delete handle 287" for this new handle (not handle 287). In fact, you are requesting that you want to delete any messages on this new handle's queue that *originated* from handle 287, and it correctly returns a 0 to indicate that 0 messages were deleted from this new handle.
If we were to use static handles, then from the command line you can do the following:
$ staf local handle create handle name dave
Response
--------
19
We just created a static handle named "dave" that was assigned handle #19.
$ staf local queue queue message hello
Response
--------
1
$ staf local queue list
Response
--------
5;20040421-10:41:51;78ntmm3;dave;19;hello
Notice on these commands that we didn't specify the handle number or name, since we set the STAF_STATIC_HANDLE (so new handles are NOT created each time you run the STAF executable).
$ staf local queue delete
Response
--------
1
Notice that the result is "1", indicating that one message was deleted from the queue.
$ staf local queue list
Response
--------
So now handle 19 no longer that the queued message.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I must be beating a dead horse on this topic by now.
Here's a test program. I set a breakpoint prior the submit() of the queue delete, then check the queue for the submitting handle. Next I step over the submit() and chech the queue again prior to program exit. The message is still on the queue. Below is the program - please point me to my error.
$ staf local queue list handle 618
Response
--------
1;20040427-16:49:35;mmahoney-dt;STAF_Process;1;STAF/PROCESS/END 619;20040427-16:49:35;0
$
#!/usr/bin/perl
BEGIN { unshift @INC, "/usr/local/staf/bin", "c:/staf/bin", "../lib", "/var/Incipient/test/lib", "c:/Incipient/test/lib"; }
use PLSTAF;
&main;
sub main
{
$| = 1;
$progHandle = STAF::STAFHandle->new("prog");
if ($progHandle->{rc} != $STAF::kOk)
{
print "Unable to get new handle\n";
exit(1);
}
So, in this example, the handle that queued the process completion message is handle 1 (not 619). When you call into the Queue service with "delete handle 619" you are asking to delete messages that originated from handle 619, and there are none.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I m having an issue unregistering a handle via a perl program. The program has two handles, numbers 162 and 163 in the example below. Handle number 162 has as STAFLog object constructed with this handle. While handle 163 was used to run a specific program (via staf PROCESS service)
Example:
$ staf local handle query all
Response
--------
1 STAF_Process InProcess 20040325-15:38:40
2 STAF/Service/STAF/ServiceLoader/1 InProcess 20040325-11:41:18
162 123.TOBIKO-DT Registered 20040325-16:11:04
163 123.TOBIKO-DT Registered 20040325-16:11:05
165 STAF/Client Registered 20040325-16:11:20
Handle 163 unregisters without an issue, but when unregistering handle 162 a return code from unRegister() is an rc = 5. The below example shows the handles that exist AFTER deleting handle 163, and prior to deleting handle 162
$ staf local handle query all
Response
--------
1 STAF_Process InProcess 20040325-15:38:40
2 STAF/Service/STAF/ServiceLoader/1 InProcess 20040325-11:41:18
162 123.TOBIKO-DT Registered 20040325-16:11:04
167 STAF/Client Registered 20040325-16:15:53
I validated that the correct handles objects were being used to unregister. Can you explain the anomoly here (aka: why handle 162 is not being unregistered/deleted and an rc = 5 is being returned by unRegister()).
Can you post the Perl code that you are using?
We are using Active Perl 5.8, with the STAF 5.8 Perl object. Below is the code logic, while a test run with stdout is at the end of this message.
my $log = $hash->{samLog};
my $ref = $hash->{jobList};
#
# Release all the staf handles
#
# Note: STAF handles are persistent after program exit on Linux,
# but not the case not on Win32 (handles are deleted upon exit)
#
if (defined($ref))
{
my @jobList = @$ref;
# Release job SAM handles
foreach my $jobHash (@jobList)
{
$log->samLog($itafConstants::ISTR, "$mTag: Deleting job handle: \"$jobHash->{handle}->{handle}\".");
$rc = $hash->{staf}->deleteStafHandle($hash, $jobHash->{handle});
if ($rc eq $itafConstants::FAILURE)
{
$exitStatus = $itafConstants::EXIT_FAILURE;
}
}
}
#
# Release SAM STAF handle
#
$handle = $hash->{staf}->getSTAFHandle();
$log->samLog($itafConstants::ISTR, "$mTag: Deleting STAF handle: \"$handle->{handle}\".");
$rc = $hash->{staf}->deleteStafHandle($hash, $handle);
if ($rc eq $itafConstants::FAILURE)
{
$exitStatus = $itafConstants::EXIT_FAILURE;
}
sub deleteStafHandle()
{
my $rc = $itafConstants::SUCCESS;
my $mTag = 'SamStaf::deleteStafHandle';
if (scalar(@_) ne 3)
{
return($itafConstants::FAILURE);
}
my $self = $_[0];
my $hash = $_[1];
my $handle = $_[2];
my $log = $hash->{samLog};
my $result = $handle->unRegister();
if ($result != $STAF::kOk)
{
my $errStr = $self->getStafErrorStr($result);
$log->samLog($itafConstants::FSTR, "$mTag: Unable to delete STAF handle: \"$handle->{handle}\". Staf RC: $errStr");
$rc = $itafConstants::FAILURE;
}
return($rc);
} # End - deleteStafHandle()
03/30/2004 09:44:05 INFO: SamStaf::getQEntry: Additional info: handle: 42 result: 1;20040330-09:43:55;mmahoney-dt;STAF_Process;1;STAF/PROCESS/END 43;20040330-09:43:55;0
03/30/2004 09:44:15 INFO: SamStaf::getQEntry: Additional info: handle: 44 result: 1;20040330-09:44:05;mmahoney-dt;STAF_Process;1;STAF/PROCESS/END 45;20040330-09:44:05;0
03/30/2004 09:44:15 INFO: SamLog::samCleanup: Deleting job handle: "42".
03/30/2004 09:44:15 INFO: SamLog::samCleanup: Deleting job handle: "44".
03/30/2004 09:44:15 FATAL: SamStaf::deleteStafHandle: Unable to delete STAF handle: "44". Staf RC:
03/30/2004 09:44:15 INFO: SamLog::samCleanup: Deleting STAF handle: "41".
03/30/2004 09:44:15 FATAL: SamStaf::deleteStafHandle: Unable to delete STAF handle: "41". Staf RC:
$
We will investigate this further (I was able to recreate using a test Perl file).
Is there a reason that you need multiple STAF handles in your Perl program? Typically an application that registers with STAF would only register one time and use that handle until the application completes (at which time it unregisters its single handle).
The reason that multiple handles are being used is because multiple jobs are being started, and are started with "notify onend", such that the result is put on the queue. If the same handle is being used to start the jobs, then there could be multiple entries on the queue, and no way (as far as I am aware) to distiquish which queue entry is for which process that was started. If there is another way, please inform me of the technique.
The RC 5 problem is a bug that will be fixed in STAF 2.6.1 (which should be released by the end of the week). See the following Bug for more details:
926738 "Multiple Perl handles: All calls made from latest handle"
I'll certainly install and use the next version of STAF when it becomes available. In the mean time, I'm still interested in learning how I should have utilized handles. Please see the note below, that is also my last message thread:
The reason that multiple handles are being used is because multiple jobs are being started, and are started with "notify onend", such that the result is put on the queue. If the same handle is being used to start the jobs, then there could be multiple entries on the queue, and no way (as far as I am aware) to distiquish which queue entry is for which process that was started. If there is another way, please inform me of the technique.
When you start a process asynchronously, you will get back a handle number. This combined with the target system identifier uniquely identifies this handle/process across all STAF systems.
When STAF on the target system detects that the process has completed, it will queue you a message if you specified NOTIFY ONEND. It seems that you are already doing this part. Now, part of this message will contain the system identifier (from the system sending the message, which is the system that you started the process on) and another part will contain the handle number of the process. Thus, with these two pieces of data, you can determine which specific process has just completed.
Section 8.12.3 of the STAF User's Guide describes the basic structure of a message delivered to a queue. It is
<Priority>;<Timestamp>;<Machine>;<Name>;<Handle>;<Message>
The <Machine> is what you need to identify the system that you started the process on.
The <Message> for a process end notification is documented in section 8.11.6 of the STAF User's Guide. It is (note, the current User's Guide fails to document the KEY string)
STAF/PROCESS/END <Handle>;<Timestamp>;<Return Code>[;KEY=<Key>][;<Returned file data>]
The <Handle> is the handle that you got back from the PROCESS START request.
Note, if you have trouble matching up system identifiers (perhaps because you are using an IP address, while STAF sends you a host name), then you can provide a KEY on the PROCESS START which will be sent to you as part of the process end notification. With this, you can generate your own unique keys to identify each process, and you don't have to do the system/handle matching.
With the above, you can easily use a single handle to handle processes getting started on any number of systems. Post again if you have more questions.
Thank you for the last bit of information regarding how the queue message is built.
The next question is how to delete a specific message on the Queue.
Ex:
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
If I want to delete the 2nd message - or even all the messages associated with this handle (again via the Perl API). The delete returns OK, but he message remains on the Queue:
$ staf local queue delete handle 169
Response
--------
0
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
I also attempted to delete via the start handle:
$ staf local queue delete handle 172
Response
--------
0
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
Can you please explain the descrepency.
Thanks!
Also attempted to delete as follows (i.e. using "contains"):
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
$ staf local queue delete contains "STAF_Process 172"
Response
--------
0
$ staf local queue list handle 169
Response
--------
1;20040416-17:05:31;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 170;20040416-17:05:31;0
1;20040416-17:06:47;mm-fi30ocq796uz;STAF_Process;1;STAF/PROCESS/END 172;20040416-17:06:47;0
$
Every time you run the STAF executable (when you run "staf local ....") that instance of the STAF client is going to be assigned a new handle number, and will have an associated queue. So, when you run "staf local queue ...." you are actually dealing with the handle that has been created for that STAF client and its associated queue.
The HANDLE option for the Queue service is used differently for LIST and DELETE commands:
--------------------------------------------------------
LIST [HANDLE <Handle>]
HANDLE specifies the handle of the process for which you want the queue contents. The default is the handle of the submitting process.
DELETE [PRIORITY <Priority>]... [MACHINE <Machine>]... [NAME <Name>]...
[HANDLE <Handle>]... [USER <User>]...
[CONTAINS <String>]... [ICONTAINS <String>]...
HANDLE specifies that you want to delete messages originating from a process with the given handle.
--------------------------------------------------------
Note that the HANDLE specified for the DELETE command is used to specify a handle that *originated* the queued message(s), but you are only deleting messages in the queue for the current STAF client (and that's why the command is returning 0, since the STAF client's queue did not contain any messages).
So, in this case, the QUEUE DELETE needs to be requested from the program that registered with STAF and was assigned handle 169.
Still having the delete message issue. The following shows all the handles that are open, with handle 286 being the program handle that later created handle 287, which is the handle to which the message was queued.
As can be seen below, I attempted to delete the message via handle 286, then attempted to delete with handle 287. What am I not understanding?
$ staf local handle query all
Response
--------
1 STAF_Process InProcess 20040420-15:39:57
2 STAF/Service/STAF/ServiceLoader/1 InProcess 20040419-09:33:03
9 STAF/Service/LOG InProcess 20040419-09:33:17
10 STAF/Service/MONITOR InProcess 20040419-09:33:03
286 123.M-DT Registered 20040420-15:39:53
287 123.M-DT Registered 20040420-15:40:43
291 STAF/Client Registered 20040420-15:41:42
$ staf m-dt queue list handle 287
Response
--------
1;20040420-15:39:57;m-dt;STAF_Process;1;STAF/PROCESS/END 288;20040420-15:39:57;0
$ staf m-dt queue delete handle 286
Response
--------
0
$ staf m-dt queue list handle 287
Response
--------
1;20040420-15:39:57;m-dt;STAF_Process;1;STAF/PROCESS/END 288;20040420-15:39:57;0
$ staf m-dt queue delete handle 287
Response
--------
0
$ staf m-dt queue list handle 287
Response
--------
1;20040420-15:39:57;m-dt;STAF_Process;1;STAF/PROCESS/END 288;20040420-15:39:57;0
$
Keep in mind that every time you call into the STAF executable (such as executing "staf m-dt queue delete handle 287"), a new, unique handle is created. So you are requesting the "queue delete handle 287" for this new handle (not handle 287). In fact, you are requesting that you want to delete any messages on this new handle's queue that *originated* from handle 287, and it correctly returns a 0 to indicate that 0 messages were deleted from this new handle.
If we were to use static handles, then from the command line you can do the following:
$ staf local handle create handle name dave
Response
--------
19
We just created a static handle named "dave" that was assigned handle #19.
$ staf local queue queue message hello
Response
--------
1
$ staf local queue list
Response
--------
5;20040421-10:41:51;78ntmm3;dave;19;hello
Notice on these commands that we didn't specify the handle number or name, since we set the STAF_STATIC_HANDLE (so new handles are NOT created each time you run the STAF executable).
$ staf local queue delete
Response
--------
1
Notice that the result is "1", indicating that one message was deleted from the queue.
$ staf local queue list
Response
--------
So now handle 19 no longer that the queued message.
In my last post I did not mention how you set STAF_STATIC_HANDLE.
After you create the static handle:
$ staf local handle create handle name dave
Response
--------
19
Run this command:
$ export STAF_STATIC_HANDLE=19
I must be beating a dead horse on this topic by now.
Here's a test program. I set a breakpoint prior the submit() of the queue delete, then check the queue for the submitting handle. Next I step over the submit() and chech the queue again prior to program exit. The message is still on the queue. Below is the program - please point me to my error.
$ staf local queue list handle 618
Response
--------
1;20040427-16:49:35;mmahoney-dt;STAF_Process;1;STAF/PROCESS/END 619;20040427-16:49:35;0
$
#!/usr/bin/perl
BEGIN { unshift @INC, "/usr/local/staf/bin", "c:/staf/bin", "../lib", "/var/Incipient/test/lib", "c:/Incipient/test/lib"; }
use PLSTAF;
&main;
sub main
{
$| = 1;
$progHandle = STAF::STAFHandle->new("prog");
if ($progHandle->{rc} != $STAF::kOk)
{
print "Unable to get new handle\n";
exit(1);
}
my $cmd = "start shell command \"/Incipient/test/bin/sleep.exe 2\" STDOUTAPPEND c:/incipient/itaf/internal/sam/3_longevity_sleep_E0.log STDERRTOSTDOUT notify WORKDIR /Incipient/test/bin/ onend priority 1 handle $progHandle->{handle} machine mmahoney-dt ASYNC ";
my $result = $progHandle->submit("local", "process", $cmd);
if ($result->{rc} != 0)
{
print "Error submitting request: RC = $result->{rc}\n".
"Additional info: $result->{result}\n";
}
print "Response\n";
print "--------\n";
print "$result->{result}\n";
my $launchHandle = $result->{result};
# Remote \n and \r
chomp($launchHandle);
$launchHandle =~ s/\r//g;
$cmdStr = "delete handle $launchHandle";
print "cmdStr: $cmdStr\n";
$result = $progHandle->submit("local", "queue", $cmdStr);
if ($result->{rc} != $STAF::kOk)
{
print "Failed to delete queue\n";
if (length($result->{result}) != 0)
{
print "Additional info: $result->{result}.";
}
}
exit(0);
}
Program output:
Response
--------
619
cmdStr: delete handle 619
The handle that sends you the process end queue message is the main STAF handle, not the process handle. Your queued message is:
1;20040427-16:49:35;mmahoney-dt;STAF_Process;1;STAF/PROCESS/END 619;20040427-16:49:35;0
The format is:
<Priority>;<Timestamp>;<Machine>;<Name>;<Handle>;<Message>
where the format for <Message> is:
STAF/PROCESS/END <Handle>;<Timestamp>;<Return Code>[;<Returned file data>]
So, in this example, the handle that queued the process completion message is handle 1 (not 619). When you call into the Queue service with "delete handle 619" you are asking to delete messages that originated from handle 619, and there are none.
Thank you. That worked.