STAF vs SSH time complexity

[Parinita]

I did a comparison of the time taken to execute the same test case in ssh and STAF. I observed that STAF calls took more time compared to SSH. Since time is the most important factor in my testing, I would like to know what shall I do to reduce the time taken by STAF calls.

[Sharon Lucas]

Were you using the STAF command line executable or STAF APIs (e.g. for Java, Python, Perl, C++) to submit your STAF service requests when you checked on timing? What STAF service request(s) were you comparing?

Using the STAF command line can be a little slower, particularly if you didn't create a STAF handle first and then use it for submitting all of your STAF service requests.

[Parinita]

I am using the command line executable for STAF. The comparison was for multithreading operation and the service was 'process start'. So the time taken for firing the commands in STAF was greater as compared to SSH. And my application cannot use the given STAF APIs as Im coding in C#.

[Sharon Lucas]

If you are submitting multiple STAF service requests (e.g. PROCESS START requests) to a system, it may be faster to submit the STAF PROCESS START requests via a .bat file or shell script, etc) to eliminate the time to start a connection, etc.

[Parinita]

Thats a good idea. However, the problem is that my calls might not be targetted to a single host. And so arises the question of how will staf be useful, incomparison with SSH?

[Sharon Lucas]

Perhaps SSH is better for your specific purposes.

[Parinita]

I will surely let you know incase I come up with a better solution with staf.

[Nixon]

[Nixon]

Hi Sharon, can you please look at my previouse comment? thanks.

[Sharon Lucas]

The staf_vs_ssh.py script that you attached shows that when using ssh, you are submitting only 1 command to ssh to a remote system and then once the connection to the remote system has been established, you are running a mount command 50 times.

However, when using STAF, you are submitting a "STAF <remoteSystem> PROCESS START COMMAND mount" request to run a mount command 50 times which means that each time it has to connect to the remote system, run the mount command and disconnect. The time it takes to connect/disconnect 50 times to the remote system is what is taking up most of the time. A more comparable test would be to submit one STAF PROCESS START request to run a script/program on the remote system. This script/program would contain a loop that runs the mount command 50 times.

For example, here's a shell script named mount50.sh that runs a "mount" command 50 times:

1 2 3 4 5 6 7 8 9 10 11 12

#!/bin/sh # Run the mount command 50 times i=1 while [ $i -le 50 ] do mount if [ $? -ne 0 ]; then echo "mount command failed with RC $?" fi (( i++ )) done exit 0

Put this shell script on the remote system (e.g. Could use a STAF FS COPY request to do this) and then run this shell script on the remote system using a single STAF PROCESS START request in your staf_vs_ssh.sh script (remove the loop in your staf_vs_ssh.sh in the STAF section and run mount50.sh instead of the mount command via a STAF PROCESS START request).

Or, you can use the STAFLoop command (provided with STAF) as follows to run this PROCESS START request on the remote system and it will output the "Total time" in seconds that it took to run this STAF command:

# STAFLoop 1 192.168.1.22 PROCESS START SHELL COMMAND "/tmp/mount50.sh" RETURNSTDOUT STDERRTOSTDOUT WAIT
Total loops: 1
Total time : 1.153
Avg. time  : 1.153
Loops/sec  : 0.867303
#

On my systems, it took just over 1 second to submit a STAF PROCESS START request to run this shell script that runs a mount command 50 times. The way you were doing it previously when using STAF was taking over 50 seconds.

And instead of using a shell script, you could write a python script to run the mount command 50 times.

[Nixon]

Thanks for the reply. I generalized the problem with a single command but the actual test case runs 50 different commands. i would not be able to run the way you are suggesting. Based on this thread, i was under the impression the connect/disconnect come into picture only if the commands are executed using STAF CLI. When the API is used, it is supposed to behave similar to ssh. That doesn't seems to be the case.

Another observation: The issue seems to be specific to Linux. If the same script is run on windows, it completes in couple of seconds (location as 'local' and command 'dir' on windows and 'ls' on Linux). Could you please why there is a difference?

Windows:

C:\tmp>type timer.bat
@echo off
echo %time%
python staf_vs_ssh.py staf
echo %time%

C:\tmp>timer
3:29:17.18
3:29:19.33

Linux:
$ time python staf_vs_ssh.py staf

real 0m56.261s
user 0m0.043s
sys 0m0.008s

[Nixon]

Any update on this issue?

[Sharon Lucas]

The PROCESS service on Linux takes additional setup time (< 1 second) when it runs a START request that specifies the WAIT option. This additional setup time is insignificant when running a command that takes a while to complete (instead of just a "ls" command which completes < 1 second) and/or when submitting the PROCESS START request to a remote system.

If you remove the WAIT option from the PROCESS START request in your staf_vs_ssh.py and run it with location = 'local' on your Linux system, you'll see that it completes in less than a second (instead of 56 seconds)

# time python staf_vs_ssh.py staf

real    0m0.203s
user    0m0.028s
sys     0m0.024s

If you don't specify the WAIT option on a PROCESS START request, then STAF starts the process asynchronously which means that the START request returns to the caller as soon as the process has begun execution. In this case, the process completion results will be saved when the process completes, and will later need to be FREE'd using a PROCESS FREE request.

If you are running commands that need to be run sequentially (not in parallel), you'll either want to specify the WAIT option or specify the NOTIFY ONEND option which tells the PROCESS service to send a process completion notification message to your STAF handle's queue when the process ends and add code that uses the QUEUE service to get the process completion notification messages off your STAF handle's queue and submit PROCESS FREE requests to free the process completion results.

[Nixon]

Thanks for the reply.

Here again if i have to get the output on command basis, it is going to take ~1 sec per command, same as the WAIT option.

import sys
from PySTAF import STAFHandle, STAFException, unmarshall

try:
handle = STAFHandle("STAF/Client/Python")
except STAFException, e:
print "Error registering with STAF, RC: %d" % e.rc
sys.exit(e.rc)

loop = 50
cmd = 'ls'
location = 'local'
service = 'process'
request = 'start shell command {0} notify onend returnstdout stderrtostdout'.format(cmd)
for i in range(loop):
result = handle.submit(location, service, request)

if (result.rc != 0):
    print "%d - Error submitting request, RC: %d" % (i, result.rc)
    sys.exit(1)

while (True):
    res = handle.submit(location, 'queue', 'get')
    if (res.rc == 0):
        resultMC = unmarshall(res.result)
        if 'message' in resultMC.rootObject:
            print resultMC.rootObject['message']
        handle.submit(location, service, "free handle " + result.result)
        break

rc = handle.unregister()

Time taken by the above script:

$ time python staf.py

real 0m50.155s
user 0m12.015s
sys 0m8.800s

If I submit all commands at one shot and then read from the queue, that takes much shorter time. but that is not my use case.

$ time python s.py

real 0m0.182s
user 0m0.067s
sys 0m0.008s

In a practical case, one of our test suite having 136 test cases takes 60minutes with STAF but only 5 minutes with SSH. would you consider fixing this issue?

[Nixon]