expectperl-discuss

[Expectperl-discuss] speed up object close

[Christoph S. <csc...@am...>]

Hello,

the tool I created using expect allows to perform shell command(s) entered 
on a web page on selected remote hosts as selected users. this works quite 
well and there is no real problem.
The only issue I would like to improve is the speed with which the expect 
handles are closed after all commands have been executed.

As sometimes a command, in parallel performed on 80 hosts take only, lets 
say 10s including logging in, the closure takes two minutes. The tool 
does:
foreach my $obj (@Objects) {
        $obj->hard_close();

Can this be speeded up somehow? Shall I try to i.e. fork 8 children of 
which each one is responsible for closing 10 objects? Why is the closure 
of an expect object relatively slow?

Thanks & Brgrds,
Christoph Schwaiger

Re: [Expectperl-discuss] speed up object close

[Austin S. <te...@of...>]

> does:
> foreach my $obj (@Objects) {
>         $obj->hard_close();
> 
> Can this be speeded up somehow? Shall I try to i.e. fork 8 children of 
> which each one is responsible for closing 10 objects? Why is the closure 
> of an expect object relatively slow?
> 

	Because it allows the process to die gracefully before it kill -9s it.
The graceful manner is usually to get the process to terminate itself and
just be ready for that. That's usually done by closing the handle, e.g.
close ($obj), or sending "exit\r" to whatever you are logged into. If the
child is ready to go away hard_close will return quickly.

	Austin

Re: [Expectperl-discuss] speed up object close

[Austin S. <te...@of...>]

On Fri, Dec 22, 2006 at 02:49:11PM +0100, Christoph Schwaiger wrote:
> Hello Austin,
> 
> thanks for your answer. Currently, I'm already using hard_close, but 
> wonder, due to the many objects I close sequentially, even hard_close 
> takes too long.
> Would you have an idea how to speed the closure up?
> 
	you can always just kill(9, $expect->pid()); and then wait(); 

	By default nothing does that because sending signals other than
9 allows the process to clean itself up before exit. But that is the fastest
way to make sure it goes away _now_. In fact you can skip on the wait() if
you don't mind leaving zombies around until your perl process exits.

	Austin

Re: [Expectperl-discuss] speed up object close

[Roland G. <rgi...@cp...>]

I'll second Austins first approach: make the spawned commands exit
cleanly. That means, send an "exit\r" or whatever is needed after you
get the output from the remote commands. This will cause the remote
side to close everything down and you will get an EOF and the spawned
command will terminate and the close() will return immediately.

BTW, why are you using Expect? You could execute the shell commands
via qx'ssh'.  Unless you're starting some interactive programs
remotely...

Regards, Roland
PS: Frohe Weihnukka! :o)

On 12/23/06, Austin Schutz <te...@of...> wrote:
> On Fri, Dec 22, 2006 at 02:49:11PM +0100, Christoph Schwaiger wrote:
> > Hello Austin,
> >
> > thanks for your answer. Currently, I'm already using hard_close, but
> > wonder, due to the many objects I close sequentially, even hard_close
> > takes too long.
> > Would you have an idea how to speed the closure up?
> >
>         you can always just kill(9, $expect->pid()); and then wait();
>
>         By default nothing does that because sending signals other than
> 9 allows the process to clean itself up before exit. But that is the fastest
> way to make sure it goes away _now_. In fact you can skip on the wait() if
> you don't mind leaving zombies around until your perl process exits.
>
>         Austin
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share your
> opinions on IT & business topics through brief surveys - and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Expectperl-discuss mailing list
> Exp...@li...
> https://lists.sourceforge.net/lists/listinfo/expectperl-discuss
>

Re: [Expectperl-discuss] speed up object close

[Christoph S. <csc...@am...>]

I think sending an exit and let the objects clean itself is the cleanest 
way and could be also the fastest (the exit is performed more or less 
concurrently in all objects as the sending should be an async action). 
I'll try this. this is so simple, why didn't I think about it :-(

BTW, why are you using Expect? You could execute the shell commands
via qx'ssh'.  Unless you're starting some interactive programs
remotely...

>> the applicaction depends on env variables which wouldn't be set in 
one-off shell commands (commands either don't work at all or behave 
differently. I did a wrapper to set the variables and performs the one off 
command, but I had to work around problems caused by starting application: 
the app contains deamons and when they are started in a one off ssh 
command with nohub, the ssh remains, as it wants all processes to stop 
prior end. As said this can be worked around with killing itself using 
different signals depending on the commands exit code - but this is 
nothing I have a good feeling with. Using expect is much cleaner as it 
uses an interactive ssh as a user would do (no difference to a human keyin 
what so ever).

Thanks to you and Austin.

Best Regards,
Christoph Schwaiger





"Roland Giersig" <rgi...@cp...> 
To
exp...@li...
cc

bcc

Subject
Re: [Expectperl-discuss] speed up object close





"Roland Giersig" <rgi...@cp...>
Sent by: exp...@li... 
23-12-06 17:44


I'll second Austins first approach: make the spawned commands exit
cleanly. That means, send an "exit\r" or whatever is needed after you
get the output from the remote commands. This will cause the remote
side to close everything down and you will get an EOF and the spawned
command will terminate and the close() will return immediately.

BTW, why are you using Expect? You could execute the shell commands
via qx'ssh'.  Unless you're starting some interactive programs
remotely...

Regards, Roland
PS: Frohe Weihnukka! :o)

On 12/23/06, Austin Schutz <te...@of...> wrote:
> On Fri, Dec 22, 2006 at 02:49:11PM +0100, Christoph Schwaiger wrote:
> > Hello Austin,
> >
> > thanks for your answer. Currently, I'm already using hard_close, but
> > wonder, due to the many objects I close sequentially, even hard_close
> > takes too long.
> > Would you have an idea how to speed the closure up?
> >
>         you can always just kill(9, $expect->pid()); and then wait();
>
>         By default nothing does that because sending signals other than
> 9 allows the process to clean itself up before exit. But that is the 
fastest
> way to make sure it goes away _now_. In fact you can skip on the wait() 
if
> you don't mind leaving zombies around until your perl process exits.
>
>         Austin
>
> 
-------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share 
your
> opinions on IT & business topics through brief surveys - and earn cash
> 
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Expectperl-discuss mailing list
> Exp...@li...
> https://lists.sourceforge.net/lists/listinfo/expectperl-discuss
>

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share 
your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Expectperl-discuss mailing list
Exp...@li...
https://lists.sourceforge.net/lists/listinfo/expectperl-discuss

Re: [Expectperl-discuss] speed up object close

[Christoph S. <csc...@am...>]

Hello Austin,

thanks for your answer. Currently, I'm already using hard_close, but 
wonder, due to the many objects I close sequentially, even hard_close 
takes too long.
Would you have an idea how to speed the closure up?

Regards & merry XMas!
Christoph Schwaiger



Austin Schutz <te...@of...> 
To
Christoph Schwaiger <csc...@am...>
cc
exp...@li...
bcc

Subject
Re: [Expectperl-discuss] speed up object close





Austin Schutz <te...@of...>
Sent by: exp...@li... 
22-12-06 03:58


> does:
> foreach my $obj (@Objects) {
>         $obj->hard_close();
> 
> Can this be speeded up somehow? Shall I try to i.e. fork 8 children of 
> which each one is responsible for closing 10 objects? Why is the closure 

> of an expect object relatively slow?
> 

                 Because it allows the process to die gracefully before it 
kill -9s it.
The graceful manner is usually to get the process to terminate itself and
just be ready for that. That's usually done by closing the handle, e.g.
close ($obj), or sending "exit\r" to whatever you are logged into. If the
child is ready to go away hard_close will return quickly.

                 Austin

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share 
your
opinions on IT & business topics through brief surveys - and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Expectperl-discuss mailing list
Exp...@li...
https://lists.sourceforge.net/lists/listinfo/expectperl-discuss