ssh-sftp-perl-users

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Mark A. F. <mar...@ea...>]

From: Ofer Nave <on...@sh...>
>there isn't the slightest mention of error handling in the docs.  How do you tell when something didn't work, 
and how do you find out what went wrong?

I don't know if we're talking about the same thing, but I put all my method calls inside an eval. I trapped the die signal in the eval and accumulated the messages in an array. After the method call (more precisely, after the eval), I detected whether the die signal handler had been invoked, and I logged the collected messages, took remedial action (like retrying the operation), etc.

I posted samples of this a year or more ago. They should be in the archive.

Mark

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Mark A. F. <mar...@ea...>]

From: "Mark A. Fuller" <mar...@ea...>
>I trapped the die signal in the eval and accumulated the messages in an array. 

I should have also said that I found it useful to accumulate messages in an array because it seemed like multiple die signals might be trapped within an eval of one method call. I found that I could end up with more than one message after the eval, when I detected that the die signal handler had been called. Sometimes the first message was more meaningful than the last.

BTW: Earlier you said it was too bad that I had to do something outside of Perl. Why's that bad? What is the downside of using Expect.pm (a Perl module) to drive an executable module (like an open-ssh sftp module)? 

It's slightly more complicated to code. A larger memory footprint?

The upside to me was that it was more "WYSIWYG." What I mean is, if I put something into production and it stopped working for some bizarre reason a year or two down the road, it was relatively straightforward to exercise the executable and verify what the driving process was encountering. With N:S:P, I found myself making a number of invalid assumptions about how my experience with the executable should be the expected experience with the Perl binding. Things got simpler for me when there was a one-to-one correlation between what I could do at the command line and via my script to automate the same connection.

I suppose psftp (the command-line emulator tool implementing N:S:P) might have alleviated my problem.

Another benefit, however is that once I went through the learning curve for Expect.pm I could apply it to *any* executable. I didn't have to learn the ins-and-outs of every Perl binding I might want to use. Or, trip over various shortcomings. The executables tend to see more widespread use and therefore the bugs are worked out. Using a single binding (Expect.pm) to drive all of them seemed like a more stable proposition *to me*. 

I'm not trying to evangelize anyone away from N:S:P. Just offering my own experience.

Mark

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Ofer N. <on...@sh...>]

Mark A. Fuller wrote:

>BTW: Earlier you said it was too bad that I had to do something outside of Perl. Why's that bad? What is the downside of using Expect.pm (a Perl module) to drive an executable module (like an open-ssh sftp module)? 
>
>  
>
Well, Expect is wonderful because in that it helps you when you're 
forced to drive an external app.  But it is a last line of defense.  
It's a hack.  I use perl because it is a powerful and expressive 
programming language.  All that goes out the window the minute I have to 
call system() and use a seperate executable (not to mention the fact 
that you fork several times needlessly in the process).  And what is my 
feedback?  An 8-bit integer, and maybe some STDOUT/STDERR output that I 
have to capture and parse?  How can you possibly compare that to the 
richness and elegance of returning complex data structures, like a 
native function can?  Or being able to throw execptions that are captured?

I only go outside perl when I have no choice.  And that is why I love 
Net::SSH::Perl. Native, pure-perl implementation of SSH.  Much needed 
and much appreciated.

-ofer

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Mark F. <mar...@ea...>]

From: "Ofer Nave" <on...@sh...>
> goes out the window the minute I have to
> call system() and use a seperate executable (not to mention the fact
> that you fork several times needlessly in the process).

Wouldn't the fact that you're executing a binary cancel out the overhead of
the system call to use it (and the forks)? I haven't noticed a performance
problem using Expect.pm to drive an sftp executable.

>And what is my feedback?  An 8-bit integer, and maybe some STDOUT/STDERR
output that I
> have to capture and parse?

I was interrogating results in N::S::P too. Also, capturing all results
(eval, die signal and accumulating messages in an array) wasn't exactly
graceful either. :) I don't see the difference. Either way a person is
probably going to want to test the results and make decisions, right?

> returning complex data structures, like a native function can?

I haven't needed that (yet). Can you give me an example of when returned
complex data structures would be useful?

>Or being able to throw execptions that are captured?

I'm not sure what you mean. If the open-ssh sftp binary encounters an error,
Expect.pm captures this. It's available to me. Are there other exceptions?

To me, the greatest benefits using Expect.pm are these three:

1. I can drive any binary in *exactly* the same manner. I'm not forced to
learn the peculiarities of every pure-perl interface. I'm probably going to
learn the binary's peculiarities since that's how I would first develop the
steps to connect and test every operation of the connection. Wrapping it
with Expect.PM is just implementing what I learned executing the binary
manually. That's seems simpler. It's like "what you see is what you get."

2. Because I'm using the binary, if anything begins failing (in the distant
future), it's relatively easy to execute the binary manually in order to
discover what (in the process of connecting and navigating) has changed. The
binary becomes an accessible reference point.

3. The binary is probably better vetted, having seen greater usage. There's
less chance of stumbling upon a bug or security risk.

Off hand, the only thing I can think of for when a pure-perl interface would
be better is if the client will execute on a variety of platforms. Then
you'd only have to learn one peculiar interface rather than each vendor's
binary. (FTP is an example of how different each vendor's binary could be).

Mark

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Ofer N. <on...@sh...>]

Mark Fuller wrote:

>From: "Ofer Nave" <on...@sh...>
>  
>
>>goes out the window the minute I have to
>>call system() and use a seperate executable (not to mention the fact
>>that you fork several times needlessly in the process).
>>    
>>
>
>Wouldn't the fact that you're executing a binary cancel out the overhead of
>the system call to use it (and the forks)? I haven't noticed a performance
>problem using Expect.pm to drive an sftp executable.
>
>  
>
You're right that the binary will be faster than a pure-perl 
implementation like Net::SSH::Perl once the code has started running.  
It doesn't "cancel" anything out cleanly, because the cost of forking is 
one constant, while the cost of a slower implementation is proportional 
to the amount of data you transfer.  But the cost of forking, and 
perhaps even the cost of a slower perl implementation, are both likely 
negligble relative to the latency of network communication.

However, the fork still does have a time cost that can add up, and it's 
harder on the OS (more task switching), and pollutes the process table, 
and is just plain ugly.  I have scripts that fork all over the place, 
but I do it when I have to, not when I have a choice. [1]

The best of both world would be a Net::SSH::XS module that wraps a C 
library - preferably whatever C library is already used by the ssh 
command (openssl?) so the same code doesn't have to be written twice.  
Native perl calls, C speed.  XSellent!

>>And what is my feedback?  An 8-bit integer, and maybe some STDOUT/STDERR
>>    
>>
>output that I
>  
>
>>have to capture and parse?
>>    
>>
>
>I was interrogating results in N::S::P too. Also, capturing all results
>(eval, die signal and accumulating messages in an array) wasn't exactly
>graceful either. :) I don't see the difference. Either way a person is
>probably going to want to test the results and make decisions, right?
>
>  
>
Yes, but perl code works well with other perl code.  If I have to use 
the system function, it's the equivalent of building a model airplane 
using a pair of salad tongs instead of your hands.  You can get some 
stuff done, but it's a pain in the ass, and you're tactile feedback is 
astonishingly limited.

>>returning complex data structures, like a native function can?
>>    
>>
>
>I haven't needed that (yet). Can you give me an example of when returned
>complex data structures would be useful?
>
>  
>
Sure.  DBI.  Imagine having to use system() to call the 'isql' utility 
to run queries against Sybase and parse the output instead of using the 
DBI library.

Oh god, I wish I hadn't said that.  I think I'm breaking out in hives now.

>>Or being able to throw execptions that are captured?
>>    
>>
>
>I'm not sure what you mean. If the open-ssh sftp binary encounters an error,
>Expect.pm captures this. It's available to me. Are there other exceptions?
>
>To me, the greatest benefits using Expect.pm are these three:
>
>1. I can drive any binary in *exactly* the same manner. I'm not forced to
>learn the peculiarities of every pure-perl interface. I'm probably going to
>learn the binary's peculiarities since that's how I would first develop the
>steps to connect and test every operation of the connection. Wrapping it
>with Expect.PM is just implementing what I learned executing the binary
>manually. That's seems simpler. It's like "what you see is what you get."
>
>2. Because I'm using the binary, if anything begins failing (in the distant
>future), it's relatively easy to execute the binary manually in order to
>discover what (in the process of connecting and navigating) has changed. The
>binary becomes an accessible reference point.
>
>3. The binary is probably better vetted, having seen greater usage. There's
>less chance of stumbling upon a bug or security risk.
>
>Off hand, the only thing I can think of for when a pure-perl interface would
>be better is if the client will execute on a variety of platforms. Then
>you'd only have to learn one peculiar interface rather than each vendor's
>binary. (FTP is an example of how different each vendor's binary could be).
>
>  
>
Does anyone else want to jump in and field these?  Or am I in the 
minority?  If I am, I'll shut up.

-ofer

[1] I lied.  I don't *have* to fork all the times that I do, but I'm too 
scared to learn multi-threaded programming.  :)

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Mark F. <mar...@ea...>]

From: "Ofer Nave" <on...@sh...>
> Yes, but perl code works well with other perl code.  If I have to use
> the system function, it's the equivalent of building a model airplane
> using a pair of salad tongs instead of your hands.  You can get some
> stuff done, but it's a pain in the ass, and you're tactile feedback is
> astonishingly limited.

I guess I still don't understand the significance of this as applied to
Expect.pm driving an open-ssh sftp compiled executable. Expect.pm is Perl.
So, it's my Perl code working well with another Perl code (which itself
abstracts all the details about system calls and trapping the output). Why
should I care that all this is happening beneath the covers (except for
perhaps the performace issues of forking and pipe creation for interprocess
communication)?

To me that's a strength. I'm not interfacing directly to the sftp command.
I'm interfacing to Expect.pm. I do Perl like I normally would. I don't have
to think too much about what's happening. And, once I learn the Expect.pm
heuristics, I can do the same thing with *anything*.

> >>returning complex data structures, like a native function can?
> >
> >I haven't needed that (yet). Can you give me an example of when returned
> >complex data structures would be useful?
> >
> Sure.  DBI.  Imagine having to use system() to call the 'isql' utility
> to run queries against Sybase and parse the output instead of using the
> DBI library.

Yes, I understand in the case of DBI (and it's rows and columns). But, in
the case of protocols like SFTP, FTP, Telnet, etc., when would this be of
value? That's what I was trying to ask.

> Does anyone else want to jump in and field these?  Or am I in the
> minority?  If I am, I'll shut up.

Sorry. I wasn't trying to challenge you. I just didn't understand why
Expect.pm would be a last resort (undesireable). I can see positives and
negatives involved with either way.

Mark

Re: [Ssh-sftp-perl-users] Net::SSH::Perl error handling

[Ofer N. <on...@sh...>]

Mark Fuller wrote:

>From: "Ofer Nave" <on...@sh...>
>  
>
>>Yes, but perl code works well with other perl code.  If I have to use
>>the system function, it's the equivalent of building a model airplane
>>using a pair of salad tongs instead of your hands.  You can get some
>>stuff done, but it's a pain in the ass, and you're tactile feedback is
>>astonishingly limited.
>>    
>>
>
>I guess I still don't understand the significance of this as applied to
>Expect.pm driving an open-ssh sftp compiled executable. Expect.pm is Perl.
>So, it's my Perl code working well with another Perl code (which itself
>abstracts all the details about system calls and trapping the output). Why
>should I care that all this is happening beneath the covers (except for
>perhaps the performace issues of forking and pipe creation for interprocess
>communication)?
>
>To me that's a strength. I'm not interfacing directly to the sftp command.
>I'm interfacing to Expect.pm. I do Perl like I normally would. I don't have
>to think too much about what's happening. And, once I learn the Expect.pm
>heuristics, I can do the same thing with *anything*.
>  
>
How does Expect work?  It works by capturing the output of a command, 
and giving you useful mechanisms for parsing it and responding to it, 
right?  But you're still parsing the human readable output of a command 
line.  How can that be consider more reliable than depending on the 
documented behavior of a perl function or module?  The output can change 
at any time.  There are differences in system utiltities from OS to OS 
(the 'ps' command is a classic example).  If you get new mail you might 
get a "new mail arrived" line thrown in unexpectedly.  Expect is 
powerful and useful, but it is a hack.

-ofer