jsch-users — For JSch users : support, suggestions, announcments, etc.

[JSch-users] Transferring large number of small files

[Luís L. <lf...@gm...>]

Hi all.

I'm trying to transfer a large number of small files to a remote server (with reasonable latency) using JSch.

The code I'm using creates a Session and a ChannelSftp and then iterates over the local files (some times thousands) to transfer them to the server, with something like (I omitted exception handling for simplicity):

Session session=...;
ChannelSftp sftp=(ChannelSftp)session.openChannel("sftp");
for(Path path:...){
    sftp.put(path.toString(), String.format("/destination/%s",path.getFileName().toString()));
}
sftp.disconnect();
session.disconnect();

The problem is this is very slow (sometimes it takes hours) compared with the something like:

rsync -avu /local/<some glob> server:/destination/ # takes seconds

Is there any faster way to transfer large number of small files with JSch?

Regards,
LL

Re: [JSch-users] Illegal data get mixed into a sent file

[Kagoshima D. A. Co., L. T.O. <ooh...@da...>]

Hello.

Thank you for contacting.

Once we had applied the patch, the problem disappeared and would no longer occur.

So we want you to answer the following questions:

1. What do you think has caused this problem?

2. Is the content of the patch a fix applicable to "JSch-0.1.49"?

3. Has the operation verification (or test) of the patch finished?
   For example, has the test finished for "JSch-0.1.54"?

[JSch-users] JSch issue with config

[CHAUMIER, R. <RAC...@bo...>]

Hi,

I'm facing an issue about ssh session with ~/.ssh/config :

I have a remote host toto.titi.fr accessible on the port 2222. Due to a limitation of the framework I'm using (which uses jsch to connect to a remote git repository), I had to define the uri like that use...@to...:path/to/my/repo<mailto:use...@to...:path/to/my/repo> and added a config file with the configuration

## redirection port
Host toto.titi.fr
     HostName toto.titi.fr
     HostKeyAlias toto.titi.fr
     Port 2222
     User username

Unfortunately when jsch try to connect (session.connect(tms) I got a NullPointerException from the method checkHost(host, port, kex) whereas it load the key list (line 801).

HostKey[] keys =
        hkr.getHostKey(chost,
                       (key_type.equals("DSA") ? "ssh-dss" : "ssh-rsa"));


Finally,  keys is null. So the line 805 (for(int j=0; j< keys.length; j++){) throws a NullPointerException.

The problem is located on the method getHostKey of HostKeyRepository. The library try to identify in the known_hosts file the remote host key. So It loops and as it finds the matched item it returns it. If no key found it returns null. In my case it returns null because the library compares the following chost  :

"[toto.titi.fr]:2222" (because of the rule (line 709 at Session.class) which modify the defaut chost value ("toto.titi.fr") when no HostKeyAlias is set and port different than 22.

to  the key in my known_hosts file which looks like :

toto.titi.fr,1.2.3.4 ssh-rsa AAAAB3Nz....C+6e/.... /9MrXset+YG/aYLMZMzG+e31PET

Why the chost is formatted like that when the port used is not the default one?


To workaround, I had to delete in my known_hosts file the key but it's not a good idea. The problem occurs if someone connect to the remote host with ssh util.

My second idea, was to used a HostKeyAlias but it not set at line 698 in the checkHost method. How can I fix the problem?

Regards

RaphC











________________________________

L'intégrité de ce message n'étant pas assurée sur internet, la société expéditrice ne peut être tenue responsable de son contenu ni de ses pièces jointes. Toute utilisation ou diffusion non autorisée est interdite. Si vous n'êtes pas destinataire de ce message, merci de le détruire et d'avertir l'expéditeur.

The integrity of this message cannot be guaranteed on the Internet. The company that sent this message cannot therefore be held liable for its content nor attachments. Any unauthorized use or dissemination is prohibited. If you are not the intended recipient of this message, then please delete it and notify the sender.

Re: [JSch-users] JSchException "timeout in wating for rekeying process." caused by waiting for user answer

[Atsuhiko Y. <ym...@jc...>]

Hi,

Sorry for my long long delay.

On Mon, Jul 6, 2015 at 5:58 PM, Alexander Koshevoy
<ale...@je...> wrote:
> We are using JSch library 0.1.52 and have a difficulty dealing with
> timeout on establishing connection. The problem is that after a user
> answers prompt dialog "The authenticity of host '***' can't be
> established. RSA key fingerprint is ***. Are you sure you want to
> continue connecting?" JSchException with "timeout in wating for rekeying
> process." message is thrown by JSch Session (under some circumstances):
...
> It seems that timeout value passed to Session#connect(int) method
> applies both to user answer time from UI and server interaction time.
> Might it be justly if user interaction had been excluded from timeout value?

Here is a patch to exclude user interaction time from timeout value.


Sincerely,
--
Atsuhiko Yamanaka
JCraft,Inc.
1-14-20 HONCHO AOBA-KU,
SENDAI, MIYAGI 980-0014 Japan.
Tel +81-22-723-2150
Skype callto://jcraft/
Twitter: http://twitter.com/ymnk
Facebook: http://facebook.com/aymnk

Re: [JSch-users] SFTP slow file transfer speed with Titan FTP

[Atsuhiko Y. <ym...@jc...>]

Hi,

Sorry for my long long delay.

On Tue, Mar 29, 2016 at 9:17 PM, Thomas Küstermann
<tho...@ou...> wrote:
> I don't know what is causing this. After the "timeout" occurred processing continued until the next outage. I tried to reproduce the "bug" with Openssh's sftp command and the -R (num_requests) argument which I think is meant by "sending multiple requests at any one time". As Openssh's sftp works well I compared the ChannelSftp implementation with what is in sftp-client.c. The only obvious difference I found is the "rq.count()==0" condition in "ChannelSftp.get(String, SftpProgressMonitor, long).new InputStream() {...}.read(byte[], int, int)". Removing the condition fixes the issue for me.
>
> Can this be considered as a bug? Is this something eligible for a next version of JSCH?
>
> The following patch is based upon jsch-0.1.53:
>
> --- src/com/jcraft/jsch/ChannelSftp.java
> +++ src/com/jcraft/jsch/ChannelSftp.java
> @@ -1374,17 +1374,16 @@
>                 len=1024;
>               }
>
> -             if(rq.count()==0) {
...

As you have suggested, we will remove "if(rq.count()==0)" check in the
next release.


Sincerely,
--
Atsuhiko Yamanaka
JCraft,Inc.
1-14-20 HONCHO AOBA-KU,
SENDAI, MIYAGI 980-0014 Japan.
Tel +81-22-723-2150
Skype callto://jcraft/
Twitter: http://twitter.com/ymnk
Facebook: http://facebook.com/aymnk

Re: [JSch-users] Illegal data get mixed into a sent file

[Atsuhiko Y. <ym...@jc...>]

Hi,

Sorry for my long long delay.

On Wed, Oct 21, 2015 at 5:22 PM, Kagoshima Data Application Co., Ltd.
T.Oohara <ooh...@da...> wrote:
> Hello.
>
> We are using JSch library version 0.1.49 and we have a problem that the illegal data will get mixed into a file we sent.
>
> If we add the following configuration to Sftp.java, which is one of the examples included with jsch-0.1.49.zip, and send a file to the SFTP Server (Titan FTP server), then the result shows the file size is the same between the sent file and the received one, but we can find the contents of the two files are different if we compare the hash values retrieved from them.
>   JSch.setConfig("mac.c2s", "hmac-sha2-256");
>   JSch.setConfig("mac.s2c", "hmac-sha2-256");
>
> This problem always happens when we sent a file which size is more than 35840 bytes. And we look into the received file, we found the data such as a header information "SSH_MSG_CHANNEL_DATA" always got mixed in with the file at every position of 35709 bytes (i.e. a size of data which JSch can send to the Titan SFTP Server in one time), we think this will cause the difference of the contents with the two files.
>
> And it seems that the following values have been set to "SSH_MSG_CHANNEL_OPEN_CONFIRMATION" from the SFTP Server:
>   initial window size:16384
>   maximum packet size:35840
>
>
> We found the same problem would occur with JSch-0.1.53, do you think this problem occurs due to a defect specific to JSch? And in that case, can you fix the defect with JSch version 0.1.49?

We have not been able to reproduce the reported problems,
but we have recognized the possibilities for such kind of problems.
Could you try the attached patch?


Sincerely,
--
Atsuhiko Yamanaka
JCraft,Inc.
1-14-20 HONCHO AOBA-KU,
SENDAI, MIYAGI 980-0014 Japan.
Tel +81-22-723-2150
Skype callto://jcraft/
Twitter: http://twitter.com/ymnk
Facebook: http://facebook.com/aymnk

[JSch-users] Illegal data get mixed into a sent file

[Kagoshima D. A. Co., L. T.O. <ooh...@da...>]

We performed the investigation on this issue by using JSch-0.1.53.

In the Read process during the first data transmit ion, "buffer.buffe" will be the newly created (extended) reference by Packet.java::shift().

  byte[] foo=new byte[s+buffer.index-5-9-len];
  buffer.buffer=foo;


During the second data transmit ion where a reference will be different from "byte[] data" in ChennelSftp.java::_put(), if the input data read from the 47th byte of "data" by ChennelSftp.java::sendWRITE() is copied to "obuf" from the 0th byte of it, the data will be different from the one which had been actually read.

* The first 47 bytes are forwarded to the SFTP server as invalid data.

  if(obuf.buffer!=data)	
    obuf.putString(data, start, _length);

* Normally, it is supposed to be a right action to start a copy from the 47th byte of "data".

So we have come up with the following two fixes, is our understanding of this issue correct? 

1. Make "byte[] data" be the same reference as "obuf.buffer" at the time before when the input file is read.

By adding the line "data=obuf.buffer;" to the 634th line in ChennelSftp.java::_put(), make "obuf.buffer" always be the same reference as "data".

2. Modify the third argument of ChennelSftp.java::sendWRITE() from "0" to "_s".

Modify the 668th line in ChennelSftp.java::_put(), that is "foo-=sendWRITE(handle, offset, data, 0, foo);", to "foo-=sendWRITE(handle, offset, data, _s, foo);".

[JSch-users] ChannelSftp usage from multiple threads

[Alon G. <alo...@gm...>]

Hello,

I'd like to perform multiple operations on an sftp server from multiple
threads simultaneously. All threads are servicing the same user connecting
to the same server sftp server.
Example:

- thread 1 performs a get() on file 1
- thread 2 performs a get() on file 2
- thread 3 performs a ls() on dir 3
- etc...

In my initial implementation I created a single session and a single
channel of type 'sftp', and shared it between all threads/requests. However
this doesn't seem to work - the InputStream used for get() can't be shared
across multiple requests and it gets closed when the second get() request
comes in.

What is the best practice for such a requirement? should each thread create
a new sftp channel and close it when done? do I also need a new session per
thread or can I still use a single user session and keep opening channels
from it on demand? is there a limit on how many sessions or channels can
(or should) be opened at the same time?

Appreciate your advice.
Alon.

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Offer B. <off...@gm...>]

well the sleep works... but that does not mean the race condition is not in
the JSCH code...
as i said i changed the loop as in the example and it still happens...

On Wed, May 18, 2016 at 1:48 PM, Tobia Conforto <tob...@gr...>
wrote:

> > but read should block until you get data or you reach EOF... it
> shouldn't return -1 if there might be anymore data...
> > as -1 (End Of File) was returned that means that no more data is
> available.
>
> You are probably right. I'm not that familiar with Java I/O streams'
> contracts.
>
> > tried it as it is in the example and i still get the same result...
> > every once in a while i get nothing back. that is the channel is closed
> and there is no available stuff to read.
>
> You might want to try executing a command like "sleep 1; echo Hello; sleep
> 1; echo World" just to rule out any race conditions in your code.
>
> Otherwise, I'll have to defer to the JSch experts here.
>
> Tobia

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Tobia C. <tob...@gr...>]

> but read should block until you get data or you reach EOF... it shouldn't return -1 if there might be anymore data...
> as -1 (End Of File) was returned that means that no more data is available.

You are probably right. I'm not that familiar with Java I/O streams' contracts.

> tried it as it is in the example and i still get the same result...
> every once in a while i get nothing back. that is the channel is closed and there is no available stuff to read.

You might want to try executing a command like "sleep 1; echo Hello; sleep 1; echo World" just to rule out any race conditions in your code.

Otherwise, I'll have to defer to the JSch experts here.

Tobia

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Offer B. <off...@gm...>]

oh.. and the channel exist status is 0.

On Wed, May 18, 2016 at 1:45 PM, Offer Baruch <off...@gm...> wrote:

> ok... tried it as it is in the example and i still get the same result...
> every once in a while i get nothing back. that is the channel is closed
> and there is no available stuff to read.
>
> On Wed, May 18, 2016 at 1:30 PM, Offer Baruch <off...@gm...>
> wrote:
>
>> but read should block until you get data or you reach EOF... it shouldn't
>> return -1 if there might be anymore data...
>> as -1 (End Of File) was returned that means that no more data is
>> available.
>>
>> anyway i will give it a try and report back.
>>
>> On Wed, May 18, 2016 at 12:48 PM, Tobia Conforto <
>> tob...@gr...> wrote:
>>
>>> Then I believe it's a race condition.
>>>
>>> You are looping until connIn.read() returns -1, but that only means that
>>> *for now* there is no more data to read. It doesn't mean that the remote
>>> command is done. So if the remote server takes a few millisecond more to
>>> run the command, your Java program will get a -1 from read() and exit the
>>> loop, instead of waiting for any more data.
>>>
>>> I suggest you study the examples provided with JSch, starting from the
>>> one about Exec[1], where you will see the only condition that can ever
>>> leave the main while(true) is basically: channel.isClosed() && !
>>> in.available()>0
>>>
>>> Tobia
>>>
>>> [1] http://www.jcraft.com/jsch/examples/Exec.java.html
>>>
>>> On 18 May 2016, at 11:38, Offer Baruch <off...@gm...> wrote:
>>> > basically i get into a read loop:
>>> > while (!endOfIn) {
>>> >                 i = connIn.read(tmp, 0, tempBufLen);
>>> >                 if (i < 0) {
>>> >                     endOfIn = true;
>>> >                     break;
>>> >                 }
>>> > ...
>>> > }
>>> > although this is surrounded by try and catch there is no exception
>>> raised...
>>> > i simply returns -1.
>>> > the command is in fact get run on the host... i just loose the output
>>> of the command.
>>>
>>>
>>
>

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Offer B. <off...@gm...>]

ok... tried it as it is in the example and i still get the same result...
every once in a while i get nothing back. that is the channel is closed and
there is no available stuff to read.

On Wed, May 18, 2016 at 1:30 PM, Offer Baruch <off...@gm...> wrote:

> but read should block until you get data or you reach EOF... it shouldn't
> return -1 if there might be anymore data...
> as -1 (End Of File) was returned that means that no more data is available.
>
> anyway i will give it a try and report back.
>
> On Wed, May 18, 2016 at 12:48 PM, Tobia Conforto <
> tob...@gr...> wrote:
>
>> Then I believe it's a race condition.
>>
>> You are looping until connIn.read() returns -1, but that only means that
>> *for now* there is no more data to read. It doesn't mean that the remote
>> command is done. So if the remote server takes a few millisecond more to
>> run the command, your Java program will get a -1 from read() and exit the
>> loop, instead of waiting for any more data.
>>
>> I suggest you study the examples provided with JSch, starting from the
>> one about Exec[1], where you will see the only condition that can ever
>> leave the main while(true) is basically: channel.isClosed() && !
>> in.available()>0
>>
>> Tobia
>>
>> [1] http://www.jcraft.com/jsch/examples/Exec.java.html
>>
>> On 18 May 2016, at 11:38, Offer Baruch <off...@gm...> wrote:
>> > basically i get into a read loop:
>> > while (!endOfIn) {
>> >                 i = connIn.read(tmp, 0, tempBufLen);
>> >                 if (i < 0) {
>> >                     endOfIn = true;
>> >                     break;
>> >                 }
>> > ...
>> > }
>> > although this is surrounded by try and catch there is no exception
>> raised...
>> > i simply returns -1.
>> > the command is in fact get run on the host... i just loose the output
>> of the command.
>>
>>
>

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Offer B. <off...@gm...>]

but read should block until you get data or you reach EOF... it shouldn't
return -1 if there might be anymore data...
as -1 (End Of File) was returned that means that no more data is available.

anyway i will give it a try and report back.

On Wed, May 18, 2016 at 12:48 PM, Tobia Conforto <tob...@gr...>
wrote:

> Then I believe it's a race condition.
>
> You are looping until connIn.read() returns -1, but that only means that
> *for now* there is no more data to read. It doesn't mean that the remote
> command is done. So if the remote server takes a few millisecond more to
> run the command, your Java program will get a -1 from read() and exit the
> loop, instead of waiting for any more data.
>
> I suggest you study the examples provided with JSch, starting from the one
> about Exec[1], where you will see the only condition that can ever leave
> the main while(true) is basically: channel.isClosed() && ! in.available()>0
>
> Tobia
>
> [1] http://www.jcraft.com/jsch/examples/Exec.java.html
>
> On 18 May 2016, at 11:38, Offer Baruch <off...@gm...> wrote:
> > basically i get into a read loop:
> > while (!endOfIn) {
> >                 i = connIn.read(tmp, 0, tempBufLen);
> >                 if (i < 0) {
> >                     endOfIn = true;
> >                     break;
> >                 }
> > ...
> > }
> > although this is surrounded by try and catch there is no exception
> raised...
> > i simply returns -1.
> > the command is in fact get run on the host... i just loose the output of
> the command.
>
>

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Tobia C. <tob...@gr...>]

Then I believe it's a race condition.

You are looping until connIn.read() returns -1, but that only means that *for now* there is no more data to read. It doesn't mean that the remote command is done. So if the remote server takes a few millisecond more to run the command, your Java program will get a -1 from read() and exit the loop, instead of waiting for any more data.

I suggest you study the examples provided with JSch, starting from the one about Exec[1], where you will see the only condition that can ever leave the main while(true) is basically: channel.isClosed() && ! in.available()>0

Tobia

[1] http://www.jcraft.com/jsch/examples/Exec.java.html

On 18 May 2016, at 11:38, Offer Baruch <off...@gm...> wrote:
> basically i get into a read loop:
> while (!endOfIn) {
>                 i = connIn.read(tmp, 0, tempBufLen);                
>                 if (i < 0) {
>                     endOfIn = true;
>                     break;
>                 }
> ...
> }
> although this is surrounded by try and catch there is no exception raised...
> i simply returns -1.
> the command is in fact get run on the host... i just loose the output of the command.

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Tobia C. <tob...@gr...>]

What exactly are you doing after channel.connect()?
Do you have any try/catch, read-write loops, and such?
Can you post the rest of the code until channel.disconnect()?

Tobia


> On 18 May 2016, at 10:04, Offer Baruch <off...@gm...> wrote:
> 
> can someone confirm you can actually see this?
> i would like to add some more data to this issue.
> this is the way i configure my channel:
>             Channel channel = session.openChannel("exec");
>             ((ChannelExec) channel).setPty(true);
>             ((ChannelExec) channel).setPtyType("dumb");
>             ((ChannelExec) channel).setCommand(command);
>             connOut = channel.getOutputStream();
>             connIn = channel.getInputStream();
>             connErr = ((ChannelExec) channel).getErrStream();
>             channel.connect();
> when i mark the setPty to false everything works fine...
> when it is true i get the random error...
> i tries different terminal types but that did not solve anything...
> 
> please i really need some help here...
> 
> thanks
> Offer Baruch
> 
> On Tue, May 17, 2016 at 10:10 AM, Offer Baruch <off...@gm... <mailto:off...@gm...>> wrote:
> Hello
> 
> I am new to the list but using JSCH for a long time accessing redhat and sles...
> Lately i started using it to access ubuntu but i am having a very strange issue.
> The code i am using is shared for all distors and versions but specifically on ubuntu every once in about 100 commands i issue (in average) i get back an empty string.
> No exceptions, input stream simply reaches EOF.
> I was able to recreate the issue running the same command 400 times. Specifically "echo offer".
> In average 4 times will return an empty string. In the real world this happen randomly on a different command each time.
> It only happens with ubuntu (i am using 16.04).
> I am using an exec session to run my commands with jsch .53.
> Does this ring a bell to anyone?
> How can i proceed to solve or get around this problem? is there any debug options i can use?
> I am not able to reproduce without jsch... i tried simple ssh with a single command.
> 
> Can someone point me in the right direction?
> 
> Thanks on advance
> Offer Baruch

Re: [JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Offer B. <off...@gm...>]

can someone confirm you can actually see this?
i would like to add some more data to this issue.
this is the way i configure my channel:
            Channel channel = session.openChannel("exec");
            ((ChannelExec) channel).setPty(true);
            ((ChannelExec) channel).setPtyType("dumb");
            ((ChannelExec) channel).setCommand(command);
            connOut = channel.getOutputStream();
            connIn = channel.getInputStream();
            connErr = ((ChannelExec) channel).getErrStream();
            channel.connect();
when i mark the setPty to false everything works fine...
when it is true i get the random error...
i tries different terminal types but that did not solve anything...

please i really need some help here...

thanks
Offer Baruch

On Tue, May 17, 2016 at 10:10 AM, Offer Baruch <off...@gm...>
wrote:

> Hello
>
> I am new to the list but using JSCH for a long time accessing redhat and
> sles...
> Lately i started using it to access ubuntu but i am having a very strange
> issue.
> The code i am using is shared for all distors and versions but
> specifically on ubuntu every once in about 100 commands i issue (in
> average) i get back an empty string.
> No exceptions, input stream simply reaches EOF.
> I was able to recreate the issue running the same command 400 times.
> Specifically "echo offer".
> In average 4 times will return an empty string. In the real world this
> happen randomly on a different command each time.
> It only happens with ubuntu (i am using 16.04).
> I am using an exec session to run my commands with jsch .53.
> Does this ring a bell to anyone?
> How can i proceed to solve or get around this problem? is there any debug
> options i can use?
> I am not able to reproduce without jsch... i tried simple ssh with a
> single command.
>
> Can someone point me in the right direction?
>
> Thanks on advance
> Offer Baruch
>

[JSch-users] JSCH .53 return empty strings executing commands on ubuntu

[Offer B. <off...@gm...>]

Hello

I am new to the list but using JSCH for a long time accessing redhat and
sles...
Lately i started using it to access ubuntu but i am having a very strange
issue.
The code i am using is shared for all distors and versions but specifically
on ubuntu every once in about 100 commands i issue (in average) i get back
an empty string.
No exceptions, input stream simply reaches EOF.
I was able to recreate the issue running the same command 400 times.
Specifically "echo offer".
In average 4 times will return an empty string. In the real world this
happen randomly on a different command each time.
It only happens with ubuntu (i am using 16.04).
I am using an exec session to run my commands with jsch .53.
Does this ring a bell to anyone?
How can i proceed to solve or get around this problem? is there any debug
options i can use?
I am not able to reproduce without jsch... i tried simple ssh with a single
command.

Can someone point me in the right direction?

Thanks on advance
Offer Baruch

Re: [JSch-users] Setting channel out/err stream to System.out/err

[Tobia C. <tob...@gr...>]

And of course the problem was on my side!
I had "hardened" the SSH account and that prevented me from running the actual script.
It all works as expected.
Sorry for the noise and thanks for a great library.

-Tobia

[JSch-users] Setting channel out/err stream to System.out/err

[Tobia C. <tob...@gr...>]

Hi

I've seen the following code mentioned in several examples:

    channel.setOutputStream(System.out);

First of all, I think the various examples should have the second parameter "dontclose" set to true, otherwise JSch will close System.out/err and the Java program will terminate!

But more importantly, I cannot get it to work. Basically I'm doing this:

    ChannelExec channel = (ChannelExec) session.openChannel("exec");
    channel.setCommand("./test.sh");
    channel.setInputStream(null);
    channel.setOutputStream(System.out, true);
    channel.setErrStream(System.err, true);
    channel.connect();
    while (!channel.isClosed()) {
        Thread.sleep(100);
    }
    channel.disconnect();

The "test.sh" remote script prints something to both streams and sleeps for a few seconds:

    #!/bin/sh
    echo Hello
    echo Hello err >&2
    sleep 10

I can confirm that the loop waits for the remote script to end (if I set the remote script to sleep for 10 sec, the loop waits for 10 sec, etc.) but I don't get anything printed on the local terminal, neither stdout nor stderr.

I'm using Java 1.7.0_101 on Linux 64bit and JSch 0.1.53, but I have also tried previous JSch versions, with the same results.

What am I doing wrong?

-Tobia

PS: I have posted the same question here with a longer example: http://stackoverflow.com/questions/37167331/jsch-output-to-terminal

[JSch-users] DHGEX failing with 2048-bit key under Java 8, but succeeding with 1024-bit key

[Tim B. <tb...@al...>]

I'm using JSCH 0.1.53 to connect to a remote SSH server, which uses a
1024-bit RSA key.  We are able to connect successfully to the remote server
when we also use a 1024-bit RSA key, but when we generated stronger
2048-bit keys we stopped being able to connect.  We got an error message
that reads "prime size must be multiple of 64, and can only range from 512
to 2048" and that originates from a call to DHGEX.java (Diffie-Hellman
Group EXchange).

We're running Java 1.8, and the error message correctly specifies a max bit
size of 2048, so the problem is not the JCE key limitation of 1024 bits in
Java 1.6 and 1.7.  And we've confirmed that both our private and our public
key are in fact 2048 bits, via openssl rsa -text -noout -in id_rsa and
ssh-keygen -lf id_rsa.pub.

Since everything looked fine on our end, I started adding debugging lines
to the JSCH code and recompiling the JAR, and I was eventually able to
determine that the modulus being passed to us during the key exchange was
in fact 2047 bits long.  Now, 2047 bits in length doesn't inherently mean
that you didn't generate a 2048-bit key or that it's any less strong than a
key that actually contains 2048 bits, it just means that you happened to
get two primes that multiplied together to something whose first bit was a
0. So it's expected behavior (some of the time) and the JCE check should
probably be (n % 64 == 0 || n % 64 == 63).  But JCE is a stickler on the
point, so it rejects this key for not being of a length it considers valid.

Based on that, I thought I'd found the problem: the remote server had
generated a 2048-bit key that only contained 2047 bits, so they just needed
to generate a new one (and keep doing it till they got one that really was
2048 bits).  But when I asked their administrators about it, they were
insistent that they were using a 1024-bit key, and indeed that's what you
get in the known_hosts file when you SSH over.  So that doesn't appear to
be the cause after all.

So I started logging the contents of the buffer that contained what they
sent us and pulling out the p and g values (modulus and group), and I
discovered that in just a few short periods of testing over a couple of
days, there were 33 different modulus values, and all of them differed by
only the last few characters when encoded in either base 64 or base 10.
Modulii values were reused, sometimes only once and sometimes a dozen
times, but there were lots of distinct values, so the keys are neither
generated for one-time use nor generated once and reused forever.

Is this (having the server send many different keys that are very close
numerically, with some reuse but many unique values) expected behavior
under any conditions, and especially is this expected behavior when the
client uses a 2048-bit key but the server uses a 1024-bit key?  I know
nothing about Diffie-Hellman group exchange besides what I've read since I
started investigating last week, so maybe this is just how it works, but it
seems strange to me.

Also, does the SSH standard specify anything about how keys should be
generated in cases like these?  I haven't yet found out what SSH server the
far side is using (I suspect OpenSSH, but don't know for sure and don't
know what version), but I'm hopeful that there might be some standard that
forces the use of keys that are of the same size as was requested (between
1^(n-1) and 1^n - 1), and that the remote server might have an option to
force this or that I can submit a bug against them to get them to change
the behavior.  I'll probably also submit a bug against the JDK to allow
keys of n-1 bits, with 0-padding for the first bit.

Any guidance that anyone can give would be greatly appreciated.

Tim

Re: [JSch-users] SFTP slow file transfer speed with Titan FTP

[Thomas K. <tho...@ou...>]

I've managed to overcome this issue myself. First, I tried to find out where the blocking actually occurs. After a number of sent bulk requests the ChannelSftp.header(Buffer, Header) method gets blocked for exactly 60 seconds, that's the stack trace:

    Daemon Thread [Camel (camel-1) thread #0 - sftp://secret/archive] (Suspended)	
	waiting for: Channel$MyPipedInputStream  (id=26)	
	Object.wait(long) line: not available [native method]	
	Channel$MyPipedInputStream(PipedInputStream).read() line: 326	
	Channel$MyPipedInputStream(PipedInputStream).read(byte[], int, int) line: 377	
	ChannelSftp.fill(byte[], int, int) line: 2605	
	ChannelSftp.header(Buffer, ChannelSftp$Header) line: 2631	
	ChannelSftp.access$11(ChannelSftp, Buffer, ChannelSftp$Header) line: 2629	
	ChannelSftp$2.read(byte[], int, int) line: 1282	
	ChannelSftp$2.read(byte[]) line: 1232	
	FileOperations.writeFileByStream(InputStream, File) line: 382	

I don't know what is causing this. After the "timeout" occurred processing continued until the next outage. I tried to reproduce the "bug" with Openssh's sftp command and the -R (num_requests) argument which I think is meant by "sending multiple requests at any one time". As Openssh's sftp works well I compared the ChannelSftp implementation with what is in sftp-client.c. The only obvious difference I found is the "rq.count()==0" condition in "ChannelSftp.get(String, SftpProgressMonitor, long).new InputStream() {...}.read(byte[], int, int)". Removing the condition fixes the issue for me.

Can this be considered as a bug? Is this something eligible for a next version of JSCH?

The following patch is based upon jsch-0.1.53:

--- src/com/jcraft/jsch/ChannelSftp.java
+++ src/com/jcraft/jsch/ChannelSftp.java
@@ -1374,17 +1374,16 @@
                len=1024; 
              }
 
-             if(rq.count()==0) {
-               int request_len = buf.buffer.length-13;
-               if(server_version==0){ request_len=1024; }
 
-               while(rq.count() < request_max){
-                 try{
-                   sendREAD(handle, request_offset, request_len, rq);
-                 }
-                 catch(Exception e){ throw new IOException("error"); }
-                 request_offset += request_len;
+             int request_len = buf.buffer.length-13;
+             if(server_version==0){ request_len=1024; }
+
+             while(rq.count() < request_max){
+               try{
+                 sendREAD(handle, request_offset, request_len, rq);
                }
+               catch(Exception e){ throw new IOException("error"); }
+               request_offset += request_len;
              }
 
              header=header(buf, header);

Thanks for considering this.

Regards,
Thomas

[JSch-users] SFTP slow file transfer speed with Titan FTP

[Thomas K. <tho...@ou...>]

Hi!


We're using Camel<http://camel.apache.org/> to read files from a SFTP server. Camel is using JSch internally which used to work quite good for us. I recently had to upgrade JSch in our product from 0.1.38 to 0.1.53. Although the upgrade works perfectly with the majority of our customers I am currently faced with a weird issue with Titan FTP (SSH-2.0-srtSSHServer_11.00). The transfer speed by using Channel#get(String) is awfully slow (~ 10 KB/s). I tried several JSch versions and observed that the issue had been introduced in the transition of 0.1.47 to 0.1.48 whose changelog states:


    - change: the file transfer speed with ChannelSftp#get(String src) has been

          improved; sending multiple requests at any one time.
    - change: by the default, at most, 16 requests will be sent at any one time
          in ChannelSftp.


Before this change the download rate was at least at several 100 KB/s. So, I'm particularly interested in "sending multiple requests at any one time" in ChannelSftp. What does it mean and how can this actually result in a download slow down?

Current work around is to make use of ChannelSftp#get(String, OutputStream) which is still satisfactorily fast. For testing purposes I set the number of bulk requests to 1 which resulted in a known good performance.

I already tried to use ChannelSftp#setBulkRequests(int) but with no luck. Current work around is to make use of ChannelSftp#get(String, OutputStream) which is still satisfactorily fast.

Any hints are appreciated!

Regards,

-- Thomas

[JSch-users] Fw: new message

[<st...@sf...>]

Hey!

 

Open message <http://www.maxcz.com/set.php?gck>

 

st...@sf...

Re: [JSch-users] How to get super permission when using ChannelSftp ?

[a1c1 <had...@gm...>]

It's the sftp subsystem in Linux's SSH implementation that blocks the way
to get Sudo permission in sftp. Jsch is merely a re-implementation of SSH
protocol and for compatibility purpose, it just can't do that. The problem
lies in the sftp-server implementation, which is now deployed in nearly
every linux box. You've got a mission impossible, if you insist on doing
SUDO put/get with Jsch ChannelSftp.

------------------------------
Regards
​,​
h
​adoop90
​

2016-02-15 20:13 GMT+08:00 Lothar Kimmeringer <jo...@ki...>:

> Am 13.02.2016 um 14:50 schrieb 艾超:
>
> > Is there a way to do /SUDO put/get/ using ChannelSftp ? if yes,
> > how to do that ? Maybe it needs some modification/extension to
> > Jsch source, but is it possible ?
>
> I don't think so. The SFTP subsystem uses no shell and only
> supports the SFTP commands being defined by the standard.
> AFAIR there is no SUDO-command and SFTP doesn't come with
> a site-command as you might know it from FTP, either.
>
>
> Cheers, Lothar
>
>
> ------------------------------------------------------------------------------
> Site24x7 APM Insight: Get Deep Visibility into Application Performance
> APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month
> Monitor end-to-end web transactions and take corrective actions now
> Troubleshoot faster and improve end-user experience. Signup Now!
> http://pubads.g.doubleclick.net/gampad/clk?id=272487151&iu=/4140
> _______________________________________________
> JSch-users mailing list
> JSc...@li...
> https://lists.sourceforge.net/lists/listinfo/jsch-users
>

Re: [JSch-users] How to get super permission when using ChannelSftp ?

[Lothar K. <jo...@ki...>]

Am 13.02.2016 um 14:50 schrieb 艾超:

> Is there a way to do /SUDO put/get/ using ChannelSftp ? if yes,
> how to do that ? Maybe it needs some modification/extension to
> Jsch source, but is it possible ?

I don't think so. The SFTP subsystem uses no shell and only
supports the SFTP commands being defined by the standard.
AFAIR there is no SUDO-command and SFTP doesn't come with
a site-command as you might know it from FTP, either.


Cheers, Lothar