Thread: RE: [JSch-users] SFTP downloads are faster than uploads
Status: Alpha
Brought to you by:
ymnk
From: Tamir G. <ta...@ta...> - 2006-02-16 14:04:19
|
Hello All, Here is some more info: I'm transferring a test file of size 80MB. Downloading the file using ChannelSFTP takes ~30sec. Uploading the file using ChannelSFTP takes ~180sec. Downloading the file using JSch's SCP example takes ~20sec. Uploading the file using JSch's SCP example takes ~20sec. Downloading the file using SecureCRT's SFTP takes ~10sec. Uploading the file using SecureCRT's SFTP takes ~10sec. As you can see, SFTP upload using ChannelSFTP is significantly slower = than any other transfer method. Am I the only one who experiences such issue with upload? Thanks in advance, --- Tamir Gal http://www.tamirgal.com/home/dev.aspx?Item=3DSharpSsh =20 -----Original Message----- From: Tamir Gal [mailto:ta...@ta...]=20 Sent: Monday, February 13, 2006 23:06 To: 'B. Scott Smith' Cc: 'jsc...@li...' Subject: RE: [JSch-users] SFTP downloads are faster than uploads Yes I am. The two machines are connected on the same switch. ________________________________________ From: jsc...@li... [mailto:jsc...@li...] On Behalf Of B. Scott = Smith Sent: Monday, February 13, 2006 22:33 To: Tamir Gal Cc: jsc...@li... Subject: Re: [JSch-users] SFTP downloads are faster than uploads After performing some tests with ChannelSFTP I noticed that all SFTP downloads are about 3 times faster than SFTP uploads. Is this a bug? =A0 Are you sure your download bandwidth isn't 3 times faster than your = upload bandwidth? |
From: <ym...@jc...> - 2006-02-16 14:15:13
|
Hi, +-From: "Tamir Gal" <ta...@ta...> -- |_Date: Thu, 16 Feb 2006 16:03:23 +0200 ___ | |I'm transferring a test file of size 80MB. |Downloading the file using ChannelSFTP takes ~30sec. |Uploading the file using ChannelSFTP takes ~180sec. ... |As you can see, SFTP upload using ChannelSFTP is significantly slower than |any other transfer method. |Am I the only one who experiences such issue with upload? I have also found that upload using ChannelSftp is really slower, but I have not found the reason why it is. I'll check the code. Thank you for your feedback. Sincerely, -- Atsuhiko Yamanaka JCraft,Inc. 4-1 OHMACHI 1-CHOME AOBA-KU, SENDAI, MIYAGI 980-0804 Japan. Tel +81-22-723-2150 +1-415-578-3454 Fax +81-22-224-8773 Skype callto://jcraft/ |
From: Al S. <as...@ar...> - 2006-02-16 15:11:45
|
Sorry, I hit the send by accident when I was half way through an edit, so the email has a few mistakes. The main one being that the using of run time as opposed to invocation counts. The 1,333 times and 6,639 times should have been; SFTP - Spends 6,639ms in the method (not 6,639 invocations) ScpTo - Spends 4,283ms in the method (1,333 invocations should have been deleted). The other noticeable thing is the read time; ScpTo - spends 200ms reading SFTP - spends 10,064ms reading My advice is to look at the protocol and whats being read, you may find that the data being passed to the encryption method for SFTP can be better optimized (i.e. align it with block boundaries for the cipher), and have a look at whats being read, it may be the case that more data can be sent out before needing to wait on a read. Sorry for the mistakes. Al. Al Sutton wrote: > Maybe I can shed some light onto this. > > I've run the ScpTo and SFTP demo programs through a profiler whilst > transferring the same 10MB file over a clean network (i.e. I have > control total control of it and know that there is no other > significant traffic). The transfer was from a 1GB Athlon64 3700+ > Windows XP (32 bit) machine to a 256MB P4 2.8GHz SuSE x86_64 machine. > > The ScpTo program calls the TripleDESCBC.update method 1,333 times, > the SFTP method calls the same method 6,639 times. From experience I > know that a near 5 fold increase in calls to a method will hurt > performance, and I would guess that there is some overhead in each > call of the method which is degrading things further. > > I've attached the call trees for those who are interested. > > Regards, > > Al. > --- > Al Sutton > Argosy TelCrest Ltd. > www.argosytelcrest.com > > > Atsuhiko Yamanaka wrote: >> Hi, >> >> +-From: "Tamir Gal" <ta...@ta...> -- >> |_Date: Thu, 16 Feb 2006 16:03:23 +0200 ___ >> | >> |I'm transferring a test file of size 80MB. >> |Downloading the file using ChannelSFTP takes ~30sec. >> |Uploading the file using ChannelSFTP takes ~180sec. >> ... >> |As you can see, SFTP upload using ChannelSFTP is significantly >> slower than >> |any other transfer method. >> |Am I the only one who experiences such issue with upload? >> >> I have also found that upload using ChannelSftp is really slower, >> but I have not found the reason why it is. >> I'll check the code. >> Thank you for your feedback. >> >> >> Sincerely, >> -- >> Atsuhiko Yamanaka >> JCraft,Inc. >> 4-1 OHMACHI 1-CHOME AOBA-KU, >> SENDAI, MIYAGI 980-0804 Japan. >> Tel +81-22-723-2150 >> +1-415-578-3454 >> Fax +81-22-224-8773 >> Skype callto://jcraft/ >> >> >> ------------------------------------------------------- >> This SF.net email is sponsored by: Splunk Inc. Do you grep through >> log files >> for problems? Stop! Download the new AJAX search engine that makes >> searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! >> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 >> _______________________________________________ >> JSch-users mailing list >> JSc...@li... >> https://lists.sourceforge.net/lists/listinfo/jsch-users >> >> > |
From: Eric M. <me...@cs...> - 2006-02-16 16:57:27
|
I did a quick profiling in NetBeans of the get and put methods and Al's advice is right on. Of the total time spent sending and getting the same files, 79.2% is spent in the TripleDESCBC.update method while sending the files. If you are interested in seeing the results, I placed a screen-shot at http://www.cs.utk.edu/~meek/images/jschProfile.png Eric Al Sutton wrote: > Sorry, I hit the send by accident when I was half way through an edit, > so the email has a few mistakes. > > The main one being that the using of run time as opposed to invocation > counts. The 1,333 times and 6,639 times should have been; > > SFTP - Spends 6,639ms in the method (not 6,639 invocations) > ScpTo - Spends 4,283ms in the method (1,333 invocations should have > been deleted). > > The other noticeable thing is the read time; > > ScpTo - spends 200ms reading > SFTP - spends 10,064ms reading > > My advice is to look at the protocol and whats being read, you may > find that the data being passed to the encryption method for SFTP can > be better optimized (i.e. align it with block boundaries for the > cipher), and have a look at whats being read, it may be the case that > more data can be sent out before needing to wait on a read. > > Sorry for the mistakes. > > Al. > > > > Al Sutton wrote: >> Maybe I can shed some light onto this. >> >> I've run the ScpTo and SFTP demo programs through a profiler whilst >> transferring the same 10MB file over a clean network (i.e. I have >> control total control of it and know that there is no other >> significant traffic). The transfer was from a 1GB Athlon64 3700+ >> Windows XP (32 bit) machine to a 256MB P4 2.8GHz SuSE x86_64 machine. >> >> The ScpTo program calls the TripleDESCBC.update method 1,333 times, >> the SFTP method calls the same method 6,639 times. From experience I >> know that a near 5 fold increase in calls to a method will hurt >> performance, and I would guess that there is some overhead in each >> call of the method which is degrading things further. >> >> I've attached the call trees for those who are interested. >> >> Regards, >> >> Al. >> --- >> Al Sutton >> Argosy TelCrest Ltd. >> www.argosytelcrest.com >> >> >> Atsuhiko Yamanaka wrote: >>> Hi, >>> >>> +-From: "Tamir Gal" <ta...@ta...> -- >>> |_Date: Thu, 16 Feb 2006 16:03:23 +0200 ___ >>> | >>> |I'm transferring a test file of size 80MB. >>> |Downloading the file using ChannelSFTP takes ~30sec. >>> |Uploading the file using ChannelSFTP takes ~180sec. >>> ... >>> |As you can see, SFTP upload using ChannelSFTP is significantly >>> slower than >>> |any other transfer method. >>> |Am I the only one who experiences such issue with upload? >>> >>> I have also found that upload using ChannelSftp is really slower, >>> but I have not found the reason why it is. >>> I'll check the code. >>> Thank you for your feedback. >>> >>> >>> Sincerely, >>> -- >>> Atsuhiko Yamanaka >>> JCraft,Inc. >>> 4-1 OHMACHI 1-CHOME AOBA-KU, >>> SENDAI, MIYAGI 980-0804 Japan. >>> Tel +81-22-723-2150 >>> +1-415-578-3454 >>> Fax +81-22-224-8773 >>> Skype callto://jcraft/ >>> >>> >>> ------------------------------------------------------- >>> This SF.net email is sponsored by: Splunk Inc. Do you grep through >>> log files >>> for problems? Stop! Download the new AJAX search engine that makes >>> searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! >>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 >>> >>> _______________________________________________ >>> JSch-users mailing list >>> JSc...@li... >>> https://lists.sourceforge.net/lists/listinfo/jsch-users >>> >>> >> > > > > ------------------------------------------------------- > This SF.net email is sponsored by: Splunk Inc. Do you grep through log > files > for problems? Stop! Download the new AJAX search engine that makes > searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 > _______________________________________________ > JSch-users mailing list > JSc...@li... > https://lists.sourceforge.net/lists/listinfo/jsch-users |
From: <ym...@jc...> - 2006-02-18 13:57:44
|
Hi, I tried to modify the code. Can you try the following patch, http://www.jcraft.com/jsch/jsch-0.1.25-sftp.patch In previous implementation, for every WRITE requests, it had waited for ack, but that patch will asynchronously read acks. Sincerely, -- Atsuhiko Yamanaka JCraft,Inc. 4-1 OHMACHI 1-CHOME AOBA-KU, SENDAI, MIYAGI 980-0804 Japan. Tel +81-22-723-2150 +1-415-578-3454 Fax +81-22-224-8773 Skype callto://jcraft/ |
From: Tamir G. <ta...@ta...> - 2006-02-20 13:17:12
|
Hi Atsuhiko, Sorry for my late response, I will only be able to try the patch tomorrow. Before I try it, I have another interesting detail about this issue. I could only reproduce the slow upload issue on OpenSSH sshd. On other sshd servers (such as freeSSHd for Windows) I didn't see any upload speed issues. -----Original Message----- From: jsc...@li... [mailto:jsc...@li...] On Behalf Of Atsuhiko Yamanaka Sent: Saturday, February 18, 2006 15:39 To: as...@ar... Cc: jsc...@li...; ta...@ta... Subject: Re: [JSch-users] SFTP downloads are faster than uploads Hi, I tried to modify the code. Can you try the following patch, http://www.jcraft.com/jsch/jsch-0.1.25-sftp.patch In previous implementation, for every WRITE requests, it had waited for ack, but that patch will asynchronously read acks. Sincerely, -- Atsuhiko Yamanaka JCraft,Inc. 4-1 OHMACHI 1-CHOME AOBA-KU, SENDAI, MIYAGI 980-0804 Japan. Tel +81-22-723-2150 +1-415-578-3454 Fax +81-22-224-8773 Skype callto://jcraft/ ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 _______________________________________________ JSch-users mailing list JSc...@li... https://lists.sourceforge.net/lists/listinfo/jsch-users |
From: Tamir G. <ta...@ta...> - 2006-02-22 08:14:37
|
I applied the patch and upload speed issues seam to be fixed. Thanks! T. -----Original Message----- From: jsc...@li... [mailto:jsc...@li...] On Behalf Of Atsuhiko Yamanaka Sent: Saturday, February 18, 2006 15:39 To: as...@ar... Cc: jsc...@li...; ta...@ta... Subject: Re: [JSch-users] SFTP downloads are faster than uploads Hi, I tried to modify the code. Can you try the following patch, http://www.jcraft.com/jsch/jsch-0.1.25-sftp.patch In previous implementation, for every WRITE requests, it had waited for ack, but that patch will asynchronously read acks. Sincerely, -- Atsuhiko Yamanaka JCraft,Inc. 4-1 OHMACHI 1-CHOME AOBA-KU, SENDAI, MIYAGI 980-0804 Japan. Tel +81-22-723-2150 +1-415-578-3454 Fax +81-22-224-8773 Skype callto://jcraft/ ------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Do you grep through log files for problems? Stop! Download the new AJAX search engine that makes searching your log files as easy as surfing the web. DOWNLOAD SPLUNK! http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642 _______________________________________________ JSch-users mailing list JSc...@li... https://lists.sourceforge.net/lists/listinfo/jsch-users |