Recent changes to Home

Discussion for Home page

Chris Rapier — Fri, 18 Jun 2021 15:10:26 -0000

It's been submitted over the years but for various reasons it was rejected by the OpenSSH team. In large part it's due to the fact that it's a relatively large patch and would require more work to maintain. They have also stated that it doesn't provide enough value to enough people to make the effort worthwhile. I understand their reasoning even if I don't agree with it.

Discussion for Home page

bustedsecret — Thu, 17 Jun 2021 14:19:35 -0000

is there any reason to not have this work included in the openssh project upstream ?

Home modified by Chris Rapier

Chris Rapier — Wed, 20 Nov 2019 16:15:04 -0000

--- v4
+++ v5
@@ -1,3 +1,5 @@
+[News and Updates Are Here](https://sourceforge.net/p/hpnssh/wiki/News%20and%20Updates/)
+
 HPN-SSH is a series of patches for OpenSSH to provide improved performance and functionality. The core of HPN-SSH is the implementation of a dynamic receive side buffer. By allowing this buffer to grow with the TCP receive buffer it's possible to attain significant throughput improvement - up to two orders of magnitude depending on the network path. Patch sets for each OpenSSH release will be found in the appropriate subfolder.

 Why does this work? OpenSSH implements SSHv2 - a multiplexed protocol that allows for multiple data connections across a single TCP stream. In order to provide necessary flow control for the individual channels that comprise this multiplexed connection it was necessary to, essentially, reimplement sliding windows on the receive side. This means that only a set amount of data can be outstanding (in transit) between the sender and receiver at any one time. The OpenSSH development team initially had this limited to 64KB (now 1024KB). Over low latency paths (and for interactive sessions) this is perfectly reasonable. However, for the transfer of large files this proves to be a significant bottleneck. This bottleneck was common to many TCP applications prior to the introduction of autotuning receive buffers in the TCP stack.

Home modified by Chris Rapier

Chris Rapier — Wed, 13 Nov 2019 20:11:17 -0000

--- v3
+++ v4
@@ -4,7 +4,7 @@

 HPN-SSH brings the current TCP receive buffer size up to the application layer and, by doing so, uses the network far more efficiently. 

-The HPN-SSH team (Ben Bennet and Mike Tasota) also developed a multi-threaded variant of the AES-CTR cipher so as to allow multicored systems to distribute the burden of computing the keystream over multiple cores. This enhancement produces a cipher stream that is indistinguishable from the default AES-CTR cipher stream. The upshot of this being that it is backwards compliant with all existing AES-CTR implementations - no need to have the multithreaded variant on both sides of the connection. Recent work has also eliminated problems with using the multi-threaded AES-CTR cipher in forked connections (common with forwarded ports) and the sandboxed privsep process. 
+The HPN-SSH team (Ben Bennet and Mike Tasota) also developed a multi-threaded variant of the AES-CTR cipher so as to allow multicored systems to distribute the burden of computing the keystream over multiple cores. This enhancement produces a cipher stream that is indistinguishable from the default AES-CTR cipher stream. The upshot of this being that it is backwards compliant with all existing AES-CTR implementations - no need to have the multithreaded variant on both sides of the connection. Recent work has also eliminated problems with using the multi-threaded AES-CTR cipher in forked connections (common with forwarded ports) and the sandboxed privsep process. In the 14.15 release a minimum of 4 threads will be created. However in systems with more than 4 cores 2 additional threads will be created for each additional pair of cores over 4. In other words: 4 cores = 4 threads, 5 cores = 4 threads, 6 cores = 6 threads, up to a total of 16 threads. This is dependent on having a linux(sysconf) or BSD(sysctl) compliant method of determing the number of CPU cores programmatically. If not the istance will be limited to 4 threads. 

 For less sensitive data the team (primarily Micahel Stevens) also developed a method to switch to the NONE cipher after a fully cryptographically secure authentication. This method forces the stream to rekey itself after authentication and only proposes the NONE cipher. This patch may significantly improve performance for users whose processes were previously CPU bound. As the NONE cipher is essentially disallowed outside of debugging in OpenSSH users wishing to make use of the NONE cipher must have this patch installed on both sides of the connection. As an aside, the multi-threaded AES-CTR cipher and the NONE cipher exhibit similar throughput enhancement albeit, in the case of the AES-CTR cipher, of significantly higher CPU load. 

@@ -12,7 +12,7 @@

 Lastly, the team also created patches to provide for enhanced server side logging and instantaneous throughput rate reporting in SCP. 

-This current patch set, HPN14v5, passes all regression tests, provides a high level of security, and significantly increased performance with minimal fuss.
+This current patch set, HPN14v20 for OpenSSH 8.1p1, passes all regression tests, provides a high level of security, and significantly increased performance with minimal fuss.
 [[project_screenshots]]
 [[project_admins]]
 [[download_button]]

Discussion for Home page

Chris Rapier — Tue, 27 May 2014 21:28:59 -0000

Note: There was a minor problem in the main patch set (the Kitchen Sink patch). Basically, I failed to pull forward a commit correctly and maintained some typos form a previous version. As of Tuesday, May 27 2014 @ 5:27pm EDT this has been resolved an fixed version of the patch is in place.

Home modified by Chris Rapier

Chris Rapier — Thu, 22 May 2014 21:11:20 -0000

--- v2
+++ v3
@@ -12,7 +12,7 @@

 Lastly, the team also created patches to provide for enhanced server side logging and instantaneous throughput rate reporting in SCP. 

-This current patch set, HPN14v1, passes all regression tests, provides a high level of security, and significantly increased performance with minimal fuss.
+This current patch set, HPN14v5, passes all regression tests, provides a high level of security, and significantly increased performance with minimal fuss.
 [[project_screenshots]]
 [[project_admins]]
 [[download_button]]

Home modified by Chris Rapier

Chris Rapier — Mon, 19 Aug 2013 17:50:31 -0000

--- v1
+++ v2
@@ -1,8 +1,18 @@
-Welcome to your wiki!
+HPN-SSH is a series of patches for OpenSSH to provide improved performance and functionality. The core of HPN-SSH is the implementation of a dynamic receive side buffer. By allowing this buffer to grow with the TCP receive buffer it's possible to attain significant throughput improvement - up to two orders of magnitude depending on the network path. Patch sets for each OpenSSH release will be found in the appropriate subfolder.

-This is the default page, edit it as you see fit. To add a new page simply reference it within brackets, e.g.: [SamplePage].
+Why does this work? OpenSSH implements SSHv2 - a multiplexed protocol that allows for multiple data connections across a single TCP stream. In order to provide necessary flow control for the individual channels that comprise this multiplexed connection it was necessary to, essentially, reimplement sliding windows on the receive side. This means that only a set amount of data can be outstanding (in transit) between the sender and receiver at any one time. The OpenSSH development team initially had this limited to 64KB (now 1024KB). Over low latency paths (and for interactive sessions) this is perfectly reasonable. However, for the transfer of large files this proves to be a significant bottleneck. This bottleneck was common to many TCP applications prior to the introduction of autotuning receive buffers in the TCP stack. 

-The wiki uses [Markdown](/p/hpnssh/wiki/markdown_syntax/) syntax.
+HPN-SSH brings the current TCP receive buffer size up to the application layer and, by doing so, uses the network far more efficiently. 

+The HPN-SSH team (Ben Bennet and Mike Tasota) also developed a multi-threaded variant of the AES-CTR cipher so as to allow multicored systems to distribute the burden of computing the keystream over multiple cores. This enhancement produces a cipher stream that is indistinguishable from the default AES-CTR cipher stream. The upshot of this being that it is backwards compliant with all existing AES-CTR implementations - no need to have the multithreaded variant on both sides of the connection. Recent work has also eliminated problems with using the multi-threaded AES-CTR cipher in forked connections (common with forwarded ports) and the sandboxed privsep process. 
+
+For less sensitive data the team (primarily Micahel Stevens) also developed a method to switch to the NONE cipher after a fully cryptographically secure authentication. This method forces the stream to rekey itself after authentication and only proposes the NONE cipher. This patch may significantly improve performance for users whose processes were previously CPU bound. As the NONE cipher is essentially disallowed outside of debugging in OpenSSH users wishing to make use of the NONE cipher must have this patch installed on both sides of the connection. As an aside, the multi-threaded AES-CTR cipher and the NONE cipher exhibit similar throughput enhancement albeit, in the case of the AES-CTR cipher, of significantly higher CPU load. 
+
+Currently the limiting performance factor, in terms of being CPU bound, is the HMAC cipher. HPN-SSH sees the HMAC cipher as being an intrinsically necessary component of SSH. While work is being conducted in order to enhance the performance of the HMAC processing there are no plans to eliminate it now or in the future. 
+
+Lastly, the team also created patches to provide for enhanced server side logging and instantaneous throughput rate reporting in SCP. 
+
+This current patch set, HPN14v1, passes all regression tests, provides a high level of security, and significantly increased performance with minimal fuss.
+[[project_screenshots]]
 [[project_admins]]
 [[download_button]]

Discussion for Home page

Chris Rapier — Mon, 19 Aug 2013 17:49:21 -0000

HPN-SSH is a series of patches for OpenSSH to provide improved performance and functionality. The core of HPN-SSH is the implementation of a dynamic receive side buffer. By allowing this buffer to grow with the TCP receive buffer it's possible to attain significant throughput improvement - up to two orders of magnitude depending on the network path. Patch sets for each OpenSSH release will be found in the appropriate subfolder.

Why does this work? OpenSSH implements SSHv2 - a multiplexed protocol that allows for multiple data connections across a single TCP stream. In order to provide necessary flow control for the individual channels that comprise this multiplexed connection it was necessary to, essentially, reimplement sliding windows on the receive side. This means that only a set amount of data can be outstanding (in transit) between the sender and receiver at any one time. The OpenSSH development team initially had this limited to 64KB (now 1024KB). Over low latency paths (and for interactive sessions) this is perfectly reasonable. However, for the transfer of large files this proves to be a significant bottleneck. This bottleneck was common to many TCP applications prior to the introduction of autotuning receive buffers in the TCP stack.

HPN-SSH brings the current TCP receive buffer size up to the application layer and, by doing so, uses the network far more efficiently.

The HPN-SSH team (Ben Bennet and Mike Tasota) also developed a multi-threaded variant of the AES-CTR cipher so as to allow multicored systems to distribute the burden of computing the keystream over multiple cores. This enhancement produces a cipher stream that is indistinguishable from the default AES-CTR cipher stream. The upshot of this being that it is backwards compliant with all existing AES-CTR implementations - no need to have the multithreaded variant on both sides of the connection. Recent work has also eliminated problems with using the multi-threaded AES-CTR cipher in forked connections (common with forwarded ports) and the sandboxed privsep process.

For less sensitive data the team (primarily Micahel Stevens) also developed a method to switch to the NONE cipher after a fully cryptographically secure authentication. This method forces the stream to rekey itself after authentication and only proposes the NONE cipher. This patch may significantly improve performance for users whose processes were previously CPU bound. As the NONE cipher is essentially disallowed outside of debugging in OpenSSH users wishing to make use of the NONE cipher must have this patch installed on both sides of the connection. As an aside, the multi-threaded AES-CTR cipher and the NONE cipher exhibit similar throughput enhancement albeit, in the case of the AES-CTR cipher, of significantly higher CPU load.

Currently the limiting performance factor, in terms of being CPU bound, is the HMAC cipher. HPN-SSH sees the HMAC cipher as being an intrinsically necessary component of SSH. While work is being conducted in order to enhance the performance of the HMAC processing there are no plans to eliminate it now or in the future.

Lastly, the team also created patches to provide for enhanced server side logging and instantaneous throughput rate reporting in SCP.

This current patch set, HPN14v1, passes all regression tests, provides a high level of security, and significantly increased performance with minimal fuss.

WikiPage Home modified by Chris Rapier

Chris Rapier — Wed, 13 Mar 2013 20:14:30 -0000

Welcome to your wiki!

This is the default page, edit it as you see fit. To add a new page simply reference it within brackets, e.g.: [SamplePage].

The wiki uses Markdown syntax.

Chris Rapier