It is easy to get the JRE to deadlock by sending
multiple disconnects. Other exceptions in message send
or key exchange can also cause this situation.
The problem is the lock handling through Tr
I found two issues with the lock handling through
TransportProtocolAlgorithmSync. Some methods could take
this lock and never release it (if there were exceptions).
After finding this problem I did a quick code
inspection and found some additional problems
TransportProtocolOutputStream.sendMessage
---------------------------------------------------
The release should be in a finally block. This was the
issue that we found in test.
TransportProtocolCommon
------------------------------
The "algorithmsIn" lock seems to fill no function since
it is never taken in TransportProtocolInputStream.
However, it is taken in onMsgNewKeys and never
released, so two calls to this method will hang the
system. Removed this lock call.
The handling of the "algorithmsOut" lock around and in
completeKeyExchange had broken exception handling and
could also release the lock even though it had not been
previously taken by this thread. Whether or not this
could actually be detected is another issue. Patched
this also - the code is still rather ugly, but I do not
understand it well enough to clean it up.
patch (see description)