[htdig-dev] Problem with SSL support

[Jim <li...@yg...>]

There appears to be a significant problem with the current SSL support in 
htdig. I have spent some time tracking down the source of a problem 
reported earlier this month (bug reports 1011830 and 1011769). It appears 
that the bug responsible for this problem is in 
SSLConnection::Read_Partial (htnet/SSLConnection.cc).

The Read_Partial method employs a select() to watch for activity on the 
socket, however this is not a reliable approach for handling an SSL 
connection. Since SSL is record based, an SSL_read() may pull more bytes 
of data from the socket than specified by the provided arguments. When 
this happens, the extra data is removed from the socket and placed in an 
internal SSL buffer. As the code is currently written, this can lead to a 
state where the process is waiting in a select() for data that has already 
been read and is currently sitting in the SSL internal buffer. A timeout 
then occurs on the select() and the SSL_read() is skipped since it appears 
that no data is available. This repeats a number of times specified by the 
max_retries attribute and then htdig gives up and reports 'connection 
down'.

I believe the solution involves adding an SSL_pending() call to check for 
data in the SSL internal buffer. One option would be to wrap the block of 
code handling the select() so that it is only entered when the internal 
buffer is empty. For example,

   if (!SSL_pending(ssl)) {
     if (timeout_value > 0) {
       FD_SET_T fds;
       FD_ZERO(&fds);
       FD_SET(sock, &fds);

       timeval tv;
       tv.tv_sec = timeout_value;
       tv.tv_usec = 0;

       int selected = select(sock+1, &fds, 0, 0, &tv);

       if (selected <= 0) {
         need_io_stop++;
       }
     }
   }

This change stood up to some very limited testing that I performed, but 
this is the first time I have touched SSL code so I have no great
confidence in this being "the" solution. Comments?

Jim