#122 JSch input stream hands on repeated reads after EOF

[Michael Osipov]

I am investigating a possible bug in JSch in use with Maven Wagon: https://issues.apache.org/jira/browse/WAGON-543

The code performed is:

protected void transfer( Resource resource, InputStream input, OutputStream output, int requestType, long maxSize )
    throws IOException
{

ByteBuffer buffer = ByteBuffer.allocate( getBufferCapacityForTransfer( resource.getContentLength() ) );
int halfBufferCapacity = buffer.capacity() / 2;

TransferEvent transferEvent = new TransferEvent( this, resource, TransferEvent.TRANSFER_PROGRESS, requestType );
transferEvent.setTimestamp( System.currentTimeMillis() );

ReadableByteChannel in = Channels.newChannel( input );

long remaining = maxSize;
while ( remaining > 0L )
{
    int read = in.read( buffer );

    if ( read == -1 )
    {
        // EOF, but some data has not been written yet.
        if ( buffer.position() != 0 )
        {
            buffer.flip();
            fireTransferProgress( transferEvent, buffer.array(), buffer.limit() );
            output.write( buffer.array(), 0, buffer.limit() );
        }

        break;
    }

    // Prevent minichunking / fragmentation: when less than half the buffer is utilized,
    // read some more bytes before writing and firing progress.
    if ( buffer.position() < halfBufferCapacity )
    {
        continue;
    }

    buffer.flip();
    fireTransferProgress( transferEvent, buffer.array(), buffer.limit() );
    output.write( buffer.array(), 0, buffer.limit() );
    remaining -= buffer.limit();
    buffer.clear();
}
output.flush();
}

input comes from JSch (Channel$MyPipedInputStream). The buffer size is 4096 bytes and the requested size is 33684 bytes. When the last remaining chunk is read 916 is expected while JSch returns 917. The buffer is not filled half of its capacity and the while loop continues. It is again tried to read from the channel expecting to read -1 bytes from it, but instread this method simply hangs.

There seems no reasonable way to detect EOF here. This works with FileInputStream or an input stream provided by Apache HttpClient.

The code in question is here: https://github.com/apache/maven-wagon/blob/master/wagon-provider-api/src/main/java/org/apache/maven/wagon/AbstractWagon.java#L583-L627
We are using JSch 0.1.54.

[Michael Osipov]

This new code has been introduced in WAGON-537. The fundamental difference is that the previous code request to stream the only remaining amount of bytes and the new one the at most the amount of bytes available to the buffer which is more than 916,

[Michael Osipov]

I have retried the very same opration with wagon-ssh-external which uses ssh(1). No hang here. So must be an issue with JSch. See details in downstream issue.