This small patch improves the throughput of direct
connections by sending all outgoing data in a packet
with a single call to write. This cuts back the amount
of TCP packets sent to the Yahoo server.
As it is now, the TCP stack (Linux) is sending the
first part of the packet as soon as the first write is
received (usually only the 4-byte magic value). Then it
waits for the server to ACK this packet before sending
the rest of the packet. There's even a delay inherent
to the TCP stack (200 ms. average on Linux) that causes
this delay to be worse, event if the round-trip to the
server is short.
I didn't want to add a new layer of buffering or a lot
of memory copying, so I modified the packet class to
reserve 20 bytes for the header at the beginning of its
buffer. This has the side effect that the connection
handlers must know of that extra padding. I fixed all
three cases where the packet's buffer is used. Direct
connection uses this new feature, but the HTTP
connection and the file sender don't, as they already
use many writes for other purposes. For the HTTP
connector buffering might make more sense, and for the
file sender it probably doesn't represent a noticeable
delay (unless you're sending lots of very small files).
Honestly, I only tested the DirectConnectionHandler,
but the other two should be alright as I only changed
one line (so the write starts after the padding).
In DirectConnectionHandler it's very easy to avoid
using a ByteArrayOS, as well as to avoid the 20-byte
copy, by using java.nio's ByteBufferS. I didn't want to
add the dependency, but I can do it for you if you
don't mind the dependency on Java 1.4.2 or higher. Of
course you can forget about them both and write the
header directly into the byte array, but that would
result in awful looking byte blitting code :-)
Hope you find this useful and, as usual, I'll be glad
to hear your comments.
