#1 performance ideas

[Jimmy P]

some things i've been thinking about to improve
performance in various places.

1. the MessageDatagram does not do a good job of
preventing parsing properties on messages that come in.
we should store standard properties in a more efficient
way that does not require parsing.

a. related to this is the fact that we are allocating
a direct buffer for every message that is instantiated
to hold properties. there should be a better way to do
this.

2. the UberMQ server reads a lot of stuff into memory
only to create garbage out of it because it is merely
moving data between participants. The server really
should be able to do bulk scatter/gather type transfers
to avoid unnecessary overhead.

a. A way to implement this would be to create a
special MessageDatagram that assumes that it has
control over the read buffer for some period of time
and does no garbage creation.

3. I have implemented durable subscriptions in such a
way that ack'ing does not merely ack into the log file.
instead it writes information at the beginning of the
log file to indicate the point up to which the client
has acked. this is bad because we do a lot of
unnecessary seeks and our throughput drops. it would be
more efficient from an IO standpoint to merely append
data into the logfile, but potentially more complicated
to implement. We'd have to have the recovery manager
deal with correlating all of those interleaved messages
and acks for compacting old data and for recovering
un'acked messages.

4. after we output a datagram, we immediately transfer
the bytes written into the write buffer onto the
channel, and then compact the buffer. we may get an
improvement in scalability if a write thread handles
outputs, or if we let them enqueue for some time.