I'd like to propose for consideration two changes to the hextile
encoding. They make use of previously unused flag bytes (of the subtile
header byte) and can be easily implemented. These ideas are at an early
stage, so I'd like some casual feedback, and then if they're not stupid,
we can take it from there! NB: I've only just finished coding my
vncserver and hextile encoder, so I may have overlooked existing
features, or misunderstood how things are "supposed" to work!
The first flag is:
FlagUnchanged: This tile is unchanged from the previous frame buffer
update. No additional bytes are used for this tile. Reason: I think it's
easier to encode one large hextile rect that might include a few stray
16x16 blocks of unchanged data than try to find a more optimal packing
of rectangles that avoid those unchanged blocks. This encoding makes it
much cheaper to "accidentally" include a few blocks of unchanged data.
Unchanged blocks of data in the middle of a frame buffer update can suck
up lots of bytes, especially if it's a photo. In other words, it's kind
of like an embedded "copyrect", where x,y,w,h are implicit.
The easiest implementation on the server side would be to track the
client's frame buffer explicitly, though some of this "unchangedness"
could be determined by more detailed bookkeeping when computing the
union of invalid rects. Implementation on client side is trivial (just
skip to the next tile).
The second flag is:
FlagRepeat: This is used on single-byte tiles (like solid colors or
Unchanged blocks), and tells the decoder to assume that the current
command byte is repeated N times. The value N is encoded in a byte
following the command byte. It's essentially adding a level of
run-length-encoding to the tile command stream. You could then concisely
say "13 unchanged tiles follow", or "27 background color tiles follow".
This flag would decrease even further the amount of bandwidth used,
especially when the application's invalidate rects are overly conservative.
If both of these flags become available, I would argue that it becomes
unnecessary to track update rects, and that the logic for performing
unions (etc) could be eliminated. A single frame buffer update could be
reasonably implemented as a single Hextile rectangle whose extent is the
entire framebuffer. (A 1600x1200 framebuffer "NOP" would require only
about 32 bytes... about the same size as a TCP packet header.)