Replicated this on Mac OS with gcc 4.2.1 and Ubuntu 14.04 on gcc 4.8.2. Buffer overflow occurs when setting a channel mode to include color then sending a message over said channel. Fixed this by increasing buf size by one. Here's a diff:
https://github.com/gtaylor/btmux/commit/62be4ba7b58c04ded730f8a83c2fa6d81911f9bf
Diff: