Haven't had much time trying to isolate this issue, I have however found a
good work around that I thought I might share.
start muse with:
taskset -p 1 muse
effectually binding muse to only the first core. In the long run this isn't
a good solution but it works very good.
I tend to run stuff outside of MusE, jamin, linuxsampler and so on so the
other CPU isn't wasted.