Matlab-shell showed a few glitches when I used it with my matlab version (R2011b). At each startup the history was lost because the version was not recognised, error messages were not displayed correctly and were not clickable, and help messages when created from clicking within another help message were not correctly displayed.
I have modified matlab.el, which works now with the R2011b version. I attach the patch file. Maybe this is useful.
patch file for matlab.el