I've had numerous problems earlier with the usb-serial dongle getting stuck and unresponsive. But after applying the attached fix, it works much better. It's not completely clear why but I guess the double retain prevents cleanup to be done properly.
It also contains some locking fixes. I guess I should open another patch for it but I suggest applying both anyway so.