#521 qemu-kvm fails to migrate properly in 0.12.2
I'm using Ganeti and encountered a problem [1] with migrating KVM virtual machines. I have confirmed that this is not a problem specific to Ganeti and is also happening to the Libvirt project [2].
When migrating from host A to host B, everything goes fine however the guest VM (running CentOS 5.4) clock changes to approximately 600 seconds in the future. When attempting to migrate back to host B, the VM locks up and stops responding. However, if you fix the clock prior to syncing back to host A, the migration happens however the clock on the VM literally stops so the guest is pretty unusable.
I tried the same scenario using Ubuntu 9.10 for the guest OS and has different issues. Upon migrating to host B, the clock on the guest VM was ahead by over 3 days into the future. Then when I migrate it back to host A the VM locks up hard.
I can confirm I have no problem using migration when using 0.11.1.
Here's the detail information about our setup:
Hardware: Intel(R) Xeon(R) CPU 5160 @ 3.00GHz
qemu-kvm: 0.12.2
kernel: 2.6.29-hardened
Linux OS: Gentoo Hardened
If you need more information, please let me know and I'll be happy to provide it.
Thanks-
[1] http://groups.google.com/group/ganeti/browse_thread/thread/9b2c556557e749cf
[2] http://thread.gmane.org/gmane.comp.emulators.libvirt/20771
This has been mentioned a few times in irc and possibly on the mailing list. No one has really come up with a fix. Can you try without kvmclock?
Are you wanting me to try this on the guest or host system?
guest
you can either use the no-kvmclock boot param or choose a different clock in:
/sys/devices/system/clocksource/clocksource0/current_clocksource
possible values are in:
/sys/devices/system/clocksource/clocksource0/available_clocksource
It seems to work if I use something like acpi_pm for the clock source (on Debian). The clock also seemed to stay in time. I haven't tested it on the other OS's but it seems that CentOS doesn't even use kvmclock (which means sense considering the age of the kernel).
I think 5.4 had kvmclock backported to it... I just think it doesn't have the same /sys adjustments for clock, so you have to use the no-kvmclock thing or possibly something else.
http://www.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/5.4/html/Virtualization_Guide/chap-Virtualization-KVM_guest_timing_management.html
has some tips for disabling it I think
possibly just clocksource=acpi_pm is needed
Any idea why kvmclock is causing this issue? Has there been any recent changes to that part of qemu-kvm?
Thanks for the tips on CentOS, I'll give that a try and see if that works as a workaround for now.
Using acpi_pm both on hosts and guests (Debian 5.0 AMD64) seems to work. I now did 20-30 live migrations or so and it worked like a charm. Thanks!
Can someone try this?
http://www.mail-archive.com/kvm@vger.kernel.org/msg28046.html
I'll give that a try sometime today
The patch you provided doesn't seem to apply cleanly on older kernels so I went ahead and used the latest stable 2.6.32.8 which seems to include the patch. Unfortunately I have the same problem as before with the time moving up by nearly three days and the VM locking up upon returning to the original host.
Has there been any progress made on this bug? I noticed that I had the same problem with 0.11.1 and just resorted to not using the kvm-clock.
There have been some fixes committed and there is another long set of patches being discussed on the list now. Unfortunately, the fixes require both host and guest updates. I don't think they will be in till 2.6.36 or so. If they are suitable for stable releases, they can probably go into 2.6.32 and the other maintained stable trees.