Some keyboard keys are mapped wrong under Xwayland. Most notably the arrow keys and right alt. One way to fix this is to do something similar that spice-gtk did in commit 28e3dc34424f:
https://github.com/freedesktop/spice-gtk/commit/28e3dc34424f5e5b0441f2341d777ae4362363f0
Patch attached. (With Xwayland the "keycodes" variable contains "(unnamed)" in lieu of "evdev".)