When sorting by RSS or VM-Size, lxtask ignores the unit (KB, MB, etc.) and orders by the value of the numerical part alone. Additionally, it ignores any fractional component, so sizes with equal integer components (eg. 1.5 MB, 1.9 MB, 1.1 MB) are frequently listed out of order.
I believe this is because of the compare_int_list_item function in interfaces.c used for comparing the items.
The attached patch corrects this behaviour by adding a new compare_mem_list_item function, but I am new to coding and have never tried to fix a bug before, so I do not know if this is a good solution.