#497 2.09 - warning: signed byte value exceeds bounds

[Anonymous]

Using NASM 2.09 instead of 2.08.1 I get the following error messages:

drivers/fat16.asm:337: warning: signed byte value exceeds bounds
drivers/fat16.asm:389: warning: signed byte value exceeds bounds
drivers/fat16.asm:667: warning: signed byte value exceeds bounds
drivers/pci.asm:115: warning: signed byte value exceeds bounds
drivers/net/rtl8139.asm:27: warning: signed byte value exceeds bounds

fat16.asm lines 337, 389, and 667:
cmp ax, 0xFFFF

pci.asm line 115:
cmp eax, 0xFFFFFFFF

rtl8139.asm line 27:
and eax, 0xFFFFFFFC

Is this a bug in the latest version of NASM?

[Victor van den Elzen]

I don't see any relevant change in 2.08..2.09 and can reproduce the problem as I see it with 2.08 as well.

The problem I see is BITS value different than operand size and fixed operand size (implicit override). In that case gencode case4(\274) does not correctly calculate the operand size.because \320 and \321 don't set ins->prefixes[PPS_OSIZE]. The calculation also has a bug in BITS 64.

The attached patch solves the above problems, but I dislike this way of calculating the operand size and the overflow check isn't precise like the EA overflow check.

[Victor van den Elzen]

possible fix

[Victor van den Elzen]

http://repo.or.cz/w/nasm.git/commit/b3cee5a57ab2cd0a99ad01ac9db73e31ae90d519 mostly fixes this, but there is still one failing case that needs some more thought.