[Nasm-devel] [PATCH] Omit warning for invalid RIP relative EA

[Liam B. <lia...@gm...>]

This addresses bug 3392797: https://bugzilla.nasm.us/show_bug.cgi?id=3392797

It adds a warning if you try to use an invalid "rel" mode in an
address. For example,
lea rbx, [rel values + rdi * 8] ; this now generates a warning, but continues to
                                             ; generate the same
instructions as before (equivalent to next line)
lea rbx, [values + rdi * 8]      ; this is a valid mode, and does not
generate a warning.

If "default rel" is set, no warning is produced for: lea rbx, [values + rdi * 8]

Initially, I put the warning alongside other warnings in process_ea(),
but that results in them being displayed twice (process_ea() is called
from calcsize() and gencode()) on the final pass. Thank you all for
such a great assembler!

Signed-off-by: Liam Bowen <Lia...@gm...>

---
 asm/assemble.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/asm/assemble.c b/asm/assemble.c
index cd3f4693..dc5ee023 100644
--- a/asm/assemble.c
+++ b/asm/assemble.c
@@ -2264,6 +2264,17 @@ static void gencode(struct out_data *data, insn *ins)
                                rfield, rflags, ins, eat, &errmsg))
                     nasm_nonfatal("%s", errmsg);

+                /* If RIP-relative, indexreg and scale must not be present:
+                 * [basereg + indexreg*scale + displacement]
+                 * https://bugzilla.nasm.us/show_bug.cgi?id=3392797
+                 */
+                if (bits == 64 && (opy->eaflags & EAF_REL) &&
+                    ((opy->indexreg != -1) || (opy->scale != -1))) {
+                  nasm_warn(WARN_OTHER | ERR_PASS2,
+                            "invalid addressing mode: RIP-relative address "
+                            "cannot contain index register or scale");
+                }
+
                 p = bytes;
                 *p++ = ea_data.modrm;
                 if (ea_data.sib_present)
-- 
2.25.1