[Valgrind-users] Unhandled instruction bytes: 0xD9 0xD0 (fnop)

[Mark <ma...@zb...>]

I am trying to port a legacy windows 32-bit delphi application (a 3d fractal image generator) to free pascal on linux. The application loads fractal formulas as machine code at runtime into memory for execution. The origin of the machine code is unclear, it may be written by hand or generated by unknown compilers, so I can't easily change it. When testing the ported application with valgrind I often encounter this message when the external code is loaded:

vex x86->IR: unhandled instruction bytes: 0xD9 0xD0 0xE9 0xAA
==00:00:00:01.399 55074== valgrind: Unrecognised instruction at address 0x405e01a.

The sequence 0xd9 0xd0 is fnop in intel x86. Checking bugzilla I found two related bugs:

   https://bugs.kde.org/show_bug.cgi?id=126256
   https://bugs.kde.org/show_bug.cgi?id=253446

There is also a reference to fnop in the valgrind git repo in docs/internals/3_1_BUGSTATUS.txt (the bug number mentioned there is 125265 but that is perhaps just a typo because it refers to a kmail bug).

I can reproduce the issue within valgrind (git master@758b0f55e) with the following test:

diff --git a/none/tests/x86/insn_fpu.def b/none/tests/x86/insn_fpu.def
index 590f5844c..f5a8d61c4 100644
--- a/none/tests/x86/insn_fpu.def
+++ b/none/tests/x86/insn_fpu.def
@@ -1,3 +1,4 @@
+fnop
  fabs st0.ps[1234.5678] : => st0.ps[1234.5678]
  fabs st0.ps[-1234.5678] : => st0.ps[1234.5678]
  fabs st0.pd[12345678.87654321] : => st0.pd[12345678.87654321]
diff --git a/none/tests/x86/insn_fpu.stdout.exp b/none/tests/x86/insn_fpu.stdout.exp
index 67128c13b..f5f4a161f 100644
--- a/none/tests/x86/insn_fpu.stdout.exp
+++ b/none/tests/x86/insn_fpu.stdout.exp
@@ -1,3 +1,4 @@
+fnop_1 ... ok
  fabs_1 ... ok
  fabs_2 ... ok
  fabs_3 ... ok

This patch fixes the issue and lets the test pass (both in valgrind and my application):

diff --git a/VEX/priv/guest_x86_toIR.c b/VEX/priv/guest_x86_toIR.c
index bd4ccd54b..710905ad1 100644
--- a/VEX/priv/guest_x86_toIR.c
+++ b/VEX/priv/guest_x86_toIR.c
@@ -4204,6 +4204,10 @@ UInt dis_FPU ( Bool* decode_ok, UChar sorb, Int delta )
                 put_ST_UNCHECKED(r_src, mkexpr(t1));
                 break;

+            case 0xD0: /* FNOP */
+               DIP("fnop\n");
+               break;
+
              case 0xE0: /* FCHS */
                 DIP("fchs\n");
                 put_ST_UNCHECKED(0, unop(Iop_NegF64, get_ST(0)));

The complete patch is included at the end of this mail including the same fix and regression test also for amd64 and a fix for the documentation typo. Would that be a proper solution or am I missing something?

Thanks,
Mark

diff --git a/VEX/priv/guest_amd64_toIR.c b/VEX/priv/guest_amd64_toIR.c
index 34ebcbdf9..0319578f2 100644
--- a/VEX/priv/guest_amd64_toIR.c
+++ b/VEX/priv/guest_amd64_toIR.c
@@ -5967,6 +5967,10 @@ ULong dis_FPU ( /*OUT*/Bool* decode_ok,
                 put_ST_UNCHECKED(r_src, mkexpr(t1));
                 break;

+            case 0xD0: /* FNOP */
+               DIP("fnop\n");
+               break;
+
              case 0xE0: /* FCHS */
                 DIP("fchs\n");
                 put_ST_UNCHECKED(0, unop(Iop_NegF64, get_ST(0)));
diff --git a/VEX/priv/guest_x86_toIR.c b/VEX/priv/guest_x86_toIR.c
index bd4ccd54b..710905ad1 100644
--- a/VEX/priv/guest_x86_toIR.c
+++ b/VEX/priv/guest_x86_toIR.c
@@ -4204,6 +4204,10 @@ UInt dis_FPU ( Bool* decode_ok, UChar sorb, Int delta )
                 put_ST_UNCHECKED(r_src, mkexpr(t1));
                 break;

+            case 0xD0: /* FNOP */
+               DIP("fnop\n");
+               break;
+
              case 0xE0: /* FCHS */
                 DIP("fchs\n");
                 put_ST_UNCHECKED(0, unop(Iop_NegF64, get_ST(0)));
diff --git a/docs/internals/3_1_BUGSTATUS.txt b/docs/internals/3_1_BUGSTATUS.txt
index 6add0ccac..4134c3cb9 100644
--- a/docs/internals/3_1_BUGSTATUS.txt
+++ b/docs/internals/3_1_BUGSTATUS.txt
@@ -54,7 +54,7 @@ v5877     fixed   126217  increase # threads
  v5880     fixed   n-i-bz  vectorise copy_address_range_state
                    n-i-bz  mpicc -fpic bug (Goedeken Richard, inbox)
  vx1611    fixed   126243  vex x86->IR: popw mem
-          low     125265  vex x86->IR: 0xD9 0xD0 (fnop)
+          low     126256  vex x86->IR: 0xD9 0xD0 (fnop)
            low     126257  vex x86->IR: 0xF2 0x0F 0xF0 0x40 (lddqu) (sse3)
            low     126258  vex x86->IR: 0xDF 0x4D (fisttp) (sse3)
                    126384  rdpmc
diff --git a/none/tests/amd64/insn_fpu.def b/none/tests/amd64/insn_fpu.def
index 525fd1b1b..219247478 100644
--- a/none/tests/amd64/insn_fpu.def
+++ b/none/tests/amd64/insn_fpu.def
@@ -1,3 +1,4 @@
+fnop
  fabs st0.ps[1234.5678] : => st0.ps[1234.5678]
  fabs st0.ps[-1234.5678] : => st0.ps[1234.5678]
  fabs st0.pd[12345678.87654321] : => st0.pd[12345678.87654321]
diff --git a/none/tests/amd64/insn_fpu.stdout.exp b/none/tests/amd64/insn_fpu.stdout.exp
index 67128c13b..f5f4a161f 100644
--- a/none/tests/amd64/insn_fpu.stdout.exp
+++ b/none/tests/amd64/insn_fpu.stdout.exp
@@ -1,3 +1,4 @@
+fnop_1 ... ok
  fabs_1 ... ok
  fabs_2 ... ok
  fabs_3 ... ok
diff --git a/none/tests/x86/insn_fpu.def b/none/tests/x86/insn_fpu.def
index 590f5844c..f5a8d61c4 100644
--- a/none/tests/x86/insn_fpu.def
+++ b/none/tests/x86/insn_fpu.def
@@ -1,3 +1,4 @@
+fnop
  fabs st0.ps[1234.5678] : => st0.ps[1234.5678]
  fabs st0.ps[-1234.5678] : => st0.ps[1234.5678]
  fabs st0.pd[12345678.87654321] : => st0.pd[12345678.87654321]
diff --git a/none/tests/x86/insn_fpu.stdout.exp b/none/tests/x86/insn_fpu.stdout.exp
index 67128c13b..f5f4a161f 100644
--- a/none/tests/x86/insn_fpu.stdout.exp
+++ b/none/tests/x86/insn_fpu.stdout.exp
@@ -1,3 +1,4 @@
+fnop_1 ... ok
  fabs_1 ... ok
  fabs_2 ... ok
  fabs_3 ... ok