Menu

#478 modfl() triggers segmentation fault for 64-bit targets

v1.0 (example)
open
nobody
None
5
2017-10-26
2015-05-07
Jason Roehm
No

modfl(), which is used to split a long double into whole and fractional parts, causes a segmentation fault with the latest mingw-w64 runtime release. The following code will show the error:

#include <math.h>

int main()
{
   long double x = 1.2;
   long double whole, frac;
   frac = modfl(x, &whole);
}

I cross-compiled the file on an OS X 10.10 host using the command line:

x86_64-w64-mingw32-gcc test.c -o test.exe

I then ran the resulting executable on a 64-bit Windows 8.1 Pro system and observe the runtime error. Here is a disassembled version of the function, with the arrow indicating the place where the fault occurs:

(gdb) disassemble modfl
Dump of assembler code for function modfl:
   0x0000000000402af0 <+0>:     sub    $0x18,%rsp
   0x0000000000402af4 <+4>:     fldt   (%rdx)
   0x0000000000402af6 <+6>:     mov    %rcx,%rax
   0x0000000000402af9 <+9>:     fld    %st(0)
   0x0000000000402afb <+11>:    fstpt  (%rsp)
   0x0000000000402afe <+14>:    sub    $0x8,%rsp
   0x0000000000402b02 <+18>:    fnstcw 0x4(%rsp)
   0x0000000000402b06 <+22>:    movzwl 0x4(%rsp),%eax
   0x0000000000402b0b <+27>:    or     $0xc,%ah
   0x0000000000402b0e <+30>:    mov    %ax,(%rsp)
   0x0000000000402b12 <+34>:    fldcw  (%rsp)
   0x0000000000402b15 <+37>:    frndint
   0x0000000000402b17 <+39>:    fldcw  0x4(%rsp)
   0x0000000000402b1b <+43>:    add    $0x8,%rsp
   0x0000000000402b1f <+47>:    test   %r8,%r8
   0x0000000000402b22 <+50>:    je     0x402b29 <modfl+57>
   0x0000000000402b24 <+52>:    fld    %st(0)
   0x0000000000402b26 <+54>:    fstpt  (%r8)
   0x0000000000402b29 <+57>:    mov    0x8(%rsp),%rdx
   0x0000000000402b2e <+62>:    and    $0x7fff,%dx
   0x0000000000402b33 <+67>:    cmp    $0x7fff,%dx
   0x0000000000402b38 <+72>:    jne    0x402b50 <modfl+96>
   0x0000000000402b3a <+74>:    mov    (%rsp),%rcx
   0x0000000000402b3e <+78>:    mov    %rcx,%rdx
   0x0000000000402b41 <+81>:    shr    $0x20,%rdx
   0x0000000000402b45 <+85>:    and    $0x7fffffff,%edx
   0x0000000000402b4b <+91>:    or     %ecx,%edx
   0x0000000000402b4d <+93>:    je     0x402b60 <modfl+112>
   0x0000000000402b4f <+95>:    nop
   0x0000000000402b50 <+96>:    fldt   (%rsp)
   0x0000000000402b53 <+99>:    fsubp  %st,%st(1)
=> 0x0000000000402b55 <+101>:   fstpt  (%rax)
   0x0000000000402b57 <+103>:   add    $0x18,%rsp
   0x0000000000402b5b <+107>:   retq
   0x0000000000402b5c <+108>:   nopl   0x0(%rax)
   0x0000000000402b60 <+112>:   fstp   %st(0)
   0x0000000000402b62 <+114>:   fldz
   0x0000000000402b64 <+116>:   jmp    0x402b55 <modfl+101>

If you inspect the contents of tax at the point of the offending instruction, it's clear that it contains a garbage value, certainly not a pointer. If I refer to the source of the modfl() function, the cause becomes apparent:

long double
modfl (long double value, long double* iptr)
{
  long double int_part = 0.0L;
  /* truncate */
#if defined(_AMD64_) || defined(__x86_64__)
  asm ("subq $8, %%rsp\n"
    "fnstcw 4(%%rsp)\n"
    "movzwl 4(%%rsp), %%eax\n"
    "orb $12, %%ah\n"
    "movw %%ax, (%%rsp)\n"
    "fldcw (%%rsp)\n"
    "frndint\n"
    "fldcw 4(%%rsp)\n"
    "addq $8, %%rsp\n" : "=t" (int_part) : "0" (value)); /* round */
#elif defined(_X86_) || defined(__i386__)
  asm ("push %%eax\n\tsubl $8, %%esp\n"
    "fnstcw 4(%%esp)\n"
    "movzwl 4(%%esp), %%eax\n"
    "orb $12, %%ah\n"
    "movw %%ax, (%%esp)\n"
    "fldcw (%%esp)\n"
    "frndint\n"
    "fldcw 4(%%esp)\n"
    "addl $8, %%esp\n\tpop %%eax\n" : "=t" (int_part) : "0" (value)); /* round */
#else
  int_part = truncl(value);
#endif
  if (iptr)
    *iptr = int_part;
  return (isinf (value) ?  0.0L : value - int_part);
}

In the 64-bit code block, it appears that eax is used without preserving its value or marking the register as being clobbered in the inline assembly call. Note that the 32-bit code path pushes and pops eax from the stack so its value doesn't get clobbered. I'm not an x86_64 ABI expert by any means, so maybe there's a reason for why this was done, but it appears to be the cause of this fault.

Discussion

  • Jason Roehm

    Jason Roehm - 2015-05-07

    Just below the disassembly dump above, it should read "If you inspect the contents of rax" instead. The above was a typo.

     
  • Jason Roehm

    Jason Roehm - 2015-05-07

    Also, while I've only seen the problem with modfl(), it appears that there are matching commits around the time this was introduced that made similar changes to modf() and modfl() as well.

     
  • David Grayson

    David Grayson - 2015-05-07

    I'm not a mingw-w64 developer, but for what it's worth, I was able to reproduce the problem on my Windows 8.1 machine using the MINGW64 shell of MSYS2. All my packages are up-to-date as of a few minutes ago. The code posted by the OP does indeed trigger a segmentation fault.

    Here is the output from gdb:

    $ gdb ./testc.exe
    GNU gdb (GDB) 7.9
    Copyright (C) 2015 Free Software Foundation, Inc.
    License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
    This is free software: you are free to change and redistribute it.
    There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
    and "show warranty" for details.
    This GDB was configured as "x86_64-w64-mingw32".
    Type "show configuration" for configuration details.
    For bug reporting instructions, please see:
    <http://www.gnu.org/software/gdb/bugs/>.
    Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.
    For help, type "help".
    Type "apropos word" to search for commands related to "word"...
    Reading symbols from ./testc.exe...done.
    (gdb) run
    Starting program: C:\Users\David\Documents\scraps\test_c\testc.exe
    [New Thread 107872.0x1a5a0]
    
    Program received signal SIGSEGV, Segmentation fault.
    0x0000000000402b65 in modfl (value=<optimized out>, iptr=0x23fe20)
        at C:/git/mingw/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/math/modfl.c:40
    40      C:/git/mingw/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/math/modfl.c: No such file or directory.
    (gdb) where
    #0  0x0000000000402b65 in modfl (value=<optimized out>, iptr=0x23fe20)
        at C:/git/mingw/mingw-w64-crt-git/src/mingw-w64/mingw-w64-crt/math/modfl.c:40
    #1  0x000000000040157f in main (argc=1, argv=0x24850) at testc.c:14
    

    GDB is claiming that the error is on line 40 of modfl.c, which is the return statement of modfl, so I am not so sure if the error is really caused by the inline assembly. (I don't know enough about x64 assembly to evaluate whether the assembly in this function is OK.) I did confirm that the error does not occur in 32-bit mode.

     

    Last edit: David Grayson 2015-05-07
  • Jason Roehm

    Jason Roehm - 2015-05-07

    The attached patch to the mingw-w64 CRT appears to make the problem go away for me. It addresses the issue with modfl(), modf(), and modff().

     
  • Dan Collins

    Dan Collins - 2016-07-19

    I am also encountering this bug, and am finding that the Perl testsuite crashes when one of the test programs makes an innocent call to modfl(). Can the patch above be reviewed and applied, and can you let us know what version it will be applied in?

     
  • FX

    FX - 2017-01-10

    The patch appears correct, but I cannot approve it formally or commit it.

     
  • Kai Tietz

    Kai Tietz - 2017-01-24

    Patch is ok. JonY could you please take care about it? it might be something for backport, too. Thanks in advance

     
  • LIU Hao

    LIU Hao - 2017-10-26

    Those asm clobber lists for i686 shall be fixed too. I posted a new patch to our ML just now.

     

Log in to post a comment.

MongoDB Logo MongoDB