#8 Prevent hang in Polygon::support()
The bug becomes apparent on x86 (i386) when using gcc-4.8.4 with -Os optimization flags (and probably above) although AFAIU it's actually a bug in the code which is luckily hides if building with certain flags (-fpmath=sse, or building on x86_64, or some other flags).
The lowlevel description of the issue follows.
In Polygon::support() gcc stores h as a 8-byte (sizeof Scalar) memory variable but optimizes d so it is stored directly in 10-byte fpu register. When initially storing h, gcc calls dot(), truncates the return value to 8-byte and stores it in h. When comparing d and h, gcc extends 8-byte h to 10-byte value and compare it with never-truncated 10-byte d.
Truncating 10-byte h to 8-byte value and then extending it back to 10-byte value may (and does!) create a value different from the original 10-byte value returned by dot(). So when dot() returns exactly the same value to store in d it differs from the same (but truncated-and-extended) value taken from h.
The actual problem is that C++ (at least, the subset currently implemented by g++ - please look at the gcc-help ml thread pointed below) does not seem to guarantee that d is truncated in the same way as h is, before they are compared.
This may lead to an infinite loop if h and d are the values of the same dot, since d != (long double)(double)d.
The patch attached adds another check in the loops to ensure no vertex is checked more than once.
Using -ffloat-store also seems to fix the issue (at least in speed-dreams) but makes code less efficient (and probably much slower).
The bug was first found and fixed in speed-dreams' bundled version of SOLID ( http://sourceforge.net/p/speed-dreams/tickets/909/ ).
A general discussion of the compiler behaviour is available at https://gcc.gnu.org/ml/gcc-help/2015-01/msg00146.html .
