More fallout from TBOOLEAN change - hidden3d broken

[Ethan M. <merritt@u.washington.edu>]

Is there a tool (or gcc compile option) that will
double check for illegal operations on our TBOOLEANs? 

In stepping through the demos to check 3.8k.3 I found
that something very ugly has happened to hidden3d.
See "surface2.dem".

Reverting to the previous version of src/syscfg.h
made the problem go away, so this was another
bit of fallout from the TBOOLEAN change.

hidden3d.c line 817:
        casenumber = (plist[new_poly].frontfacing != 0)
            + 2 * (plist[old_poly].frontfacing != 0);

frontfacing is a TBOOLEAN.

Even worse:
            TBOOLEAN v1_inside_p, v2_inside_p;
             unsigned int classification[]
-- 
            v1_inside_p = ~ (classification[0]
                             | classification[1]
                             | classification[2]);
--
            v2_inside_p = (v1_inside_p >> 1) & 0x1;
            v1_inside_p = v1_inside_p & 0x1;

I have corrected the syntax involving frontfacing,
and changed v?_inside_p to be (unsigned int).
This fixes surface2.dem, but other problems may lurk.

           

-- 
Ethan A Merritt       merritt@u.washington.edu
Biomolecular Structure Center
Mailstop 357742
University of Washington, Seattle, WA 98195