#8 Fix endian handling

[Ryan Carsten Schmidt]

Compiling DevIL 1.7.8 fails on OS X 10.9 Mavericks:

libtool: compile:  ccache /usr/bin/clang++ -DHAVE_CONFIG_H -I. -I../include/IL -I ./../src-IL/include -I ./../include -isystem/opt/local/include -fgnu89-inline -msse -msse2 -msse3 -pipe -Os -arch x86_64 -MT libIL_la-il_utx.lo -MD -MP -MF .deps/libIL_la-il_utx.Tpo -c ./../src-IL/src/il_utx.cpp  -fno-common -DPIC -o .libs/libIL_la-il_utx.o
In file included from ./../src-IL/src/il_utx.cpp:17:
In file included from ./../src-IL/include/il_utx.h:31:
In file included from /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/memory:588:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/__config:31:22: error: expected value in expression
#if __LITTLE_ENDIAN__
                     ^
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/../lib/c++/v1/__config:86:11: fatal error: 'endian.h' file not found
# include <endian.h>
          ^
2 errors generated.
make[1]: *** [libIL_la-il_utx.lo] Error 1

This is because the system headers expect that if __LITTLE_ENDIAN__ is defined, then it is defined to a truthy value, and this DevIL header breaks that assumption by undefining it and then redefining it as empty. The fix is to remove the lines that undefine and redefine __LITTLE_ENDIAN__. Since there were already lines defining __BIG_ENDIAN__ to 1 on big-endian systems if undefined, I added lines to do the same for __LITTLE_ENDIAN__ on little-endian systems, though they're not needed to build successfully on OS X because OS X already defines __LITTLE_ENDIAN__ to 1.

While looking into fixing this, I noticed that DevIL will never correctly detect big-endian systems because it compares __BYTE_ORDER__ against __BIG_ENDIAN__, which is the wrong constant. The fix is to compare against __ORDER_BIG_ENDIAN__.

I fixed this in MacPorts with this patch:

https://trac.macports.org/browser/trunk/dports/devel/libdevil/files/patch-src-IL-include-il_endian.h.diff?rev=112877

ResIL 1.7.9 does not have the same build failure, but applying the same patch to ResIL nonetheless seems correct, at least to correct the detection of big-endian systems.

[Ryan Carsten Schmidt]

In fact, identifiers containing __ are reserved by the implementation. You are not permitted to define or undefine or redefine them. In addition to undefining __BIG_ENDIAN__ and __LITTLE_ENDIAN__ in il_endian.h, there are many places in the ResIL code where such identifiers are defined. There are other identifier naming patterns that are also reserved. You should check if you're using any of those as well.