ScummVM / Bugs / #6673 IRIX: build and runtime failures when building with MIPSPro compiler

digitall - 2014-07-04

summary: IRIX build / runtime failures due to -static-libgcc, -export-dynamic and -whole-archive --> IRIX: build and runtime failures when building with MIPSPro compiler
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

digitall - 2014-07-04

The attached patch is not quite the right solution.

On IRIX, you can build with either free GCC compiler or the commercial SGI MIPSPro compiler.

We do test for MIPSPro vs. GCC compiler earlier in the configure script:
https://github.com/scummvm/scummvm/blob/master/configure#L1727

It appears that these GCC-only options were added breaking MIPSPro building.

To fix these, rather than removing, they should be guarded by the following or similar:
if test "$have_gcc" = yes; then

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

canavan - 2014-07-05

The updated patch incorporates your suggestion and uses different options for gcc and other compilers (i.e. MIPSPro). I have verified that the generated config.mk, config.h and config.log are identical (minus timestamps) between the patched configure and a stock 1.7-branch configure.

However, that doesn't appear to help with gcc, since the scummvm binary produced by gcc 4.7 fails to load dynamic plugins. I've verified this with a fresh checkout of branch-1-7, and I get identical errors there:

WARNING: Failed loading plugin 'plugins/libmortevielle.so' (Failed loading plugins/libmortevielle.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libmortevielle.so: _ZN3GUI8DebuggerD2Ev)!
WARNING: Failed loading plugin 'plugins/libscumm.so' (Failed loading plugins/libscumm.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libscumm.so: _ZN5Audio13makeVOCStreamEPN6Common18SeekableRead)!
WARNING: Failed loading plugin 'plugins/libqueen.so' (Failed loading plugins/libqueen.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libqueen.so: _ZN5Audio14makeRjp1StreamEPN6Common18SeekableRea)!
WARNING: Failed loading plugin 'plugins/libneverhood.so' (Failed loading plugins/libneverhood.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libneverhood.so: _ZTI18AdvancedMetaEngine)!
WARNING: Failed loading plugin 'plugins/libtinsel.so' (Failed loading plugins/libtinsel.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libtinsel.so: _ZNK18AdvancedMetaEngine17getSupportedGamesEv)!
WARNING: Failed loading plugin 'plugins/libtouche.so' (Failed loading plugins/libtouche.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libtouche.so: _ZNK18AdvancedMetaEngine19detectGameFilebasedER)!
WARNING: Failed loading plugin 'plugins/libvoyeur.so' (Failed loading plugins/libvoyeur.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libvoyeur.so: _ZNK5Video12VideoDecoder11getCurFrameEv)!
WARNING: Failed loading plugin 'plugins/libwintermute.so' (Failed loading plugins/libwintermute.so: 50184:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libwintermute.so: _ZN8Graphics15TransformStructC1EiiNS_16TSpr)!

scummvm-configure-irix.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

digitall - 2014-07-05

OK. That new patch looks much better and should be acceptable to merge as-is. However, will wait until we are sure we have no other changes for IRIX to merge from this bug.

So the current status is firstly that IRIX builds with either MIPSPro or GCC fails for static builds due to a linker problem ld32: FATAL 2: Internal: at ../../ld/pass1.c Total size of all GPREL sections exceeds 64K)

canavan: Can you please try earlier release versions and see if this is a regression and when it started occuring?

The last released version was v1.4.1 for IRIX (6.5.*, n32, mips3 and mips4):
http://prdownloads.sourceforge.net/scummvm/scummvm-1.4.1-irix-mips3.tar.gz?download
http://prdownloads.sourceforge.net/scummvm/scummvm-1.4.1-irix-mips4.tar.gz?download

Therefore if you could start with testing the v1.4.1 and v1.5.0 sources please?
http://prdownloads.sourceforge.net/scummvm/scummvm-1.4.1.tar.gz?download
http://prdownloads.sourceforge.net/scummvm/scummvm-1.5.0.tar.gz?download

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

digitall - 2014-07-05

canavan: secondly, IRIX builds built with MIPSPro compiler with dynamic plugins and your second patch are now fine? Please confirm.

So the remaining issue is with IRIX builds built with GCC compiler with dynamic plugins... The missing symbols look like libscummvm.so or the similar "core" library plugin is not being loaded / resolved by the runtime linker successfully... I wonder if this is an issue where the "base" library is not copied to the right directory and is thus not in the linker path. shrug Will wait for your updates on first two issues before we go on with this.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Johannes Schickel - 2014-07-05

The missing symbols look like libscummvm.so or the similar "core" library plugin is not being loaded / resolved by the runtime linker successfully...

There is nothing like a "core" library plugin/libscummvm.so. All these symbols which cannot be resolved are (or should be) in the main executable.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

digitall - 2014-07-05

lordhoto: Mea Culpa. I was talking "off the top of my head" thinking of the Android port's usage of plugins... but of course, that uses a "core" library due to the Java / Native code split.

canavan: Ignore that last speculation. As lordhoto rightly said, those symbols should be in the main scummvm executable. Not sure how the IRIX linker works and how to debug this.. You could start with seeing if those symbols are actually present in the compiled scummvm executable I suppose with "objdump -t scummvm" or similar..

lordhoto: Any further suggestions about debugging this?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

canavan - 2014-07-06

Static builds with gcc with just a few engines work, but fail with a different message: Please use -LD_LAYOUT:lgot_buffer=50. Adding that to the link step succeeds and results in a working binary. Static builds with MIPSPro are actually OK, since the linker apparently automatically uses Multigot as long as one doesn't link with -IPA (which doesn't help anyway).

IRIX builds built with MIPSPro compiler with dynamic plugins and my second patch are indeed fine (aside from a bunch of FORBIDDEN_SYMBOL_EXCEPTIONs I have to set which probably should also be in the patch, obviously for dynamic and static builds, namely -DFORBIDDEN_SYMBOL_EXCEPTION_clock=1 -DFORBIDDEN_SYMBOL_EXCEPTION_difftime=1 -DFORBIDDEN_SYMBOL_EXCEPTION_mktime=1 -DFORBIDDEN_SYMBOL_EXCEPTION_asctime=1 -DFORBIDDEN_SYMBOL_EXCEPTION_time=1 -DFORBIDDEN_SYMBOL_EXCEPTION_ctime=1 -DFORBIDDEN_SYMBOL_EXCEPTION_gmtime=1 -DFORBIDDEN_SYMBOL_EXCEPTION_localtime=1 -DFORBIDDEN_SYMBOL_EXCEPTION_getdate=1 -DFORBIDDEN_SYMBOL_EXCEPTION_isnumber=1)

Dynamic builds with scummvm 1.4.1 and GCC fail as well:
scummvm-1.4.1 :) $ ./scummvm
WARNING: Failed loading plugin 'plugins/libscumm.so' (Failed loading plugins/libscumm.so: 15940:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libscumm.so: _ZN10MidiParser13resetTrackingEv)!
WARNING: Failed loading plugin 'plugins/libqueen.so' (Failed loading plugins/libqueen.so: 15940:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libqueen.so: _ZN3GUI8DebuggerC2Ev)!
WARNING: Failed loading plugin 'plugins/libtinsel.so' (Failed loading plugins/libtinsel.so: 15940:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libtinsel.so: _ZN18AdvancedMetaEngineC2EPKvjPK19PlainGameDesc)!
WARNING: Failed loading plugin 'plugins/libtouche.so' (Failed loading plugins/libtouche.so: 15940:./scummvm: rld: Fatal Error: unresolvable symbol in plugins/libtouche.so: _ZN5Audio10MidiPlayer4sendEj)!
WARNING: File 'translations.dat' has a mismatching version, expected was 2 but you got 3. Skipping this file!
WARNING: You are missing a valid 'translations.dat' file. GUI translation will not be available!
WARNING: Could not find theme 'scummmodern' falling back to builtin!
User picked target 'simon1-win-1' (gameid 'simon1')...
Looking for a plugin supporting this gameid... failed
WARNING: simon1 is an invalid gameid. Use the --list-games option to list supported gameid!
User picked target 'comi' (gameid 'comi')...
Looking for a plugin supporting this gameid... failed
WARNING: comi is an invalid gameid. Use the --list-games option to list supported gameid!
User picked target 't7g' (gameid 'groovie')...
Looking for a plugin supporting this gameid... failed
WARNING: groovie is an invalid gameid. Use the --list-games option to list supported gameid!
scummvm-1.4.1 :) $ nm scummvm |grep _ZN18AdvancedMetaEngineC2EPKvjPK19PlainGameDesc
scummvm-1.4.1 :( $

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

canavan - 2014-07-10

The updated patch below produces working static and dynamic scummvm binaries both with GCC and MIPSPro.

It will however most likely break cross-compiling, as the GNU linker, which is not used on IRIX, even by GCC, isn't as accepting of unknown options (fails and exits with -1) as the IRIX linker (which just prints a warning and ignores them). Since the error message is unmistakable (unrecognized option '-all') and those who dare to cross-compile should know the risks and how to deal with this problem, I'd consider this acceptable.

Can this patch please be applied before the 1.7.0 release?

scummvm-configure-irix.patch

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Johannes Schickel - 2014-07-10

All these "-DFORBIDDEN_SYMBOL_EXCEPTION_*" look wrong. Where do you get issues when compiling without these? These defines should really be only set in system specific code at the backend side.

You also seem to define SYSTEM_NOT_SUPPORTING_D_TYPE twice now.

Also just for clarification: This "-all" and "-notall" for the linker is the equivalent of "-whole-archive"/"-no-whole-archive" of GNU ld? In that case, shouldn't you also set "-W,-notall" in POST_OBJS_FLAGS in the MIPSPro case?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

canavan - 2014-07-10

"-all" and "-notall" for the linker are indeed the equivalent of "-whole-archive"/"-no-whole-archive" of GNU ld.

And you're right "-W,-notall" in POST_OBJS_FLAGS in the MIPSPro case looks like a good idea, but my first test build just failed with some undefined symbols when linking the scummvm binary.

When not setting the -DFORBIDDEN_SYMBOL_EXCEPTION_*, compiling posix-main.cpp (which is apparently the second file to be compiled) fails:

/usr/nekoware/gcc-4.7/bin/g++ -MMD -MF "backends/platform/sdl/posix/.deps/posix-main.d" -MQ "backends/platform/sdl/posix/posix-main.o" -MP -Wall -O3 -DIP32 -DIRIX -I/usr/nekoware/include/freetype2 -I/usr/nekoware/include/ -I/usr/nekoware/include/SDL -ansi -W -Wno-unused-parameter -Wno-empty-body -pedantic -Wno-long-long -O2 -Wuninitialized -fpic -Wno-long-long -Wno-multichar -Wno-unknown-pragmas -Wno-reorder -Wpointer-arith -Wcast-qual -Wshadow -Wnon-virtual-dtor -Wwrite-strings -fno-exceptions -fcheck-new -DHAVE_CONFIG_H -DRELEASE_BUILD -DMIPS_TARGET -DIRIX -DSYSTEM_NOT_SUPPORTING_D_TYPE -DSDL_BACKEND -DPOSIX -DDATA_PATH=\"/usr/nekoware/share/scummvm\" -DPLUGIN_DIRECTORY=\"/usr/nekoware/lib/scummvm\" -DENABLE_SCUMM=DYNAMIC_PLUGIN -DENABLE_SCUMM_7_8 -DENABLE_HE -DENABLE_AGI=DYNAMIC_PLUGIN -DENABLE_AGOS=DYNAMIC_PLUGIN -DENABLE_AGOS2 -DENABLE_CGE=DYNAMIC_PLUGIN -DENABLE_CINE=DYNAMIC_PLUGIN -DENABLE_COMPOSER=DYNAMIC_PLUGIN -DENABLE_CRUISE=DYNAMIC_PLUGIN -DENABLE_DRACI=DYNAMIC_PLUGIN -DENABLE_DRASCULA=DYNAMIC_PLUGIN -DENABLE_DREAMWEB=DYNAMIC_PLUGIN -DENABLE_GOB=DYNAMIC_PLUGIN -DENABLE_GROOVIE=DYNAMIC_PLUGIN -DENABLE_HOPKINS=DYNAMIC_PLUGIN -DENABLE_HUGO=DYNAMIC_PLUGIN -DENABLE_KYRA=DYNAMIC_PLUGIN -DENABLE_LOL -DENABLE_EOB -DENABLE_LURE=DYNAMIC_PLUGIN -DENABLE_MADE=DYNAMIC_PLUGIN -DENABLE_MOHAWK=DYNAMIC_PLUGIN -DENABLE_MORTEVIELLE=DYNAMIC_PLUGIN -DENABLE_NEVERHOOD=DYNAMIC_PLUGIN -DENABLE_PARALLACTION=DYNAMIC_PLUGIN -DENABLE_PEGASUS=DYNAMIC_PLUGIN -DENABLE_QUEEN=DYNAMIC_PLUGIN -DENABLE_SAGA=DYNAMIC_PLUGIN -DENABLE_IHNM -DENABLE_SCI=DYNAMIC_PLUGIN -DENABLE_SKY=DYNAMIC_PLUGIN -DENABLE_SWORD1=DYNAMIC_PLUGIN -DENABLE_SWORD2=DYNAMIC_PLUGIN -DENABLE_TEENAGENT=DYNAMIC_PLUGIN -DENABLE_TINSEL=DYNAMIC_PLUGIN -DENABLE_TOLTECS=DYNAMIC_PLUGIN -DENABLE_TONY=DYNAMIC_PLUGIN -DENABLE_TOON=DYNAMIC_PLUGIN -DENABLE_TOUCHE=DYNAMIC_PLUGIN -DENABLE_TSAGE=DYNAMIC_PLUGIN -DENABLE_TUCKER=DYNAMIC_PLUGIN -DENABLE_VOYEUR=DYNAMIC_PLUGIN -DENABLE_WINTERMUTE=DYNAMIC_PLUGIN -I. -I. -I./engines -I/usr/nekoware/include/SDL -D_GNU_SOURCE=1 -D_SGI_MP_SOURCE -I/usr/nekoware/include/freetype2 -c backends/platform/sdl/posix/posix-main.cpp -o backends/platform/sdl/posix/posix-main.o
In file included from /usr/include/time.h:4:0,
from /usr/include/wchar.h:6,
from /usr/include/inttypes.h:243,
from /usr/nekoware/include/SDL/SDL_stdinc.h:66,
from /usr/nekoware/include/SDL/SDL_main.h:26,
from /usr/nekoware/include/SDL/SDL.h:30,
from ./backends/platform/sdl/sdl-sys.h:58,
from ./backends/platform/sdl/sdl.h:26,
from ./backends/platform/sdl/posix/posix.h:26,
from backends/platform/sdl/posix/posix-main.cpp:27:
/usr/include/internal/time_core.h:133:26: error: macro "clock" passed 1 arguments, but takes just 0
In file included from /usr/include/time.h:4:0,
from /usr/include/wchar.h:6,
from /usr/include/inttypes.h:243,
from /usr/nekoware/include/SDL/SDL_stdinc.h:66,
from /usr/nekoware/include/SDL/SDL_main.h:26,
from /usr/nekoware/include/SDL/SDL.h:30,
from ./backends/platform/sdl/sdl-sys.h:58,
from ./backends/platform/sdl/sdl.h:26,
from ./backends/platform/sdl/posix/posix.h:26,
from backends/platform/sdl/posix/posix-main.cpp:27:
/usr/include/internal/time_core.h:134:15: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:135:15: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:136:15: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:140:14: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:141:14: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:142:19: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:143:19: error: expected initializer before 'SYMBOL'
/usr/include/internal/time_core.h:216:19: error: expected initializer before 'SYMBOL'
In file included from /usr/include/wctype.h:25:0,
from /usr/include/wchar.h:9,
from /usr/include/inttypes.h:243,
from /usr/nekoware/include/SDL/SDL_stdinc.h:66,
from /usr/nekoware/include/SDL/SDL_main.h:26,
from /usr/nekoware/include/SDL/SDL.h:30,
from ./backends/platform/sdl/sdl-sys.h:58,
from ./backends/platform/sdl/sdl.h:26,
from ./backends/platform/sdl/posix/posix.h:26,
from backends/platform/sdl/posix/posix-main.cpp:27:
/usr/include/internal/wctype_core.h:186:12: error: expected initializer before 'SYMBOL'
gmake: *** [backends/platform/sdl/posix/posix-main.o] Error 1

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

canavan - 2014-07-10

It turns out that POST_OBJS_FLAGS := -Wl,-notall reproducably builds a scummvm binary that can't load plugins due to undefined symbols, e.g. Audio::MidiPlayer::sendToChannel(unsigned char,unsigned int) or GUI::Debugger::attach(const char*) (both demangled).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Johannes Schickel - 2014-07-10

Try defining "FORBIDDEN_SYMBOL_ALLOW_ALL" before "#include "common/scummsys.h"" in posix-main.cpp.

I suppose the IRIX linker works differently then. It probably does only respect the last option of "-all"/"-noall" instead of GNU ld which will link all objects after such an option in the manner the option enables. It will make sense to create a comment about that to make sure nobody adds it back in the future.

This makes me wonder: For building with GCC this really works with the "-noall" in "POST_OBJS_FLAGS"?

I am not sure I got it correctly but: Static builds fail even when running configure without any arguments? In that case it probably also makes sense to default to dynamic plugins in case of IRIX.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

canavan - 2014-07-11

backends/platform/sdl/posix/posix.cpp with

+#define FORBIDDEN_SYMBOL_ALLOW_ALL 1
#include "common/scummsys.h"

without the -DFORBIDDEN* causes the same error as not having FORBIDDEN_SYMBOL_ALLOW_ALL defined:

cc-1065 CC: ERROR File = /usr/include/internal/wctype_core.h, Line = 186
A semicolon is expected at this point.

extern int isnumber(__SGI_LIBC_NAMESPACE_QUALIFIER wint_t);

The Documentation for -all and -notall would lead me to believe that they work just as the GNU ld equivalents, i.e. just for the following archives and objects:

-all Links in all of the objects from all archives following this option. This option is commonly used in conjunction with the -shared option to create a shared object out of an existing archive. (C, C++, F77, F90) -notall Turns off the action of the -all option. This pulls objects out of the archive into a DSO if the objects resolve unresolved symbols in the already included object, but these objects are included without exporting their symbols by default. (C, C++, F77, F90)

This makes me wonder: For building with GCC this really works with the "-noall" in "POST_OBJS_FLAGS"?

I just tried this once more, and the dynamic scummvm binary built with GCC nad -noall works, while one with MIPSPro and -noall fails due to unresolvable symbols.

Static build with MIPSPro actually work as well, I just always had -IPA in my linker flags and without that, the build succeeds and the resulting scummvm works.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Johannes Schickel - 2014-07-21

Sorry for the delay, but I have been busy and it seems nobody else took a look at this...

If "#define FORBIDDEN_SYMBOL_ALLOW_ALL" works (you should leave out the " 1" here for consistency), you should use that in your patch.

Do you have any idea why there is a difference in the behavior when adding "-noall" with MIPSPro and GCC? If not, we should probably at the very least make a comment about it...

If you can attach an updated patch, I'll look at pushing it to master and branch-1-7 (in case we create a 1.7.1 release) soon.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Johannes Schickel - 2014-07-21

[EDIT: OK, it seems that my first message was posted despite the bug tracker throwing an error... forget about this one...]

Last edit: Johannes Schickel 2014-07-21

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

IRIX: build and runtime failures when building with MIPSPro compiler

A cross-platform interpreter for many point-and-click adventure games

Group

Searches

Help

#6673 IRIX: build and runtime failures when building with MIPSPro compiler

Discussion