#44 suite3270-4.4ga6 can not build parallel for unix

[Dr. Werner Fink]

Currently running make with parallel builds (e.g. with make -j 8) enabled does mostly crash with messages about local_parallel is a missed symbol while using about.o at linkage time. Sometimes other sysmbols as well do not have any reference.

[   43s] gcc -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g -o x3270 Cme.o CmeBSB.o CmeLine.o CmplxMenu.o Husk.o about.o dialog.o display8.o display_charsets.o display_charsets_dbcs.o ft_gui.o host_gui.o idle_gui.o keymap.o keypad.o keysym2ucs.o menubar.o nvt_gui.o popups.o print_gui.o print_window.o printer_gui.o resources.o save.o screen.o select.o tls_passwd_gui.o status.o stmenu.o trace_gui.o x3270.o xaa.o xactions.o xkybd.o xtables.o xutil.o fallbacks.o version.o -L/usr/lib64  -L/usr/lib64 -L../../../obj/x86_64-suse-linux-gnu/lib3270 -l3270 -L../../../obj/x86_64-suse-linux-gnu/lib3270i -l3270i -L../../../obj/x86_64-suse-linux-gnu/lib32xx -l32xx -lssl -lcrypto   -L../../../obj/x86_64-suse-linux-gnu/lib3270stubs -l3270stubs -lXaw -lXmu -lXt -lX11 
[   43s] /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: about.o: warning: relocation against `local_process' in read-only section `.text'
[   43s] /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: about.o: in function `popup_about_status':
[   43s] /home/abuild/rpmbuild/BUILD/x3270-4.4-build/suite3270-4.4/obj/x86_64-suse-linux-gnu/x3270/../../../x3270/about.c:542:(.text+0x2019): undefined reference to `local_process'
[   43s] /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: /home/abuild/rpmbuild/BUILD/x3270-4.4-build/suite3270-4.4/obj/x86_64-suse-linux-gnu/x3270/../../../x3270/about.c:552:(.text+0x24d8): undefined reference to `local_process'
[   43s] /usr/lib64/gcc/x86_64-suse-linux/14/../../../../x86_64-suse-linux/bin/ld: warning: creating DT_TEXTREL in a PIE
[   43s] collect2: error: ld returned 1 exit status

[Paul Mattes]

I've been using parallel make for many years and I do not see this locally. Can you attach the entire output from a failed build?

[Dr. Werner Fink]

Looks like a race condition. For this I've run several times a build here and have two logs ... here the first one

[Dr. Werner Fink]

and here the second one

[Dr. Werner Fink]

AFAICS the e.g. lib3270.a will build more than one time and with parallel building it might happen that several stages running on the same lib3270.a

werner/x3270> grep 'ar cr lib' /abuild/oscbuild/openSUSE_Factory/.build.log | less -S
[   32s] ar cr lib3270i.a scroll.o
[   34s] ar cr lib3270stubs.a ft_gui_stubs.o glue_gui_stubs1.o glue_gui_stubs2.o glue_gui_stubs3.o host_gui_stubs.o menubar_stubs.o model_stubs.o nvt_gui_stubs.o popups_stubs.o pr3287_se>
[   35s] ar cr lib32xx.a apl.o asprintf.o base64.o boolstr.o copyright.o indent_s.o min_version.o popup_an_error.o popup_separator.o popups_glue.o pr3287_session.o prefer.o proxy.o proxy>
[   38s] ar cr lib3270.a Malloc.o XtGlue.o actions.o b8.o bind-opt.o child.o childscript.o codepage.o cookiefile.o ctlr.o devname.o event.o favicon.o fprint_screen.o ft.o ft_cut.o ft_dft>
[   38s] ar cr lib3270.a Malloc.o XtGlue.o actions.o b8.o bind-opt.o child.o childscript.o codepage.o cookiefile.o ctlr.o devname.o event.o favicon.o fprint_screen.o ft.o ft_cut.o ft_dft>

in best case it works ... in worst case two ar trying to create lib3270.a ... or ar conflicts with an other ranlib ... there are several dependencies paths in the Makefile rules like direct with lib3270 as well as over x3270 and the final Makefile.obj for lib3270.a does not check if the jobs is already done

[Dr. Werner Fink]

Also it looks like that some object files for pr3287 are recompiled which seems that some symbols become missed in the new lib3270.a which are required for x3270

[Dr. Werner Fink]

It seems by luck if the correct conf.h gets included ... just verified that with

obj/x86_64-suse-linux-gnu/lib3270> gcc -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g -Wall -Wsign-compare -std=c99 -pedantic -Wno-variadic-macros -MMD -MP  -I../../../lib/3270 -I../../../lib/3270/../include/unix -I../../../lib/3270/../include -I../../../include -DLIBX3270DIR=\"/etc/x3270\"  -D_POSIX_SOURCE -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_XOPEN_SOURCE -D_GNU_SOURCE -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g   -c -o telnet.o ../../../Common/telnet.c
obj/x86_64-suse-linux-gnu/lib3270> nm telnet.o | grep local_process
0000000000000038 B local_process

the symbol local_process is included but with the second compile

obj/x86_64-suse-linux-gnu/lib3270> gcc -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g -Wall -Wsign-compare -std=c99 -pedantic -Wno-variadic-macros -MMD -MP  -I../../../mitm -I../../../include  -D_POSIX_SOURCE -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g -Wall -Wsign-compare -std=c99 -pedantic -Wno-variadic-macros -MMD -MP  -I../../../lib/3270 -I../../../lib/3270/../include/unix -I../../../lib/3270/../include -I../../../include -DLIBX3270DIR=\"/etc/x3270\"  -D_POSIX_SOURCE -D_BSD_SOURCE -D_DEFAULT_SOURCE -D_XOPEN_SOURCE -D_GNU_SOURCE -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type  -g   -c -o telnet.o ../../../Common/telnet.c
obj/x86_64-suse-linux-gnu/lib3270> nm telnet.o | grep local_process
obj/x86_64-suse-linux-gnu/lib3270> 

it is missed

[Dr. Werner Fink]

OK ... after patching around during debugging I've a patch which makes it build ... nevertheless I suggest only one conf.h.in below include/ to avoid that common C files see different configurations.

[Paul Mattes]

Thank you -- you've done quite a bit of work for me!

It's going to take me a while to get all the way through your patch. I will let you know if I have any questions.

[Paul Mattes]

I was able to reproduce the strange behavior here, where gmake tries building lib3270.a twice. It happens only on gmake 4.4. It does not happen on gmake 4.3. Whether this is a 4.4 bug I do not know yet.

[Paul Mattes]

I have a solution, and it's quite simple.

The Makefile for the mitm utility mistakenly lists lib3270 as a dependency, rather than lib32xx. My guess is that gmake has gotten quite a bit smarter in 4.4. If it dispatches a process to build mitm before lib3270 is built as a dependency of other targets, gmake (amazingly) figures out how to build lib3270 for mitm. It gets that mostly right, except that it apparently includes the mitm directory in the gcc -I options, which is definitely incorrect.

I corrected mitm/Makefile.obj.in to specify a dependency on lib32xx instead of on lib3270, and I no longer see multiple builds of lib3270 on OpenSUSE.

I will publish that fix shortly.

[Paul Mattes]

Fix committed to the 4.4 and main branches.

[Dr. Werner Fink]

There are two -f on directories which should become -C that is x3270if and ibm_hosts

[Paul Mattes]

Fixed. And thank you for the suggestion of using -C and multiple targets at once.