Schily Tools / Tickets / #5 2018.08.07: libschily-mapvers is empty when building

Jörg Schilling - 2018-08-07

If you cannot explain why the mapfile was created empty, I have no idea how I can help you.

It worked for me on Linux.

You could remove the intermediate mapfile and run smake -d do disable the silent mode.

Last edit: Jörg Schilling 2018-08-07

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

For testing, I edited lines 65/66 in RULES/rules.shl
so that the command is not hidden. The relevant output from the make run is

...echo "       ==> LOCALIZING mapfile \"OBJ/amd64-linux-cc/pic/libschily-mapvers\"";
        ==> LOCALIZING mapfile "OBJ/amd64-linux-cc/pic/libschily-mapvers"
...x86_64-pc-linux-gnu-gcc -E -DSCHILY_BUILD -IOBJ/amd64-linux-cc/pic -I../incs/amd64-linux-cc -I../include  -Istdio -DUSE_SCANSTACK     -DPORT_ONLY        -D_GNU_SOURCE  libschily-mapvers | tr -d '"' | sed 's/^SCHILY/+SCHILY/' | tr '\012' '@' | tr '+' '\012' | sort -t. -n -k1 -k2 -k3 |  tr '@' '\012' > OBJ/amd64-linux-cc/pic/libschily-mapvers || /bin/rm -f OBJ/amd64-linux-cc/pic/libschily-mapvers
x86_64-pc-linux-gnu-gcc: warning: libschily-mapvers: linker input file unused because linking not done

I also inserted two ls commands before and after that lines:
Before the command, there is no corresponding file OBJ/amd64-linux-cc/pic/libschily-mapvers, after the command this file exists, but its length is 0.

martin.vaeth - 2018-08-07

The output of the above compiler command is empty (i.e. even if I remove all the rest of the pipe).

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-08

Your compiler installation is defective.

GCC is known to not behave correctly with -E (it does not accept any type of file) and for this reason, configure checks for a program "cpp" in the same installation directory as the GCC in use.

It seems that you did not install this "cpp" and as a result suffer from the well known GCC -E problem.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Thanks. There does exist cpp and alsox86_64-pc-linux-gnu-cpp in the same directory, but gentoo's default environment exports some environment variables in which gcc -E is preferred.
After exporting an appropriate CPP, compilation seems to proceed further, but I get another linking problem:

+ x86_64-pc-linux-gnu-gcc -o OBJ/amd64-linux-cc/mkisofs OBJ/amd64-linux-cc/mkisofs.o OBJ/amd64-linux-cc/tree.o OBJ/amd64-linux-cc/write.o OBJ/amd64-linux-cc/hash.o OBJ/amd64-linux-cc/rock.o OBJ/amd64-linux-cc/
inode.o OBJ/amd64-linux-cc/udf.o OBJ/amd64-linux-cc/multi.o OBJ/amd64-linux-cc/joliet.o OBJ/amd64-linux-cc/match.o OBJ/amd64-linux-cc/name.o OBJ/amd64-linux-cc/eltorito.o OBJ/amd64-linux-cc/boot.o OBJ/amd64-li
nux-cc/isonum.o OBJ/amd64-linux-cc/scsi.o OBJ/amd64-linux-cc/apple.o OBJ/amd64-linux-cc/volume.o OBJ/amd64-linux-cc/desktop.o OBJ/amd64-linux-cc/mac_label.o OBJ/amd64-linux-cc/stream.o OBJ/amd64-linux-cc/ifo_r
ead.o OBJ/amd64-linux-cc/dvd_file.o OBJ/amd64-linux-cc/dvd_reader.o OBJ/amd64-linux-cc/walk.o -L../libs/amd64-linux-cc/pic -L../libs/amd64-linux-cc -lhfs -lfile -lsiconv -lscgcmd -lrscg -lscg -lcdrdeflt -ldefl
t -lfind -lmdigest -lschily -lcap
OBJ/amd64-linux-cc/multi.o:multi.c:function merge_remaining_entries: error: undefined reference to 'strlcat'                                                                                                    
OBJ/amd64-linux-cc/multi.o:multi.c:function merge_remaining_entries: error: undefined reference to 'strlcat'                                                                                                    
OBJ/amd64-linux-cc/multi.o:multi.c:function merge_remaining_entries: error: undefined reference to 'strlcat'                                                                                                    
OBJ/amd64-linux-cc/multi.o:multi.c:function merge_remaining_entries: error: undefined reference to 'strlcat'

Let me know if you prefer that I open a separate ticket for this.

Jörg Schilling - 2018-08-08

How does Gentoo cause this problem?

What is in the environment that causes this unwanted behavior?

Is there a need to block such kind of environment?

BTW: versioned libraries work four you!

strlcat() and many others was missing in the linker map file

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

martin.vaeth - 2018-08-08

I was actually wrong. (I had instructed gentoo's toolchain functions to export CPP, and observed that they export "gcc -E". But this was not the problem).

The actual problem is that your method of test conflicts with gentoo's ccache wrapper:
This is a directory /usr/lib/ccache/bin which preceeds /usr/bin in PATH and which for every compiler executable supported by ccache contains a corresponding symlink to /usr/bin/ccache
(which in turn will call the corresponding executable if it cannot find the generated .o file in the cache). Of course, cpp is not supported by ccache and thus has no corresponding symlink, but your check will only look in /usr/lib/ccache/bin.

If I set neither and instruct portage to omit the ccache wrapping, your heuristics find indeed /usr/bin/cpp (although perhaps it should better find x86_64-pc-linux-gnu-cpp in that case, corresponding to the compiler it already detected).

Anyway, it is not a big deal since in the ebuild I can just omit the problem by exporting CPP or CPPX correctly.

However, even without the ccache wrapper the second link issue remains.

strlcat() and many others was missing in the linker map file

I do not understand that comment. Should I check some file or manually add something to some file?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-08

OK, so there IS a broken compiler installation if the first entry in PATH has the compiler but not cpp.

"was missing" obviously mean that I recently added it to the linker map file because it was missing but needed.

This is something that needs an #ifdef as sane OS platforms come with strlcat() in libc and this is a fallback implementation in libschily.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

martin.vaeth - 2018-08-08

so there IS a broken compiler installation

Is there a standard with the requirement that certain binaries belong to the same entry of PATH?
I would actually expect that it is not so rare that a particular binary is overridden by an earlier /usr/local/bin entry.
Perhaps simply "cpp" (or first ${CC%gcc}cpp) could be used as a fallback if it is executable via PATH.

Anyway, for my purpose I consider the problem as solved.

mean that I recently added it to the linker map file

I see. Thanks. So I will wait for the next release.

In one run, I also had some linker messages with some functions from libcap not being found (although libcap is installed), but this is hard to reproduce (since usually the other error occurs first). Unless you tell me something else, I will wait for the next release.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-08

If you wait for the next release, you enhance the chance that there are more problems that are not fixed before the next release is published.

I have no linux installation that still has libcap. The related functions have been moved to libc.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

martin.vaeth - 2018-08-08

Is there a way to get the recent changes you mentioned before the release?

I have no linux installation that still has libcap

I do not know about the relation of these libraries; they have similar version numbers, but glibc depends on libcap if USE=caps is specified.

The error message was this

OBJ/amd64-linux-cc/pic/remote.:remote.c:function rmtgetconn: error: undefined reference to 'cap_get_proc' OBJ/amd64-linux-cc/pic/remote.:remote.c:function rmtgetconn: error: undefined reference to 'cap_get_flag' OBJ/amd64-linux-cc/pic/remote.:remote.c:function rmtgetconn: error: undefined reference to 'cap_get_flag'

I suppose that some -lcap has to be added somewhere. With the previous release of schily-tools there was no such error.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

martin.vaeth - 2018-08-09

The related functions have been moved to libc

The NEWS in the glibc-2.26 tarball did not contain such an information. (Which I would have expected since the latest version of libcap appears to be 2.25.)
Also Debian seems to have a separate package for libcap:
https://packages.debian.org/stretch/libcap-dev
Apparently, the package with the binary library is called libcap2 in Debian.
Also on kernel.org, the library seems to be called libcap2 (although it installs libcap.so.2):
https://mirrors.edge.kernel.org/pub/linux/libs/security/linux-privs/libcap2/

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-09

After adding strlcat to the libschily map file, schilytools compile and link fine with:

smake LINKMODE=dynamic

Did you try to compile schilytools inside a sane login environment?

My impression is that everything beyond the missing entry in the map file is caused by Gentoo specific problems, so it might be the right way to ask Gentoo people for help.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

martin.vaeth - 2018-08-09

This was a tough one: It turned out that the CPPX I passed contained an embedded newline. This in turn caused an sed error, leading to a non-generation of rules.cnf, and so half of the build system went havoc. Suprisingly that the missing -lcaps options (and actually also missing -ldl options) were the only visible effect.

Summarizing: All is fine now except for the missing patch to strlcat (and the not satisfactory heuristics for the CPPX default).

Last edit: martin.vaeth 2018-08-09

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-10

The next version of schilytools will include an enhanced configure test that addresses the Gentoo compiler missinstallation. So I think we may close this.

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-10

status: open --> closed
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

martin.vaeth - 2018-08-12

BTW, "gcc -E -" and "clang -E -" works within pipes (i.e. if "-" is used as the file name).
(Also GNU's cpp and clang-cpp accept the - argument).

I suppose that "gcc -E -" or "clang -E -" are preferrable over "cpp" or "clang-cpp", because at least cpp produces output which avoids some GNU features, see https://stackoverflow.com/questions/43967458/difference-between-cpp-and-gcc-e

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Jörg Schilling - 2018-08-13

Unfortunately, you cannot grant that this already works with GCC-1, so it cannot be used as the preferred method. The configure tool in schilytools however uses gcc -E - as a fallback since last Wednesday...

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

2018.08.07: libschily-mapvers is empty when building

A Tool Box with tools written or managed by Jörg Schilling

Milestone

Searches

Help

#5 2018.08.07: libschily-mapvers is empty when building

Discussion