patch: add missing -l for building shared objects
Brought to you by:
rwhaley,
tonyc040457
Menu
▾
▴
#13 patch: add missing -l for building shared objects
Hi,
Following patch allows generating shared objects containing appropriate "NEEDED" tags (in the ELF dynamic section) for all the libraries which are actually used (directly called) by the object.
It also adds the called libraries as dependencies to the make target (this allows auto-detection of dependencies in cases where you build a specific object directly).
I strongly recommend that you also add the following option to the ld(1) command.
--unresolved-symbols=ignore-in-shared-libs
This causes errors in case there are more libraries which are directly called by the so, but not specified in my patch.
(I did not test the patch against current dev sources, so it is necessary to use such option at least for the debugging stage).
thanks for all the good work,
Amit
patch for AtlasBase/Clint/atlas-make.base
Amit,
Thanks for the patch! I have moved it to the patch tracker; all user requests should start on the support or patch tracker, and I will escalate to the bug tracker once I have confirmed that the bug exists.
I want to understand better what your patch is supposed to do. Is this patch for Linux?
What benefit is there for the "NEEDED" flag?
I notice that you show liblapack dependent on libcblas, what happens when the user wants to use libptcblas instead?
Thanks,
Clint
> all user requests should start on the support or patch tracker
my mistake - this is clearly not a bug - I did not fully understand the logic of the tracker system...
> Is this patch for Linux?
Yes, but I imagine it should work with non-gnu linkers as well. The ELF format supports storing this information, so linkers should be able to fill it in.
> What benefit is there for the "NEEDED" flag?
Sorry if was not clear.
I was referring to DT_NEEDED tags of the ELF binary. See http://www.caldera.com/developers/devspecs/gabi41.pdf section 5-19.
The benefit is that if, say, libblas.so binary has a tag declaring it needs libatlas.so, then the user can link her application to libblas.so (-lblas) alone without needing to specify -latlas as well. This sort of "hides the internals" and allows easier configuration of user apps (users have to declare only stuff they directly use).
> I notice that you show liblapack dependent on libcblas, what happens when
> the user wants to use libptcblas instead?
Did not think about that. I guess we should probably have two separate rules. The dependency is there because I had used --unresolved-symbols=ignore-in-shared-libs (not included in my patch) to figure out all the dependencies actually used by the library - but it can not check if there is another library providing the same function. Let me know if you want me to send another patch.
thanks,
Amit
The way linking is supposed to work is that lapack & atlas libs are the same for threaded and serial, and you select whether you want threaded or serial performance by choosing the serial or threaded BLAS API library (libcblas for C interface, libf77blas for fortran). Here's the FAQ entry describing it:
http://math-atlas.sourceforge.net/errata.html#LINK
I would like shared object to work the same way if possible. How will this work, if at all, in your given implicit link scenario? Say we applied your patch, so that lapack was by default linked to libptf77/cblas. If the user asked for libcblas instead, would the serial interface get used, or would the default done by your patch override the user selection?
Are you saying that we must really have two lapack libs, one that uses ptc/f77blas and one that uses c/f77blas?
Thanks,
Clint
From your description, I assume that the lapack functions (tri/trf) themselves do not have SMP support (I suppose that they just use the cblas interface) - please correct me if I'm wrong.
I can think of several ways (not necessarily mutually exclusive) to design the shared objects. For simplicity, I'll only discuss the c interface.
* The one closest to the way things work now is by having the threaded/non-threaded version of each library have the same soname, and link lapack with it (i.e. libptcblas.so would be linked with -soname=libcblas.so, and lapack with -lcblas). When the application is linked, it can specify -lptcblas to indicate threaded version, and the linker would still consider the dependency satisfied.
* However, I still think that if someone just wants to do LU decomposition, he should not be required to link directly with blas, so maybe it would be best to provide a libptlapack.so (possibly with -soname=liblapack.so) which would depend on the threaded cblas. This allows using a single binary application for benchmarking with different configurations, simply by changing the LD_LIBRARY_PATH.
* One more point: maybe the user wants to build only-threaded or only non-threaded version, in which case there should be no problems with the namings. It might be acceptable to support 'threaded-only' build and 'non-threaded-only' build but not both (you can always build, copy libs elsewhere, then build again).
The above points only concern the linkage commands. I'll have to 'man make' a bit to see how to construct makefile rules that would support the above build strategies.
Amit
p.s. I tried looking for what packagers providing bre-built ATLAS are doing.
It seems that the libs from deb packages (Debian and Ubuntu) do have DT_NEEDED tags (similar to result of my patch) - but they provide only non-threaded binaries. The rpm's I was able to find contain static libs only (no shared objects).
Amit,
In a few developer releases from now, I think I'm going to be forced redesign the lapack lib, and fission it into a liblapack.a and libptlapack.a. When I have that done, maybe that will go some way to solving this problem? I may want to ask some questions on this, since I don't know squat about dynamic libraries . . .
Thanks,
Clint
I guess you refer to the problem of supporting a threaded lapack shared object (which my patch did not provide).
Note that this can probably be solved in the current design as well (see my prev comment, the stuff about the soname in particular). I did not try implementing it myself back then due to lack of time.
Still, the change you describe might make things simpler.
I'll be glad to help if needed, but bear in mind that I am not an expert on shared objects either, and did not try building ATLAS from dev sources for more than a year. I also can not guarantee a quick response.