From: Alan W. I. <ir...@be...> - 2006-09-22 01:26:36
|
I recently expressed a "bright idea" on plplot-general for dealing with the ENABLE_DYNDRIVERS OFF case and the C++ device drivers. The idea was to split off the C++ drivers into their own library rather than mixing that C++ object code into the core C library. Only problem is this causes cross-dependencies; this proposed library defined symbols required by our core library and vice versa. Such cross-dependencies should be avoided since they cause linking issues on many platforms. The autotools book even has a special section on avoiding cross or circular library dependencies. Unfortunately, I wasted a lot of time on the idea until I figured out it was not so bright. That's when I thought to post to the CMake list about whether there was any real problem with the alternative of mixing C and C++ object code in the same library, and I have been assured there that KDE does that with no problems which is consistent with Geoffrey's post on the same issue. Therefore, I have decided to make the experiment of allowing both the psttf and wxwidgets device drivers when ENABLE_DYNDRIVERS OFF. Some changes had to be made to get this to work, but the result passes the usual ctest results for the following possibilities: (1) default (-DBUILD_SHARED_LIBS=ON[default] -DENABLE_DYNDRIVERS=ON[default]) (2) disabled dynamic drivers (-DBUILD_SHARED_LIBS=ON[default] -DENABLE_DYNDRIVERS=OFF) (3) static build (-DBUILD_SHARED_LIBS=OFF -DENABLE_DYNDRIVERS=OFF[this value forced when BUILD_SHARED_LIBS is OFF]) N.B. case (3) still uses shared external libraries, although the PLplot libraries are static. The first two also passed the install tree examples build and execute test so I have recently committed all my work. Please test these changes to our CBS on the platforms accessible to you. The current status is there is a library ordering problem with the build of the installed examples for case (3) above which I should be able to sort out tomorrow. Once that works, then I will be in a good position to finally answer the interesting (paraphrased) question on plplot-general of whether for Linux systems it is possible to make a completely static build of PLplot. That can be implemented by the combination of case (3) above plus use of the linker option -Bstatic to force static linking to external libraries. I think the answer to the question must be yes, but it may take some time to get to that answer since library ordering is even more critical for that case then it is for case (3) above. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the Yorick front-end to PLplot (yplot.sf.net); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Geoffrey F. <fu...@li...> - 2006-09-22 14:42:02
|
Alan W. Irwin writes: > I recently expressed a "bright idea" on plplot-general for dealing with the > ENABLE_DYNDRIVERS OFF case and the C++ device drivers. > > The idea was to split off the C++ drivers into their own library rather than > mixing that C++ object code into the core C library. Only problem is this > causes cross-dependencies; this proposed library defined symbols required by > our core library and vice versa. Such cross-dependencies should be avoided > since they cause linking issues on many platforms. The autotools book even > has a special section on avoiding cross or circular library dependencies. > Unfortunately, I wasted a lot of time on the idea until I figured out it > was not so bright. This is a useful segway to something I've been meaning to bring (back) up for a while. With the encouraging reports of an improved developer experience with CBS, I've been contemplating additional development activities that might be interesting and useful. What I would really, personally, most like to do, is to push into areas involving new plotting features. Let me come back to that though, because it's an involved topic. But I do have one more extant "project concept" that is still kind of back in the area of plplot organization/structure. And that is to actually *split* the drivers from the core library. Some might remember that I've talked about this in the past. I actually got started on it at one point, but it became effectively an abandoned project. Combination of what was for me daunting AM/LT comlpexity, and a seemingly irreversible transition into perpetual mania on my day job. Anticipating that we are building up to a seemingly increasingly likely decision to cutover with finality to CBS, and given that linkage issues remain a constant source of exasperation for users, I'm thinking its perhaps a more interesting time to be pushing the core/driver split concept. This idea flows out of, or could be thought of as an extentions of, the dynamic drivers thing, which was realisticaly, my last successful contribution to the project. And the idea is simply this: Application programs link to PLplot via: -lplplot(d) That's it. No more linkage complexity. How would this work? Hee Hee. Simple, actually. Just pull the drivers all out to dynamically loadable drivelets (driver modules), each with their own linkage specification to external libraries (which would be handled by PLplot during build time), and, specifically, *disallow* dependence of the drivers back upon linkage to libplplot. "But wait!". you might say, the drivers have to make calls to the core to do thier work! True, but this does not require linkage to libplplot. It only requires knowledge of the addresses of the core functions that are needed. This can be accomodated by introducing a structure which contains members which are the function pointers of the core library functions needed by drivers. struct PLEnv { void (*f1)( ... ); void (*f2)( ... ); void (*f3)( ... ); ... }; Then, you pass in the (initailized) PLEnv struct to the driver when you load/call it, and from there it all flows. This would have the effect of decoupling the drivers from the library, so that the library would not incur the linkage demands of hte drivers, and, moreover, the drivers themselves would not even be linked to the core library themselves either. Then, when PLplot is built, if you used dynamic drivers, you could make the core library either shared or static, and link to that, but nothing else. Granted, if application code wanted to get involved with direct use of the Tcl extensions like the Tcl extension matrix command, then you'd have to pick up -ltclmatrixd, but that would be only in special situations where the user knows they're doing this. Or, heck, we could even recombine libtclmatrix with libplplot for Tcl-enabled PLplot builds. Anyway the core idea here is to decouple (in the sense of linkage requirements) libplplot and the various driver modules. A full static build, in which dynamic drivers would not be employed, would continue to impute the driver dependencies to the core library, as today. But I think in this organizational model, most users would simply build with dynamic drivers, and benefit from a massively reduced linkage complexity situation for the core library. I'd like to hear what others think about this idea. |
From: Arjen M. <arj...@wl...> - 2006-09-26 12:00:05
|
Geoffrey Furnish wrote: >Some might remember that I've talked about this in the past. I actually got >started on it at one point, but it became effectively an abandoned project. >Combination of what was for me daunting AM/LT comlpexity, and a seemingly >irreversible transition into perpetual mania on my day job. > >Anticipating that we are building up to a seemingly increasingly likely >decision to cutover with finality to CBS, and given that linkage issues >remain a constant source of exasperation for users, I'm thinking its perhaps >a more interesting time to be pushing the core/driver split concept. > >This idea flows out of, or could be thought of as an extentions of, the >dynamic drivers thing, which was realisticaly, my last successful >contribution to the project. And the idea is simply this: > >Application programs link to PLplot via: -lplplot(d) > >That's it. No more linkage complexity. > >How would this work? Hee Hee. Simple, actually. Just pull the drivers all >out to dynamically loadable drivelets (driver modules), each with their own >linkage specification to external libraries (which would be handled by PLplot >during build time), and, specifically, *disallow* dependence of the drivers >back upon linkage to libplplot. > >"But wait!". you might say, the drivers have to make calls to the core to do >thier work! True, but this does not require linkage to libplplot. It only >requires knowledge of the addresses of the core functions that are needed. >This can be accomodated by introducing a structure which contains members which >are the function pointers of the core library functions needed by drivers. > >struct PLEnv >{ > void (*f1)( ... ); > void (*f2)( ... ); > void (*f3)( ... ); >... >}; > >Then, you pass in the (initailized) PLEnv struct to the driver when you >load/call it, and from there it all flows. > >This would have the effect of decoupling the drivers from the library, so >that the library would not incur the linkage demands of hte drivers, and, >moreover, the drivers themselves would not even be linked to the core library >themselves either. > > > Hi Geoffrey, the architecture you describe reminds me very much of the Tcl stubs facility. There you have exactly the same problem: loading extensions is one thing, but they need to be able to point back into the Tcl library itself. I suggest you have a look at how that is done: it has been around for many years now (at least since Tcl 8.1) and I think it would be easy to adopt for PLplot's way of dealing with drivers. Regards, Arjen |
From: Alan W. I. <ir...@be...> - 2006-09-22 17:34:24
|
On 2006-09-22 09:41-0500 Geoffrey Furnish wrote: > [...] Anyway the core idea here is > to decouple (in the sense of linkage requirements) libplplot and the various > driver modules. > > A full static build, in which dynamic drivers would not be employed, would > continue to impute the driver dependencies to the core library, as today. Actually, cmake is a fairly decent language which allows you a fairly high level of abstraction. The result is each device driver with special needs (e.g., psttf) fills in some standard variable names prefixed by the device name. Those variables are processed in a foreach loop over each driver name. To see the state of the art of this abstraction, please have a quick look at drivers/CMakeLists.txt. So the actual linking of dynamic device drivers is pretty straightforward once you have filled in all the appropriate variable names (in the device-related modules such as psttf.cmake in cmake/modules). The result of your decoupling efforts for the dynamic devices case would be to remove libplplot from the linking of each of those dynamic devices. That is a significant simplification of the final link commands that result for each dynamic device, but to be honest it would not simplify the linking of libplplot which would remain as simple as it is now (dependencies on the csiro libraries, libfreetype, and libltdl [see discussion below] but that is about it). The case is a bit different when dynamic devices are disabled. Currently for that case all device code must be put into libplplot which extraordinarily swells its linking dependencies, but if you can manage to remove the dependency of devices on libplplot, then at least that allows you the opportunity to put all device code into its own library without that library depending on libplplot thanks to your decoupling efforts. The software to implement plotting devices is quite dissimilar from software to implement plotting algorithms for the various kinds of plots we support so I generally favor splitting the two if we can. We already do that for the dynamic devices case, but your decoupling would also allow us to do that when dynamic devices were disabled which would be a nice result indeed. Also note that disabled dynamic devices is an important case on all the various windows platforms. Dynamic devices never worked for Cygwin and MinGW/MSYS for our ABS, and similarly dynamic devices will probably not work for Cygwin, MinGW/MSYS, and bare windows for our CBS for quite some time. The reason is that regardless of our build system, our code relies internally on the libltdl wrapper library to dynamically load device code with the same API on all platforms, and libltdl probably needs maturation to work properly for the various windows platforms. Finally, a benefit of your proposed decoupling efforts is you are generally reducing interactions between various pieces of our code, and any time you do that it should make our code easier to understand and debug. In sum, your proposed decoupling would reduce linking complexity for the dynamic drivers case, allow separating (into separate libraries and/or plug-ins) plotting device implementations from plotting algorithm implementations in all cases, and finally (probably the real biggie), generally make the code easier to understand and debug. So I am all for it. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the Yorick front-end to PLplot (yplot.sf.net); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |
From: Alan W. I. <ir...@be...> - 2006-09-23 04:20:29
|
On 2006-09-21 18:25-0700 Alan W. Irwin wrote: > (3) static build (-DBUILD_SHARED_LIBS=OFF -DENABLE_DYNDRIVERS=OFF[this value > forced when BUILD_SHARED_LIBS is OFF]) > > N.B. case (3) still uses shared external libraries, although the PLplot > libraries are static. > > The current status is there is a library ordering problem with the build of > the installed examples for case (3) above which I should be able to sort out > tomorrow. Done. The problem before was our CBS was configuring the pkg-config file using a combined list of libraries in the absolute path form (e.g., /path/to/lib/libwhatever.so [or .a]) and also in the -L/path/to/lib -lwhatever form. The combined list was in the correct order, but pkg-config would reorder it to put all the absolute path forms first which was the wrong order for static linking. The fix was to use string(REGEX REPLACE ....) (an extremely powerful CMake facility, by the way) to convert everything to the -L/path/to/lib -lwhatever form. pkg-config is much happier with the configured file, and now produces a much cleaner result. It now rejects duplicates and standard system library locations completely (which speeds up linking of the installed examples) and also keeps all libraries in the correct order (which keeps static linking happy). > Once that works, then I will be in a good position to finally > answer the interesting (paraphrased) question on plplot-general of whether > for Linux systems it is possible to make a completely static build of > PLplot. The answer to that question is yes, but it is a lot of work resolving static libraries that have to be included even when you turn off everything but one device (the wxwidgets device, in the case I studied to answer Joe Chuma's question). I will write up the finish of this static linking investigation on plplot-general. Alan __________________________ Alan W. Irwin Astronomical research affiliation with Department of Physics and Astronomy, University of Victoria (astrowww.phys.uvic.ca). Programming affiliations with the FreeEOS equation-of-state implementation for stellar interiors (freeeos.sf.net); PLplot scientific plotting software package (plplot.org); the Yorick front-end to PLplot (yplot.sf.net); the Loads of Linux Links project (loll.sf.net); and the Linux Brochure Project (lbproject.sf.net). __________________________ Linux-powered Science __________________________ |