plplot-devel

[Plplot-devel] tkwin doesn't work as dynamic driver :(

[Alan W. I. <ir...@be...>]

As an experiment I tried the following local changes to turn tkwin into
a dynamic driver (i.e., a tiny dll that is loaded as needed by the core
plplot routines):

cvs diff
Index: configure.in
===================================================================
RCS file: /cvsroot/plplot/plplot/cf/configure.in,v
retrieving revision 1.134
diff -u -3 -p -r1.134 configure.in
--- configure.in        11 Jul 2002 20:05:52 -0000      1.134
+++ configure.in        12 Jul 2002 00:47:18 -0000
@@ -133,12 +133,12 @@ hp7470, hp7580, lj_hpgl, ljiip, imp, xwi
 # Note specifically which drivers are known to be loadable.  Eventually,
 # hopefully, every driver will show up on this list.  However, we add them one
 # at a time since each will have its own peculiarities in the build process
-# (only ones missing at present: xwin and tk and tkwin).
+# (only ones missing at present: xwin and tk).

 define([PL_DYNAMIC_DRIVER_LIST],
 [plmeta null xterm tek4010 tek4010f tek4107 tek4107f mskermit \
 conex linuxvga vlt versaterm dg300 png jpeg cgm ps psc xfig ljii \
-hp7470 hp7580 lj_hpgl ljiip imp pbm gnome pstex ntk])
+hp7470 hp7580 lj_hpgl ljiip imp pbm gnome pstex ntk tkwin])

 # We think of a "device" as the output medium.  Could be a machine
 # style (Tektronix, X11), or a file format (Postscript).
Index: dyndrv.in
===================================================================
RCS file: /cvsroot/plplot/plplot/cf/dyndrv.in,v
retrieving revision 1.24
diff -u -3 -p -r1.24 dyndrv.in
--- dyndrv.in   9 Jul 2002 19:52:58 -0000       1.24
+++ dyndrv.in   12 Jul 2002 00:47:18 -0000
@@ -61,8 +61,8 @@ drivers/impress$(LIB_TAG).drv : shared/i
 drivers/gnome$(LIB_TAG).drv : shared/gnome$O $(PLLIBS)
        $(SHLIB_BUILD) $@ $< @GNOMELIBS@ $(driverlibs)

-#drivers/tkwin$(LIB_TAG).drv : shared/tkwin$O $(PLLIBS)
-#      $(SHLIB_BUILD) $@ $< @TKLIBS@ $(driverlibs)
+drivers/tkwin$(LIB_TAG).drv : shared/tkwin$O $(PLLIBS)
+       $(SHLIB_BUILD) $@ $< @TKLIBS@ $(driverlibs)

 drivers/ntk$(LIB_TAG).drv : shared/ntk$O $(PLLIBS)
        $(SHLIB_BUILD) $@ $< @TKLIBS@ $(driverlibs)

However, these changes do not work (so I won't commit them).

The problem is that the link step for applications shows undefined symbols:

make
gcc -g   plrender.o -L. -lplplotd -ltclmatrixd -o plrender \
         -litk3.1 -ltk8.3 -litcl3.1 -ltcl8.3 -L/usr/X11R6/lib -lX11 -ldl -lm
-lg2c -Wl,-rpath -Wl,/home/software/plplot_cvs/HEAD/plplot_working3/tmp
./libplplotd.so: undefined reference to PLColor_from_TkColor_Changed'
./libplplotd.so: undefined reference to plplot_tkwin_ccmap'
./libplplotd.so: undefined reference to plD_dispatch_init_tkwin'
./libplplotd.so: undefined reference to plD_open_tkwin'
./libplplotd.so: undefined reference to pltkwin_setBGFG'
collect2: ld returned 1 exit status
make: *** [plrender] Error 1

These symbols are referred to by plplotter.c, and putting plplotter.o into
libplplot is causing the problem.

My understanding is that similar problems (Tcl/Tk stuff integrated into
libplplot) keep the tk driver (and therefore xwin) from being dynamic.

I think it is crazy to have Tcl/Tk stuff (or code from any other front end)
inside libplplot so I hope we can rationalize this situation.  Of course,
once the Tcl/Tk front end code is in its own separate library, then the next
step is to turn that into a dll which could be dynamically loaded into wish
or tclsh (using the TEA-based approach, I presume). Separating out the
python front-end stuff into dll's is exactly the python approach we have
used both with the old and present python interfaces.  That idea works very
well so I am anxious to see it tried for the Tcl/Tk case, and a side benefit
I understand is it allows us to convert our remaining static drivers (tk,
xwin, and tkxwin) optionally into dynamic drivers as well.

I believe we have concluded previously that a separate Tcl/Tk library and
ultimately dll was the way to go. I don't have the expertise to do this
myself, but if somebody is inspired to take responsibility for this change,
I will certainly cheer from the sidelines, and also I will be more than
happy to help with the configuration and testing of the result!

Alan

email: ir...@be...
phone: 250-727-2902	FAX: 250-721-7715
snail-mail:
Dr. Alan W. Irwin
Department of Physics and Astronomy,
University of Victoria, P.O. Box 3055,
Victoria, British Columbia, Canada, V8W 3P6
__________________________

Linux-powered astrophysics
__________________________

Re: [Plplot-devel] tkwin doesn't work as dynamic driver :(

[Vince D. <vi...@sa...>]

Try putting plplotter.c in the tiny dll with tkwin.c...

-- Vince

<http://www.santafe.edu/~vince>


On Thu, 11 Jul 2002, Alan W. Irwin wrote:

> As an experiment I tried the following local changes to turn tkwin into
> a dynamic driver (i.e., a tiny dll that is loaded as needed by the core
> plplot routines):
> 
> cvs diff
> Index: configure.in
> ===================================================================
> RCS file: /cvsroot/plplot/plplot/cf/configure.in,v
> retrieving revision 1.134
> diff -u -3 -p -r1.134 configure.in
> --- configure.in        11 Jul 2002 20:05:52 -0000      1.134
> +++ configure.in        12 Jul 2002 00:47:18 -0000
> @@ -133,12 +133,12 @@ hp7470, hp7580, lj_hpgl, ljiip, imp, xwi
>  # Note specifically which drivers are known to be loadable.  Eventually,
>  # hopefully, every driver will show up on this list.  However, we add them one
>  # at a time since each will have its own peculiarities in the build process
> -# (only ones missing at present: xwin and tk and tkwin).
> +# (only ones missing at present: xwin and tk).
> 
>  define([PL_DYNAMIC_DRIVER_LIST],
>  [plmeta null xterm tek4010 tek4010f tek4107 tek4107f mskermit \
>  conex linuxvga vlt versaterm dg300 png jpeg cgm ps psc xfig ljii \
> -hp7470 hp7580 lj_hpgl ljiip imp pbm gnome pstex ntk])
> +hp7470 hp7580 lj_hpgl ljiip imp pbm gnome pstex ntk tkwin])
> 
>  # We think of a "device" as the output medium.  Could be a machine
>  # style (Tektronix, X11), or a file format (Postscript).
> Index: dyndrv.in
> ===================================================================
> RCS file: /cvsroot/plplot/plplot/cf/dyndrv.in,v
> retrieving revision 1.24
> diff -u -3 -p -r1.24 dyndrv.in
> --- dyndrv.in   9 Jul 2002 19:52:58 -0000       1.24
> +++ dyndrv.in   12 Jul 2002 00:47:18 -0000
> @@ -61,8 +61,8 @@ drivers/impress$(LIB_TAG).drv : shared/i
>  drivers/gnome$(LIB_TAG).drv : shared/gnome$O $(PLLIBS)
>         $(SHLIB_BUILD) $@ $< @GNOMELIBS@ $(driverlibs)
> 
> -#drivers/tkwin$(LIB_TAG).drv : shared/tkwin$O $(PLLIBS)
> -#      $(SHLIB_BUILD) $@ $< @TKLIBS@ $(driverlibs)
> +drivers/tkwin$(LIB_TAG).drv : shared/tkwin$O $(PLLIBS)
> +       $(SHLIB_BUILD) $@ $< @TKLIBS@ $(driverlibs)
> 
>  drivers/ntk$(LIB_TAG).drv : shared/ntk$O $(PLLIBS)
>         $(SHLIB_BUILD) $@ $< @TKLIBS@ $(driverlibs)
> 
> However, these changes do not work (so I won't commit them).
> 
> The problem is that the link step for applications shows undefined symbols:
> 
> make
> gcc -g   plrender.o -L. -lplplotd -ltclmatrixd -o plrender \
>          -litk3.1 -ltk8.3 -litcl3.1 -ltcl8.3 -L/usr/X11R6/lib -lX11 -ldl -lm
> -lg2c -Wl,-rpath -Wl,/home/software/plplot_cvs/HEAD/plplot_working3/tmp
> ./libplplotd.so: undefined reference to PLColor_from_TkColor_Changed'
> ./libplplotd.so: undefined reference to plplot_tkwin_ccmap'
> ./libplplotd.so: undefined reference to plD_dispatch_init_tkwin'
> ./libplplotd.so: undefined reference to plD_open_tkwin'
> ./libplplotd.so: undefined reference to pltkwin_setBGFG'
> collect2: ld returned 1 exit status
> make: *** [plrender] Error 1
> 
> These symbols are referred to by plplotter.c, and putting plplotter.o into
> libplplot is causing the problem.
> 
> My understanding is that similar problems (Tcl/Tk stuff integrated into
> libplplot) keep the tk driver (and therefore xwin) from being dynamic.
> 
> I think it is crazy to have Tcl/Tk stuff (or code from any other front end)
> inside libplplot so I hope we can rationalize this situation.  Of course,
> once the Tcl/Tk front end code is in its own separate library, then the next
> step is to turn that into a dll which could be dynamically loaded into wish
> or tclsh (using the TEA-based approach, I presume). Separating out the
> python front-end stuff into dll's is exactly the python approach we have
> used both with the old and present python interfaces.  That idea works very
> well so I am anxious to see it tried for the Tcl/Tk case, and a side benefit
> I understand is it allows us to convert our remaining static drivers (tk,
> xwin, and tkxwin) optionally into dynamic drivers as well.
> 
> I believe we have concluded previously that a separate Tcl/Tk library and
> ultimately dll was the way to go. I don't have the expertise to do this
> myself, but if somebody is inspired to take responsibility for this change,
> I will certainly cheer from the sidelines, and also I will be more than
> happy to help with the configuration and testing of the result!
> 
> Alan
> 
> email: ir...@be...
> phone: 250-727-2902	FAX: 250-721-7715
> snail-mail:
> Dr. Alan W. Irwin
> Department of Physics and Astronomy,
> University of Victoria, P.O. Box 3055,
> Victoria, British Columbia, Canada, V8W 3P6
> __________________________
> 
> Linux-powered astrophysics
> __________________________
> 
> 
> 
> -------------------------------------------------------
> This sf.net email is sponsored by:ThinkGeek
> PC Mods, Computing goodies, cases & more
> http://thinkgeek.com/sf
> _______________________________________________
> Plplot-devel mailing list
> Plp...@li...
> https://lists.sourceforge.net/lists/listinfo/plplot-devel
> 

Re: [Plplot-devel] tkwin doesn't work as dynamic driver :(

[Alan W. I. <ir...@be...>]

On Fri, 12 Jul 2002, Vince Darley wrote:

> Try putting plplotter.c in the tiny dll with tkwin.c...
>

I think that might work. Here is my best guess at the mental model of the
linking, subject to correction by the experts here.

(a) You would need some tcl/tk mechanism to dlopen that combined dll.  I
assume that would be where TEA came in.

(b) the dll itself would be linked to libplplot as a shared library so that
library gets automatically loaded as well.  (At least that is what seems
to happen for the python plplot extension module which is a dll that calls
libplplot functions).

(c) libplplot has calls to tkwin.o itself which it would dlopen according to
the dynamic driver mechanism in plcore.c.  Under python this just works, and
I assume the same would be true under tcl/tk except the dlopen would
automatically turn into an effective no-op since that dll would have already
been loaded by the tcl dlopen.

(d) libplplot is currently linked against the shared libtclmatrix library.
From our recent experience with "load libplplotd.so Plplotter" under wish
that linking against libtclmatrix is all that is required to make the
libtclmatrix functions available to the tcl examples that were run under
wish with runAllDemos.tcl.

So I think this linking scenario you suggest would work.

However, IMO a much better idea is to split off everything (not just
plplotter.o) concerning tcl/tk helper functions from libplplot and put that
into a dll.  The upside of that for other front ends such as python is it
makes libplplot substantially smaller.  Also, I think reserving libplplot
strictly for plotting functions is just good organization that makes
everything much easier to maintain.  Splitting off the tcl/tk helper
functions would be a great step in the right direction, and I know Maurice
intends to split off the fortran stuff as well for similar organizational
reasons.

There is another less important issue.  It would probably be worthwhile to
keep the tkwin dll separate just to be in conformity with the way the rest
of the device dll's are treated.  So with these two changes (a) gets changed
to

(a') tcl dlopens tcl/tk helper function dll *and* tkwin dll under TEA

and (c) gets changed to

(c') the *tkwin* dll gets dlopened by plcore which would effectively be a
no-op.

Vince (or anybody else here), don't hesitate to comment on list about
refinements of this mental model of the way the linking happens if I was
unclear or got something wrong.  Of course our understanding of what goes on
in (b) and (c') and (d) doesn't really matter (so long as they work) because
those parts are all done automatically.

So the next question in my mind is how do you implement (a')?  In analogy
with the setup.py way of building the python interface dll, should the
tcl/tk helper function dll (and tkwin dll) be built under TEA control?  If
so, and given that I collect a list of tcl/tk helper C source files that
should be compiled and put into the tcl/tk helper dll rather than libplplot,
what is the cookbook?

Alan

Re: [Plplot-devel] tkwin doesn't work as dynamic driver :(

[Vince D. <vi...@sa...>]

I agree with all the below, except that 'TEA' is misleading.  All TEA is
is a collection fof stuff: a suggested way of building, installing, a
suggested directory structure, code conventions, etc.  None of it is
necessary.

Now that we have built a shared library with all of this, and it works, we
are "done".  Obviously there is cleanup that can happen, but TEA isn't
going to give us anything else (Ok: this isn't quite true, I mentioned
that we should ultimately compile with USE_TCL_STUBS and USE_TK_STUBS so
that we don't care what version of Tcl/Tk we load into, but again that can
wait).

So: to do what you want, you just need to build two separate dlls.  As you
say, there should be a base plplot dll and a separate tk-plplot shared
library with all the tcl-tk related stuff (in the future we might want
several of these, but lets ignore that for the moment).

On Fri, 12 Jul 2002, Alan W. Irwin wrote:
> (a) You would need some tcl/tk mechanism to dlopen that combined dll.  I
> assume that would be where TEA came in.

That will happen automatically.  No tea.

> (b) the dll itself would be linked to libplplot as a shared library so that
> library gets automatically loaded as well.  (At least that is what seems
> to happen for the python plplot extension module which is a dll that calls
> libplplot functions).

Exactly the same for Tcl.

> (c) libplplot has calls to tkwin.o itself which it would dlopen according to
> the dynamic driver mechanism in plcore.c.  Under python this just works, and
> I assume the same would be true under tcl/tk except the dlopen would
> automatically turn into an effective no-op since that dll would have already
> been loaded by the tcl dlopen.

Yup.

> However, IMO a much better idea is to split off everything (not just
> plplotter.o) concerning tcl/tk helper functions from libplplot and put that
> into a dll.  The upside of that for other front ends such as python is it
> makes libplplot substantially smaller.  Also, I think reserving libplplot
> strictly for plotting functions is just good organization that makes
> everything much easier to maintain.  Splitting off the tcl/tk helper
> functions would be a great step in the right direction, and I know Maurice
> intends to split off the fortran stuff as well for similar organizational
> reasons.

Yes!

> There is another less important issue.  It would probably be worthwhile to
> keep the tkwin dll separate just to be in conformity with the way the rest
> of the device dll's are treated.  So with these two changes (a) gets changed
> to

I don't see the need to complicate things this much, but feel free ;-)

> So the next question in my mind is how do you implement (a')?  In analogy
> with the setup.py way of building the python interface dll, should the
> tcl/tk helper function dll (and tkwin dll) be built under TEA control?  If
> so, and given that I collect a list of tcl/tk helper C source files that
> should be compiled and put into the tcl/tk helper dll rather than libplplot,
> what is the cookbook?

You don't need any helper functions at all.  Just build a shared library
and off you go.

cheers,

Vince.

Re: [Plplot-devel] tkwin doesn't work as dynamic driver :(

[Alan W. I. <ir...@be...>]

On Fri, 12 Jul 2002, Vince Darley wrote:

> Just build a shared library and off you go.

The light dawns....  Before I thought dll's and shared libraries were
structurally different (i.e., compiled with different flags).  However, a
google search on the phrase "linux dll" turned up an article which said they
were the same, and a look at the flags for creating, e.g., the ntk device
dll, ntkd.drv, confirm this.

So I guess the only remaining distinction to make in Linux is whether the
dll is loaded under your code's direct control by a call to dlopen (as
occurs in plcore.c, for example with the device dll's) or whether it is
loaded automatically when your programme is first loaded (or as an indirect
result of calling dlopen for a library that is linked against another
library).

That suggests an easy solution for the problems I was having with making
tkwin a dll.  As an experiment I will link libplplot against tkwind.drv.
I believe that will resolve the symbols that were unresolved before.

If/when that part of the mental model is confirmed, then I will look at
splitting off the tcl/tk stuff (as libtclplplot, say) from libplplot.  This
will mean I will drop the experiment, e.g., libplplot will no longer be
linked against tkwind.drv.  Instead, libtclplplot will be linked against
tkwind.drv and libplplot.  Then applications will only be linked against
libtclplplot in special cases (such as pltcl and plserver) where tcl/tk is
actually being used.  And if the mental model is correct, then simply
loading libtclplplot from wish is all that will be required to automatically
load every dll that will be needed.

Let me know if there are any problems with the revised mental model of
the dynamic linking process.

Assuming there are no such problems I will go ahead with the experiment and
if that works the splitting, and if that also works under Linux I will
commit the results to CVS HEAD so others will be able try it out.

Alan