plplot-devel

[Plplot-devel] New java configuration completed and tested and other configuration news

[Alan W. I. <ir...@be...>]

I have now configured our java interface with the new configuration approach.
The new wrapper library is called libplplotjava.so.  This completes the
move of all interface wrapper code from libplplot to special wrapper libraries
which are built or not depending on the configuration options.

I have done some limited testing of a few java examples to make sure they
are installed okay and can be built properly with the instructions in
$prefix/lib/java/plplot/examples/README.javademos.

Note, Java is still experimental (principally because of the limited API
that we have at this time) so this configuration effort was not release
critical at all, but I decided to do it anyway while the required
new configuration for our other interfaces was fresh in my mind.

The only other interface not configured with the new approach at this time
is perl, but I am going to leave that strictly to Rafael because the current
version of the perl interface to PLplot is incomplete, and Rafael has
tentative plans to change it over to PDL in any case.  All drivers
(configured both dynamically and statically) are also now configured with
the new scheme. So there should be no further major additions to the current
configuration scheme until substantially after the release (and only then if
Rafael finishes with perl or somebody donates a new interface or driver).

The only other configuration news is that Vince has recently tried the new
configuration scheme on the cygwin platform.  It got all the way through the
compilation stage, but failed at the link stage because I had forgotten an
autoconf macro that is important for cygwin linking according to the
autobook. So we are now on the second iteration there.

We still have an important need for testing of the new configuration scheme
on solaris with tcl/tk, but reports on any other Unix platforms (e.g., MAC
OS X, HPUX, IRIX, AIX, additional OSF1, and additional Linux) would be most
appreciated as well.

We still also have an important need to get the -dev tk problem solved.
(tcl/tk cannot seem to find the scripts in $prefix/lib/plplot-5.1.0/tcl under
these circumstances although there are no problems in other circumstances.)
Is there any plplot tcl guru here who is willing to take this on?

Alan

email: ir...@be...
phone: 250-727-2902	FAX: 250-721-7715
snail-mail:
Dr. Alan W. Irwin
Department of Physics and Astronomy,
University of Victoria, P.O. Box 3055,
Victoria, British Columbia, Canada, V8W 3P6
__________________________

Linux-powered astrophysics
__________________________

Re: [Plplot-devel] New java configuration completed and tested and other configuration news

[<jc...@fe...>]

On Tuesday 17 December 2002 05:51, Alan W. Irwin wrote:
=2E..
| All drivers (configured both dynamically and statically) are also now
| configured with the new scheme. So there should be no further major
| additions to the current configuration scheme

I don't quite agree. If your AT effort does simplify our (developper)=20
job, by automatically providing plplot on (potentially) new platforms,=20
it does turns the development cycle longer and tedious, because the=20
need to "make install".

The problem is how to develop in the build tree without the need to=20
install. There are three situations:

1-finding data like fonts, etc. This should be pretty easy, so I would=20
solve it last.

2-finding libraries: libraries will be found by executables, thanks to a=20
wrapper with the same name of the executable that points to the .libs=20
hidden libraries. This point is also solved, but we must be sure that=20
the build tree library is searched first than an installed library by=20
the executable wrappers.

3-finding drivers: this currently does not work. drivers must be=20
installed, and this means "make && make install" after each=20
modification to a driver. As "make install" also installs other files,=20
this might add difficulties to a debugging cycle, see bellow.

4-interpreted languages that dlopen they own plplot extensions.=20
Currently Octave only works with plplot installed. I don't know if=20
other languages has the same problem, but to avoid misunderstandings,=20
please "rm -rf <prefix>" before reporting a working binding: in my=20
case, I do "make && make install &&  rm -rf /usr/local/test/bin=20
/usr/local/test/include/ /usr/local/test/python/ /usr/local/test/share/=20
/usr/local/test/lib/lib* /usr/local/test/lib/plplot5.1.0/tcl". When=20
point 1 and 3 above becomes solved, than a plain "rm -rf <prefix>" will=20
be enough.

Thus, while I agree that AT is a good thing, there is still work to do=20
to make our developper life confortable.

Joao

PS-No, I will not upgrade my 700MHz P4 :-)

Re: [Plplot-devel] New java configuration completed and tested and other configuration news

[Alan W. I. <ir...@be...>]

On Tue, 17 Dec 2002, [iso-8859-1] Jo=E3o Cardoso wrote:

> On Tuesday 17 December 2002 05:51, Alan W. Irwin wrote:
> ...
> | All drivers (configured both dynamically and statically) are also now
> | configured with the new scheme. So there should be no further major
> | additions to the current configuration scheme
>
> I don't quite agree. If your AT effort does simplify our (developper)
> job, by automatically providing plplot on (potentially) new platforms,
> it does turns the development cycle longer and tedious, because the
> need to "make install".
>
> The problem is how to develop in the build tree without the need to
> install. There are three situations:
>
> 1-finding data like fonts, etc. This should be pretty easy, so I would
> solve it last.
>
> 2-finding libraries: libraries will be found by executables, thanks to a
> wrapper with the same name of the executable that points to the .libs
> hidden libraries. This point is also solved, but we must be sure that
> the build tree library is searched first than an installed library by
> the executable wrappers.
>
> 3-finding drivers: this currently does not work. drivers must be
> installed, and this means "make && make install" after each
> modification to a driver. As "make install" also installs other files,
> this might add difficulties to a debugging cycle, see bellow.

This should be as simple as finding the build (as opposed to installed)
fonts, maps, etc.  The drivers.db file is generated (by configure) in
plplot/drivers, and all the xxx_drv.so files are built in plplot/drivers/.l=
ib.

>
> 4-interpreted languages that dlopen they own plplot extensions.
> Currently Octave only works with plplot installed. I don't know if
> other languages has the same problem.

One issue is that for every scripting language you have to tell it the
location of the extension.  But we know exactly where all of these
extensions are built so it is a straightforward problem to solve.  For
example, in the python case the installed extensions are found in the
following way for my configured setup:

(This is the totality of plplot/examples/plplot_python_start.py).
# Append to effective python path so that can find plplot modules.
import sys
sys.path.insert (0, "/usr/local/plplot_at/lib/python2.1/site-packages")

So it is a matter of changing this file if you want to refer to the build
location of the extension.  For octave, I believe you use the -p parameter
on the octave command to specify the location of the extension. The
automatic searching for extensions by tcl is still an art rather than a
science so that will be the most problematic one.  BTW, for tcl there is an
additional issue of finding the tcl startup scripts in the build location
rather than the install location.

An additional issue is the rpath location of libplplot that all extensions
must link to. That is automatically taken care of for all extensions that
are built by libtool; at build time the build location of libplplot is used
and at install time everything is automatically relinked to use the
installed location of libplplot.  The only exception to this is the octave
plplot extension because currently that particular extension is not built b=
y
libtool.  However, privately we discussed how you could build it with
libtool, and I believe that would completely solve this issue.

A final issue with getting things to work both in the build and install
areas is confusion over which you are using.  I don't know how many times i=
n
the past I got burned by this issue.  Thus, I strongly suggest we have a
configuration option which allows us to make the choice between either the
build area or installed area with no possible confusion between the two.

> but to avoid misunderstandings,
> please "rm -rf <prefix>" before reporting a working binding: in my
> case, I do "make && make install &&  rm -rf /usr/local/test/bin
> /usr/local/test/include/ /usr/local/test/python/ /usr/local/test/share/
> /usr/local/test/lib/lib* /usr/local/test/lib/plplot5.1.0/tcl". When
> point 1 and 3 above becomes solved, than a plain "rm -rf <prefix>" will
> be enough.

These complicated directions are *only* required if you are using "make
check".  For now to avoid these complications I would advise avoiding "make
check". Instead, I suggest sticking strictly with using installed results
(i.e., the current development cycle should consist of a programme change,
rm -rf <prefix>, make install, do test in the installed area). By the way,
we are only talking an additional 20 seconds here (at least on my 600MHz
machine) to do "make install", and it is substantially less if you limit th=
e
configuration to just the features and driver(s) you are testing.

>
> Thus, while I agree that AT is a good thing, there is still work to do
> to make our developper life confortable.

Although only an additional 20 seconds in the test cycle is involved for
developers, such latency is often annoying.  Thus, if you would like to wor=
k
on issues 1-4 and the additional issues with using the build area for tests
that I have discussed, then please go ahead.  However, I don't view this
effort as release critical since it doesn't affect our users, and it is
becoming critical to make the release so our users can enjoy all the bug
fixes and new features already on CVS HEAD compared to what was available
for plplot-5.1.0.  So if you are aware of user-related issues that you coul=
d
fix or improve before the release, I hope you will give them higher priorit=
y
than the removal of this developer cycle latency.

Also, please don't freeze PLplot improvements at this time in anticipation
of a release. Christmas is fast approaching so I have now resigned myself t=
o
some time in January as the earliest possible release date, and it could be
indefinitely later than that because nobody at this time has stepped forwar=
d
and taken responsibility for the -dev tk issue or the solaris with tcl/tk
test that are currently holding up the release.

Alan

email: ir...@be...
phone: 250-727-2902=09FAX: 250-721-7715
snail-mail:
Dr. Alan W. Irwin
Department of Physics and Astronomy,
University of Victoria, P.O. Box 3055,
Victoria, British Columbia, Canada, V8W 3P6
__________________________

Linux-powered astrophysics
__________________________

Re: [Plplot-devel] New java configuration completed and tested and other configuration news

[Joao C. <jc...@fe...>]

On Tuesday 17 December 2002 19:54, Alan W. Irwin wrote:
> On Tue, 17 Dec 2002, [iso-8859-1] Jo=E3o Cardoso wrote:
> > On Tuesday 17 December 2002 05:51, Alan W. Irwin wrote:
> > ...
> >
> > | All drivers (configured both dynamically and statically) are also n=
ow
> > | configured with the new scheme. So there should be no further major
> > | additions to the current configuration scheme
> >
> > I don't quite agree. If your AT effort does simplify our (developper)
> > job, by automatically providing plplot on (potentially) new platforms=
,
> > it does turns the development cycle longer and tedious, because the
> > need to "make install".
> >
> > The problem is how to develop in the build tree without the need to
> > install. There are three situations:
> >
> > 1-finding data like fonts, etc. This should be pretty easy, so I woul=
d
> > solve it last.
> >
> > 2-finding libraries: libraries will be found by executables, thanks t=
o a
> > wrapper with the same name of the executable that points to the .libs
> > hidden libraries. This point is also solved, but we must be sure that
> > the build tree library is searched first than an installed library by
> > the executable wrappers.
> >
> > 3-finding drivers: this currently does not work. drivers must be
> > installed, and this means "make && make install" after each
> > modification to a driver. As "make install" also installs other files=
,
> > this might add difficulties to a debugging cycle, see bellow.
>
> This should be as simple as finding the build (as opposed to installed)
> fonts, maps, etc.  The drivers.db file is generated (by configure) in
> plplot/drivers, and all the xxx_drv.so files are built in
> plplot/drivers/.lib.

But I would expect AT to make that. It's hard to belive that developpers =
have=20
to rely on a hidden directory files... and as you are our AT expert...

> > 4-interpreted languages that dlopen they own plplot extensions.
> > Currently Octave only works with plplot installed. I don't know if
> > other languages has the same problem.
>
> One issue is that for every scripting language you have to tell it the
> location of the extension.  But we know exactly where all of these
> extensions are built so it is a straightforward problem to solve.  For
> example, in the python case the installed extensions are found in the
> following way for my configured setup:
>
> (This is the totality of plplot/examples/plplot_python_start.py).
> # Append to effective python path so that can find plplot modules.
> import sys
> sys.path.insert (0, "/usr/local/plplot_at/lib/python2.1/site-packages")
>
> So it is a matter of changing this file if you want to refer to the bui=
ld
> location of the extension.  For octave, I believe you use the -p parame=
ter
> on the octave command to specify the location of the extension. The
> automatic searching for extensions by tcl is still an art rather than a
> science so that will be the most problematic one.  BTW, for tcl there i=
s an
> additional issue of finding the tcl startup scripts in the build locati=
on
> rather than the install location.
>
> An additional issue is the rpath location of libplplot that all extensi=
ons
> must link to.

This is my issue.

> That is automatically taken care of for all extensions that
> are built by libtool; at build time the build location of libplplot is =
used
> and at install time everything is automatically relinked to use the
> installed location of libplplot.  The only exception to this is the oct=
ave
> plplot extension because currently that particular extension is not bui=
lt
> by libtool.

Ah, you mean that if octave modules extension was .so instead of .oct tha=
n the=20
problem would be solved? I could try to use and establish a link.

>  However, privately we discussed how you could build it with
> libtool, and I believe that would completely solve this issue.

Yes, but I didn't gave you the final anwser: mkoctfile is really build fr=
om=20
mkoctfile.in at configure time, after configure and libtool detects the=20
systems peculiarities. Also, mkoctfile if for ordinary users to use, not=20
necessarily to developpers, and all details are thus hidden. So mkoctfile=
=20
enables to build octave modules in all unix systems, win2000, djgpp, win9=
8,=20
macOS, etc. That's why I want to use it -- whatever the system is, using=20
mkoctfile guarantees that a module will be correctly built and linked.
To use it, it must be compiled twice, once for the build tree and again f=
or=20
the install tree. This could be simplified, I think, and only relink, not=
=20
recompile.

> A final issue with getting things to work both in the build and install
> areas is confusion over which you are using.  I don't know how many tim=
es
> in the past I got burned by this issue.  Thus, I strongly suggest we ha=
ve a
> configuration option which allows us to make the choice between either =
the
> build area or installed area with no possible confusion between the two=
=2E

No problem.

> > but to avoid misunderstandings,
> > please "rm -rf <prefix>" before reporting a working binding: in my
> > case, I do "make && make install &&  rm -rf /usr/local/test/bin
> > /usr/local/test/include/ /usr/local/test/python/ /usr/local/test/shar=
e/
> > /usr/local/test/lib/lib* /usr/local/test/lib/plplot5.1.0/tcl". When
> > point 1 and 3 above becomes solved, than a plain "rm -rf <prefix>" wi=
ll
> > be enough.
>
> These complicated directions are *only* required if you are using "make
> check".

How the hell can I test the library, drivers, bindings, and examples? I m=
ust=20
do a "make check"!

By the way, "make check" is generally used to auto-check, where besides b=
eing=20
compiled, the tests are run and its output compared with know good result=
s.=20
In our case the target should be called "make demos". And to avoid the ne=
ed=20
to make all demos, we should have several targets, make cdemos, fdemos,=20
cxxdemos, alldemos, etc.

>  For now to avoid these complications I would advise avoiding "make
> check". Instead, I suggest sticking strictly with using installed resul=
ts
> (i.e., the current development cycle should consist of a programme chan=
ge,
> rm -rf <prefix>, make install, do test in the installed area).

But this is exactly what I'm saying: I'm not satisfied with the current A=
M=20
scheme!

> By the way,
> we are only talking an additional 20 seconds here (at least on my 600MH=
z
> machine) to do "make install", and it is substantially less if you limi=
t
> the configuration to just the features and driver(s) you are testing.

You are talking as a sales man, selling the lower figures :-).

In a real situation, where normally  I have the plplot tree configured fo=
r all=20
drivers and bindings, if I ocasionaly detect something that needs a chang=
e, I=20
must wait much longer than that. Only on very specific ocasions I have a=20
minimum set of drivers/bindings configured.

> > Thus, while I agree that AT is a good thing, there is still work to d=
o
> > to make our developper life confortable.
>
> Although only an additional 20 seconds in the test cycle is involved fo=
r
> developers, such latency is often annoying.  Thus, if you would like to
> work

If *we* agree that it is annoying, than *we* should work... sorry, you st=
arted=20
it...

And I like to reach a consensus of what to do and how before going forwar=
d.=20
When the decisions affects others. I dont ask when I correct a bug or=20
introduce a feature, that makes no sense; but on global issues, I like to=
=20
agree on what to do. This is also the reason why I'm not changing things.=
 (=20
and the total unknowledge on AT, which I would like to keep :()

But I don't want an ad hoc solution such as searching in the .libs=20
directories. Only if it is necessary, and its hard to believe that  AT, w=
hich=20
where made to easy developpers life will end up complicating it.

Anyhow, I don't have the time right now. And at the begining of the next =
year=20
I have to make ready a new discipline course. So it looks that AM will st=
ay=20
as is now.

> on issues 1-4 and the additional issues with using the build area for
> tests that I have discussed, then please go ahead.  However, I don't vi=
ew
> this effort as release critical since it doesn't affect our users,

It will be if they want to test before install. You have to tell them=20
"configure with a test prefix, make, make install. If you like, than=20
configure again with the righ prefix, then make clean, make and make inst=
all=20
again."  I have already used this argument. Not critical, but annoying.=20
Nonetheless this issue I agree with a release at this (near) time.

> and it
> is becoming critical to make the release so our users can enjoy all the=
 bug
> fixes and new features already on CVS HEAD compared to what was availab=
le
> for plplot-5.1.0.  So if you are aware of user-related issues that you
> could fix or improve before the release, I hope you will give them high=
er
> priority than the removal of this developer cycle latency.
>
> Also, please don't freeze PLplot improvements at this time in anticipat=
ion
> of a release. Christmas is fast approaching so I have now resigned myse=
lf
> to some time in January as the earliest possible release date, and it c=
ould
> be indefinitely later than that because nobody at this time has stepped
> forward and taken responsibility for the -dev tk issue or the solaris w=
ith
> tcl/tk test that are currently holding up the release.
>
> Alan
>
> email: ir...@be...
> phone: 250-727-2902=09FAX: 250-721-7715
> snail-mail:
> Dr. Alan W. Irwin
> Department of Physics and Astronomy,
> University of Victoria, P.O. Box 3055,
> Victoria, British Columbia, Canada, V8W 3P6
> __________________________
>
> Linux-powered astrophysics
> __________________________
>
>
>
>
>
> -------------------------------------------------------
> This sf.net email is sponsored by:
> With Great Power, Comes Great Responsibility
> Learn to use your power at OSDN's High Performance Computing Channel
> http://hpc.devchannel.org/
> _______________________________________________
> Plplot-devel mailing list
> Plp...@li...
> https://lists.sourceforge.net/lists/listinfo/plplot-devel

Re: [Plplot-devel] New java configuration completed and tested and other configuration news

[Alan W. I. <ir...@be...>]

On Wed, 18 Dec 2002, Joao Cardoso wrote:

> > An additional issue is the rpath location of libplplot that all extensions
> > must link to.
>
> This is my issue.
>
> > That is automatically taken care of for all extensions that
> > are built by libtool; at build time the build location of libplplot is used
> > and at install time everything is automatically relinked to use the
> > installed location of libplplot.  The only exception to this is the octave
> > plplot extension because currently that particular extension is not built
> > by libtool.
>
> [...] That's why I want to use it -- whatever the system is, using
> mkoctfile guarantees that a module will be correctly built and linked.
> To use it, it must be compiled twice, once for the build tree and again for
> the install tree. This could be simplified, I think, and only relink, not
> recompile.

If you want to stick with mkoctfile that is okay with me, but then you have
to explicitly find the solution for linking a version at build time with a
build rpath and relinking at install time with a install rpath.  I am sure
there is a reasonably straightforward way to do this with the appropriate
automake syntax and time stamp files, but I haven't thought very much about
it since I was concentrating on a different solution.

> > These complicated directions are *only* required if you are using "make
> > check".
>
> How the hell can I test the library, drivers, bindings, and examples? I must
> do a "make check"!

This claim weakens your argument since make check is definitely not
necessary for testing. Instead do the following: cd to the install location
of the c examples, $prefix/lib/plplot5.1.0/examples/c. "make -f
Makefile.examples x01c". (I have mentioned this step before to the list, but
perhaps you failed to notice it?) Run x01c with the variety of drivers you
want to test.  This procedure tests the library, drivers, c binding, and
example. If you want to test other bindings then go to the appropriate
directory in the installed examples instead.  If you want to test all the
non-interactive examples simply run plplot-test.sh, but that is overkill and
running single examples usually does all the testing you need.

Have you ever tried testing this way in the installed location?  If you try
it you may find it is not that bad, and then the whole argument goes away.

>
> > By the way,
> > we are only talking an additional 20 seconds here (at least on my 600MHz
> > machine) to do "make install", and it is substantially less if you limit
> > the configuration to just the features and driver(s) you are testing.
>
> You are talking as a sales man, selling the lower figures :-).

Nope.  I am not trying to sell this at all.  Instead, I am trying to give
you information about my real recent experiences with testing programme
changes with the new configuration scheme.  To expand this information I
decided to time all the steps accurately for you and the others so there is
no mistake about the relatively small amount of time we are discussing here.
Everything done after a rm -rf /usr/local/plplot_at/* and make distclean on
my AMD 600MHz machine.

(1) time ./bootstrap.sh >& bootstrap.out
18.640u 0.390s 0:20.35 93.5%    0+0k 0+0io 7383pf+0w

(2) time ./configure --prefix=/usr/local/plplot_at  \
--enable-dyndrivers --enable-octave --enable-java >& configure.out
17.270u 10.440s 0:32.67 84.8%   0+0k 0+0io 406819pf+0w

Note this configure includes every driver except for linuxvga (missing library
on my system) and gnome and ntk (turned off by default).  Also
tcl/tk, python, c++, f77, java, and octave.

(3) time make >& make.out
174.740u 17.870s 3:45.53 85.4%  0+0k 0+0io 556384pf+0w

(4) time make install >& make_install.out
14.640u 8.040s 0:26.71 84.9%    0+0k 0+0io 282441pf+0w

Before I got ~20 seconds (as I stated) rather than 27 seconds for this step,
but then I probably wasn't installing java and octave.

(5) touch src/plcore.c (emulate a change in plcore.c).

(6) rm -rf /usr/local/plplot_at/*

(7)  time make install >& make_install.out
19.830u 8.910s 0:32.35 88.8%    0+0k 0+0io 309732pf+0w

Steps 5-7 emulate a change in source code. As expected, the make install is
a bit longer than step 4 since plcore.c had to be compiled (I checked this
occurred).  But if you don't configure a lot of different components and
drivers, this ~30 second near worst case latency can be made substantially
smaller. Also note that autotools does very good dependency tracking so when
you change a file a minimum of recompilation and relinking is done for
the install.


> In a real situation, where normally  I have the plplot tree configured for all
> drivers and bindings, if I ocasionaly detect something that needs a change, I
> must wait much longer than that. Only on very specific ocasions I have a
> minimum set of drivers/bindings configured.

See real numbers above for near worst case ~30 sec latency from make install
after a change.  If you do the same tests do you really get much slower
results on your 700MHz machine? Note neither machine is a front end machine
any more and in any case you can get substantially lower latency if you cut
down on unused drivers and front ends.  So my conclusion is not many
developers are going to be bothered that much by this latency.

>
> > > Thus, while I agree that AT is a good thing, there is still work to do
> > > to make our developper life confortable.
> >
> > Although only an additional 20 seconds in the test cycle is involved for
> > developers, such latency is often annoying.  Thus, if you would like to
> > work
>
> If *we* agree that it is annoying, than *we* should work... sorry, you started
> it...

I was trying to "lean over backwards" to understand your concerns.  However,
your argument forced me to look at the timing numbers again, and they
confirm this is not a high priority for my needs (or most other developer's
needs). However, since you obviously do feel this is a high priority, then I
have been willing to cooperate by laying out for you what you have to do in
a fair amount of detail which has taken a substantial amount of my time.
Much of it is C code change such as telling plcore.c where the drivers are
built (see below) or where the font and map data are located.

>
> But I don't want an ad hoc solution such as searching in the .libs
> directories. Only if it is necessary, and its hard to believe that  AT, which
> where made to easy developpers life will end up complicating it.

There is nothing magical or ad hoc about .libs.  That is the standard
subdirectory location in any particular directory where libtool puts its
results.  If that's where the drivers are built, then you have to tell
plcore.c to look in that location.  It is exactly analogous to the old days
where we had plcore look in plplot/tmp as well as the install locations.  It
really is as simple as that.

Alan

email: ir...@be...
phone: 250-727-2902	FAX: 250-721-7715
snail-mail:
Dr. Alan W. Irwin
Department of Physics and Astronomy,
University of Victoria, P.O. Box 3055,
Victoria, British Columbia, Canada, V8W 3P6
__________________________

Linux-powered astrophysics
__________________________

Re: [Plplot-devel] New java configuration completed and tested and other configuration news

[Joao C. <jc...@fe...>]

On Wednesday 18 December 2002 05:13, Alan W. Irwin wrote:
> On Wed, 18 Dec 2002, Joao Cardoso wrote:

I don't like the way how AT  is setup, but I also don't want to spend one=
 or=20
two weeks learning how to change it.
Do I have to live with it? I don't yet know the answer to this question.

Joao