Re: [Plplot-devel] New way of getting device info from dyndrivers

[Joao C. <jc...@fe...>]

On Tuesday 04 February 2003 21:37, Rafael Laboissiere wrote:
> Preamble
> =3D=3D=3D=3D=3D=3D=3D=3D
>
> The big recent cvs commit regarding the dyndrivers database was on the =
top
> of my TODO list.  It is a necessary step towards clean
> configuration/building and also packaging for Debian.
>
> I am not yet completely happy with my implementation, but my changes
> (apparently) did not break PLplot.  My simple test script succeeded, at
> least.  I am committing without previous discussion here because, as Al=
an
> uses to say, getting novelties in cvs HEAD is the best way to foster
> discussions.

That's true, but the target should be to get a full-working 5.2.1, that s=
hould=20
be, in my opinion, a bug-correcting release.
Introducing such stuff can compromise it.
But we have not defined what 5.2.1 should be, and I'm too conservative.

I have myself lots of new stuff, and I'm refraining myself to commit them=
=2E I=20
hope that 5.2.2 or even 5.3.0 follows shortly.

>
> The Problem
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> The way PLplot used to get information about the available devices prov=
ided
> by the dyndrivers was through the DATA_DIR/drivers.db file.  This file =
was
> generated at configuration time and parsed when the library was
> initialized.
>
> This approach has two drawbacks:
>
> 1) Information about the devices are scattered in different places (nam=
ely
>    in the driver source file and in configure.ac).  This is ugly and ma=
y
>    result in unnecessary maintenance burden.
>
> 2) Since the list of available devices is hardcoded in the drivers.db i=
t is
>    almost impossible to do clean packaging of Plplot.  In Debian, for
>    instance, packaging is granular in order to reduce the dependencies:
>    plplot-xwin, plplot-tk, plplot-gd, etc.  Users can install a subset =
of
>    the available packages at will.  However, they will always get the f=
ull
>    list of available devices when plinit is called.  That is not a crit=
ical
>    problem, but annoying.
>
>
> The Solution
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
>
> I have elaborated a full fix for this problem, but I just committed an
> intermediary solution for it.  Here is how it works:
>
> 1) drivers.db does not exist anymore.
>
> 2) In each driver file, there is a global declaration like this:
>
>    char* DEVICE_INFO_gd =3D "jpeg:JPEG file:0:gd:40:jpeg\n"
>                           "png:PNG file:0:gd:39:png";

OK, but we need a registration text file, manually maintained, to guide n=
ew=20
drivers writers on how to obtain (and register) new ids for the new drive=
rs.=20

>
>    containing the entries that used to go into drivers.db. If a driver
>    provides more than one device, their entries must be separated by a
>    newline character ('\n').
>
> 3) When the library is initialized, if dyndrivers are enabled, the driv=
ers
>    directory is scanned for the *.la files.  Each found driver is dlope=
ned
>    and the DEVICE_INFO_<driver> symbol is read and put in a temporary f=
ile.
>
> 4) This temporary file plays the role of the old drivers.db file.

Why is the tmp file needed? Why not keep the collected info in an interna=
l=20
structure?

> Noticed that I did minimal changes to Geoff's code in plcore.c.  In the
> plInitDispatchTable function, I replaced the initial code (where the
> drivers.db were scanned) by the scanning of the drivers directory descr=
ibed
> in point 3 above.

Ah, you use a tmp file just to reuse Geoff's code. But there is no reason=
 to=20
not change that also latter, right?

> Also, all references to drivers.db and DRIVERS_DB have disappeared from=
 the
> sources.
>
>
> Drawbacks and improvements
> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D
>
> I see two potential problems with my approach:
>
> 1) Portability.  I am using new libc functions (POSIX tmpfile, opendir,
>    readdir and closedir).  Although am I following the recommended
> procedure found in the Autoconf docs (i.e. using AC_HEADER_DIRENT and a
> couple of tests with HAVE_DIRENT_H), I am sure that there will be some
> weird system out there (MacIntosh, say) for which my code won't compile=
=2E=20
> If that happens, we have to port that part of the code.

Don't create the file :)

> 2) With my approach, I have to open all and every module before using t=
hem.
>    This may appear as a regression as regards the "cache file" approach
>    provided by the use of drivers.db.  In terms of performance, with ou=
r
>    current computer power, the overload is negligible.  However, as I w=
rote
>    above, my original design was much better, but harder to implement (=
of
>    course).  It looks like this:
>
>    a) Forget about that entries a la drivers.db.
>
>    b) In each driver file <driver>.c define symbols DEV_DESC_*, DEV_SEQ=
_*,
>       DEV_TAG_*, etc.  These symbols can be used when filling fields in
>       function plD_dispatch_init_<device>.  This would further reduce t=
he
>       maintenance problem due to duplication of information.
>
>    c) At build time (not configuration time), a small C program dlopen =
the
>       <driver>.c files,

You mean "open" and not dlopen(), right?

> get the symbols

No, you mean dlopen() the <driver>.so (I don't know the current suffix)

What do you really mean? Anyway, see bellow.

> described above and write the
>       associated device entries in <driver>.rc (or whichever name).
>
>    d) Those <driver>.rc are installed in the drivers directory, along w=
ith
>       the <driver>.la and <driver>.so files.
>
>    e) When PLplot is initialized, the <driver>.rc files are scanned.
>
>    If I have some time before the 5.2.1 release, I will try to implemen=
t
>    this idea.

I prefer the full dynamic one, i.e., reading the already build drivers. I=
=20
don't think that performance will suffer. Do you have some figures or onl=
y =20
feelings?

Your second idea implies that the small program that scans the source fil=
es=20
needs information from the configure step to know what drivers are desire=
d by=20
the user and supported by the system. This complicates matters.
With the first idea, by contrary, only already build drivers, i.e, user=20
desired and system supported, are scanned.

Also, with the second idea I think that the driver-ids (the magic numbers=
) can=20
go away -- ah, no, for historical reasons the xwin driver must be number =
one,=20
the tk driver number 2, etc. hmm, there is another solution for this but =
let=20
discuss it latter.

If performance is really an issue, then one could implements a mixing of =
the=20
two ideas: don't generate drivers.db at configure nor build time. At run =
time=20
if drivers.db does not exists, scan the drivers and build it. This way th=
e=20
performance loss will only occurs once.
If a new driver is latter added to the directory (not probable, but possi=
ble,=20
after all this is the only advantage of dyndrivers), one could first comp=
are=20
the time-stamp of all drivers versus the drivers.db file, and rebuild=20
drivers.db if a driver is more recent. Reading time-stamp is fast, I beli=
eve.

Joao