[Plplot-devel] Re: Embedded single precision in the matrix library

[Alan W. I. <ir...@be...>]

Thanks for your fix.  At first I thought there was a 12-digit limit to the
matrix precision, but that is just the default value of tcl_precision.  If I
set it to 17 I get full double-precision results.

BTW, tcl_precision doesn't seem to have universal application.  My tests
indicated the internal precision of ordinary variables in pltcl was 17
digits regardless of what tcl_precision value was set.  For example with
the default (12) value of tcl_precision I get the following results:

pltcl> set y 3.12345678901234567890
3.12345678901234567890
pltcl> puts [expr $y - 3.123456]
7.89012345681e-07

I would prefer matrix to act the same way regardless ot tcl_precision.
Therefore, could you just set the matrix precision to 17 digits when PLFLT
is double and 7 digits otherwise?  The other alternative I am considering is
to set tcl_precision to 17 in each of the examples, but I prefer not to do
that if there is an automatic PLFLT-style solution you can use for
libmatrix.

Alan

>  > Why aren't there more significant digits in the matrix results?
>
> It's because standard sprintf was being used.  But have no fear, I've
> converted over to using the internal Tcl print (double) function, which uses
> the tcl_precision variable to decide how long to make it.
>
> --
> Maurice LeBrun    mj...@ga...
>