Re: Tcl precision [was: Re: [Plplot-devel] Java changes to support plsurf3d()]

[Maurice L. <mj...@ga...>]

Alan W. Irwin writes:
 > As I recall the result of that thread, Maurice implemented something like 12
 > (or 13?) significant digits as a stop-gap measure to the tcl precision
 > problems.  There was some reason why he did not want to increase the
 > precision to 17 digits.  In any case, he had a better solution in mind,
 > which I don't think has been implemented yet.

Correct, not implmented yet (it will be a while).  BTW here is my original
commentary on the problem:

| The story becomes more interesting.  I believe the two representations
| (variable or tclMatrix) would've given the same result in pre-Tcl8.0.  The
| reason being that in the old Tcl, "everything's a string".  But no longer --
| the variable that you create with "set" is "dual-ported", meaning it has both
| a binary and string representation.  The binary one is used in expression
| evaluations and so is more accurate.
| 
| I've been poring over the documentation, and I think it's feasible to change
| tclMatrix to emit objects instead of strings, which seems to be the right way
| to really fix this.  Also will help a lot with performance.  The object is
| equipped with procedure calls to convert it to strings when necessary.
| Unfortunately this is a pretty major undertaking, and not one I care to get
| into right now.
| 
| So as for a stopgap measure, I'll change it to carry 12 digits of precision,
| both internally and externally.  I tried 17 but didn't like the result.  E.g.
| 
| pltcl> matrix c f 3 = {3.2 4.5 5.7}
| c
| pltcl> puts [c *]
| 3.2000000000000002 4.5 5.7000000000000002
| 
| which the tclvars man page warns about.

-- 
Maurice LeBrun    mj...@ga...
Research Organization for Information Science and Technology of Japan (RIST)