From: Alan W. Irwin <irwin@be...>  20020115 18:29:37

Thanks for your fix. At first I thought there was a 12digit limit to the matrix precision, but that is just the default value of tcl_precision. If I set it to 17 I get full doubleprecision results. BTW, tcl_precision doesn't seem to have universal application. My tests indicated the internal precision of ordinary variables in pltcl was 17 digits regardless of what tcl_precision value was set. For example with the default (12) value of tcl_precision I get the following results: pltcl> set y 3.12345678901234567890 3.12345678901234567890 pltcl> puts [expr $y  3.123456] 7.89012345681e07 I would prefer matrix to act the same way regardless ot tcl_precision. Therefore, could you just set the matrix precision to 17 digits when PLFLT is double and 7 digits otherwise? The other alternative I am considering is to set tcl_precision to 17 in each of the examples, but I prefer not to do that if there is an automatic PLFLTstyle solution you can use for libmatrix. Alan > > Why aren't there more significant digits in the matrix results? > > It's because standard sprintf was being used. But have no fear, I've > converted over to using the internal Tcl print (double) function, which uses > the tcl_precision variable to decide how long to make it. > >  > Maurice LeBrun mjl@... > 
From: Alan W. Irwin <irwin@be...>  20020115 18:33:54

P.S. Maurice, an even simpler alternative is to always use 17 figures internally rather than 17 or 7 depending on PLFLT. The choice is up to you. Alan email: irwin@... phone: 2507272902 FAX: 2507217715 snailmail: Dr. Alan W. Irwin Department of Physics and Astronomy, University of Victoria, P.O. Box 3055, Victoria, British Columbia, Canada, V8W 3P6 __________________________ Linuxpowered astrophysics __________________________ 
From: Maurice LeBrun <mjl@ga...>  20020115 21:58:14

Alan W. Irwin writes: > Thanks for your fix. At first I thought there was a 12digit limit to the > matrix precision, but that is just the default value of tcl_precision. If I > set it to 17 I get full doubleprecision results. > > BTW, tcl_precision doesn't seem to have universal application. My tests > indicated the internal precision of ordinary variables in pltcl was 17 > digits regardless of what tcl_precision value was set. For example with > the default (12) value of tcl_precision I get the following results: > > pltcl> set y 3.12345678901234567890 > 3.12345678901234567890 > pltcl> puts [expr $y  3.123456] > 7.89012345681e07 1 234567890ab ..i.e. 12 digits of precision. Further examples: pltcl> set tcl_precision 6 6 pltcl> puts [expr $y  3.123456] 7.89012e07 pltcl> set tcl_precision 4 4 pltcl> puts [expr $y  3.123456] 7.89e07 > I would prefer matrix to act the same way regardless ot tcl_precision. > Therefore, could you just set the matrix precision to 17 digits when PLFLT > is double and 7 digits otherwise? The other alternative I am considering is > to set tcl_precision to 17 in each of the examples, but I prefer not to do > that if there is an automatic PLFLTstyle solution you can use for > libmatrix. I really think tying it to tcl_precision is the right thing to do. It doesn't have any effect on internal representation, just the number of digits emitted when converting the PLFLT to a string.  Maurice LeBrun mjl@... 
From: Alan W. Irwin <irwin@be...>  20020116 00:45:57

On Tue, 15 Jan 2002, Maurice LeBrun wrote: > Alan W. Irwin writes: > > Thanks for your fix. At first I thought there was a 12digit limit to the > > matrix precision, but that is just the default value of tcl_precision. If I > > set it to 17 I get full doubleprecision results. > > > > BTW, tcl_precision doesn't seem to have universal application. My tests > > indicated the internal precision of ordinary variables in pltcl was 17 > > digits regardless of what tcl_precision value was set. For example with > > the default (12) value of tcl_precision I get the following results: > > > > pltcl> set y 3.12345678901234567890 > > 3.12345678901234567890 > > pltcl> puts [expr $y  3.123456] > > 7.89012345681e07 > 1 234567890ab > > ..i.e. 12 digits of precision. Point well taken, but I have investigated further, and there is more to this story that I didn't understand before. From the above example $ y  3.123456 has access to the full ~17 digit precision of y before subtracting the constant from it so there are actually still 10 significant digits left in the result (note the last two "81" digits are incorrect because the internal precision of y is roughly 17 digits.) But look how the corresponding matrix result does not act this way. pltcl> set tcl_precision 12 12 pltcl> matrix a f 1 = {3.1234567890123456789} a pltcl> puts [expr [a 0]  3.123456] 7.89010000002e07 There are 5 fewer significant digits in this result than for the y  const result. My mental model of what is going on here is as follows: unlike $y  const, [a 0]  const only has access to tcl_precision digits of precision for matrix values. In other words set tcl_precision effectively sets the internal precision of matrix calculations rather than simply limiting the number of digits emitted for results. > > I would prefer matrix to act the same way regardless ot tcl_precision. > > Therefore, could you just set the matrix precision to 17 digits when PLFLT > > is double and 7 digits otherwise? The other alternative I am considering is > > to set tcl_precision to 17 in each of the examples, but I prefer not to do > > that if there is an automatic PLFLTstyle solution you can use for > > libmatrix. > > I really think tying it to tcl_precision is the right thing to do. > It doesn't have any effect on internal representation, just the number > of digits emitted when converting the PLFLT to a string. That would be okay with me, but as you can see from the above example it is affecting the (effective) internal precision, and not just the digits being emitted. Hope you can get this matrix problem sorted out. Alan 
From: Maurice LeBrun <mjl@ga...>  20020116 02:36:00

Alan W. Irwin writes: > Point well taken, but I have investigated further, and there is more to this > story that I didn't understand before. From the above example $ y  > 3.123456 has access to the full ~17 digit precision of y before subtracting the > constant from it so there are actually still 10 significant digits left in > the result (note the last two "81" digits are incorrect because the internal > precision of y is roughly 17 digits.) > > But look how the corresponding matrix result does not act this way. > > pltcl> set tcl_precision 12 > 12 > pltcl> matrix a f 1 = {3.1234567890123456789} > a > pltcl> puts [expr [a 0]  3.123456] > 7.89010000002e07 > > There are 5 fewer significant digits in this result than for the y  const > result. > > My mental model of what is going on here is as follows: unlike $y  const, > [a 0]  const only has access to tcl_precision digits of precision for > matrix values. In other words set tcl_precision effectively sets the > internal precision of matrix calculations rather than simply limiting the > number of digits emitted for results. You're right, it's still not really right. Will look into it some more.  Maurice LeBrun mjl@... 
From: Maurice LeBrun <mjl@ga...>  20020116 06:00:07

Alan W. Irwin writes: > Point well taken, but I have investigated further, and there is more to this > story that I didn't understand before. From the above example $ y  > 3.123456 has access to the full ~17 digit precision of y before subtracting the > constant from it so there are actually still 10 significant digits left in > the result (note the last two "81" digits are incorrect because the internal > precision of y is roughly 17 digits.) > > But look how the corresponding matrix result does not act this way. > > pltcl> set tcl_precision 12 > 12 > pltcl> matrix a f 1 = {3.1234567890123456789} > a > pltcl> puts [expr [a 0]  3.123456] > 7.89010000002e07 > > There are 5 fewer significant digits in this result than for the y  const > result. > > My mental model of what is going on here is as follows: unlike $y  const, > [a 0]  const only has access to tcl_precision digits of precision for > matrix values. In other words set tcl_precision effectively sets the > internal precision of matrix calculations rather than simply limiting the > number of digits emitted for results. The story becomes more interesting. I believe the two representations (variable or tclMatrix) would've given the same result in preTcl8.0. The reason being that in the old Tcl, "everything's a string". But no longer  the variable that you create with "set" is "dualported", meaning it has both a binary and string representation. The binary one is used in expression evaluations and so is more accurate. I've been poring over the documentation, and I think it's feasible to change tclMatrix to emit objects instead of strings, which seems to be the right way to really fix this. Also will help a lot with performance. The object is equipped with procedure calls to convert it to strings when necessary. Unfortunately this is a pretty major undertaking, and not one I care to get into right now. So as for a stopgap measure, I'll change it to carry 12 digits of precision, both internally and externally. I tried 17 but didn't like the result. E.g. pltcl> matrix c f 3 = {3.2 4.5 5.7} c pltcl> puts [c *] 3.2000000000000002 4.5 5.7000000000000002 which the tclvars man page warns about.  Maurice LeBrun mjl@... 