q-lang-users

[q-lang-users] Q 7.2 RC2

[Albert G. <Dr....@t-...>]

Dear all,

I've just finished uploading RC2, here is where you can get it:
http://sourceforge.net/project/showfiles.php?group_id=96881&package_id=188958&release_id=425798

I think I've fixed all issues with RC1 brought up by John and others (in 
particular, IEEE floating point arithmetic should now work as John 
suggested), and I worked through my TODO list, so this should be real 
close to 7.2 final now. If you notice anything that I forgot to fix 
please let me know.

John, it would be nice if you (and everyone else who cares about the 
numeric stuff) could once again apply a bit of torture to the 
float/rational/complex arithmetic to see if everything now works as you 
proposed and we discussed. ;-) There have been quite a few changes, so 
I'd really appreciate your feedback.

Rob is still busy working on some final updates to his "Q+Q" library and 
documentation, but this will hopefully be finished soon so that it can 
be included in the final release. For the time being, I've included an 
updated version of Q+Q (0.3) which will work with the new '%' operator.

Thanks for all the suggestions, contributions and discussions, and happy 
computing with RC2. :)

Cheers,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Q 7.2 RC2

[John C. <co...@cc...>]

Albert Graef scripsit:

> John, it would be nice if you (and everyone else who cares about the 
> numeric stuff) could once again apply a bit of torture to the 
> float/rational/complex arithmetic to see if everything now works as you 
> proposed and we discussed. ;-) There have been quite a few changes, so 
> I'd really appreciate your feedback.

I've tested some corner cases against RC3; I never did test RC2.

1) Comparisons against nan still return false (per IEEE) rather than
failing, which we had agreed is more Q-ish.

2) 3/0 and 3%0 both return the normal form 3/0.  Since / is now supposed
to always return inexacts, I think these should return inf and 3%0
respectively.

3) Making sure that those really were normal forms made me aware of
just how useful it would be to be able to turn off unparsing and then
turn it on again easily.  I had to use my Chicken interface to make
sure I wasn't being fooled.  An interpreter command would make sense.

4) You had said earlier that you would provide unparsing rules for dict
and the other container types.

5) Nan is returning true for isratval and isintval, when it should return
false; by convention, a NaN is real but not rational.

6) When % is called on floats, the result is a float.  Ideally, 2.5%1
should return 5%2, though I don't insist that the best possible fraction
should be returned, just some reasonable fraction.  This may be best
done in C, where you can call useful functions like frexp().

7) This brings up the issue of coercions from exact to inexact (easy,
just need a special case for complex numbers) and for inexact to exact.
These should be in the library somewhere.

-- 
There are three kinds of people in the world:   John Cowan
those who can count,                            co...@cc...
and those who can't.

Re: [q-lang-users] Q 7.2 RC2

[Albert G. <Dr....@t-...>]

John Cowan wrote:
> I've tested some corner cases against RC3; I never did test RC2.

Hi John, thanks a lot for taking the time to test again.

> 1) Comparisons against nan still return false (per IEEE) rather than
> failing, which we had agreed is more Q-ish.

Yes, this was one of those ideas that sound nice in theory but turn out 
awful in practice. ;-) Having nan comparisons fail means that most 
definitions involving comparisons with numbers would have to check for 
nan values to avoid exceptions on failing conditions, which turns even 
the simplest numeric algorithms into a big unholy mess. So I decided to 
do it the IEEE way instead.

> 2) 3/0 and 3%0 both return the normal form 3/0.  Since / is now supposed
> to always return inexacts, I think these should return inf and 3%0
> respectively.

Yes, I have to agree that 3/0 should now return inf instead of failing, 
although that introduces a big incompatibility with earlier versions. :(

I'm not so sure about 3%0, though. As I argue under item 6 below, for me 
the right way to implement (%) on inexact numbers is the way it is done 
now, i.e., to fall back to inexact division. But inexact division 
returns (or rather will return when I fixed it) inf in this case. So 
there is a case for having 3%0 evaluate to inf, too. X%0 should still 
fail if X/0 does (the current definition isn't working correctly in that 
case, but I've fixed that now). However, AFAICT this case wouldn't 
actually arise any more.

> 3) Making sure that those really were normal forms made me aware of
> just how useful it would be to be able to turn off unparsing and then
> turn it on again easily.  I had to use my Chicken interface to make
> sure I wasn't being fooled.  An interpreter command would make sense.

Well, as I pointed out earlier there's already the possibility to just 
add a definition like "unparse = ();" to your script which has the 
effect of disabling all custom unparsings. But I agree that there should 
be a simpler way. Probably an "unparse" command taking an on/off arg, 
like "debug".

> 4) You had said earlier that you would provide unparsing rules for dict
> and the other container types.

I'm still undecided on this, because everything I do in the library sets 
a precedent that needs to be followed later. Numbers are a rather 
specific case, and it certainly makes sense to include default unparsing 
rules for complex and rational numbers in the library. But I'm not so 
sure whether this is the right thing for the container data structures 
yet, so I just postponed that decision. For the time being, you can 
easily add your own unparsing definitions for those data structures if 
you want/need them; they're all one-liners. :)

> 5) Nan is returning true for isratval and isintval, when it should return
> false; by convention, a NaN is real but not rational.

isintval nan correctly returns false over here, but isratval nan 
incorrectly returned true. That has been fixed.

> 6) When % is called on floats, the result is a float.  Ideally, 2.5%1
> should return 5%2, though I don't insist that the best possible fraction
> should be returned, just some reasonable fraction.  This may be best
> done in C, where you can call useful functions like frexp().

No, I have to disagree. That would promote the result to a more specific 
type than the operands and we all agreed that this should never be done. 
Therefore, if (%) is to work on all numbers (which is a good thing, 
IMHO) then the right way to do this is to fall back to inexact division 
on inexact operands.

Besides, there's also the issue that there's no single "canonical" way 
to do the rational approximation. So I think that it's much better to 
leave that to a separate module (like Rob's ratutils.q) which can do a 
much more comprehensive job and provide different kinds of specialized 
approximation algorithms.

> 7) This brings up the issue of coercions from exact to inexact (easy,
> just need a special case for complex numbers) and for inexact to exact.
> These should be in the library somewhere.

In fact they are. There's float to convert any Real number to a Float, 
there are round, trunc, floor and ceil to go the opposite direction, and 
there are Rob's Float -> Rational approximation algorithms. If needed, 
these can be applied to the real and imaginary components of Complex. 
What else do you need?

Granted, you can't just apply, say, round to Complex, but usually that 
doesn't make mathematical sense anyway. The library cannot know whether 
you'd prefer to round according to, e.g., the 1-, 2- or maybe the \infty 
norm, that should be up to the application.

Thanks,
Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Q 7.2 RC2

[Albert G. <Dr....@t-...>]

I wrote:
> isintval nan correctly returns false over here, but isratval nan 
> incorrectly returned true. That has been fixed.

Oops, I just noticed that isintval inf incorrectly returns true, I'll 
have to fix that, too.

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Q 7.2 RC2

[Rob H. <hub...@gm...>]

Dear All,

On 26/06/06, Albert Graef <Dr....@t-...> wrote:
> John Cowan wrote:

> > 4) You had said earlier that you would provide unparsing rules for dict
> > and the other container types.
>
> I'm still undecided on this, because everything I do in the library sets
> a precedent that needs to be followed later. Numbers are a rather
> specific case, and it certainly makes sense to include default unparsing
> rules for complex and rational numbers in the library. But I'm not so
> sure whether this is the right thing for the container data structures
> yet, so I just postponed that decision. For the time being, you can
> easily add your own unparsing definitions for those data structures if
> you want/need them; they're all one-liners. :)

>From my experience with C++, containers are a real pain when
debugging. I don't care how a set is implemented - for me it's a black
box - just tell me what's in it.

Thus, I believe unparsing containers is the right thing to do.

> > 6) When % is called on floats, the result is a float.  Ideally, 2.5%1
> > should return 5%2, though I don't insist that the best possible fraction
> > should be returned, just some reasonable fraction.  This may be best
> > done in C, where you can call useful functions like frexp().
>
> No, I have to disagree. That would promote the result to a more specific
> type than the operands and we all agreed that this should never be done.
> Therefore, if (%) is to work on all numbers (which is a good thing,
> IMHO) then the right way to do this is to fall back to inexact division
> on inexact operands.
>
> Besides, there's also the issue that there's no single "canonical" way
> to do the rational approximation.

2.5%1 should be Float. (%) should be exact *if possible*. But 2.5 is
not exact (it is "2.5+epsilon". To make the result Rational would be
to hide inexactness.

Rob.

Re: [q-lang-users] Q 7.2 RC2

[Albert G. <Dr....@t-...>]

Rob Hubbard wrote:
>>From my experience with C++, containers are a real pain when
> debugging. I don't care how a set is implemented - for me it's a black
> box - just tell me what's in it.

Well, in the debugger custom unparsings must be disabled anyway, since 
otherwise the output from the debugger, i.e. the printed rules and 
reductions, looks just plain wrong. (I've actually tried that.)

OTOH, the debugger gives you the option to reduce the level of detail, 
and it's possible to evaluate arbitrary expressions in the context of 
the current rule, so that you can inspect exactly those parts of the 
data structure that you want to see, in the representation that you 
choose. Q's debugger is much more powerful than most C/C++ debuggers in 
this respect.

> Thus, I believe unparsing containers is the right thing to do.

Ok, I guess that I can just add this to the prelude as an experimental 
feature, so that we can give it a try and see how everybody likes it.

> 2.5%1 should be Float. (%) should be exact *if possible*. But 2.5 is
> not exact (it is "2.5+epsilon". To make the result Rational would be
> to hide inexactness.

I fully agree with that. The only alternative I see would be to make (%) 
undefined on inexact numbers which would be very inconvenient.

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag

Re: [q-lang-users] Q 7.2 RC2

[John C. <co...@cc...>]

Albert Graef scripsit:

> Yes, I have to agree that 3/0 should now return inf instead of failing, 
> although that introduces a big incompatibility with earlier versions. :(

I think it's one that everyone can live with.

> I'm not so sure about 3%0, though. As I argue under item 6 below, for me 
> the right way to implement (%) on inexact numbers is the way it is done 
> now, i.e., to fall back to inexact division. But inexact division 
> returns (or rather will return when I fixed it) inf in this case. So 
> there is a case for having 3%0 evaluate to inf, too. X%0 should still 
> fail if X/0 does (the current definition isn't working correctly in that 
> case, but I've fixed that now). However, AFAICT this case wouldn't 
> actually arise any more.

I now agree with this:  % is not to be understood as exact division
(of the div-and-rem sort), but exactness-contagion division, just as +
- * are exactness-contagion operators: they return inexact results iff
at least one argument is inexact.  / on the other hand always produces
inexact results.  So both 3%0 and 3/0 should return inf.

> Therefore, if (%) is to work on all numbers (which is a good thing, 
> IMHO) then the right way to do this is to fall back to inexact division 
> on inexact operands.

This one was a consequence of my misunderstanding of %, and I agree it
should be as you say.  

> Besides, there's also the issue that there's no single "canonical" way 
> to do the rational approximation. So I think that it's much better to 
> leave that to a separate module (like Rob's ratutils.q) which can do a 
> much more comprehensive job and provide different kinds of specialized 
> approximation algorithms.

For the record, R5RS Scheme provides a "rationalize" procedure of two
arguments which returns the simplest rational number that differs from
X by no more than Y, and adds:

        A rational number r1 is simpler than another rational number r2
        if r1 = p1/q1 and r2 = p2/q2 (in lowest terms) and |p1| <= |p2|
        and |q1| <= |q2|. Thus 3/5 is simpler than 4/7. Although not all
        rationals are comparable in this ordering (consider 2/7 and 3/5)
        any interval contains a rational number that is simpler than
        every other rational number in that interval (the simpler 2/5
        lies between 2/7 and 3/5). Note that 0 = 0/1 is the simplest
        rational of all.

Rationalize, like almost all R5RS procedures, is exactness-contagious.

> > 7) This brings up the issue of coercions from exact to inexact (easy,
> > just need a special case for complex numbers) and for inexact to exact.
> > These should be in the library somewhere.
> 
> In fact they are. There's float to convert any Real number to a Float, 
> there are round, trunc, floor and ceil to go the opposite direction, and 
> there are Rob's Float -> Rational approximation algorithms. If needed, 
> these can be applied to the real and imaginary components of Complex. 
> What else do you need?
> 
> Granted, you can't just apply, say, round to Complex, but usually that 
> doesn't make mathematical sense anyway. The library cannot know whether 
> you'd prefer to round according to, e.g., the 1-, 2- or maybe the \infty 
> norm, that should be up to the application.

R5RS uses the phrase "{inexact,exact} [representable] number that is
numerically closest to the argument", presumably in the sense of
having the smallest difference from the argument.  Still, I agree
that flexibility probably beats simplicity here, especially in the
case of general complex numbers.

-- 
John Cowan      co...@cc...        http://www.ccil.org/~cowan
        Is it not written, "That which is written, is written"?

Re: [q-lang-users] Q 7.2 RC2

[Albert G. <Dr....@t-...>]

All right, it looks like we once again agree on all issues, very nice. 
So I can now finish fixing the bugs John reported and update Q+Q to 
Rob's latest codebase, after which Q 7.2 should be ready to be released.

Albert

-- 
Dr. Albert Gr"af
Dept. of Music-Informatics, University of Mainz, Germany
Email:  Dr....@t-..., ag...@mu...
WWW:    http://www.musikinformatik.uni-mainz.de/ag