Re: [Webwork-devel] Problem comparing large numbers

[Maurice P. <Ma...@Vi...>]

Fixed.  Thanks for the catch Johan!

-Maurice


On Friday, January 18, 2002, at 01:25 AM, Wirell Johan wrote:

> Hi,
>
> There seems to be a small problem in the webwork expression
> language when comparing two numbers. The current code converts
> both numbers to float values before comparing them. This may be
> a problem when comparing long values since precision could be
> lost when converted to a float. The following two numbers will
> because of this be considered equal for example:
>
> 1011170535000
> 1011170557000
>
> An easy solution would be to change the comparison
> code in parser.jj to be "double" based instead:
>
>               if ( operand1 instanceof Number && operand2 instanceof 
> Number ) {
>
>                   double number1 = ((Number)operand1).doubleValue();
>                   double number2 = ((Number)operand2).doubleValue();
>
>                   int comp = -1; // less than
>                   if ( number1 > number2 ) {
>                       comp = 1;  // greater than
>                   } else if ( number1 == number2 ) {
>                       long longBits1 = Double.doubleToLongBits(number1);
>                       long longBits2 = Double.doubleToLongBits(number2);
>                       if ( longBits1 > longBits2 ) {
>                           comp = 1;
>                       } else if ( longBits1 == longBits2 ) {
>                           comp = 0;
>                       }
>                   }
>
>                   return resolve(comp, token);
>
>               }
>
> Regards,
> Johan
>
> _______________________________________________
> Webwork-devel mailing list
> Web...@li...
> https://lists.sourceforge.net/lists/listinfo/webwork-devel
>