[Webwork-devel] Problem comparing large numbers

[Wirell J. <joh...@co...>]

Hi,

There seems to be a small problem in the webwork expression
language when comparing two numbers. The current code converts
both numbers to float values before comparing them. This may be
a problem when comparing long values since precision could be
lost when converted to a float. The following two numbers will
because of this be considered equal for example:

1011170535000
1011170557000

An easy solution would be to change the comparison
code in parser.jj to be "double" based instead:

              if ( operand1 instanceof Number && operand2 instanceof Number ) {

                  double number1 = ((Number)operand1).doubleValue();
                  double number2 = ((Number)operand2).doubleValue();

                  int comp = -1; // less than
                  if ( number1 > number2 ) {
                      comp = 1;  // greater than
                  } else if ( number1 == number2 ) {
                      long longBits1 = Double.doubleToLongBits(number1);
                      long longBits2 = Double.doubleToLongBits(number2);
                      if ( longBits1 > longBits2 ) {
                          comp = 1;
                      } else if ( longBits1 == longBits2 ) {
                          comp = 0;
                      }
                  }

                  return resolve(comp, token);

              }

Regards,
Johan