Re: [WWdevel] Re: extraAnswerEvaluators.pl

[John J. <jj...@as...>]

Davide P.Cervone wrote:

> John and Gavin:
>
> I'm rerouting this to the developer mailing list, since it now relates 
> to feature decisions.
>
> On Aug 24, 2005, at 12:19 AM, John Jones wrote:
>
>> Davide P. Cervone wrote:
>>
>>>> For the num_cmp() evaluator, setting 'tol'=>0.5 gives the result
>>>>   Entered    Answer Preview    Correct        Result
>>>>   0            0        1/sqrt(7)    correct
>>>> and taking out the tol gives
>>>>   Entered    Answer Preview    Correct        Result
>>>>   0.25        0.25        1/sqrt(7)    incorrect
>>>>
>>>> For the number_list_cmp evaluator, with the 'tol' option I get
>>>>   Entered    Answer Preview    Correct    Result
>>>>     -0, 0       -0,0    0, 0    correct
>>>> and without,
>>>>   Entered    Answer Preview    Correct            Result
>>>>   -0.25, 0.25   -0.25,0.25    -0.37796.., 0.37796..    incorrect
>>>
>>>
>>>
>>> The reason that the parser shows the student's answer as 0 is that 
>>> when it stringifies a real number, it shows it as zero if it is 
>>> "equal" to zero according to the current equality conditions.  Since 
>>> your answers actually ARE equal to zero (within an absolute 
>>> tolerance of 0.5), they get printed as zeros.  When a RELATIVE 
>>> tolerance is used, there is a special tolerance for zero that gets 
>>> used for checking against zero.  This is not the case for absolute 
>>> tolerance, because you are specifying exactly how close you want the 
>>> answer to be.
>>
>>
>> I am not so sure it is good to simplify a student's answer to 0 for a 
>> few reasons.  It exposes some of webwork's internal tricks to the 
>> students.  In general, other answers are not simplified in this way.  
>> For example, 1.0001 is not simplified to 1.  And, it would be 
>> confusing to a student to see their answer not register.  I tried 
>> setting tolType=>'strict' in num_cmp and taking the default setting.  
>> Then, I submitted 0.09, webwork told me I submitted 0.
>>
>
> Well, first of all, tolType should be one of 'relative' or 'absolute' 
> from what I can see.  (I am unable to find a reference to tolType 
> being 'strict' anywhere.)  num_cmp doesn't check the value of tolType 
> to make sure it is legal, and perhaps it should.  What happens when 
> you tolType to something other than 'relative' or 'absolute' is 
> somewhat random, as some tests are for tolType equal to 'relative' and 
> others are for 'absolute' and both assume if it is not the one 
> checked, then it is the other.  So when set to 'strict' you will 
> sometimes get the absolute branch of the test and other times the 
> relative branch.  For the example above, you end up getting an 
> absolute test with the tolerance set to 1, so the answer 0.09 is 
> properly considered to be 0.

My mistake on using strict instead of absolute for the tolType, and you 
do have to be closer to 0 to see the rounding effect with absolute.

I tried

ANS( num_cmp(5, tolType=>'absolute') );

and entered 0.000999.  Webwork tells me that I entered 0 and the preview 
is 0.

I can see that tolType relative acts differently.  With

ANS( num_cmp(5) );

webwork will correctly report what I enter until I get down to 
0.0000000000009 which is interpreted as 0.

I understand that students get confused when a number is listed in 
scientific notation.  We get that complaint, but only when the value 
appearing in the problem text is in scientific notation (it happens by 
accident, and so we just fix the problem).

But, one of the frustrations I hear about is "webwork won't take my 
answer".  If someone enters a non-zero answer and the system says they 
entered 0, a common perception would be that the number they entered 
didn't take.  Then they try to submit the same thing 10 more times with 
the same result, and get mad at the system.

The defaults for relative comparison seem ok.  If someone enters 
0.0000000000009, I think you can reasonably tell them that the computer 
rounded it off because it was so close to 0.  The main reason we 
currently use Parser-based answer checkers is that they make things less 
confusing for students.  But on the absolute error example above, 
entering 0.0009 and being told it is 0 seems confusing.

Now that I am thinking about it, I don't understand why 0 is special at 
all when using absolute error.  I see why it is special when dealing 
with relative error, but it should be just like any other number when we 
have absolute error, or am I missing something?

John