Re: [WWdevel] Re: extraAnswerEvaluators.pl

[P G. L. <gl...@um...>]

Hi Davide,

Just one additional comment about this behavior.  I agree that in cases 
where an answer is of smaller magnitude than the absolute numerical 
tolerance, it is mathematically reasonable to treat the answer as zero. 
I am a little worried that students will think that the fact that they 
entered an answer that is non-zero and it is previewed as zero will make 
them think that the system isn't working correctly, however.  For example, 
if the correct answer is evaluated with num_cmp(sqrt(7)) and the student 
enters 1/sqrt(7), the system reports

     Entered	Answer Preview	Result
     0		 1/sqrt(7)	incorrect

I suspect the zero might be confusing to some students.

The one case where I use absolute tolerances like this is when I have 
students estimating values from data or from a graph.  In this case I can 
determine what maximum error I expect them to have, and code that as an 
absolute tolerance.  I suppose I could construct the problems to avoid 
answers that are zero within the tolerance, but that seems like an 
artificial solution.

Thanks,
Gavin

On Wed, 24 Aug 2005 at 07:39 Davide P.Cervone wrote:

> John and Gavin:
>
> I'm rerouting this to the developer mailing list, since it now relates to 
> feature decisions.
>
> Well, first of all, tolType should be one of 'relative' or 'absolute' from 
> what I can see.  (I am unable to find a reference to tolType being 'strict' 
> anywhere.)  num_cmp doesn't check the value of tolType to make sure it is 
> legal, and perhaps it should.  What happens when you tolType to something 
> other than 'relative' or 'absolute' is somewhat random, as some tests are for 
> tolType equal to 'relative' and others are for 'absolute' and both assume if 
> it is not the one checked, then it is the other.  So when set to 'strict' you 
> will sometimes get the absolute branch of the test and other times the 
> relative branch.  For the example above, you end up getting an absolute test 
> with the tolerance set to 1, so the answer 0.09 is properly considered to be 
> 0.
>
> The intent of changing the student answer to zero was to change answers of 
> the form 1E-13 (which are confusing to some students) to 0.  In the case were 
> RELATIVE tolerances are used, this is in fact the effect, since the relative 
> checks have a special and much more restricted test for zero than absolute 
> checks do (this is when the zeroLevel and zeroLevelTol values come into 
> play), so only values less than 1E-12 get affected.  But when people are 
> setting absolute tolerances on the order of 1 or .5, the behavior is a bit 
> less satisfying.  I think the real problem with these examples is that the 
> absolute tolerance is artificially large.
>
> On the other hand, since it seems that it is easy to get absolute tolerances 
> accidentally, I think it might be a good idea if the problem author were made 
> aware of this by the fact that small answers are being treated as zero, 
> otherwise he or she will be surprised when a student's answer of 0 is counted 
> as correct for an answer of .09 (as happened in Gavin's problem).  The fact 
> that the Parser shows you what is happening could be considered a GOOD thing 
> in terms of problem authoring.  Certainly it alerted Gavin to the error in 
> his problem.
>
> Perhaps changing to zero should be suppressed when absolute checks are being 
> made, but I still think it works for relative checks.  I had considered using 
> the tolerance when stringifying the number so that it would appear with the 
> number of digits that where appropriate for the tolerance, so that an 
> absolute tolerance of .01 would print 1.0001 as 1.00 and relative tolerance 
> of 1 percent would print 10001 as 10000, but couldn't decide what a tolerance 
> of .05 meant in this setting, and thought that perhaps it was too slick.  But 
> it has always bothered me that the answers showed large numbers of digits as 
> though they were significant, when they aren't.  I'd be interested in hearing 
> what Mike, Arnie and Sam have to say about these issues.
>
> Davide

-- 
P. Gavin LaRose, Ph.D.  Program Manager (Instructional Tech.)
Math Dept., University of Michigan
gl...@um...                        "There's no use in trying," [Alice]
734.764.6454                             said. "One Can't believe impossible
http://www.math.lsa.umich.edu/~glarose/  things." "I daresay you haven't had
                                          much practice," said the Queen.
                                                               - Lewis Carrol