Re: [WWdevel] New diagnostic capabilities in function answer checker

[Sam H. <sh...@ma...>]

On Oct 16, 2005, at 11:44, Davide P. Cervone wrote:

> Folks:
>
> A few weeks ago, I mentioned an idea for helping problem writers  
> analyze whether their functions were using good input ranges and  
> tolerance values.  I finally had the chance to put something  
> together for that, and have updated the Parser package to allow you  
> to get diagnostic information for the Formula answer checker.   
> Since this is used by the Parser-based version of fun_cmp(), it is  
> also available to it.
>
> The diagnostics include a graph of the student answer and the  
> correct answer (one one set of axes), plus graphs of the absolute  
> and relative errors between the two.  It also shows the points  
> actually used for the function comparison and the numeric values of  
> the functions and errors.  Any of these items can be enabled/ 
> disabled separately.

This sounds like a great debugging tool.

> [snip log message]
>
> Note that the diagnostics are added to the page via the "warn"  
> command, so the screen will show pink when they are enabled.  As a  
> side effect, the diagnostics will go into the httpd error log as  
> well, and since they are pretty long, this can increase the size of  
> your log file if you use it a lot.  Is there a better way to add  
> this kind of data to the output?  I can't just use TEXT, because it  
> is generated when the answer checker runs (not when the page is set  
> up), and I think that it is too late for that (though I didn't  
> actually try it).

"warn" doesn't seem quite right to me -- the diagnostics aren't an  
indication that something has necessarily gone wrong. Perhaps it  
would be appropriate to add a new "slot" in the content that's passed  
back from the translator for data such as this. It could then be  
displayed conditionally by Problem.pm (i.e. only show it to  
professors?).

I've never looked closely at the answer evaluation process, so I  
don't know what would be easiest or make the most sense  
architecturally, but maybe this could be added as a "diagnostics"  
field to the value returned by each answer evaluator (called  
rh_ans_evaluation_result in process_answers in Translator.pm).

Mike may wish to comment further since he is most familiar with this  
mechanism.
-sam