Menu
▾
▴
[WWcvs] pg: Added a new experimental diagonstic function for the function answer
|
From: dpvc v. a. <we...@ma...> - 2005-10-16 03:28:21
|
Log Message:
-----------
Added a new experimental diagonstic function for the function answer
checker. When enabled, it will produce graphs of the correct answer,
the student answer, and the absolute and relative errors, and will
list the data points used in the comparison, plus the numerical values
of the results and errors.
To enable the diagnostic, use
ANS(fun_cmp($f,diagnostics=>1));
Note that only single-variable functions can be graphed at the moment,
so if you are using a multi-variable check, you need to disable the
graphing. To do this use
ANS(fun_cmp($f,vars=>['x','y'],diagnostics=>[showGraphs=>0]));
The diagnostic mode is only available for the Parser-based versions of
the function checker, and (of course) with the native Parser objects
as well:
ANS(Formula($f)->cmp(diagnostics=>1));
There are now Context settings to control the diagnostics, which can
be set through Context()->diagnostics->set(). For example
Context()->diagnostics->set(formulas=>{showGraphs=>0});
would turn off graphs for all functions comparisons. Some of the
other values you can set are:
formulas => {
showTestPoints => 1, # show the test points and function values
showRelativeErrors => 1, # show the relative errors for the student answer
showAbsoluteErrors => 1, # show the absolute errors for the student answer
showGraphs => 1, # show the various graphs
graphRelativeErrors => 1, # show the relative error graph
graphAbsoluteErrors => 1, # show the absolute error graph
clipRelativeError => 5, # don't show relative errors above 5
clipAbsoluteError => 5, # don't show absolute errors above 5
plotTestPoints => 1, # include dots at the test points
combineGraphs => 1, # show correct and student graphs in one image
},
graphs => {
divisions => 75, # the number of data points to plot
limits => [-2,2], # the lower and upper limit of the plot
# (taken from the function limits if not provided)
size => 250, # pixel size of the image (could be [width,height])
grid => [10,10], # number of grid lines in each direction
axes => [0,0], # where to put axes relative to origin
}
Any of these can be set in the Context(), or in the answer checker
itself. If you set diagnostics to an array reference, the entries in
the array refer to element of the formulas hash. If you set
diagonstics to a hash reference, then you can set values in either the
formulas or graphs hashes, as in:
ANS(Formula($f)->cmp(diagnostics=>{
formulas => {showAbsoluteErrors=>0},
graphs => {size=>300, divisions=>100},
}));
If you want all function checkers to show diagnostics, use
Context()->diagonstics->set(formulas=>{show=>1});
The image file names are modified to include the current time so that
the names will be unique. This avoids problems with the browser cache
showing a old image when a new one has been generated. But this also
means that the temporary image directory will fill up fast, so you may
need to empty it if you use the diagnostic images frequently.
This is just a first attempt at a diagnostic feature. I think it will
help when you are not sure if the tolerances are set properly, or if
you think a student answer should be markes correct but isn't, as it
will point out which point(s) are not being accepted.
Modified Files:
--------------
pg/lib/Value:
AnswerChecker.pm
pg/macros:
PGanswermacros.pl
Revision Data
-------------
Index: AnswerChecker.pm
===================================================================
RCS file: /webwork/cvs/system/pg/lib/Value/AnswerChecker.pm,v
retrieving revision 1.70
retrieving revision 1.71
diff -Llib/Value/AnswerChecker.pm -Llib/Value/AnswerChecker.pm -u -r1.70 -r1.71
--- lib/Value/AnswerChecker.pm
+++ lib/Value/AnswerChecker.pm
@@ -72,10 +72,12 @@
$ans->{debug} = $ans->{rh_ans}{debug};
$ans->install_evaluator(sub {$ans = shift; $ans->{correct_value}->cmp_parse($ans)});
$ans->install_pre_filter('erase') if $self->{ans_name}; # don't do blank check if answer_array
+ $self->cmp_diagnostics($ans);
return $ans;
}
sub correct_ans {protectHTML(shift->string)}
+sub cmp_diagnostics {}
#
# Parse the student answer and compute its value,
@@ -130,6 +132,7 @@
if ($self->cmp_collect($ans)) {
$self->cmp_equal($ans);
$self->cmp_postprocess($ans) if !$ans->{error_message};
+ $self->cmp_diagnostics($ans);
}
} else {
$self->cmp_collect($ans);
@@ -1314,6 +1317,8 @@
#
# Handle removal of outermost parens in a list.
+# Evaluate answer, if the eval option is used.
+# Handle the UpToConstant option.
#
sub cmp {
my $self = shift;
@@ -1382,6 +1387,231 @@
}
#
+# Diagnostics for Formulas
+#
+sub cmp_diagnostics {
+ my $self = shift; my $ans = shift;
+ my $isEvaluator = (ref($ans) =~ /Evaluator/)? 1: 0;
+ my $hash = $isEvaluator? $ans->rh_ans : $ans;
+ my $diagnostics = $self->{context}->diagnostics->merge("formulas",$self,$hash);
+ my $formulas = $diagnostics->{formulas};
+ return unless $formulas->{show};
+
+ my $output = "";
+ if ($isEvaluator) {
+ #
+ # The tests to be performed with the answer checker is created
+ #
+ $self->getPG('loadMacros("PGgraphmacros.pl")');
+ my ($inputs) = $self->getPG('$inputs_ref');
+ my $process = $inputs->{checkAnswers} || $inputs->{previewAnswers} || $inputs->{submitAnswers};
+ if ($formulas->{checkNumericStability} && !$process) {
+ ### still needs to be written
+ }
+ } else {
+ #
+ # The checks to be performed when an answer is submitted
+ #
+ my $student = $ans->{student_formula};
+ my $points = [map {$_->[0]} @{$self->{test_points}}];
+
+ #
+ # The graphs of the functions and errors
+ #
+ if ($formulas->{showGraphs}) {
+ my @G = ();
+ if ($formulas->{combineGraphs}) {
+ push(@G,$self->cmp_graph($diagnostics,[$student,$self],
+ title=>'Student Answer (red)<BR>Correct Answer (green)<BR>',
+ points=>$points,showDomain=>1));
+ } else {
+ push(@G,$self->cmp_graph($diagnostics,$self,title=>'Correct Answer'));
+ push(@G,$self->cmp_graph($diagnostics,$student,title=>'Student Answer'));
+ }
+ my $cutoff = Value::Formula->new($self->getFlag('tolerance'));
+ if ($formulas->{graphAbsoluteErrors}) {
+ push(@G,$self->cmp_graph($diagnostics,[abs($self-$student),$cutoff],
+ clip=>$formulas->{clipAbsoluteError},
+ title=>'Absolute Error',points=>$points));
+ }
+ if ($formulas->{graphRelativeErrors}) {
+ push(@G,$self->cmp_graph($diagnostics,[abs(($self-$student)/$self),$cutoff],
+ clip=>$formulas->{clipRelativeError},
+ title=>'Relative Error',points=>$points));
+ }
+ $output .= '<TABLE BORDER="0" CELLSPACING="0" CELLPADDING="0">'
+ . '<TR VALIGN="TOP">'.join('<TD WIDTH="20"></TD>',@G).'</TR></TABLE>';
+ }
+
+ #
+ # The test points and values
+ #
+ my @rows = (); my $colsep = '</TD><TD WIDTH="20"></TD><TD ALIGN="RIGHT">';
+ my @P = (map {(scalar(@{$_}) == 1)? $_->[0]: Value::Point->make(@{$_})} @{$self->{test_points}});
+ my @i = sort {$P[$a] <=> $P[$b]} (0..$#P);
+ if ($formulas->{showTestPoints}) {
+ $student->createPointValues($self->{test_points},0,1,1) unless $student->{test_values};
+ my @p = ("Input:",(map {$P[$i[$_]]} (0..$#P)));
+ push(@rows,'<TR><TD ALIGN="RIGHT">'.join($colsep,@p).'</TD></TR>');
+ push(@rows,'<TR><TD ALIGN="RIGHT">'.join($colsep,("<HR>")x scalar(@p)).'</TD></TR>');
+ push(@rows,'<TR><TD ALIGN="RIGHT">'
+ .join($colsep,"Correct Answer:", map {$self->{test_values}[$i[$_]]} (0..$#P))
+ .'</TD></TR>');
+ my $test = $student->{test_values};
+ push(@rows,'<TR><TD ALIGN="RIGHT">'
+ .join($colsep,"Student Answer:", map {Value::isNumber($test->[$i[$_]])? $test->[$i[$_]]: "undefined"} (0..$#P))
+ .'</TD></TR>');
+ }
+ #
+ # The absolute errors (colored by whether they are ok or too big)
+ #
+ if ($formulas->{showAbsoluteErrors}) {
+ my @p = ("Absolute Error:");
+ my $tolerance = $self->getFlag('tolerance');
+ my $tolType = $self->getFlag('tolType'); my $error;
+ foreach my $j (0..$#P) {
+ if (Value::isNumber($student->{test_values}[$i[$j]])) {
+ $error = abs($self->{test_values}[$i[$j]]-$student->{test_values}[$i[$j]]);
+ $error = '<SPAN STYLE="color:#'.($error<$tolerance ? '00AA00': 'AA0000').'">'.$error.'</SPAN>'
+ if $tolType eq 'absolute';
+ } else {$error = "---"}
+ push(@p,$error);
+ }
+ push(@rows,'<TR><TD ALIGN="RIGHT">'.join($colsep,@p).'</TD></TR>');
+ }
+ #
+ # The relative errors (colored by whether they are OK ro too big)
+ #
+ if ($formulas->{showRelativeErrors}) {
+ my @p = ("Relative Error:");
+ my $tolerance = $self->getFlag('tolerance');
+ my $tolType = $self->getFlag('tolType'); my $error;
+ foreach my $j (0..$#P) {
+ if (Value::isNumber($student->{test_values}[$i[$j]])) {
+ $error = abs(($self->{test_values}[$i[$j]]-$student->{test_values}[$i[$j]])/
+ ($self->{test_values}[$i[$j]]||1E-10));
+ $error = '<SPAN STYLE="color:#'.($error<$tolerance ? '00AA00': 'AA0000').'">'.$error.'</SPAN>'
+ if $tolType eq 'relative';
+ } else {$error = "---"}
+ push(@p,$error);
+ }
+ push(@rows,'<TR><TD ALIGN="RIGHT">'.join($colsep,@p).'</TD></TR>');
+ }
+ #
+ # Put the data into a table
+ #
+ if (scalar(@rows)) {
+ $output .= '<p><HR><p><TABLE BORDER="0" CELLSPACING="0" CELLPADDING="0">'
+ . join('<TR><TD HEIGHT="3"></TD>',@rows)
+ . '</TABLE>';
+ }
+ }
+ #
+ # Put all the diagnostic output into a frame
+ #
+ return unless $output;
+ $output
+ = '<TABLE BORDER="1" CELLSPACING="2" CELLPADDING="20" BGCOLOR="#F0F0F0">'
+ . '<TR><TD ALIGN="LEFT"><B>Diagnostics for '.$self->string .':</B>'
+ . '<P><CENTER>' . $output . '</CENTER></TD></TR></TABLE><P>';
+ warn $output;
+}
+
+#
+# Draw a graph from a given Formula object
+#
+sub cmp_graph {
+ my $self = shift; my $diagnostics = shift;
+ my $F1 = shift; my $F2; ($F1,$F2) = @{$F1} if (ref($F1) eq 'ARRAY');
+ #
+ # Get the various options
+ #
+ my %options = (title=>'',points=>[],@_);
+ my $graphs = $diagnostics->{graphs};
+ my $limits = $graphs->{limits}; $limits = $self->getFlag('limits',[-2,2]) unless $limits;
+ $limits = $limits->[0] if ref($limits) eq 'ARRAY' && ref($limits->[0]) eq 'ARRAY';
+ my $size = $graphs->{size}; $size = [$size,$size] unless ref($size) eq 'ARRAY';
+ my $steps = $graphs->{divisions};
+ my $points = $options{points}; my $clip = $options{clip};
+ my ($my,$My) = (0,0); my ($mx,$Mx) = @{$limits};
+ my $dx = ($Mx-$mx)/$steps; my $f; my $y;
+
+ #
+ # Find the max and min values of the function
+ #
+ foreach $f ($F1,$F2) {
+ next unless defined($f);
+ unless (scalar(keys(%{$f->{variables}})) < 2) {
+ warn "Only formulas with one variable can be graphed";
+ return "";
+ }
+ if ($f->isConstant) {
+ $y = $f->eval;
+ $my = $y if $y < $my; $My = $y if $y > $My;
+ } else {
+ my $F = $f->perlFunction;
+ foreach my $i (0..$steps-1) {
+ $y = eval {&{$F}($mx+$i*$dx)}; next unless defined($y) && Value::isNumber($y);
+ $my = $y if $y < $my; $My = $y if $y > $My;
+ }
+ }
+ }
+ $My = 1 if abs($My - $my) < 1E-5;
+ $my *= 1.1; $My *= 1.1;
+ if ($clip) {
+ $my = -$clip if $my < -$clip;
+ $My = $clip if $My > $clip;
+ }
+ $my = -$My/10 if $my > -$My/10; $My = -$my/10 if $My < -$my/10;
+ my $a = Value::Real->new(($My-$my)/($Mx-$mx));
+
+ #
+ # Create the graph itself, with suitable title
+ #
+ my $grf = $self->getPG('$_grf_ = {n => 0}');
+ $grf->{Goptions} = [
+ $mx,$my,$Mx,$My,
+ axes => $graphs->{axes},
+ grid => $graphs->{grid},
+ size => $size,
+ ];
+ $grf->{G} = $self->getPG('init_graph(@{$_grf_->{Goptions}})');
+ $grf->{G}->imageName($grf->{G}->imageName.'-'.time()); # avoid browser cache
+ $self->cmp_graph_function($grf,$F2,"green",$steps,$points) if defined($F2);
+ $self->cmp_graph_function($grf,$F1,"red",$steps,$points);
+ my $image = $self->getPG('alias(insertGraph($_grf_->{G}))');
+ $image = '<IMG SRC="'.$image.'" WIDTH="'.$size->[0].'" HEIGHT="'.$size->[1].'" BORDER="0" STYLE="margin-bottom:5px">';
+ my $title = $options{title}; $title .= '<DIV STYLE="margin-top:5px"></DIV>' if $title;
+ $title .= "<SMALL>Domain: [$mx,$Mx]</SMALL><BR>" if $options{showDomain};
+ $title .= "<SMALL>Range: [$my,$My]<BR>Aspect ratio: $a:1</SMALL>";
+ return '<TD ALIGN="CENTER" VALIGN="TOP" NOWRAP>'.$image.'<BR>'.$title.'</TD>';
+}
+
+#
+# Add a function to a graph object, and plot the points
+# that are used to test the function
+#
+sub cmp_graph_function {
+ my $self = shift; my $grf = shift; my $F = shift;
+ my $color = shift; my $steps = shift; my $points = shift;
+ $grf->{n}++; my $Fn = "F".$grf->{n}; $grf->{$Fn} = $F; my $f;
+ if ($F->isConstant) {
+ my $y = $F->eval;
+ $f = $self->getPG('new Fun(sub {'.$y.'},$_grf_->{G})');
+ } else {
+ my $X = (keys %{$F->{variables}})[0];
+ $f = $self->getPG('new Fun(sub {Parser::Evaluate($_grf_->{'.$Fn.'},'.$X.'=>shift)},$_grf_->{G})');
+ foreach my $x (@{$points}) {
+ my $y = Parser::Evaluate($F,($X)=>$x); next unless defined($y) && Value::isNumber($y);
+ $grf->{x} = $x; $grf->{y} = $y;
+ my $C = $self->getPG('new Circle($_grf_->{x},$_grf_->{y},4,"'.$color.'","'.$color.'")');
+ $grf->{G}->stamps($C);
+ }
+ }
+ $f->color($color); $f->weight(2); $f->steps($steps);
+}
+
+#
# If an answer array was used, get the data from the
# Matrix, Vector or Point, and format the array of
# data using the original parameter
Index: PGanswermacros.pl
===================================================================
RCS file: /webwork/cvs/system/pg/macros/PGanswermacros.pl,v
retrieving revision 1.39
retrieving revision 1.40
diff -Lmacros/PGanswermacros.pl -Lmacros/PGanswermacros.pl -u -r1.39 -r1.40
--- macros/PGanswermacros.pl
+++ macros/PGanswermacros.pl
@@ -1138,7 +1138,6 @@
if ref($rh_ans->{student_value});
return $rh_ans;
});
- $cmp->{debug} = $num_params{debug};
&$Context($oldContext);
return $cmp;
@@ -1502,6 +1501,7 @@
'zeroLevel' => $functZeroLevelDefault,
'zeroLevelTol' => $functZeroLevelTolDefault,
'debug' => 0,
+ 'diagnostics' => undef,
);
# allow var => 'x' as an abbreviation for var => ['x']
@@ -1552,6 +1552,7 @@
'zeroLevelTol' => $out_options{'zeroLevelTol'},
'params' => $out_options{'params'},
'debug' => $out_options{'debug'},
+ 'diagnostics' => $out_options{'diagnostics'} ,
),
);
}
@@ -1674,6 +1675,7 @@
'zeroLevel' => $functZeroLevelDefault,
'zeroLevelTol' => $functZeroLevelTolDefault,
'debug' => 0,
+ 'diagnostics' => undef,
);
my $var_ref = $options{'vars'};
@@ -1697,6 +1699,7 @@
'scale_norm' => 1,
'params' => $ra_params,
'debug' => $options{debug} ,
+ 'diagnostics' => $options{diagnostics} ,
);
}
@@ -1934,7 +1937,10 @@
# End of cleanup of calling parameters
########################################################
- my %options = (debug => $func_params{'debug'});
+ my %options = (
+ debug => $func_params{'debug'},
+ diagnostics => $func_params{'diagnostics'},
+ );
#
# Initialize the context for the formula
@@ -1975,7 +1981,6 @@
$f->{limits} = $func_params{'limits'};
$f->{test_points} = $func_params{'test_points'};
my $cmp = $f->cmp(%options);
- $cmp->{debug} = 1 if $func_params{'debug'};
&$Context($oldContext);
#
|
