Menu
▾
▴
#8 Provide better information upon validation failure
Currently when a benchmark fails validation, the user just gets a message stating that validation failed, and the observed and expected checksums.
Ideally we would have captured the expected output and when validaton fails, we output the diff between expected and observed outputs.
This feature requested by Alexei Svitkine.
No benchmarks currently have the expected output verbatim recorded since the checksum is a summary of that information. This would require keeping the expected output for all output files and stderr and stdout that are currently processed by the digest and buffering or logging the output for diffs.