I've updated the Cucumber code for our smoke tests, adding a test comparing for tab delimited tables to known good versions. The test compares line by line, and if a line differs, it compares that line field by field. If a field is a valid floating point number, it compares the fields numerically to see if they match within a specified tolerance. Implementing this tests reduces the number of failings tests from 13 to 8. The remaining failing tests have small floating point differences and a change in the order of the lines.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I committed a patch that partially fixed the smoke tests on OS X by add floating point tests within a specified relative error. Unfortunately my first implementation used the ruby "csv" module, and the interface to that module has changed dramatically over the last few versions. It worked on OS X, but doesn't work on our local Linux boxes, which have a much older version of Ruby installed for use in Cucumber. I'll have to re-implement using a more stable API.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I've updated the Cucumber code for our smoke tests, adding a test comparing for tab delimited tables to known good versions. The test compares line by line, and if a line differs, it compares that line field by field. If a field is a valid floating point number, it compares the fields numerically to see if they match within a specified tolerance. Implementing this tests reduces the number of failings tests from 13 to 8. The remaining failing tests have small floating point differences and a change in the order of the lines.
I committed a patch that partially fixed the smoke tests on OS X by add floating point tests within a specified relative error. Unfortunately my first implementation used the ruby "csv" module, and the interface to that module has changed dramatically over the last few versions. It worked on OS X, but doesn't work on our local Linux boxes, which have a much older version of Ruby installed for use in Cucumber. I'll have to re-implement using a more stable API.