I'd like to add some end-to-end testing to OOoPy to gain further trust in the tool (especially the Mail Merge part) and spot future incompatibilities with upcoming versions of Open Office / Libre Office.
What I imagine is something like this:
1) have some demo data (odt templates + csv data), then fist run mailmerge; after that run Open Office in headless mode and export in plain text format and ensure that the substitutions happened just as expected. This should cover basic cases, but also problematic inputs (umlauts, spaces, different encodings, ...)
2) same as above, but test for pixel-perfect output. This time do not export to plain text, but to pixel data (Bitmap/TIFF/PNG/...) and then use imaging software (or PIL/Python Imaging) to determine what pixels differ from a reference bitmap. Then save a difference bitmap to let a human user spot the differences easily. It might be necessary to specify multiple "acceptable" reference bitmaps if there are inevitable slight differences when using different operating systems/open office releases, but that number should be limited and small.
Do you think this approach is reasonable? Would you be interested in the relevant code?
Best Regards
Michael
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
On Sun, Sep 22, 2013 at 02:57:26PM +0000, Michael Nagel wrote:
What I imagine is something like this:
1) have some demo data (odt templates + csv data), then fist run
mailmerge; after that run Open Office in headless mode and export in
plain text format and ensure that the substitutions happened just as
expected. This should cover basic cases, but also problematic inputs
(umlauts, spaces, different encodings, ...)
You should look into the ooopy doctests. They do something similar: Do
the mailmerge part and then verify that the resulting XML inside the
open office document has the intended structure and contents. This is in
a way superior to just test that the text matches.
2) same as above, but test for pixel-perfect output. This time do not
export to plain text, but to pixel data (Bitmap/TIFF/PNG/...) and then
use imaging software (or PIL/Python Imaging) to determine what pixels
differ from a reference bitmap. Then save a difference bitmap to let a
human user spot the differences easily. It might be necessary to
specify multiple "acceptable" reference bitmaps if there are
inevitable slight differences when using different operating
systems/open office releases, but that number should be limited and
small.
That would certainly be interesting. But since the open document format
is something that leave some interpretation to the client you will
probably end up modifying your tests for each new version of Open/Libre
Office.
Ralf
Ralf Schlatterbeck email: ralf@zoo.priv.at
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Hi there,
I'd like to add some end-to-end testing to OOoPy to gain further trust in the tool (especially the Mail Merge part) and spot future incompatibilities with upcoming versions of Open Office / Libre Office.
What I imagine is something like this:
1) have some demo data (odt templates + csv data), then fist run mailmerge; after that run Open Office in headless mode and export in plain text format and ensure that the substitutions happened just as expected. This should cover basic cases, but also problematic inputs (umlauts, spaces, different encodings, ...)
2) same as above, but test for pixel-perfect output. This time do not export to plain text, but to pixel data (Bitmap/TIFF/PNG/...) and then use imaging software (or PIL/Python Imaging) to determine what pixels differ from a reference bitmap. Then save a difference bitmap to let a human user spot the differences easily. It might be necessary to specify multiple "acceptable" reference bitmaps if there are inevitable slight differences when using different operating systems/open office releases, but that number should be limited and small.
Do you think this approach is reasonable? Would you be interested in the relevant code?
Best Regards
Michael
On Sun, Sep 22, 2013 at 02:57:26PM +0000, Michael Nagel wrote:
You should look into the ooopy doctests. They do something similar: Do
the mailmerge part and then verify that the resulting XML inside the
open office document has the intended structure and contents. This is in
a way superior to just test that the text matches.
That would certainly be interesting. But since the open document format
is something that leave some interpretation to the client you will
probably end up modifying your tests for each new version of Open/Libre
Office.
Ralf
Ralf Schlatterbeck email: ralf@zoo.priv.at