#222 Put binary test data somewhere else, use better test control

[Ulf Lorenz]

What and Why

Currently, the demos double as tests. That is fine. The reference output data is placed in the corresponding demo data, and also doubles as input for the Wavepacket wiki (e.g., animations are referenced from there).

With the advent of more and more demos involving plotting, we run into the problem that this test data slowly becomes rather large. Being binary data, it cannot be efficiently compressed and bloats the git repository. This is not a high-priority problem, but these are regression tests, so they are going to change on a regular basis. The problem can be partially mended if you commit changes to the test data only when you are done with a branch, but this again requires care from the developer.

Another problem looming at the horizon is that updating the reference data is painful. It involves copying certain files around to certain locations. Currently, this can still be handled, but it starts to become a problem.

There should be generic solutions for this problem, I am pretty sure Wavepacket is not the first project that uses regression tests. One such solution should preferably be implemented, maybe together with CI.

---

Mostly fixed as part of [#226]. The plotting output is no longer used for binary comparison, but only its existence (and size > 1kB) is verified. While this does not quite solve all the issues here (plots may change over time), this makes the issue way less important.