#317 joe needs a regression test suite

[Joe Allen]

We should have some kind of automated testing. This would make the whole project more professional: at the very least it ensures that once a bug has been fixed, it stays fixed in future releases.

I'm not sure if a tool exists for this, but it should: simulate user interaction with the editor from a macro or script and compare the results (file and screen contents) with a golden version.

Also there should be some kind of random testing (random editing of a large file would be good check of the buffer code for example).

[John J. Jordan]

In addition to the benefits offered by regression testing, this would also provide a basis for repeatable performance testing, and opens the door to using profile-guided optimization.

I don't know if you're open to using Python here, but I think its unittest framework alongside a terminal emulator library would be well-suited to this purpose. It also provides an abstraction for fixtures, which could be used to setup input files and the joerc appropriately. I had actually messed around with something like this a few months ago... I had just barely gotten pyte (terminal emulator library) talking to JOE and not much else. I managed to dig out that code and flesh it out a little bit. It's incomplete, and probably outright wrong in some spots, but I think it demonstrates the idea.

[Egmont Koblinger]

This might be useful; don't know, haven't taken a closer look yet:
https://bugzilla.gnome.org/show_bug.cgi?id=745045

[John J. Jordan]

Updated, with a few more tests for recently-fixed regressions.

There's some timing issues somewhere in JOE. If bytes get sent to it early, they end up in the wrong place, e.g. in the buffer window rather than the prompt I was just trying to pull up. In general, I've tried to perform actions slowly -- do something, wait for an indicator that it's ready, and then proceed. This still winds up being relatively fast, but you can't just blast out data without a bit of caution.

Also, I had to throttle output on a test for wordwrap because the typeahead check fix disabled it ;-)

There's more I want to do framework-wise, and make my naming convention consistent (this always bites me in Python -- doesn't help that the standard library itself is schitzo there)... but it's... close?

EDIT: + cleanup.

[John J. Jordan]

This has been in JOE for a while now. It does not cover enough quite yet but the framework is there and proven, imo. Closing.