Re: [Rails-devel] Automated testing

[Erik V. <eri...@xs...>]

It's a unfortunate that even small report format changes render test cases
invalid, and it makes this way of testing much less valuable for me.
I understand that this is the only simple way, but perhaps we should start
thinking about a somewhat more intelligent test evaluation method.

When I do my own regression testing with saved games, the main thing I'm
looking for is that the game runs to the end without throwing exceptions,
and that the end result looks reasonable.
An automated test would of course need to be a bit more thorough:
- are the player and companies turn sequences identical,
- are the cash amounts the same,
And perhaps a few more. 

Only checking the status overview at the end of each round would already do
a lot.  Now that overview only reports cash, and I'm wondering if it would
help to add other possessions,
to create a complete written status overview (like the zines publish about
the games they run), and check only that?

However, that is for the future.  For now, I can't guarantee anything.  I'm
myself occasionally wrestling with saved games that no longer load as a
result of some improvement.
For instance, removing redundant actions (where only 'Done' is possible) has
had that effect in some cases. I still occasionally have to remove 'Done'
actions from saved games to make these loadable again.  Fixing bugs has
equally well invalidated test cases in the past.  So did implementation of
the dynamic company operating order.

I can't tell if any such thing would apply to the current test set, which I
haven't studied. The more recent the file is, the larger the chance that it
is OK.

By private mail I'm sending you some files of the kind that I regularly use
to check if things still work, and that cover some interesting aspects, in
particular in relation to train trading. 
But in general I think recently completed games are more valuable (trusting
that the players haven't missed any errors!).

Erik.

> -----Original Message-----
> From: Stefan Frey [mailto:ste...@we...]
> Sent: Monday, August 01, 2011 7:27 AM
> To: 'Development list for Rails: an 18xx game'
> Subject: [Rails-devel] Automated testing
> 
> I have added the games that I have received to automated game testing.
> Thanks to all contributing save files.
> There is also a table of those in the wiki (under developer information).
> 
> In the next few days (as time permits) I will add the existing information
on
> game testing (from older mails) to the wiki and I hope it will increase
the
> usage of those tests.
> 
> Currently all tests fail as Erik has changed many things that impacted the
> Game Report (phase names and train obsolete messages).
> 
> Question to Erik: If you think all changes are merely phase name changes
and
> adding/removing additional messages and you are sure that everything
> works ok, give me a nod and then the game reports can be updated to the
> current states and all tests will show green again.
> 
> Unfortunately my master still automatically merges on pull, I will setup
the
> master fresh as soon as I have pushed all my current local branches.
> 
> Stefan
> 
> 
> 
>
----------------------------------------------------------------------------
--
> Got Input?   Slashdot Needs You.
> Take our quick survey online.  Come on, we don't ask for help often.
> Plus, you'll get a chance to win $100 to spend on ThinkGeek.
> http://p.sf.net/sfu/slashdot-survey
> _______________________________________________
> Rails-devel mailing list
> Rai...@li...
> https://lists.sourceforge.net/lists/listinfo/rails-devel