gamedevlists-general

[GD-General] Unit tests

[Jason B. <bay...@ho...>]

I'm very curious to explore how (or whether) the XP-stoked unit test 
paradigm can work in games. Are any of you are using automated unit tests in 
your game engine? If so, do you have any suggestions to help me along? Can 
you point me at any printed or online resources about unit testing that 
would apply to real time games?

I've been making a concerted effort lately to work some small automated 
tests into our game engine code to do simple things like checking for 
alignment of related tables, checking math-related functions, etc. So far, 
it's been limited to a group of special test functions that get run when the 
game is launched, and which will assert if anything goes wrong.

But I'm at a loss as to how go about constructing unit tests for most of the 
game functions, such as menu navigation, interacting with AI, etc. It seems 
like this would require me to write some kind of module to simulate user 
input (cursor moves, button presses, etc.) and I'm not sure how I could 
possibly write something that could come close to simulating the input 
patterns of a human. Plus, I'm writing for resource-limited handheld 
consoles (currently the Nintendo GBA and DS), so I can't really use a 
separate app to poke at my game while it's running, as I might be able to do 
on a PC.

If you have any suggestions, anecdotes, or resources, I'd love to hear them.

Jason Bay
Griptonite Games

(P.S. I sent this message earlier in the week, but it didn't appear to show 
up on the list. My apologies if this turns out to be a duplicate.)

Re: [GD-General] Unit tests

[Kent Q. <ken...@co...>]

Much testing can be done if you architect the application properly. In 
particular, it's very useful to create a very thin interface through which 
all variable input flows. This includes:

* Keystrokes
* Controllers (mouse, joystick, touchpad)
* Timing information
* Random numbers

You can then record and play back this stream to provide certain types of 
tests (as well as a handy macro capability).

You can also write simulators which can act like anything from random 
banging on the keyboard to structured testing.

On the other side of the same interface, you can write a simulator that 
checks to see if your input functions are generating the expected results.

If you're also careful to separate your user interface using some sort of 
pattern like Model-View-Controller, it becomes possible to write a View 
that has no graphics and simply reports results. This gives you the ability 
to record and play back user interface code without having to watch a screen.

And so forth. I must confess I've never successfully implemented all of 
this. Different bits on different projects, yes. But priorities vary per 
project.

Don't forget that most internal functions have no user interface and can be 
independently tested if you're clever about it.

         Kent

At 01:28 PM 10/14/2004, you wrote:
>I'm very curious to explore how (or whether) the XP-stoked unit test 
>paradigm can work in games. Are any of you are using automated unit tests 
>in your game engine? If so, do you have any suggestions to help me along? 
>Can you point me at any printed or online resources about unit testing 
>that would apply to real time games?
>
>I've been making a concerted effort lately to work some small automated 
>tests into our game engine code to do simple things like checking for 
>alignment of related tables, checking math-related functions, etc. So far, 
>it's been limited to a group of special test functions that get run when 
>the game is launched, and which will assert if anything goes wrong.
>
>But I'm at a loss as to how go about constructing unit tests for most of 
>the game functions, such as menu navigation, interacting with AI, etc. It 
>seems like this would require me to write some kind of module to simulate 
>user input (cursor moves, button presses, etc.) and I'm not sure how I 
>could possibly write something that could come close to simulating the 
>input patterns of a human. Plus, I'm writing for resource-limited handheld 
>consoles (currently the Nintendo GBA and DS), so I can't really use a 
>separate app to poke at my game while it's running, as I might be able to 
>do on a PC.
>
>If you have any suggestions, anecdotes, or resources, I'd love to hear them.
>
>Jason Bay
>Griptonite Games
>
>(P.S. I sent this message earlier in the week, but it didn't appear to 
>show up on the list. My apologies if this turns out to be a duplicate.)
>
>
>
>
>-------------------------------------------------------
>This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
>Use IT products in your business? Tell us what you think of them. Give us
>Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out more
>http://productguide.itmanagersjournal.com/guidepromo.tmpl
>_______________________________________________
>Gamedevlists-general mailing list
>Gam...@li...
>https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
>Archives:
>http://sourceforge.net/mailarchive/forum.php?forum_id=557

----
Kent Quirk
CTO, CogniToy

Re: [GD-General] Unit tests

[Chris R. <c....@gm...>]

I found unit-testing cumbersome as soon as you come to debugging your
unit-test code.=20

As soon as you come to terms with more abstract and highlevel design
aspects of your engine, the time and effort spent in unit-testing (for
us at least) did not outweigh the benefits.

For low-level (math-, mesh/geometry-, animation related functionality,
where behaviour is well defined and results can be checked easily) there
was a huge benefit as unit tests resulted in catching code changes
breaking existing code paths.

For high level, abstact functionality we abandoned the unit test
approach and implemented a nightly build script, which checked if the
engine would compile and would load all finished content and compare the
standard output against wanted results (e.g, it would send a mail to all
developers if the word "error:" appeared in the output while loading the
content).

This of course did not catch the bugs when the content was displayed
incorrectly, or if other functionality was broken (e.g, ai behaving
wrong, renderer displaying strange images), but these bugs cannot be
caught by unit tests either but by a developer who tests his code
extensively by running it and looking at the results.=20

just mho and my 2 cents,=20

regards,=20
Chris=20



On Thu, 2004-10-14 at 19:28, Jason Bay wrote:
> I'm very curious to explore how (or whether) the XP-stoked unit test=20
> paradigm can work in games. Are any of you are using automated unit tests=
 in=20
> your game engine? If so, do you have any suggestions to help me along? Ca=
n=20
> you point me at any printed or online resources about unit testing that=20
> would apply to real time games?
>=20
> I've been making a concerted effort lately to work some small automated=20
> tests into our game engine code to do simple things like checking for=20
> alignment of related tables, checking math-related functions, etc. So far=
,=20
> it's been limited to a group of special test functions that get run when =
the=20
> game is launched, and which will assert if anything goes wrong.
>=20
> But I'm at a loss as to how go about constructing unit tests for most of =
the=20
> game functions, such as menu navigation, interacting with AI, etc. It see=
ms=20
> like this would require me to write some kind of module to simulate user=20
> input (cursor moves, button presses, etc.) and I'm not sure how I could=20
> possibly write something that could come close to simulating the input=20
> patterns of a human. Plus, I'm writing for resource-limited handheld=20
> consoles (currently the Nintendo GBA and DS), so I can't really use a=20
> separate app to poke at my game while it's running, as I might be able to=
 do=20
> on a PC.
>=20
> If you have any suggestions, anecdotes, or resources, I'd love to hear th=
em.
>=20
> Jason Bay
> Griptonite Games
>=20
> (P.S. I sent this message earlier in the week, but it didn't appear to sh=
ow=20
> up on the list. My apologies if this turns out to be a duplicate.)
>=20
>=20
>=20
>=20
> -------------------------------------------------------
> This SF.net email is sponsored by: IT Product Guide on ITManagersJournal
> Use IT products in your business? Tell us what you think of them. Give us
> Your Opinions, Get Free ThinkGeek Gift Certificates! Click to find out mo=
re
> http://productguide.itmanagersjournal.com/guidepromo.tmpl
> _______________________________________________
> Gamedevlists-general mailing list
> Gam...@li...
> https://lists.sourceforge.net/lists/listinfo/gamedevlists-general
> Archives:
> http://sourceforge.net/mailarchive/forum.php?forum_id=3D557
>=20

Re: [GD-General] Unit tests

[Jorrit T. <jor...@gm...>]

On Fri, 15 Oct 2004 13:44:17 +0200, Chris Raine <c....@gm...> wrote:
> This of course did not catch the bugs when the content was displayed
> incorrectly, or if other functionality was broken (e.g, ai behaving
> wrong, renderer displaying strange images), but these bugs cannot be
> caught by unit tests either but by a developer who tests his code
> extensively by running it and looking at the results.

It would be possible to do output tests too if you make a screenshot
from the rendering and compare that to a previous rendering. You can
use image comparison tools that do 'fuzzy' comparisons so that small
deviations are not flagged as errors.

Greetings,

Re: [GD-General] Unit tests

[Noel L. <nl...@co...>]

On Friday 15 October 2004 04:44 am, Chris Raine wrote:
> I found unit-testing cumbersome as soon as you come to debugging your
> unit-test code.

I don't understand how unit tests can be cumbersome to debug. A unit test 
should be extremely simple and be just a few lines long. You will hardly ever 
have to debug (as in, step in the debugger) a unit test itself. The problem 
should always be obvious it if fails because a unit test tests one and only 
one thing.

> As soon as you come to terms with more abstract and highlevel design
> aspects of your engine, the time and effort spent in unit-testing (for
> us at least) did not outweigh the benefits.

It sounds like you didn't have a good architecture. It's easier to start 
testing low-level features, but to test the high-level code, you really need 
a modular architecture with a minimum of dependencies.

The best way to arrive at this architecture is to use test-driven development 
(TDD). I can't recommend it highly enough. If you develop things that way, it 
will become immediately obvious how to structure your AI code so it's easily 
testable. You'll find you can create just about any object on the stack and 
manipulate it in isolation from other objects/libraries.

> For high level, abstact functionality we abandoned the unit test
> approach and implemented a nightly build script

Automated builds and unit tests are completely different things. You really 
should have both, not one or the other. Same thing with functional tests, 
which test whole game behaviors instead of small units of functionality. 
Ideally you would do all three.


--Noel
Games from Within
http://www.gamesfromwithin.com