Re: [Supertuxkart-devel] Test before commit?

[Joerg H. <jo...@lu...>]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi Andrew,

thanks a lot for your email!

Andrew James wrote:
> I've noticed recently that a few of the commits are causing bugs in STK.
>  Shouldn't updates be tested before they are commited and, if there are
> problems a description of the problem should be sent to the mailing list
> (or forum).  
Well, that's basically the case :)  But there are a few things to
consider: we are currently doing some major work on the internals of
STK. Auria doing a fantastic job on refactoring the whole race handling
to make it easier to add new racing modes, and I am (besides working on
the physics which only affects a minor part of the code) adding LAN
support. Both of these changes affect a significant part of the code,
and are likely to overlap. So if we hold off committing our patches we
will most likely occur a huge overhead in having to resolve conflicts.
By frequent commits we reduce this overhead. Additionally, it gives us a
backup as well (I admit that this is not a good reason to use SVN, but
just ask Coz, who once lost his whole widget_manager code because of a
harddrive crash, and he didn't commit an in-between version that would
already have worked, but wasn't perfect).

Auria had concerns about the stability of STK during this phase, and I
made the decision that we can consider current SVN as really unstable,
since it will save us a lot of unnecessary work long term. We both try
to test our patches (and if necessary post a warning):
1) STK should at all times compile (well, I can't test Macs, I can't
   adjust the xcode project file of course, so Auria has to fix things,
   and I on the other hand do the same for linux and windows for her
   commits).
2) It should always be possible to do a single race - this way no other
   development work (e.g. be it LAN support, physics, work done by
   Stephen on the audio) is really affect, all work can go on
   concurrently.

This approach actually has an additional advantage: it is a known fact
that it is difficult to find your own bugs (that's why in professional
SW development you usually have testing and quality assurance
departments). So consider STK's situation: currently there are two
coders, and a lot of non-coders. Two coders is just not enough, so how
can we make better use of the non-coder? Well, by making them our
testing department.

I am not saying that we don't care if our patches contain a bug or that
we don't test, but the amount of testing can be huge. E.g. I recently
committed a one line patch that changed one of the physics parameters.
To really test this patch I would have had to run a race with each kart
(since the karts have different size they are affected differently by
the physics parameters) on each track (since each track might have
slightly different bumps on the road), potentially several times (since
you will test different parts of the track, e.g. left side/right side,
...). This amount of testing is just impossible to do. So by committing
early, we can get additional help from all the non-coding people here.
And get bug reports very early on. And while we try to make sure that
STK always works, there is always the chance of us missing a bug (after
all, no one commits a bug on purpose :) ).

There is a (somewhat sneaky) further reason: people might be more
motivated to submit a small but fix than doing a much more time
consuming development of a new feature. E.g. see the patch we recently
got for some LAN menu fixes, 16-bit support. So again we get more help
this way :)  (though of course that doesn't mean that we introduce bugs
on purpose, or don't do any testing because of this!)

So, I hope you can understand why STK is 'more unstable' atm than it
usually is: not so much because we are lazy or don't care, but because
we make huge changes (be it the amount of code, or the impact of the
code), it's just not possible to test everything before a commit (and we
might just miss a bug as well), and we get more help this way. In an
ideal world I'd love to have unit testing etc. (forcing a full test
before a commit), but that's just not possible with the current number
of coders.

> I'm only saying this because, for example, the explosions
> no longer display properly and to fix this someone will have to work out
> why, and then search through all the code (possibly).  If whoever
That's not that much of an issue. I am quite certain that once we know
that the explosions don't work, one of us will immediately know who did
it and why :)

> committed the update that caused this problem had tested before they
> committed it and told the mailing list, we would only have to look
> through the code from that commit to find/fix the problem {which would
> take less time and give more time to adding new features :) }
Fixing that bug is not that much work compared with the amount of time
needed to do the testing. That's an advantage of frequent small commits:
we are more likely to be informed of bugs early on.

Besides, we might get rid of explosions completely, once we replace the
last rocket type ;)


> It's okay to commit things that mess up other code if it's something
> new/good such as battle mode, but people shouldn't be committing updates
> that break previously working code without telling anyone.
But we are (usually) not aware of some of the bugs. That's why we need
all your help. Submitting a bug report (or telling us here on the list)
is excellent support for us, and help us speeding up the development
process!


> I think STK is doing really well and the updates are really good - I'm
> not blaming or complaining or anything like that, it's just a suggestion
>  - I just don't want anyone taking this message the wrong way :)
Thanks for your concern, and the polite way of telling us. I think even
with some additional bugs being introduced recently, we are making very
good progress, and that our approach is (atm with the given
circumstances) the right one. Hopefully soon we will enter a more stable
phase again, leading to the 0.6 release.

Cheers,
   Joerg


- --
- ----------------------------------------------------------------
Joerg Henrichs
Luding Administration
e-mail: jo...@lu...
URL:    http://luding.org

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFJAUP0LC0mrNKFwF4RAiqUAKC0vFvQpUUAmaZfpdlzm5ceYvJRywCfcRuA
mgrbfW2jz4nnQyW3Rn0dh0w=
=nTon
-----END PGP SIGNATURE-----