From: Chris C. <ca...@al...> - 2005-11-13 21:35:48
|
On Saturday 12 Nov 2005 23:20, Silvan wrote: > On Saturday 12 November 2005 04:43 pm, Chris Cannam wrote: > > We never test the things we write ourselves enough.) > > You two don't. I test my stuff. Guillaume probably does too, although I do sometimes feel he's happy if it compiles. The problem is (with all developers -- I bet this would include you if you did a few more of the bigger interactive features like your selection filter dialog) that the things you test turn out not to be the things people actually do. That's why it's important that projects have dedicated testers who are not developers, and that's somewhere that open source development almost always fails. Not that I'm arguing about who tests more, you understand. > Right, there's nothing wrong with the theory. The practice is that > it has generally proven that that is, in fact, what we do, even > though it has wound up going out chock full of bugs that weren't > discovered until afterwards, because nobody actually tested the damn > thing. This was most especially true for the pre-1.0 releases. Yup. Some of the 0.9.whatever RCs actually seemed to get quite a bit of testing despite being short-lived, and I think that fooled us for a while. > On that last point, Rich had the right idea years ago by trying to > put together formal testing procedures for people to run through. > That ball got dropped. Because it's difficult and hard work to do. Relatedly, I do now have a regression test programme for playback and recording, and there won't be another release until we've made sure all the boxes are ticked (or explained away!). You can find it in a spreadsheet at head/docs/test/play-record-regression.gnumeric. Feel free to fill in any fields with observations from current CVS and commit if you like; I'll run a concerted plan in a few (Chris-)days. Chris |