Hi Mark and other Webware developers-
Here are my first thoughts on organizing automated test cases. To make=20=
it easy for people to contribute automated tests, I think we need:
=95 Test discussion - A way to get feedback when you are =
thinking about=20
writing a test. Although a test might be only a few lines of code,=20
it's affect on Webware is significant. Writing a test implies that all=20=
of Webware must now comply with this test and the "proper" result may=20
not be obvious for some cases. So people who are new might need help=20
to know what to test for, especially taking into account other=20
platforms. Simply discussing on the development list is probably=20
sufficient, but the issue should be mentioned in the documentation.
=95 Easy to run tests - When writing a test, I want to know that =
I am=20
not breaking things on other platforms that I cannot test on. So there=20=
should be a way to get feedback quickly after checking in a new test. =20=
Conversely, all the people who are developing with Webware are=20
reluctant to download new versions and run tests if they break things=20
and disrupt their own development. So I think Webware needs to be=20
arranged so you can have two separate versions on your machine. One=20
that you develop with, and a second that you can download often just to=20=
run the tests. This implies that we will not use disutils to install=20
Webware into the Python standard places, and that we have a way to=20
control the python path to determine which version of Webware is being=20=
used.
=95 Testing Configuration - With an automated testing system, we =
need a=20
way to specify what tests run on my computer, e.g. I am on a Mac so=20
don't run COMKit tests, or I have only PostgreSQL so don't run MySQL or=20=
MSSQL tests, or I have Apache with mod_kit or I use the cgi adapter.
=95 Testing daemon - Eventually, we might setup a "continuous =
testing=20
daemon" that people can run on their machine which will check for new=20
versions in CVS, download them to a separate place, run the tests, and=20=
email any failures to a mailing list. I've heard of one for Ant,=20
perhaps we could use that, or write a simple one in Python. That would=20=
allow developers to get feedback on new tests from all platforms within=20=
minutes of committing a new test.
So to accomplish this we need:
=95 Documentation describing the need to discuss a test on the=20=
webware-devel list, and describing how to add a test, run the tests,=20
etc.
=95=A0alltests.py in each kit - We need a standard module that =
contains=20
all the automated tests for that component. I have always called it=20
alltests.py. It has a suite() function that returns the suite of=20
unittest.TestCases for that component.
=95 Test Launcher - A script that will set the python path, =
choose the=20
local machine's configuration, and then run all or just some tests.
How does that sound?
-winston
On Nov 29, 2004, at 6:06 PM, Mark Phillips wrote:
> On Nov 29, 2004, at 12:13 PM, Winston Wolff wrote:
>
>> I have used WebUnit very successfully for high level tests. =20
>> (http://mechanicalcat.net/tech/webunit)
>>
>> I haven't been able to figure out how to unit-test pages since=20
>> anything that extends Page requires the AppServer to be running. =20
>> Perhaps part of the dependency can be eliminated with a mock=20
>> HTTPTransaction, but I'm not sure.
>>
>> I think it would be good to organize a working group of people=20
>> willing to work on tests. I know several people including myself=20
>> have volunteered to work on the release, but not to lead it=20
>> themselves citing inexperience. However I have not been "called to=20=
>> service" yet. A difficulty with volunteering to do a release is to=20=
>> know if I've made a mistake in the release process. Hence the need=20=
>> for tests. Once we have a group together, we can pick a few=20
>> priorities in the tests, and settle anything needed to let people get=20=
>> to work individually, such as how our tests will fit into the larger=20=
>> set of test cases.
>>
>> -winston
>>
>
> Thanks for the reply, Winston. I will add your commentary and take a=20=
> look at WebUnit when I get the chance.
>
> I am open to discussing how a "release team" might be organized and=20
> operate. Feel free to send me a note off-list if you wish.
>
> Mark Phillips
_________________________________________
winston wolff - (646) 827-2242 - http://www.stratolab.com - learning by=20
creating
|