Re: [Springframework-developer] Testing

[<tri...@tr...>]

Rod & All,

I think automated integration testing is a great idea.  I would suggest that we
start with a JBoss/HSQL combo - easy install and the db is integrated in the app
server.  Later on we could add a WebLogic/Oracle combo.  Here we might not want
to install the server software for each test, but we could create a WebLogic
domain and an Oracle instance/database using scripts for each test.

Speaking of tests - I have not been able to run the entire test suite lately. 
It keeps failing on the org.springframework.metadata.bcel.AttributeWriterTests.
 I can run this test by itself OK under Eclipse, but not using the 'tests'
target in the build script.  Has anybody else seen this?  Here is the error:
Testsuite: org.springframework.metadata.bcel.AttributeWriterTests
Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 0.771 sec

Testcase: testAttributes took 0.771 sec
	FAILED
Expected one custom class attribute expected:<1> but was:<0>
junit.framework.AssertionFailedError: Expected one custom class attribute
expected:<1> but was:<0>
	at
org.springframework.metadata.bcel.AttributeWriterTests.testAttributes(AttributeWriterTests.java:74)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

Testcase: testAttributes

Not sure what is going on here.

Thomas

> Hi Guys,
> 
> I've been thinking about how to improve our testing strategy further.
> 
> Currently we have excellent--and continually improving--unit test coverage.
> This is essential and, I believe, the most important form of testing for a
> framework such as Spring.
> 
> However it would also be good to have automated integration tests that help
> to verify that code changes won't break user applications. This is
> especially important before releases, but would be really useful possibly as
> part of a nightly build process. (I don't see any point running unit tests
> in a nightly build: developers shouldn't commit without doing that.) I think
> it's important to move beyond ad hoc testing, not just for unit testing.
> 
> I recently had lunch with Patrick Linskey of SolarMetric (Kodo JDO). He was
> telling me about their testing, which is impressive. They have multiple
> machines devoted to continually testing Kodo on different application
> servers and against different databases. They start from a clean install of
> the app server, then install Kodo and their test applications.
> 
> Perhaps we could apply a similar approach as we move towards 1.0 final.
> 
> The basic process would probably involve an additional Ant script, and look
> something like this:
> 
> - be able to do a checkout from a tag or the head
> - compile and build deployment units
> - take a clean application server (perhaps tarred) and deploy the spring
> Jars (and dependencies) from CVS along with the target application from CVS
> (one of our samples, at least to start with). We'd also need a database set
> up similarly--probably HSQL or such an embedded database, including create
> - wait until deployment is complete, unless the target app server can tell
> us
> - run tests against the app. This might involve HTTP requests with something
> like HTTPUnit. Or maybe there are better tools. If there are any EJB
> endpoints in any app we could do RMI against them.
> - clean up so that no temporary files would break the next run
> 
> There are a number of challenges and a lot of work to get to this point, but
> I think it would be worth it. I would love to get Spring a reputation for
> stability and lack of bugs. Tasks would include:
> 
> - Writing build script
> - Deciding which app servers to target? I'd suggest Tomcat and JBoss or
> WebLogic to start off with. Also, where do we keep the app server tarballs?
> We can't check JBoss or WebLogic into CVS.
> - Deciding where do these tests run? How often do they run? Who looks at the
> results? Maybe email would be the best way to go.
> - Writing test scripts for deployed apps, and deciding on the best tool.
> 
> Ideally the scripts would allow adding new app servers or sample
> applications + test scripts, within a consistent framework.
> 
> I'd like some views on this. Does anyone have experience of doing something
> similar?
> 
> Also, a volunteer to take ownersip would be great, assuming we agree it's a
> good idea! This might be a good opportunity for someone who doesn't
> presently have CVS access but wants to be a Spring developer, as it wouldn't
> involve changing existing code. (Not that I want to discourage any present
> developer.)
> 
> Regards,
> Rod
> 
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> SourceForge.net hosts over 70,000 Open Source Projects.
> See the people who have HELPED US provide better services:
> Click here: http://sourceforge.net/supporters.php
> _______________________________________________
> Springframework-developer mailing list
> Spr...@li...
> https://lists.sourceforge.net/lists/listinfo/springframework-developer
>