Thread: [Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

Brought to you by: newtonm, nihility, patriot1burke, starksm

jboss-dev-forums

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <rya...@jb...> - 2005-04-27 21:55:00

I'll start with the easiest questions first :-)

"tom...@jb..." wrote : I am still not totally clear on the source tags and what exactly they are defining (can these be any free form strings, or do they have to be 'main', 'test', etc.?).  
  | 

These are just the names of the directories under your src directory.  They have no special meaning to the build system.  So if you had a src/foo directory, you could generate compile targets for it like this:

<source id="foo"/>

which would result in a build.foo target.

"tom...@jb..." wrote : Am not sure how to specify a string expression for the files I want excluded.

Yes, that's certainly a limitation.  Is this an actual need at this point?  If so, we could change that attribute to be an Ant patternset refid instead of the actual pattern.

anonymous wrote : So now just need to be able to specify exactly the jars included on the classpath for the test run and how to pass the jvmarg parameter.  I see that runtest uses pathElements for the classpath, but don't know where the values come from.  
  | 

The pathElements tag is a special tag which resolves to the inputs of your source - and the jars in the Classpath of your inputs manfest.  So if you have:

<source id="tests">
  |    <input="jboss-common.jar"/>
  | </source>

The  will include jboss-common.jar in your test's classpath, as well as all of jboss-common.jar's dependencies.  You can see this working by using the special show target:

ant -f jbossbuild.xml show -Dshow=runtests.test

This will let you will see the target that is generated, and what the  actually resolves to.  Also, see the bottom of this page: 
http://www.jboss.org/index.html?module=bb&op=viewtopic&t=58993&postdays=0&postorder=asc&start=10


View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875671#3875671

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875671

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <rya...@jb...> - 2005-04-27 22:15:19

anonymous wrote : 
  | I guess the jvmarg can be added to the runtest taskdef as well, but am not sure how to make it conditional depending of if there is a variable defined for it within the jbossbuild.xml.    
  | 

Usually,  you can add an attribute to an element like this, and then do conditional processing based on the presence of that attribute.  However, in this case we have an optional parameter (your jvm arguments), which the template processing can't handle.  I think we will need to extend the template language to allow you to do something like this:

   <source id="test" tests="*TestCase.java">
  |       <input id="main"/>
  |       <jvmarg value="-Dloader.path=${output.lib.dir}/jboss-remoting-loading-test.jar"/>
  |    </source>

and then in our tasks.xml where we make the JUnit call we could have:

   <junit ...
  |        <pathelements>
  |        <jvmargs/>
  |   </junit>

And the above would expand to include all the jmvarg elements you defined. This would be a generic mechanism for any kind of element.  

Adrian?  Do you see any problem with having dynamic elements like this?    I haven't investigated deeply, but ant's DynamicElement seems like it might fit the bill. 

We need to focus on getting the build complete, but I will add this as an enhancement in JIRA if Adrian agrees.

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875675#3875675

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875675

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <ad...@jb...> - 2005-04-27 22:41:21

I've never liked the idea of passing arguments to tests this way.

For a start, it makes it difficult to run the those tests in IDEs. 

But more importantly, it also doesn't scale when more than one test 
in the same suite wants to do a similar but incompatible thing.

I prefer the mechanism where the test bootstraps its own environment, though
I've never found a good generic solution to this problem.
i.e. being able to run the tests as unit tests inside the home project, and also
bundling the tests for binary distribution into some larger testsuite or QA
environment.

I guess this is one for the custom TestRunner we want to write for other reasons
like being able to simulate a JBoss crash or reboot?

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875678#3875678

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875678

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <ad...@jb...> - 2005-04-27 22:54:35

Objections and ideals over.

To reproduce what already exists, like you suggested, 
you could add specialised parameters to the source definition that can affect the macro generating the junit task.

But this would require some coding. 

Probably the most general solution is to allow any element to  be added
to a macro by making each type implement DynamicElement not just DynamicAttribute?

  | <source id="tests" test="whatever">
  |     <junit-elements> <!-- This is just held inside the definition as a DynamicElement -->
  | ...
  |     </junit-elements>
  | </source>
  | 

Then insert the junit-elements directly into the junit task (which might be empty).


  |       <targetdef target="runtest">
  |          <component/>
  |          <source when="@{test}">
  |             <mkdir dir="@{testDir}"/>
  |             <junit fork="true"
  |                    printSummary="true">
  |                <formatter type="plain"/>
  |                <classpath>
  |                   <pathElements/>
  |                </classpath>
  |                <batchtest todir="@{testDir}">
  |                   <fileset dir="@{sourceDir}" includes="@{test}"/>
  |                </batchtest>
  |                <junit-elements/> <!-- HERE source/junit-elements -->
  |             </junit>
  |          </source>
  |       </targetdef>
  | 

The downside is the loss of validation if you mistype and sub-element, e.g.

  | <source id="tests">
  |    <incude id="main"/> // OOPS! this is now a DynamicElement when it should be a parser exception
  | </source>

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875681#3875681

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875681

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <ad...@jb...> - 2005-04-27 23:15:50

To make this easy to parse you could add an "elements" attribute
identify what you want to replace.


  |          <source when="@{test}" elements="junit-elements">
  |             <mkdir dir="@{testDir}"/>
  |             <junit fork="true"
  |                    printSummary="true">
  |                <formatter type="plain"/>
  |                <classpath>
  |                   <pathElements/>
  |                </classpath>
  |                <batchtest todir="@{testDir}">
  |                   <fileset dir="@{sourceDir}" includes="@{test}"/>
  |                </batchtest>
  |                <junit-elements/> <!-- HERE source/junit-elements -->
  |             </junit>
  |          </source>
  | 

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875683#3875683

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875683

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <tom...@jb...> - 2005-04-28 20:54:42

anonymous wrote : Yes, that's certainly a limitation. Is this an actual need at this point? If so, we could change that attribute to be an Ant patternset refid instead of the actual pattern.

I really only need this to segment out the performance tests, which use the same XXXTestCase naming convention and is under the same org.jboss.test.remoting root package as the rest of the other functional remoting tests. This is because I don't always want to run the performance tests when running functional tests, as they takes a long time.

I would not be opposed to just changing the package structure to be something like org.jboss.test.remoting.functional and org.jboss.test.remoting.performance so I can segment out the different tests. The only problem I see with this, is there is no way that I know of to create another source element for this (because both would live under src/tests/ directory, both would have to be named 'tests'). I could create a src/performance/ directory and put the tests there, but this does not seem appropriate.

anonymous wrote : The pathElements tag is a special tag which resolves to the inputs of your source - and the jars in the Classpath of your inputs manfest. So if you have:

Using the show target for the runtests.test, didn't work for me:

However, using ant -v shows that '..\remoting\output\classes\main' (which I would expect due to having ) and '..\remoting\output\classes\tests' is include in the JUnit classpath when running the tests. So trying to exclude certain test jars won't matter anyways. Am not sure how to find a solution to my particular problem of scoping the classes I want to include in test runs, but maybe a solution to the previous problem (performance vs functional tests) will present a solution to this as well.

anonymous wrote : I've never liked the idea of passing arguments to tests this way.

So given that my test example may not be a good example of why jvmarg is needed, I have another, more valid one. I have been working on SSL transport for remoting. This means that I'll need to be able to declare where the key store is located via a system property (i.e. -Djavax.net.ssl.keyStore=.keystore). This one is outside my control :)

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875842#3875842

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875842

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <ad...@jb...> - 2005-04-28 21:18:43

"tom...@jb..." wrote : 
  | anonymous wrote : I've never liked the idea of passing arguments to tests this way. 
  |   
  | So given that my test example may not be a good example of why jvmarg is needed, I have another, more valid one.  I have been working on SSL transport for remoting.  This means that I'll need to be able to declare where the key store is located via a system property (i.e. -Djavax.net.ssl.keyStore=.keystore).  This one is outside my control :)
  | 
  | 

TestCase.setup() and System.setProperty()
or 
docs/guide/security/jsse/JSSERefGuide.html#CustomizingStores
For mocking something up that can be used in a testcase while still maintaining the
ability for the test to be run with other tests that also want to do SSL with different trust
configuration and/or checking failure conditions like expriing the certificate
or removing one from the trusted chain.

e.g.
http://cvs.sourceforge.net/viewcvs.py/jboss/jboss/src/main/org/jboss/invocation/http/interfaces/AnyhostVerifier.java?rev=1.6&view=markup

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875843#3875843

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875843

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <rya...@jb...> - 2005-04-28 22:45:21

"ad...@jb..." wrote : Probably the most general solution is to allow any element to  be added to a macro by making each type implement DynamicElement not just DynamicAttribute?
  | 
Yes, that's what I was thinking.

I like the junit-elements idea.

anonymous wrote : The downside is the loss of validation if you mistype and sub-element.

Yes, I think is a serious flaw.  I could add a  in tasks.xml which would list the allowed dynamic elements to prevent silent errors such as the one above.


  |    <dynamicElements>
  |       <dynamicElement name="junit-element"/>
  |    </dynamicElements>
  | 

I don't think I should add this at the targetdef/template level because a given targetdef won't necessarily know/care about all dynamic elements.

http://jira.jboss.com/jira/browse/JBBUILD-70

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875847#3875847

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875847

[Jboss-dev-forums] [Design of JBoss Build System] - Re: test target

From: <rya...@jb...> - 2005-04-29 00:06:51

"tom...@jb..." wrote : The only problem I see with this, is there is no way that I know of to create another source element for this (because both would live under src/tests/ directory, both would have to be named 'tests').  

For performance tests, you could add a seperate target which calls Junit directly.  It would need JBBUILD-71 implemented to be able reference the classspath and other things you'll need.  It is something we need in general, but will help with this particular case.

anonymous wrote : Using the show target for the runtests.test, didn't work for me:

Sorry, that should have been 

ant -f jbossbuild.xml show -Dshow=runtest.tests
  |   | 
  |   | It is based on the ID of your source.  target.id
  |   | 
  |   | anonymous wrote :  Am not sure how to find a solution to my particular problem of scoping the classes I want to include in test runs, but maybe a solution to the previous problem (performance vs functional tests) will present a solution to this as well.
  |   | 
  |   | Honestly, all of this is making me wonder how much work we should put into making testing a core part of jbossbuild.   I've already raised  2 JIRA issues for you to be able to run your tests, so this tells me we may need to rethink how we define tests for a project.  I know aop has an extensive testsuite which seems like it would accept no generic solution.  
  |   | 
  |   | Perhaps projects should define their own test targets, at least until we get the build part of jbossbuild complete.
  |   | 

View the original post : http://www.jboss.org/index.html?module=bb&op=viewtopic&p=3875850#3875850

Reply to the post : http://www.jboss.org/index.html?module=bb&op=posting&mode=reply&p=3875850