From: Marius K. <am...@gm...> - 2011-06-10 23:57:25
Attachments:
Parameterized.java
|
Hi, I've just discovered and fell in love with Parameterized tests (with: @RunWith(value = Parameterized.class) ) However in the first two use cases I have I have tests that does not use the parameterization mixed in with test that do use it (in the same class). I don't seem to be the only one missing this feature: http://stackoverflow.com/q/3361239/381083 It appears that TestNG does not suffer from this problem. http://techinsides.blogspot.com/2010/09/dont-tie-yourself-up-parameterized.html I'm not so desperate to switch to TestNG, so I modified the builtin Parameterized class to support this feature (see attached). Just annotate applicable tests as @NonParameterized. Note that this class only works with its on annotations, i.e. check your imports. If this has any chance of getting merged into junit I can turn it into a branch add tests and documentation etc. let me know. -- regards <>< Marius ><> |
From: Marius K. <am...@gm...> - 2011-06-11 02:40:25
Attachments:
Parameterized.java
|
Here is a version with improved validation and error handling. |
From: David S. <da...@sa...> - 2011-06-13 19:25:49
|
Would it be possible to quickly code up a test showing the new runner in action? Thanks, David On Fri, Jun 10, 2011 at 7:56 PM, Marius Kruger <am...@gm...> wrote: > Hi, > I've just discovered and fell in love with Parameterized tests (with: > @RunWith(value = Parameterized.class) ) > > However in the first two use cases I have I have tests that does not > use the parameterization > mixed in with test that do use it (in the same class). > I don't seem to be the only one missing this feature: > http://stackoverflow.com/q/3361239/381083 > > It appears that TestNG does not suffer from this problem. > http://techinsides.blogspot.com/2010/09/dont-tie-yourself-up-parameterized.html > > I'm not so desperate to switch to TestNG, so I modified the builtin > Parameterized class to support this feature (see attached). > Just annotate applicable tests as @NonParameterized. Note that this > class only works with its on annotations, i.e. check your imports. > > If this has any chance of getting merged into junit I can turn it into > a branch add tests and documentation etc. let me know. > > -- > regards > <>< Marius ><> > > ------------------------------------------------------------------------------ > EditLive Enterprise is the world's most technically advanced content > authoring tool. Experience the power of Track Changes, Inline Image > Editing and ensure content is compliant with Accessibility Checking. > http://p.sf.net/sfu/ephox-dev2dev > _______________________________________________ > Junit-devel mailing list > Jun...@li... > https://lists.sourceforge.net/lists/listinfo/junit-devel > > |
From: David S. <da...@sa...> - 2011-06-14 13:59:36
|
Marius, How hard would it be to open this up as a github branch? Each time I want to view one of your attachments, I end up having to go to the web-based list archives and dig it up again. I'm not sure why your attachments aren't coming through. If you're familiar with git, and it wouldn't be too hard to have the discussion there, it would be great. Otherwise, we'll continue here, but more slowly. David On Tue, Jun 14, 2011 at 8:16 AM, Marius Kruger <am...@gm...> wrote: > On 13 June 2011 21:25, David Saff <da...@sa...> wrote: >> Would it be possible to quickly code up a test showing the new runner >> in action? Thanks, > > I've attached a simple maven (and eclipse) project that uses this. > Testing that it works with and without parameterized tests. > > Note that at the moment non-parameterized tests will be runned only > once with the first parameter and only if there is at least one > parameter. > It may be possible to change this to always run but then it will need > 2 constructors and thus I need to modify that validation too. > > -- > <>< Marius ><> > |
From: Marius K. <am...@gm...> - 2011-06-14 23:18:46
|
On 14 June 2011 15:59, David Saff <da...@sa...> wrote: > How hard would it be to open this up as a github branch? I'm a bazaar guy so I first had to figure out this github thing first. I pushed my bazaar branches to github now: https://am...@gi.../amanica/junit-nonparameterized.git https://am...@gi.../amanica/nonparameterized_noparams.git (this only include the maven projects not the eclipse stuff) I don't know how to push different branches into the same git repo (I could not even fork it), so I just made separate repos:( Would it have been ok if I pushed it into launchpad instead? -- <>< Marius ><> |
From: David S. <da...@sa...> - 2011-06-16 14:45:58
|
Marius, Thanks, now I understand where this is going. Genuinely sorry for the git busywork. I still don't know why the attachments aren't getting passed along. As to the feature itself, I'm confused--what's the cost of moving the non-parameterized tests to their own class? You can also use @RunWith(Enclosed.class) to have several inner classes, with different parameterizations, inside one file. David Saff On Tue, Jun 14, 2011 at 7:18 PM, Marius Kruger <am...@gm...> wrote: > On 14 June 2011 15:59, David Saff <da...@sa...> wrote: >> How hard would it be to open this up as a github branch? > > I'm a bazaar guy so I first had to figure out this github thing first. > I pushed my bazaar branches to github now: > https://am...@gi.../amanica/junit-nonparameterized.git > https://am...@gi.../amanica/nonparameterized_noparams.git > (this only include the maven projects not the eclipse stuff) > > I don't know how to push different branches into the same git repo (I > could not even fork it), so I just made separate repos:( > Would it have been ok if I pushed it into launchpad instead? > > -- > <>< Marius ><> > |
From: Marius K. <am...@gm...> - 2011-06-17 20:20:16
|
On 16 June 2011 16:45, David Saff <da...@sa...> wrote: > Marius, > > Thanks, now I understand where this is going. Genuinely sorry for the > git busywork. I still don't know why the attachments aren't getting > passed along. They were probably just a little too big and needed moderation. That's why I mailed it to you directly too. > As to the feature itself, I'm confused--what's the cost of moving the > non-parameterized tests to their own class? You can also use > @RunWith(Enclosed.class) to have several inner classes, with different > parameterizations, inside one file. In short that is not how I'd like to use it, is there any reason why you don't want this feature? I like to group tests that test the same functionality and require the same setup in the same class. So if it makes sense for some tests to run several times and some only once and they all use the same setup, I'd like them in one class. If there are one or two tests that does not need to execute multiple times then I'll rather let them execute multiple times than split them out into their own class because they belong with the others. But if it is easy to mark them to not execute multiple times then I want to do that. Multiple nested classes will be much uglier than a simple annotation, IMHO. The first 3 classes that I wanted to use Parameterized on I also wanted partially @NonParameterized. I'd really rather have this in the core of junit in stead of maintaining my own junit utils elsewhere. -- thanks for listening :-) <>< Marius ><> |
From: David S. <da...@sa...> - 2011-06-20 19:58:03
|
On Fri, Jun 17, 2011 at 4:19 PM, Marius Kruger <am...@gm...> wrote: > On 16 June 2011 16:45, David Saff <da...@sa...> wrote: >> Marius, >> >> Thanks, now I understand where this is going. Genuinely sorry for the >> git busywork. I still don't know why the attachments aren't getting >> passed along. > > They were probably just a little too big and needed moderation. That's > why I mailed it to you directly too. > >> As to the feature itself, I'm confused--what's the cost of moving the >> non-parameterized tests to their own class? You can also use >> @RunWith(Enclosed.class) to have several inner classes, with different >> parameterizations, inside one file. > > In short that is not how I'd like to use it, is there any reason why > you don't want this feature? Parameterized.class currently allows one parameterization per test class. This patch enables two parameterizations, one of which must be the special case of "no parameters". A runner like JUnitParams (http://code.google.com/p/junitparams/) enables an arbitrary number of parameterizations, including "no parameters". I'd like to consider how to handle the arbitrary case, like JUnitParams, and would rather not create a special case for "no parameters" that users would need to migrate to the new framework when it's ready. Does that make sense? David Saff > I like to group tests that test the same functionality and require the > same setup in the same class. > So if it makes sense for some tests to run several times and some only once > and they all use the same setup, I'd like them in one class. > If there are one or two tests that does not need to execute multiple > times then I'll rather let them > execute multiple times than split them out into their own class > because they belong with the others. > But if it is easy to mark them to not execute multiple times then I > want to do that. > Multiple nested classes will be much uglier than a simple annotation, IMHO. > The first 3 classes that I wanted to use Parameterized on I also > wanted partially @NonParameterized. > I'd really rather have this in the core of junit in stead of > maintaining my own junit utils elsewhere. > > -- > thanks for listening :-) > <>< Marius ><> > |
From: Marius K. <am...@gm...> - 2011-06-25 19:19:37
|
On 20 June 2011 21:57, David Saff <da...@sa...> wrote: ... > Parameterized.class currently allows one parameterization per test > class. This patch enables two parameterizations, one of which must be > the special case of "no parameters". A runner like JUnitParams > (http://code.google.com/p/junitparams/) enables an arbitrary number of > parameterizations, including "no parameters". I'd like to consider > how to handle the arbitrary case, like JUnitParams, and would rather > not create a special case for "no parameters" that users would need to > migrate to the new framework when it's ready. > > Does that make sense? Yes now that I looked at JUnitParams. It is awesome - much more than I needed so far. I'll be using that then in the mean time. Thanks a lot for pointing me this way. -- sorry for the late reply, it was a busy week. <>< Marius ><> |
From: Marius K. <am...@gm...> - 2011-06-25 20:11:56
|
On 25 June 2011 21:19, Marius Kruger <am...@gm...> wrote: > On 20 June 2011 21:57, David Saff <da...@sa...> wrote: > ... >> Parameterized.class currently allows one parameterization per test >> class. This patch enables two parameterizations, one of which must be >> the special case of "no parameters". A runner like JUnitParams >> (http://code.google.com/p/junitparams/) enables an arbitrary number of >> parameterizations, including "no parameters". I'd like to consider >> how to handle the arbitrary case, like JUnitParams, and would rather >> not create a special case for "no parameters" that users would need to >> migrate to the new framework when it's ready. >> >> Does that make sense? > > Yes now that I looked at JUnitParams. It is awesome - much more than I > needed so far. I now found a case that was handled much nicer with my patched version: I just want to setUp(@Before) the tests according to a parameter. This is possible with the standard JUnit Parameterized runner and my version because we get the value in the constructor. eg.: A)== @RunWith(JUnitParamsRunner.class) public class MyJUnitParamsTest { protected AbstractHomeLayout homeLayout; public void setupHomeLayout(int layoutVersion) { homeLayout = init(layoutVersion); } protected Object[] getLayoutVersions() { return new Object[][] { { 0 }, { 1 } }; } @Test @Parameters(method = "getLayoutVersions") public void testParameterized(int layoutVersion) throws IOException { setupHomeLayout(layoutVersion); //test stuff } @Test public void testNonParameterized() throws IOException { //test stuff } // 10 more tests like these } == B)== @RunWith(value = Parameterized.class) public class MyParameterizedTest { int layoutVersion public HomeLayoutTest(int layoutVersion) { this.layoutVersion = layoutVersion; } protected AbstractHomeLayout homeLayout; @Before public void setupHomeLayout() { homeLayout = init(layoutVersion); } @Parameters public static Collection getLayoutVersions() { Object[][] data = new Object[][] { { 0 }, { 1 } }; return Arrays.asList(data); } @Test public void testParameterized() throws IOException { setupHomeLayout(layoutVersion); //test stuff } @Test @NonParameterized public void testNonParameterized() throws IOException { //test stuff } // 10 more tests like these } == So with A you have to modify three lines for every single parameterized test: @Parameters(method = "getLayoutVersions") public void testParameterized(int layoutVersion) throws IOException { setupHomeLayout(layoutVersion); but in B you just annotate the non-parameterized tests... -- <>< Marius ><> |
From: David S. <da...@sa...> - 2011-06-27 15:40:17
|
Good point. Here's how I'm thinking about moving forward: we currently have @Rule, which provides a way to override the rules for _running_ a method, for _every @Test_ method in the class. I'm considering a similar construct, which will allow overriding: 1) Which methods trigger tests 2) How many tests per method 3) Validation of test methods 4) Test-instance creation 5) Test-method invoking Given this, it should be possible to enable something very close to @NonParameterized as a drop-in Rule-like object, rather than as a core modification to the overall Parameterized runner. One issue: I can't think of a name for this new construct. Suggestions welcome. David Saff On Sat, Jun 25, 2011 at 4:11 PM, Marius Kruger <am...@gm...> wrote: > On 25 June 2011 21:19, Marius Kruger <am...@gm...> wrote: >> On 20 June 2011 21:57, David Saff <da...@sa...> wrote: >> ... >>> Parameterized.class currently allows one parameterization per test >>> class. This patch enables two parameterizations, one of which must be >>> the special case of "no parameters". A runner like JUnitParams >>> (http://code.google.com/p/junitparams/) enables an arbitrary number of >>> parameterizations, including "no parameters". I'd like to consider >>> how to handle the arbitrary case, like JUnitParams, and would rather >>> not create a special case for "no parameters" that users would need to >>> migrate to the new framework when it's ready. >>> >>> Does that make sense? >> >> Yes now that I looked at JUnitParams. It is awesome - much more than I >> needed so far. > > I now found a case that was handled much nicer with my patched version: > I just want to setUp(@Before) the tests according to a parameter. This > is possible with the standard JUnit Parameterized runner and my > version because we get the value in the constructor. eg.: > > A)== > @RunWith(JUnitParamsRunner.class) > public class MyJUnitParamsTest { > > protected AbstractHomeLayout homeLayout; > > public void setupHomeLayout(int layoutVersion) { > homeLayout = init(layoutVersion); > } > > protected Object[] getLayoutVersions() { > return new Object[][] { { 0 }, { 1 } }; > } > > @Test > @Parameters(method = "getLayoutVersions") > public void testParameterized(int layoutVersion) throws IOException { > setupHomeLayout(layoutVersion); > //test stuff > } > > @Test > public void testNonParameterized() throws IOException { > //test stuff > } > > // 10 more tests like these > } > == > > B)== > @RunWith(value = Parameterized.class) > public class MyParameterizedTest { > > int layoutVersion > public HomeLayoutTest(int layoutVersion) { > this.layoutVersion = layoutVersion; > } > > protected AbstractHomeLayout homeLayout; > > @Before > public void setupHomeLayout() { > homeLayout = init(layoutVersion); > } > > @Parameters > public static Collection getLayoutVersions() { > Object[][] data = new Object[][] { { 0 }, { 1 } }; > return Arrays.asList(data); > } > > @Test > public void testParameterized() throws IOException { > setupHomeLayout(layoutVersion); > //test stuff > } > > @Test > @NonParameterized > public void testNonParameterized() throws IOException { > //test stuff > } > > // 10 more tests like these > } > == > > > So with A you have to modify three lines for every single parameterized test: > @Parameters(method = "getLayoutVersions") > public void testParameterized(int layoutVersion) throws IOException { > setupHomeLayout(layoutVersion); > but in B you just annotate the non-parameterized tests... > > > > -- > <>< Marius ><> > |
From: Marius K. <am...@gm...> - 2011-07-14 19:29:57
|
On 27 June 2011 17:40, David Saff <da...@sa...> wrote: > Good point. > > Here's how I'm thinking about moving forward: we currently have @Rule, > which provides a way to override the rules for _running_ a method, for > _every @Test_ method in the class. I'm considering a similar > construct, which will allow overriding: > > 1) Which methods trigger tests > 2) How many tests per method > 3) Validation of test methods > 4) Test-instance creation > 5) Test-method invoking > > Given this, it should be possible to enable something very close to > @NonParameterized as a drop-in Rule-like object, rather than as a core > modification to the overall Parameterized runner. > It's a bit abstract for me, I can not imagine/picture how my use case will look like with this. With the above features it sounds like it will be quite useful. If you like you can make a sudo example to show how you imagine it to be used. > One issue: I can't think of a name for this new construct. Suggestions > welcome. > @TestController @TestOrchestrator @TestManager One problem that I noticed with @RunWith(JUnitParamsRunner.class) tests is that IDE support lags a bit or maybe its a bug, but from eclipse I can't re-run a single method or one of it's permutations or jump to its location by double-clicking on it in the junit panel. It works fine with @RunWith(value = Parameterized.class) which I'm glad to see. Before I knew about this, I made my own suite implementation to sort of do this, and with some tweaking I was able to get eclipse to be able to re-run those too. -- <>< Marius ><> |
From: David S. <da...@sa...> - 2011-07-19 18:56:17
|
I've gotten a bit too buried in day work to fully answer this. Two points: 1) I've opened an issue on github, so this doesn't get lost: https://github.com/KentBeck/junit/issues/264 2) Also, if you'd like to contribute a patched version of Parameterized to the junit.contrib project, I'd be happy to work with you on that. David Saff On Thu, Jul 14, 2011 at 3:29 PM, Marius Kruger <am...@gm...> wrote: > On 27 June 2011 17:40, David Saff <da...@sa...> wrote: >> >> Good point. >> >> Here's how I'm thinking about moving forward: we currently have @Rule, >> which provides a way to override the rules for _running_ a method, for >> _every @Test_ method in the class. I'm considering a similar >> construct, which will allow overriding: >> >> 1) Which methods trigger tests >> 2) How many tests per method >> 3) Validation of test methods >> 4) Test-instance creation >> 5) Test-method invoking >> >> Given this, it should be possible to enable something very close to >> @NonParameterized as a drop-in Rule-like object, rather than as a core >> modification to the overall Parameterized runner. > > It's a bit abstract for me, I can not imagine/picture how my use case will > look like with this. With the above features it sounds like it will be quite > useful. > If you like you can make a sudo example to show how you imagine it to be > used. > >> >> One issue: I can't think of a name for this new construct. Suggestions >> welcome. > > @TestController > @TestOrchestrator > @TestManager > > One problem that I noticed with @RunWith(JUnitParamsRunner.class) tests is > that > IDE support lags a bit or maybe its a bug, but from eclipse I can't re-run a > single method or one of it's permutations or jump to its location by > double-clicking on it in the junit panel. It works fine with @RunWith(value > = Parameterized.class) which I'm glad to see. > Before I knew about this, I made my own suite implementation to sort of do > this, > and with some tweaking I was able to get eclipse to be able to re-run those > too. > -- > <>< Marius ><> > > |
From: Marius K. <am...@gm...> - 2011-07-19 19:21:47
|
On 19 July 2011 20:56, David Saff <da...@sa...> wrote: > I've gotten a bit too buried in day work to fully answer this. yes I'm also busy, I understand. > Two points: > > 1) I've opened an issue on github, so this doesn't get lost: > https://github.com/KentBeck/junit/issues/264 thanks > 2) Also, if you'd like to contribute a patched version of > Parameterized to the junit.contrib project, I'd be happy to work with > you on that. Sure, if you think it will be too long to wait for the other new features. Should I submit this somewhere else then? When you have time let me know what I need to fix to get it in, or point me to instructions somewhere. -- <>< Marius ><> |
From: David S. <da...@sa...> - 2011-07-20 02:16:45
|
On Tue, Jul 19, 2011 at 3:21 PM, Marius Kruger <am...@gm...> wrote: > On 19 July 2011 20:56, David Saff <da...@sa...> wrote: >> I've gotten a bit too buried in day work to fully answer this. > > yes I'm also busy, I understand. > >> Two points: >> >> 1) I've opened an issue on github, so this doesn't get lost: >> https://github.com/KentBeck/junit/issues/264 > > thanks > >> 2) Also, if you'd like to contribute a patched version of >> Parameterized to the junit.contrib project, I'd be happy to work with >> you on that. > > Sure, if you think it will be too long to wait for the other new features. > Should I submit this somewhere else then? > When you have time let me know what I need to fix to get it in, or > point me to instructions somewhere. I need to write up better instructions (since no one has actually issued a junit.contrib pull request yet), but the idea is: 1) Fork http://github.com/dsaff/junit.contrib 2) Create a folder with your project name (nonparameterized?) 3) Put your work there, in a standard Maven source layout. 4) Put an Apache 2.0 license agreement in as a LICENSE file 5) Issue a pull request. I'd be happy to work with you to resolve any confusion. Thanks, David > > -- > <>< Marius ><> > |