Menu
▾
▴
dacapobench-researchers
|
From: Andreas S. <se...@st...> - 2012-08-30 10:59:03
Attachments:
signature.asc
|
Hi all, I recently started refactoring the DaCapo harness a bit. The guiding principle is thereby to further decouple the harness and the actual benchmarks -- both in terms of code and build infrastructure. The DaCapo suite already took a step in the right direction by having self-describing benchmarks (via .cnf files), but didn't go all the way towards freely combineable benchmark and harness JARs. To simplify the build infrastructure I propose that each benchmark's Ant script should produce its own JAR (complete with cnf, dat, and jar directories). Only in a final step these benchmark JARs would then be merged with the JAR produced for the harness to produce a single, standalone dacapo-12.x.jar, which is what most end-users will use. To make the most out of having individual benchmark JARs and to facilitate testing them, the harness should be made flexible enough to pick up any number of benchmarks from the classpath. In other words, java -cp avrora-benchmark.jar:batik-benchmark.jar:harness.jar Harness should work, with the harness picking up both (self-describing) benchmarks from the classpath. (Currently this doesn't quite work, only the first benchmark is picked up.) Thoughts? (Hopefully, there will soon be a branch at <https://bitbucket.org/sewe/dacapobench> demonstrating this, but I will be travelling this weekend, so don't hold your breath.) Best wishes, Andreas |
|
From: Steve B. <Ste...@an...> - 2012-08-30 12:57:53
|
Hi Andreas, This sounds good, but one problem that we had to deal with in the past was duplication, and IIRC, some of what you're describing was designed specifically to avoid duplication of common elements. tradebeans and tradesoap are an obvious example --- they share almost everything. I'm not sure how what you're proposing deals with that (or perhaps you're arguing it's not worth worrying about).... --Steve On 30/08/2012, at 8:58 PM, Andreas Sewe wrote: > Hi all, > > I recently started refactoring the DaCapo harness a bit. The guiding > principle is thereby to further decouple the harness and the actual > benchmarks -- both in terms of code and build infrastructure. > > The DaCapo suite already took a step in the right direction by having > self-describing benchmarks (via .cnf files), but didn't go all the way > towards freely combineable benchmark and harness JARs. > > To simplify the build infrastructure I propose that each benchmark's Ant > script should produce its own JAR (complete with cnf, dat, and jar > directories). Only in a final step these benchmark JARs would then be > merged with the JAR produced for the harness to produce a single, > standalone dacapo-12.x.jar, which is what most end-users will use. > > To make the most out of having individual benchmark JARs and to > facilitate testing them, the harness should be made flexible enough to > pick up any number of benchmarks from the classpath. In other words, > > java -cp avrora-benchmark.jar:batik-benchmark.jar:harness.jar Harness > > should work, with the harness picking up both (self-describing) > benchmarks from the classpath. (Currently this doesn't quite work, only > the first benchmark is picked up.) > > Thoughts? > > (Hopefully, there will soon be a branch at > <https://bitbucket.org/sewe/dacapobench> demonstrating this, but I will > be travelling this weekend, so don't hold your breath.) > > Best wishes, > > Andreas > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/_______________________________________________ > dacapobench-researchers mailing list > dac...@li... > https://lists.sourceforge.net/lists/listinfo/dacapobench-researchers |
|
From: Eliot M. <mo...@cs...> - 2012-08-30 13:08:30
|
Here's a wondering: Would it be that much more complicated (or maybe this is what is currently done) to have the notions of: harness benchmark application library where the libraries are things the benchmarks can choose to use (i.e., to have on their class path)? If a library is used by only one benchmark, then it would not matter, though it might be uniform to put a library in the library group rather then the benchmark anyway. But if a library is used by two or more benchmarks, then you would save space. If two benchmarks want different versions of the same library, they could still include the library in the benchmark ... Regards -- Eliot |
|
From: Andreas S. <se...@st...> - 2012-08-30 13:30:47
Attachments:
signature.asc
|
Hi Eliot,
thanks for your comments.
> Here's a wondering:
>
> Would it be that much more complicated (or maybe this is what is
> currently done) to have the notions of:
>
> harness benchmark application library
Yes, that's pretty much the three logical entities we currently have.
> where the libraries are things the benchmarks can choose to use
> (i.e., to have on their class path)? If a library is used by only
> one benchmark, then it would not matter, though it might be uniform
> to put a library in the library group rather then the benchmark
> anyway. But if a library is used by two or more benchmarks, then you
> would save space.
>
> If two benchmarks want different versions of the same library, they
> could still include the library in the benchmark ...
What I propose is to build one JAR for the harness and one JAR per
benchmark. The latter JARs would also contain the application libraries
they use. Here's an example:
harness.jar
*.class
a.jar
HarnessA.class
cnf/a.cnf
dat/a.zip
jar/application-library-1.0.jar
jar/another-library-1.0.jar
jar/third-library-1.0.jar
b.jar
HarnessB.class
cnf/b.cnf
dat/b.zip
jar/application-library-1.0.jar
jar/another-library-1.1.jar
When you merge these three JARs into one big, standalone JAR you
arrive at the following:
dacapo.jar
*.class
HarnessA.class
HarnessB.class
cnf/a.cnf
b.cnf
dat/a.zip
b.zip
jar/application-library-1.0.jar
jar/another-library-1.0.jar
jar/another-library-1.1.jar
jar/third-library-1.0.jar
So, while a.jar and b.jar taken together larger than needed (as they
both contain application-library-1.0.jar, the resulting merged JAR will
only contain a single such JAR. Different benchmarks using different
versions of an application library are also not an issue, as long as the
library's version is part of its name (another-library-1.0.jar,
another-library-1.1.jar). This only depends on the same naming
convention being following across benchmarks.
BTW, this is a setup I already successfully use for my Scala Benchmark
Suite (where Maven enforces the naming convention for me).
Best wishes,
Andreas
|
|
From: Eliot M. <mo...@cs...> - 2012-08-30 13:40:54
|
On 8/30/2012 9:30 AM, Andreas Sewe wrote: > What I propose is to build one JAR for the harness and one JAR per > benchmark. The latter JARs would also contain the application libraries > they use. Here's an example: > > harness.jar > *.class > > a.jar > HarnessA.class > cnf/a.cnf > dat/a.zip > jar/application-library-1.0.jar > jar/another-library-1.0.jar > jar/third-library-1.0.jar > > b.jar > HarnessB.class > cnf/b.cnf > dat/b.zip > jar/application-library-1.0.jar > jar/another-library-1.1.jar > > When you merge these three JARs into one big, standalone JAR you > arrive at the following: > > dacapo.jar > *.class > HarnessA.class > HarnessB.class > cnf/a.cnf > b.cnf > dat/a.zip > b.zip > jar/application-library-1.0.jar > jar/another-library-1.0.jar > jar/another-library-1.1.jar > jar/third-library-1.0.jar > > So, while a.jar and b.jar taken together larger than needed (as they > both contain application-library-1.0.jar, the resulting merged JAR will > only contain a single such JAR. Different benchmarks using different > versions of an application library are also not an issue, as long as the > library's version is part of its name (another-library-1.0.jar, > another-library-1.1.jar). This only depends on the same naming > convention being following across benchmarks. > > BTW, this is a setup I already successfully use for my Scala Benchmark > Suite (where Maven enforces the naming convention for me). Ok, I understand the proposal much better now! Seems a sensible move -- Eliot |
|
From: Andreas S. <se...@st...> - 2012-08-30 13:41:34
Attachments:
signature.asc
|
Hi Steve, > This sounds good, but one problem that we had to deal with in the > past was duplication, and IIRC, some of what you're describing was > designed specifically to avoid duplication of common elements. > tradebeans and tradesoap are an obvious example --- they share almost > everything. I'm not sure how what you're proposing deals with that > (or perhaps you're arguing it's not worth worrying about).... this depends on whether you are talking about duplicated dependencies in the final dacapo.jar or duplication of build logic. The former can be avoided or rather removed upon merging the benchmark JARs into the final dacapo.jar (see my mail to Eliot). The latter may be done by refactoring out common bits in the Ant scripts (if desired). But frankly, I haven't had a very deep look at trade* yet; these two benchmarks don't even build properly on my machine. :-( Best wishes, Andreas |
|
From: Steve B. <Ste...@an...> - 2012-08-30 21:00:56
|
On 30/08/2012, at 11:40 PM, Andreas Sewe wrote: > this depends on whether you are talking about duplicated dependencies in > the final dacapo.jar or duplication of build logic. Both are important, but I was meaning the content of the jars. > The former can be avoided or rather removed upon merging the benchmark > JARs into the final dacapo.jar (see my mail to Eliot). OK. That sounds fine. > The latter may be > done by refactoring out common bits in the Ant scripts (if desired). > > But frankly, I haven't had a very deep look at trade* yet; these two > benchmarks don't even build properly on my machine. :-( They are huge and they only differ in the harness. --Steve |
|
From: Andreas S. <se...@st...> - 2012-09-06 14:54:58
Attachments:
signature.asc
|
Hi again, > I recently started refactoring the DaCapo harness a bit. The guiding > principle is thereby to further decouple the harness and the actual > benchmarks -- both in terms of code and build infrastructure. *snip* > To make the most out of having individual benchmark JARs and to > facilitate testing them, the harness should be made flexible enough to > pick up any number of benchmarks from the classpath. In other words, > > java -cp avrora-benchmark.jar:batik-benchmark.jar:harness.jar Harness > > should work, with the harness picking up both (self-describing) > benchmarks from the classpath. (Currently this doesn't quite work, only > the first benchmark is picked up.) My current version of the harness <https://bitbucket.org/sewe/dacapobench/changesets/tip/..bookmark%28%22combinable-jars%22%29> successfully picks up cnf/*.cnf files from all classpath elements rather than just the first one. To give it a try, best grab a pre-build benchmark snapshot from <http://repo.scalabench.org/snapshots/org/scalabench/benchmarks/> and put it on the classpath as well. Now, the next step is getting Ant to build such benchmark JARs for the DaCapo benchmarks as well. I'll keep you informed about any progress I make. Best wishes, Andreas |
Thanks for helping keep SourceForge clean.
X