The DaCapo Benchmark Suite

Hi Eric,

> I was just told by Feng Chen from UIUC that apparently bloat reads as
> an input one of its own class files.

I'd have to go through the sources carefully to be sure, but yes, I  
believe bloat performs transformations over bloat (AFAIK, the  
underlying benchmark predates DaCapo and was known as bloat-bloat).

> Now if people like us happen to
> do static optimizations on benchmarks, this will obviously transform
> class files and hence transform the input to bloat.

It sounds like you're not only doing static analysis, but furthermore  
you're doing it at the bytecode level, and further, you're doing it  
by re-writing the said class files and overwriting the originals (as  
opposed to class-loading time etc). Correct?

>  We have seen
> really crazy slowdowns on bloat and this might actually explain those
> problems...

I agree that this could be the source of your problem.

> Could you please confirm whether this is really currently the case

I won't be able to do that immediately unfortunately as I'm swamped  
with reviewing duties at this moment.  It should not be too hard to  
check, though, if you have the sources at hand.

> and
> if so, would it be possible to change the benchmark so that it reads
> some other class file?

This should be easy enough to do.  However, we have made it clear  
that we will NOT change the workload between major releases of the  
suite.  The obvious choice in this case is to have the said classes  
in a separate jar, or similar.  The scenario of overwriting source  
class files only affects those who use the source or xdeps releases.   
It should not be too hard to engineer a work around for you.

> (this class file should optimally not be part
> of any benchmark or if it is, it should only be a library class at
> least.

Not sure what you're getting at here.  I'm not sure why it  
particularly matters what the origins of the input classes are.  The  
only issue, as far as I can tell, is trying to avoid users  
inadvertently modifying the input set.  This is an engineering  
question and somewhat independent of the origins or nature of said  
classes.

--Steve

> It sounds like you're not only doing static analysis, but furthermore
> you're doing it at the bytecode level, and further, you're doing it
> by re-writing the said class files and overwriting the originals (as
> opposed to class-loading time etc). Correct?

Yes, all correct. We instrument the DaCapo benchmarks on a binary
level and then measure the instrumentation overhead.

> > Could you please confirm whether this is really currently the case
>
> I won't be able to do that immediately unfortunately as I'm swamped
> with reviewing duties at this moment.  It should not be too hard to
> check, though, if you have the sources at hand.

That's ok, I can investigate by myself. I just thought you might
already be aware of this.

> > and
> > if so, would it be possible to change the benchmark so that it reads
> > some other class file?
>
> This should be easy enough to do.  However, we have made it clear
> that we will NOT change the workload between major releases of the
> suite.  The obvious choice in this case is to have the said classes
> in a separate jar, or similar.  The scenario of overwriting source
> class files only affects those who use the source or xdeps releases.
> It should not be too hard to engineer a work around for you.

Agreed. Yet, I would propose to have that changed for the next major
release then.

> > (this class file should optimally not be part
> > of any benchmark or if it is, it should only be a library class at
> > least.
>
> Not sure what you're getting at here.  I'm not sure why it
> particularly matters what the origins of the input classes are.  The
> only issue, as far as I can tell, is trying to avoid users
> inadvertently modifying the input set.  This is an engineering
> question and somewhat independent of the origins or nature of said
> classes.

Yes, I agree. One basically has to make sure that the input does not
change. As long as bloat is processing itself, this may sometimes be
hard to achieve. So there should at least be a way of having it
process another (unmodified) copy of itself. Maybe it suffices to
change some build/configuration scripts. I'll look into it and let you
know.

Eric

--=20
Eric Bodden
Sable Research Group
McGill University, Montr=E9al, Canada

On 21/04/2007, at 12:41 PM, Eric Bodden wrote:
>> This should be easy enough to do.  However, we have made it clear
>> that we will NOT change the workload between major releases of the
>> suite.  The obvious choice in this case is to have the said classes
>> in a separate jar, or similar.  The scenario of overwriting source
>> class files only affects those who use the source or xdeps releases.
>> It should not be too hard to engineer a work around for you.
>
> Agreed. Yet, I would propose to have that changed for the next major
> release then.

To be clear,  changing the mechanics of the harness to ensure you get  
an unaffected version of the classes should be fine for a maintenance  
release. I was just saying we can't move to using a different set of  
classes (different workload) during a release cycle.  But I don't  
think that's what we need, so I think we're fine.

Let's see if we can get the harness fixed to work around the problem  
and get the fix into a maintenance release in the next few weeks.

Obviously if you or anyone else is aware of any other minor problem,  
we should attack them and fold them into the MR too.

Cheers,

--Steve

On 20/04/07, Steve Blackburn <Steve.Blackburn@...> wrote:
> Let's see if we can get the harness fixed to work around the problem
> and get the fix into a maintenance release in the next few weeks.

Sounds perfect to me. I'll let you know as soon as I can give you more
information.

Eric

--=20
Eric Bodden
Sable Research Group
McGill University, Montr=E9al, Canada

My suggestion would be to have a separate copy of the relevant files for
use as input to bloat. The fact that they happen to be bloat itself would
then not matter. This might mean that the path bloat uses to find its input
would be different from the path used for regular loading, etc.

-- Eliot

Hi all.

I just tried to implement this. However, this raises the following problem:

In order to tell bloat where to look for those class files, I need to
set it's "-classpath" command line parameter. However, as soon as I do
so, it cannot find java.lang.Object any more because rt.jar is not
contained in this classpath. So I need some way to tell the harness to
append rt.jar to the classpath or something...

Any thoughts?

Eric

On 21/04/07, Eliot Moss <moss@...> wrote:
> My suggestion would be to have a separate copy of the relevant files for
> use as input to bloat. The fact that they happen to be bloat itself would
> then not matter. This might mean that the path bloat uses to find its inp=
ut
> would be different from the path used for regular loading, etc.
>
> -- Eliot
>
> -------------------------------------------------------------------------
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> dacapobench-researchers mailing list
> dacapobench-researchers@...
> https://lists.sourceforge.net/lists/listinfo/dacapobench-researchers
>


--=20
Eric Bodden
Sable Research Group
McGill University, Montr=E9al, Canada

Hi Eric,

Perhaps just systematically rename all the bloat classes with a =20
DaCapo prefix.  A tool like jreplace should make that trivial. The =20
original ones get executed; the renamed ones become input.

Cheers,

--Steve

On 25/04/2007, at 3:19 AM, Eric Bodden wrote:

> Hi all.
>
> I just tried to implement this. However, this raises the following =20
> problem:
>
> In order to tell bloat where to look for those class files, I need to
> set it's "-classpath" command line parameter. However, as soon as I do
> so, it cannot find java.lang.Object any more because rt.jar is not
> contained in this classpath. So I need some way to tell the harness to
> append rt.jar to the classpath or something...
>
> Any thoughts?
>
> Eric
>
> On 21/04/07, Eliot Moss <moss@...> wrote:
>> My suggestion would be to have a separate copy of the relevant =20
>> files for
>> use as input to bloat. The fact that they happen to be bloat =20
>> itself would
>> then not matter. This might mean that the path bloat uses to find =20
>> its input
>> would be different from the path used for regular loading, etc.
>>
>> -- Eliot
>>
>> ---------------------------------------------------------------------=20=

>> ----
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> dacapobench-researchers mailing list
>> dacapobench-researchers@...
>> https://lists.sourceforge.net/lists/listinfo/dacapobench-researchers
>>
>
>
> --=20
> Eric Bodden
> Sable Research Group
> McGill University, Montr=E9al, Canada
>
> ----------------------------------------------------------------------=20=

> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/
> _______________________________________________
> dacapobench-researchers mailing list
> dacapobench-researchers@...
> https://lists.sourceforge.net/lists/listinfo/dacapobench-researchers

Hi Eric,

IMO the problem is that when bloat parses its command line option, it 
does this:

         } else if (args[i].equals("-classpath")) {
           if (++i >= args.length) {
             usage();
           }

           String classpath = args[i];
           loader.setClassPath(classpath);

replacing the current classpath completely.  You could try changing this 
to "loader.prependClassPath" ?

cheers

Eric Bodden wrote:
> Hi all.
> 
> I just tried to implement this. However, this raises the following problem:
> 
> In order to tell bloat where to look for those class files, I need to
> set it's "-classpath" command line parameter. However, as soon as I do
> so, it cannot find java.lang.Object any more because rt.jar is not
> contained in this classpath. So I need some way to tell the harness to
> append rt.jar to the classpath or something...
> 
> Any thoughts?
> 
> Eric
> 
> On 21/04/07, Eliot Moss <moss@...> wrote:
>> My suggestion would be to have a separate copy of the relevant files for
>> use as input to bloat. The fact that they happen to be bloat itself would
>> then not matter. This might mean that the path bloat uses to find its input
>> would be different from the path used for regular loading, etc.
>>
>> -- Eliot
>>
>> -------------------------------------------------------------------------
>> This SF.net email is sponsored by DB2 Express
>> Download DB2 Express C - the FREE version of DB2 express and take
>> control of your XML. No limits. Just data. Click to get it now.
>> http://sourceforge.net/powerbar/db2/
>> _______________________________________________
>> dacapobench-researchers mailing list
>> dacapobench-researchers@...
>> https://lists.sourceforge.net/lists/listinfo/dacapobench-researchers
>>
> 
> 


-- 
Robin Garner
Dept. of Computer Science
Australian National University
http://cs.anu.edu.au/people/Robin.Garner/

Ok, it turns out that bloat actually has an option "-classpath/p" to
preprend something to the classpath already. So this is perfect and
seems to work.

Attached is a patch against the current trunk version that
incorporates all the necessary changes.

Eric

--=20
Eric Bodden
Sable Research Group
McGill University, Montr=E9al, Canada

Thanks Eric.

Robin and I are swamped right now, but we'll incorporate your patch =20
and ensure it gets into a maintenance release RSN.

Thanks again,

--Steve

On 26/04/2007, at 7:58 AM, Eric Bodden wrote:

> Ok, it turns out that bloat actually has an option "-classpath/p" to
> preprend something to the classpath already. So this is perfect and
> seems to work.
>
> Attached is a patch against the current trunk version that
> incorporates all the necessary changes.
>
> Eric
>
> --=20
> Eric Bodden
> Sable Research Group
> McGill University, Montr=E9al, Canada
> <bloat.patch>
> ----------------------------------------------------------------------=20=

> ---
> This SF.net email is sponsored by DB2 Express
> Download DB2 Express C - the FREE version of DB2 express and take
> control of your XML. No limits. Just data. Click to get it now.
> http://sourceforge.net/powerbar/db2/=20
> _______________________________________________
> dacapobench-researchers mailing list
> dacapobench-researchers@...
> https://lists.sourceforge.net/lists/listinfo/dacapobench-researchers