Thread: [Sablevm-developer] regression testing staging@r1575 and sablevm-classpath@r1575 against 1.0.9
Brought to you by:
egagnon
Menu
▾
▴
sablevm-developer
|
From: Chris P. <chr...@ma...> - 2004-02-12 18:56:03
|
Hi, I'm trying to do regression tests of staging and sablevm-classpath at r1575 against 1.0.9. I'll run SPECjvm98, JOlden, and JGF single- and multi-threaded. If you commit things after 1575 besides scripts and documentation, they won't get regression-tested by me. Greg wants to move to bugfree if the regression tests are fine. I updated sablevm-classpath, but now I can't run configure anymore, even with --no-gtk specified. ./configure: line 21672: syntax error near unexpected token `PKG_CHECK_MODULES(GTHREAD,' ./configure: line 21672: ` PKG_CHECK_MODULES(GTHREAD, gthread-2.0 >= 2.2)' The mail about this is here: http://mail.gnu.org/archive/html/commit-classpath/2004-02/msg00001.html I have libtool-1.5.2, automake-1.8.2, and autoconf-2.59 ... I can take out this code for now to compile things. Chris |
|
From: Grzegorz B. P. <ga...@de...> - 2004-02-13 01:57:58
|
W li=B6cie z czw, 12-02-2004, godz. 13:56, Chris Pickett pisze:=20 > I updated sablevm-classpath, but now I can't run configure anymore, eve= n=20 > with --no-gtk specified. >=20 > ./configure: line 21672: syntax error near unexpected token=20 > `PKG_CHECK_MODULES(GTHREAD,' > ./configure: line 21672: ` PKG_CHECK_MODULES(GTHREAD, gthread-2.0 >= =3D=20 > 2.2)' I haven't seen these problems, at least last weekend, and I don't think much has been changed since then. Try these: aclocal-1.7 && libtoolize --force && autoconf && autoheader && automake-1.7 --foreign -a -c If you use Debian, and have the right versions of the tools installed - the above programs should be found. HTH GBP PS: I looked at the patch you pointed out and it seems to be fine. --=20 Grzegorz B. Prokopski <ga...@de...> Debian GNU/Linux http://www.debian.org SableVM - LGPLed JVM http://www.sablevm.org Why SableVM ?!? http://devel.sablevm.org/wiki/WhySableVM |
|
From: Chris P. <chr...@ma...> - 2004-02-13 04:05:21
|
Grzegorz B. Prokopski wrote: > W li=B6cie z czw, 12-02-2004, godz. 13:56, Chris Pickett pisze:=20 >=20 >>I updated sablevm-classpath, but now I can't run configure anymore, eve= n=20 >>with --no-gtk specified. >> >>./configure: line 21672: syntax error near unexpected token=20 >>`PKG_CHECK_MODULES(GTHREAD,' >>./configure: line 21672: ` PKG_CHECK_MODULES(GTHREAD, gthread-2.0 >= =3D=20 >>2.2)' >=20 >=20 > I haven't seen these problems, at least last weekend, and I don't think= > much has been changed since then. Try these: >=20 > aclocal-1.7 && libtoolize --force && autoconf && autoheader && > automake-1.7 --foreign -a -c >=20 > If you use Debian, and have the right versions of the tools installed > - the above programs should be found. >=20 So, it compiled. But I couldn't get it to work without dnl'ing the=20 lines in question (I tried the above commands). It needs to be fixed=20 before moving into bugfree, IMHO, but I don't know how. Feel free to=20 verify it on one of the Sable machines with the versions I have=20 (aclocal-1.8.2, libtoolize-1.5.2, autoconf-2.59, autoheader-2.59,=20 automake-1.8.2, which are the latest versions -- I just installed them). You can see the results of the benchmark runs in: http://sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/ *_times_* are the raw benchmark times *_report_* are reports, with user times and pass/fail status (see first) *_diff_* are diffs that caused failure in the reports *_paranoid_diff_* are diffs that don't ignore run-specific information Basically: 1) I found no regressions 2) mtrt seems to have been fixed for switch-debug (no blank line) 3) JGFSyncBench, JGFLUFactBench, and JGFRayTracerBench fail (single- and multi-threaded errors) 4) JGFSerialBench fails, but it fails for java as well (verification error) 5) In general things are faster, but there are a couple of exceptions. *** I didn't run the SizeB and SizeC benchmarks from JGF, since they=20 take too long *** This is what I think is needed for bugfree: 1) A list of known issues (incl. the above benchmark failures and the=20 fact that synchronized methods may crash the VM) 2) A fix for the classpath configure problem (I think the easiest is to=20 undo Mark's configure.ac patch, since apparently this was only for JamVM)= =2E Cheers, Chris |
|
From: Chris P. <chr...@ma...> - 2004-02-13 04:09:58
|
Chris Pickett wrote: > Grzegorz B. Prokopski wrote: >=20 >> W li=B6cie z czw, 12-02-2004, godz. 13:56, Chris Pickett pisze: >> >>> I updated sablevm-classpath, but now I can't run configure anymore,=20 >>> even with --no-gtk specified. >>> >>> ./configure: line 21672: syntax error near unexpected token=20 >>> `PKG_CHECK_MODULES(GTHREAD,' >>> ./configure: line 21672: ` PKG_CHECK_MODULES(GTHREAD, gthread-2.0= =20 >>> >=3D 2.2)' >> >> >> >> I haven't seen these problems, at least last weekend, and I don't thin= k >> much has been changed since then. Try these: >> >> aclocal-1.7 && libtoolize --force && autoconf && autoheader && >> automake-1.7 --foreign -a -c >> >> If you use Debian, and have the right versions of the tools installed >> - the above programs should be found. >> >=20 > So, it compiled. But I couldn't get it to work without dnl'ing the=20 > lines in question (I tried the above commands). It needs to be fixed=20 > before moving into bugfree, IMHO, but I don't know how. Feel free to=20 > verify it on one of the Sable machines with the versions I have=20 > (aclocal-1.8.2, libtoolize-1.5.2, autoconf-2.59, autoheader-2.59,=20 > automake-1.8.2, which are the latest versions -- I just installed them)= =2E >=20 > You can see the results of the benchmark runs in: >=20 > http://sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/ Sorry. I never seem to get url's right :/ http://www.sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/ Chris |
|
From: Grzegorz B. P. <ga...@de...> - 2004-02-13 05:51:06
|
W li=B6cie z czw, 12-02-2004, godz. 23:04, Chris Pickett pisze:=20 > You can see the results of the benchmark runs in: >=20 > http://sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/ Great! I've just looked at them, see below. > *_times_* are the raw benchmark times > *_report_* are reports, with user times and pass/fail status (see first= ) > *_diff_* are diffs that caused failure in the reports > *_paranoid_diff_* are diffs that don't ignore run-specific information >=20 > Basically: >=20 > 1) I found no regressions I have to believe you, because I looked at the diffs and firstly: I don't know what I am comaring with what. I mean, that you used just diff, instead of ex. 'diff -u', which would put filenames into the header. Second thing, partially being result of the first one: I looked at http://www.sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/grande_section1= _diff_feb12 and it seems that errors are not identical. I guess that both cases qualify as "doesn't work", but still - I'd love to know which errors do we get now. Do I get it right, that w/ 1.0.9 we get link error, and now we get security provider error (so it's more CP problem)? > 2) mtrt seems to have been fixed for switch-debug (no blank line) oh, and how about switch w/o signals? It might be interesting to see. > 3) JGFSyncBench, JGFLUFactBench, and JGFRayTracerBench fail > (single- and multi-threaded errors) > 4) JGFSerialBench fails, but it fails for java as well > (verification error) > 5) In general things are faster, but there are a couple of exceptions. Working on it. Inlined is staging is ATM not reliable in terms of speed. There are good chances to get fixed this weekend. > 1) A list of known issues (incl. the above benchmark failures and the=20 > fact that synchronized methods may crash the VM) > 2) A fix for the classpath configure problem (I think the easiest is to= =20 > undo Mark's configure.ac patch, since apparently this was only for JamV= M). I guess we can just undo the change for this moment, but it'd be best to ask on gnu classpath ML. Hmmm... I think he's on SableVM list. Mark? :-) I think Chirs, that you've just pushed 1.1.0 a lot to make it reality. Thanks a lot! Cheers, Grzegorz B. Prokopski --=20 Grzegorz B. Prokopski <ga...@de...> Debian GNU/Linux http://www.debian.org SableVM - LGPLed JVM http://www.sablevm.org Why SableVM ?!? http://devel.sablevm.org/wiki/WhySableVM |
|
From: Mark W. <ma...@kl...> - 2004-02-13 12:01:51
|
Hi, On Fri, 2004-02-13 at 06:21, Grzegorz B. Prokopski wrote: > W li=B6cie z czw, 12-02-2004, godz. 23:04, Chris Pickett pisze:=20 > > 2) A fix for the classpath configure problem (I think the easiest is to= =20 > > undo Mark's configure.ac patch, since apparently this was only for JamV= M). > I guess we can just undo the change for this moment, but it'd be > best to ask on gnu classpath ML. Hmmm... I think he's on SableVM list. >=20 > Mark? :-) Something like that patch is definitely needed. Not just for JamVM, libgcj needed the same to be able to run AWT programs, see: http://gcc.gnu.org/ml/java-patches/2004-q1/msg00411.html Is pkg.m4 available on the machine, in sablevm? Seems the problem is that the PKG_CHECK_MODULES macro from that file isn't defined. On a debian system it comes from the package 'pkg-config'. Cheers, Mark |
|
From: Chris P. <chr...@ma...> - 2004-02-13 19:35:34
|
Mark Wielaard wrote: > Hi, >=20 > On Fri, 2004-02-13 at 06:21, Grzegorz B. Prokopski wrote: >=20 >>W li=B6cie z czw, 12-02-2004, godz. 23:04, Chris Pickett pisze:=20 >> >>>2) A fix for the classpath configure problem (I think the easiest is t= o=20 >>>undo Mark's configure.ac patch, since apparently this was only for Jam= VM). >> >>I guess we can just undo the change for this moment, but it'd be >>best to ask on gnu classpath ML. Hmmm... I think he's on SableVM list. >> >>Mark? :-) >=20 >=20 > Something like that patch is definitely needed. Not just for JamVM, > libgcj needed the same to be able to run AWT programs, see: > http://gcc.gnu.org/ml/java-patches/2004-q1/msg00411.html >=20 > Is pkg.m4 available on the machine, in sablevm? > Seems the problem is that the PKG_CHECK_MODULES macro from that file > isn't defined. On a debian system it comes from the package > 'pkg-config'. I installed pkg-config-0.15.0 and it worked. I didn't test earlier=20 versions. I think pkg-config should be added to the list of classpath=20 dependencies in the `Suggested Software' part of INSTALL, and that we=20 should put a little note in our version of classpath in the interim. Cheers, Chris |
|
From: Chris P. <chr...@ma...> - 2004-02-13 19:35:32
|
Grzegorz B. Prokopski wrote: >>Basically: >> >>1) I found no regressions > > I have to believe you, because I looked at the diffs and firstly: > > I don't know what I am comaring with what. I mean, that you used > just diff, instead of ex. 'diff -u', which would put filenames > into the header. Each SableVM output is compared against java.out, which is supposed to be the correct output (from sun's 1.4.1-b21 java). If there's a problem, the name of the SableVM output file is printed, and then the diff. e.g. ./grande/section1/sablevm-1.0.9-switch-debug-JGFSerialBench.out 1c1,12 < Exception in thread "main" java.lang.VerifyError: (class: JGFSerialBench, method: JGFrun signature: ()V) Incompatible types for storing into array of arrays or objects --- > *** Couldn't bind native method Java_java_lang_Class_getDeclaredFields *** > *** or Java_java_lang_Class_getDeclaredFields__ *** > java.lang.UnsatisfiedLinkError If they're identical, save for timing information (grep -i -v -E 'time|total|msec|returned value'), the result is a pass in the report file. The `paranoid' diffs are those that have everything (but still worth scanning over -- one time I found negative numbers reported for timing information with my spmt stuff). I ran the SableVM's with '-Y'. If you want, it's very easy to get a 'diff -u'. Let me know (I don't think it will provide more useful information, just nicer formatting). > Second thing, partially being result of the first one: I looked at > http://www.sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/grande_section1_diff_feb12 > and it seems that errors are not identical. I guess that both cases > qualify as "doesn't work", but still - I'd love to know which errors > do we get now. Do I get it right, that w/ 1.0.9 we get link error, > and now we get security provider error (so it's more CP problem)? Without having looked at any code, yes, I think so. "Doesn't work" is sort of abused in this case, because it "doesn't work" for Sun's Java either. But you're right, there is an improvement on JGFSerialBench: the error is handled more gracefully. >>2) mtrt seems to have been fixed for switch-debug (no blank line) > > oh, and how about switch w/o signals? It might be interesting to see. $ diff sablevm-staging-r1575-switch-nosig.out java.out and $ diff sablevm-1.0.9-switch-nosig.out java.out both give me nothing, which is interesting. N.B. I only did it once -- the error may be intermittent. I think for now we need to focus on micro synchronization problems first (e.g. JGFSyncBench). > >>3) JGFSyncBench, JGFLUFactBench, and JGFRayTracerBench fail >> (single- and multi-threaded errors) >>4) JGFSerialBench fails, but it fails for java as well >> (verification error) >>5) In general things are faster, but there are a couple of exceptions. > > Working on it. Inlined is staging is ATM not reliable in terms of speed. > There are good chances to get fixed this weekend. Okay. I'll put the scripts for doing the regression testing in ~/sablevm/regression in my home directory -- they need some unification to become truly reusable, but as long as you have the same directory layout for the benchmarks as me, and run 4 different SableVM's, they should be fairly easily adjusted (using some editor's find-and-replace). >>1) A list of known issues (incl. the above benchmark failures and the >>fact that synchronized methods may crash the VM) >>2) A fix for the classpath configure problem (I think the easiest is to >>undo Mark's configure.ac patch, since apparently this was only for JamVM). > > I guess we can just undo the change for this moment, but it'd be > best to ask on gnu classpath ML. Hmmm... I think he's on SableVM list. > > Mark? :-) Fixed (see other email to Mark). > I think Chirs, that you've just pushed 1.1.0 a lot to make it reality. > Thanks a lot! No problem :) Most of the time went into compilation (configure wasn't seeing gcc-3.3.2 for staging, and once that was fixed I had to go back and specify gcc-2.95.4 for 1.0.9), installing dependencies, and adjusting benchmarking scripts. The actual benchmarks took about 6 hours total to run (4 different machines, I think), but that could be sped up if the grande benchmarks were broken into section1, section2, and section3, and creating the reports is trivial once the scripts are set up. I want a release personally too -- if anything, so that I have a supposedly more stable codebase to work against if I want it. According to the policy, we have to try and fix regressions as soon as possible, if they appear. I think "bugfree" is a bit of a misnomer, it should really be thought of as "regressionfree", since no software is truly bug-free. One last thing: I think it would be helpful to include David's build script for classpath, and if we're not putting one in for SableVM, at least put a note that says "the default configure options are: --with-threading=inlined, etc. etc." We also need a note about gcc. Cheers, Chris |
|
From: Chris P. <chr...@ma...> - 2004-02-13 22:55:29
|
Chris Pickett wrote: > Okay. I'll put the scripts for doing the regression testing in > ~/sablevm/regression in my home directory -- they need some unification > to become truly reusable, but as long as you have the same directory > layout for the benchmarks as me, and run 4 different SableVM's, they > should be fairly easily adjusted (using some editor's find-and-replace). Sorry, I meant sandbox (~/sablevm IS the working copy of my sandbox). They were just committed. I'm stopping work on progress towards bugfree now, unless there's something else specific you'd like me to do. By the way, the link to the benchmark results on the news page is dead. http://www.sable.mcgill.ca/~cpicke/sablevm/r1575-vs-1.0.9/ Cheers, Chris |
×
Want the latest updates on software, tech news, and AI?
Get latest updates about software, tech news, and AI from SourceForge directly in your inbox once a month.
Thanks for helping keep SourceForge clean.
X