From: <ste...@an...> - 2007-06-12 14:43:57
|
=A0 =3E =3E 1=2E A =22core=22 set of workloads=2E =3E = =3E This is what I was going with for the =27core=27 test run=2E I would= also =3E like to see gcmap-sanity merged into this once we fix the gcmap bugs= =3E that still affect O1=2E You=27ve misunderstood my intent=2E=A0=A0 I am trying to separate concer= ns=2E What I=27ve described (it was just a suggestion) for this particul= ar test set is one that is=3A =A0a) user-driven (the set we want to tell our users *always* works=2C a= nd is presumably based on what we think our users care about in very sim= ple terms)=2C =A0b) always passing=2C and =A0c) one we can run super-frequently=2C continuously (ie around 1hr or = so per run on the fast machines) There are sure to be better names=2E Perhaps =22never-fail=22=2E=A0 gcma= p sanity fits none of these criteria=2E=A0 I want the role of these test= sets to be crystal clear=2E=A0 The gcmap sanity test is critical=2C I a= gree=2E=A0 It=27s just a completely different thing to a never-fail user= workload=2C so should fit somewhere else in the regression name space=2E= =3E Can we please name this =22pre-commit=22 - the name =22commit=22 jus= t = =3E annoys me =3B) I don=27t mind=2E =3E I would like to clear out any configurations that don=27t really = =3E seem to =3E be maintained=2E NoGC comes to mind as most of the test-run =22failu= res=22 =3E are false positives and could by avoided by properly configuring the= =3E test run or removing certain tests from test run=2E +1 (NoGC is an important/fundamental test=2C but obviously can only work on= some very limited workloads) =3E I would still like to aim for a baseline of 0 failures for this = =3E test run aswell=2E I share your sentiment=2E=A0=A0 On the other hand=2C we should always be= stretching ourselves with new workloads which chase out corner cases=2E= =A0 One could argue (not sure if I agree)=2C that 0 failures indicates w= e=27re not testing hard enough=2E=A0 I used to love it in the good ol da= ys when every now and again the regression email would come in with the = subject line =22SUCCESS!=22 (ie 100=25)=2E=A0 I think may get 100=25 tod= ay on that test set=3B we test for a lot more cases now=2E Cheers=2C --Steve |