[Kerncomp-devel] Re: [ANNOUNCE] Automated Kernel Build Regression Testing

[Jan D. <jdi...@pp...>]

Ian Wienand wrote:
> On Sat, May 14, 2005 at 06:17:18PM +0200, Jan Dittmer wrote:
> 
>>What I plan to implement, hopefully till next weekend, is a system
>>which discovers new kernel versions and builds a table of
>>kernel/version/arch triplets that want to be tested.
> 
> 
> Ok, that's good.  However, I would say the most important thing for
> *us* is application of patches and building from a git tree every
> night.  Currently the rate of change in the git tree isn't really like
> what the old bitkeeper trees was, but I'm sure it will get there.
> This has really what kerncomp was used for from the start, it
> originated right at the time IA64 was moving from being kept out of
> tree to being merged into mainline.
> 
> For people who aren't trying to maintain out of tree patches this
> really isn't important.  

Yes, and I won't prevent you from doing that. I'm just providing a very
general system to view the results from such tests.
You can still test whatever version you want, as long as you have a unique
version string.

>>Against these versions any number of tests can run: compile, run
>>time test, whatever, ... and report the results back, without much
>>of formal requirements.  Basically a big storage of test
>>results. Would that work out for you?
> 
> 
> It sounds like it would; I guess the devil is in the details :)
>  
> 
>>... and write git-SELECT to search through them? :) Really sql has
>>lots of advantages when your store even slightly more
>>complex/relational things.
> 
> 
> Personally, I'd really need to be conviced on the need of database
> here.  We do all our builds on a big machine that is largly unused
> overnight, and can then simply rsync the results over to the
> webserver.  If I want to put a status as to why the build failed, I
> just add it in the summary.log file and re-rsync.  If I screw
> something up, I just delete the subdirectory.  Everything is kept in
> order via standard unix permissions.  If I need to sort through the
> output, I use sed and awk.  We just bzip the files to archive them (we
> could use rzip across all of the files if space was really an issue).

Well, you convinced me. I now build a version which stores the build
results in a directory tree named:

version/arch/configs/testtype/*

* being files for each data item stored there. Something like this:

db/2.6.11-rc4-bk6/alpha/defconfig
db/2.6.11-rc4-bk6/alpha/defconfig/compile
db/2.6.11-rc4-bk6/alpha/defconfig/compile/startdate
db/2.6.11-rc4-bk6/alpha/defconfig/compile/enddate
db/2.6.11-rc4-bk6/alpha/defconfig/compile/log
db/2.6.11-rc4-bk6/alpha/defconfig/compile/success
db/2.6.11-rc4-bk6/alpha/defconfig/compile/warnings
db/2.6.11-rc4-bk6/alpha/defconfig/compile/errors
db/2.6.11-rc4-bk6/alpha/defconfig/compile/config
db/2.6.11-rc4-bk6/alpha/defconfig/compile/compiler
db/2.6.11-rc4-bk6/alpha/defconfig/compile/version
db/2.6.11-rc4-bk6/alpha/defconfig/compile/num_errors

And rewrote the front-end to access the database. Big pro: You can just
rsync results from different places together, as long as they have
unique 'testtype' stamps. You can even use git to manage this database.
I'll put the beta interface up tomorrow.

-- 
Jan