[Kerncomp-devel] Re: [ANNOUNCE] Automated Kernel Build Regression Testing

[Jan D. <jdi...@pp...>]

Ian Wienand wrote:
> On Tue, May 17, 2005 at 12:48:10AM +0200, Jan Dittmer wrote:
> 
>>Well, you convinced me. I now build a version which stores the build
>>results in a directory tree named:
>>
>>version/arch/configs/testtype/*
>>
>>* being files for each data item stored there. Something like this:
>>
>>db/2.6.11-rc4-bk6/alpha/defconfig
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/startdate
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/enddate
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/log
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/success
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/warnings
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/errors
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/config
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/compiler
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/version
>>db/2.6.11-rc4-bk6/alpha/defconfig/compile/num_errors
>>
>>And rewrote the front-end to access the database. Big pro: You can just
>>rsync results from different places together, as long as they have
>>unique 'testtype' stamps. You can even use git to manage this database.
>>I'll put the beta interface up tomorrow.
> 
> 
> I like it :)

Well, and I've redone it once again :-). The problem with the format
being, that it's hard to switch around.
So what I'm doing now, because hashes are cool nowadays, I keep a
database of results in db/objects/<sha1hashofresults>/<resultfiles>
and a symlink tree in db/cache/, which looks like the above, but
can be completly different.
So you have directories like:
db/objects/00/10/0010fa08869201472b0903d23bed23c2fe9a71dc/
containing lots of files. I've also added a gpg signature file
(signature), which signs the hash of all files in the directory,
without 'signature' itself.
Therefore it's possible that I could start to automatically
import tests from other people (By email, rsync, ...).


> I also got in contact with Michael Still who also does some
> autobuilding (http://www.stillhq.com/linux/automated/)
> 
> He's got a nice general architecture behind his.  You write scripts
> that are supposed to operate on a clean tree, and they get called by
> an overseer type script.
> 
> I think this could translate into something where you specify a
> 'procedures' file which has a list of scripts to run over a clean
> tree.  You give each one a few options.  The generic ones would be
> 'patch with this patchset', 'build this config', 'boot this on the ski
> simulator', etc.  If one step fails, the others don't complete.
> 
> So your procedure file might look like
> 
> -- lvhpt patches --
> apply_patch /home/ianw/patches/lvhpt/series
> build_kernel defconfig
> count_warnings
> sim_boot
> -- zx1_defconfig build --
> build_kernel zx1_defconfig
> -- arm build --
> cross_build_kernel arm defconfig
> 
> You can create your procedures file on the fly nightly too, I guess,
> using something like m4.

That's great. I thought about something like this, but if it's already
finished...

> It should be quite easy to add new procedure steps by writing your
> scripts using all the generic functions provided by kerncomp.  It
> should handle redirecting output to html files in directories, etc
> etc.

Well, that would perfectly fit with my result storage I think. The final
'procedure' would just sign the results and submit them to the results
database.



-- 
Jan