[Pyret-devel] PyReT design document

[Nigel H. <nh...@us...>]

Greetings,

I work with James Kenefick on Linux kernel testing.  I read the PyReT=20
design document (2nd draft, 11/4/05); and if I understanding it correctly, =

you?re designing an infrastructure in which you can build a regression=20
hunting engine.  Your document is *not* meant to describe a specific=20
regression hunting algorithm ... right?  Here are some initial comments=20
based on this assumption.

It appears to be a blackboard-like (work allocation) approach where the=20
Master analyzes the Search Space and creates Jobs to run tests.  Hunters=20
idle until they find new jobs.  Jobs have the general structure: Load=20
modules, Transport files, Build, ... . Hunters run jobs and create results =

files for the Master to process.=20

Automating kernel testing is a great pursuit and your basic mechanism=20
seems okay.  I think this is a good start.  In general your paper needs a=20
much better introductory description of the problem and overview of the=20
solution.  You jump into the solution model too soon.  The definitions in=20
the glossary should be incorporated into a design overview that precedes=20
the diagrams.  Also, a simple working example would help the reader=20
understand how to use the services provided by your mechanism.=20

The most important thing missing from the document is a clear description=20
of how the ?search space? is used to drive the whole process.  Also, the=20
analysis loop is not explicitly shown.  I see blocks for ?Process Search=20
Space?, ?Create Jobs?, but ?Evaluate Results? appears to be outside that=20
loop and terminates without feedback into the system.

In the Hunter Flow Diagrams (note you have a typo in the diagram name),=20
there are some blocks with multiple out-going archs that are unlabeled.=20
For example, Build, Patch, Install, all have an arch to ?Update Hunter=20
Status? as well as an arch to the next stage.  Are both of these archs=20
taken sequentially, or is ?Update Hunter Status? an error path?


Nit picks:=20

The reference to an implementation specific protocol, NFS, seems out of=20
place given the otherwise high-level description of your diagrams.

The use of Python (also implementation specific) in the project name could =

lead some to presume you?re testing Python.



nigel.