Menu
▾
▴
Re: [LTP] [RFC][PATCH 4/4][RT] Documentation and a tool for XML library
|
From: Darren H. <dv...@us...> - 2008-11-20 20:01:17
|
Gilles Carry wrote:
> Documentation: just the basics on how to
> - run
> - code
> - compile
>
> Tool: reference all xml files in collection.xml
> - xml_collect.sh
>
> Also an example of XML dump.
This is a nice doc, thanks for starting it with the patch early on.
>
> Signed-off-by: Gilles Carry <gil...@bu...>
> ---
> testcases/realtime/doc/XML | 286 ++++++++++++++++++++++++++++
> testcases/realtime/scripts/xml_collect.xml | 15 ++
> 2 files changed, 301 insertions(+), 0 deletions(-)
> create mode 100644 testcases/realtime/doc/XML
> create mode 100755 testcases/realtime/scripts/xml_collect.xml
>
> diff --git a/testcases/realtime/doc/XML b/testcases/realtime/doc/XML
> new file mode 100644
> index 0000000..e814c1c
> --- /dev/null
> +++ b/testcases/realtime/doc/XML
> @@ -0,0 +1,286 @@
> +XML library
> +===========
> +
> +
> +Content:
> +
> +1. Why dump results and stats with XML?
> +2. Run test with XML
> +3. Develop with libxml
> +4. Example of generated XML
> +5. Post processing
> +
> +
> +
> +1. Why dump results and stats with XML?
> +---------------------------------------
> +
> +RT tests already generate stats and plot files. What XML can give more?
> +
> +Post processing:
> +XML provides an easy way to transform the data without the need to change and
> +recompile the test. Just create or modify an Xquery to get the requested output.
> +
> +Regression/comparison/correlation:
> +With XML data, it's easy to gather several tests outputs and see what changed
> +from one test to another.
> +Plotting combined graphic can be made very easy using the proper Xquery.
> +
> +
> +2. Run test with XML
> +--------------------
> +
> +Usage:
> + Run test with -x option.
> + Example:
> + mtrix_mult -x my-kernel-setting-1.1
> +
> +For each run, an XML dump file is created.
> +The file is names <testname>.<timestamp>.xml
> +
> +Note:
> + This library does not offer any post-processing tool.
> +
> +
> +3. Develop with libxml
> +----------------------
> +
> +Synopsis:
> + xml_stream_t *xs;
> +
> + rt_init(option_string, parse_args, argc, argv);
> + ...
> + xs = xml_stream_init("foo", "ltp-run", "foo test");
> +
> +
> + /* Begin of application XML dump. */
> + /* Use according to taste. */
> + xml_start_tag (xs, "this-node");
> + ...
> + xml_start_tag (xs, "other-node");
> + ...
> + xml_end_tag (xs, "other-node");
> + ...
> + xml_end_tag (xs, "this-node");
> + ...
> + xml_entry (xs, "data", "%s %ld", mystring, mylong);
> + ...
> + xml_empty_tag (xs, "empty-node");
> + ...
> + /* End of application XML dump. */
> +
> + xml_stream_close(xs, "ltp-run");
> +
> +
> +Note:
> + rt_init must be called before xml_stream_init as the former sets global
> + variables used by xml_stream_init in order to setup headers in XML
> + document.
> +
> +
> +Warning:
> + This library does not perform any check: document well-formed and
> + grammar are programmer's responsibility.
> +
> +
> +Compilation:
> + To build tests with libxml run:
> + LIB_XML=1 make
> +
> +
> +Adding XML dump to your test:
> + Make XML conditional: use #ifdef LIB_XML to include specific code.
> +
> + To initialize the XML stream:
> + xs = xml_stream_init("myXMLfile", "myRootTag", "The test title here");
> +
> + This creates a new XML document by opening a file descriptor and adding
> + all proper headers (using global variables set by rt_init:
> + user_free_id).
> +
> +
> + To write a node start tag <mynode> to XML document:
> + xml_start_tag (xs, "mynode");
> +
> +
> + To write a node end tag </mynode> to XML document:
> + xml_end_tag (xs, "mynode");
> +
> +
> + To write data enclosed by tags <mytag>anystring</mytag> to XML document:
> + xml_entry (xs, "mytag", "printf style format", args...);
> +
> +
> + To write an empty node tag </mynode> to XML document:
> + void xml_empty_tag (xs, "mynode");
> +
> +
> + To terminate the XML stream:
> + xml_stream_close(xs, "myRootTag");
> + myRootTag must be the same as used in xml_stream_init.
> +
> +
> +XML document header:
> + Basically, this is to compare test runs performed on differents
> + architectures/kernels/configurations and spot regressions.
> + Hence, it is vital to identify the conditions in which the test was
> + run. This library automatically adds information to allow this
> + identification.
> +
> + In the header we can find the following elements:
> + <title>:
> + <start-time>:
> + <systeminfo>:
> + Though it is not possible to completely identify what's in the
> + box/kernel, this compound element gives an overall view of the
> + architecure.
> + Subelements descritpion:
> + <uname>: report data as 'uname -a'
> + <user-free-id>: a free field set by user to help spot the test runs.
> + This is set by the -x arg in the command line.
> + <online-cpus>: obvious
> + <sys>: a structure of tags that imitate /sys filesystem, though
> + only featuring the useful information that describe the system's
> + topology: how many cpu nodes/cores/threads.
> + As an example, two system featuring each 8 online cpus may
> + offer very different performances in thread parallelization.
> + Looking at <systeminfo> gives the answer as one is a
> + 4-dualthread-cpu box while the other is an 8-unthreaded-cpu box.
> +
> +
> +XML document application dump:
> + Application (the test itself) should not only dump results and samples
> + but also information about the condition in which the test has been run.
> + In the example below:
> + <parameters>
> + <iterations>
> + <calc-per-iter>
> +
> +
> +4. Example of generated XML
> +---------------------------
> +
> +<?xml version="1.0" encoding="UTF-8"?>
> +<ltp-run>
> + <title>matrix-mult</title>
> + <start-time>2008-10-23.10-06-43</start-time>
> + <systeminfo>
> + <uname>
> + <sysname>Linux</sysname>
> + <nodename>excalibur</nodename>
> + <release>2.6.26.6-rt11</release>
> + <version>#146 SMP PREEMPT RT Tue Oct 14 09:15:00 CEST 2008</version>
> + <machine>ppc64</machine>
> + </uname>
> + <sys>
> + <devices>
> + <system>
> + <node>
> + <node0>
> + <cpu0/>
> + <cpu1/>
> + <cpu2/>
> + <cpu3/>
> + </node0>
> + <node1>
> + <cpu4/>
> + <cpu5/>
> + <cpu6/>
> + <cpu7/>
> + </node1>
> + </node>
> + <cpu>
> + <cpu0>
> + <topology>
> + <thread_siblings_list>0-1</thread_siblings_list>
> + </topology>
> + </cpu0>
> + <cpu1>
> + <topology>
> + <thread_siblings_list>0-1</thread_siblings_list>
> + </topology>
> + </cpu1>
> + <cpu2>
> + <topology>
> + <thread_siblings_list>2-3</thread_siblings_list>
> + </topology>
> + </cpu2>
> + <cpu3>
> + <topology>
> + <thread_siblings_list>2-3</thread_siblings_list>
> + </topology>
> + </cpu3>
> + <cpu4>
> + <topology>
> + <thread_siblings_list>4-5</thread_siblings_list>
> + </topology>
> + </cpu4>
> + <cpu5>
> + <topology>
> + <thread_siblings_list>4-5</thread_siblings_list>
> + </topology>
> + </cpu5>
> + <cpu6>
> + <topology>
> + <thread_siblings_list>6-7</thread_siblings_list>
> + </topology>
> + </cpu6>
> + <cpu7>
> + <topology>
> + <thread_siblings_list>6-7</thread_siblings_list>
> + </topology>
> + </cpu7>
> + </cpu>
> + </system>
> + </devices>
> + </sys>
> + <user-free-id>special-hack-v2</user-free-id>
> + <online-cpus>8</online-cpus>
> + </systeminfo>
> + <testname>matrix_mult</testname>
> + <title>Matrix Multiplication (SMP Performance)</title>
> +
> + <!-- Application dump here -->
> + <parameters>
> + <iterations>128</iterations>
> + <calc-per-iter>8</calc-per-iter>
> + </parameters>
> + ...snip...
> +
> +</ltp-run>
> +
> +
> +5. Post processing
> +------------------
> +
> +Of course, anyone is free to use whatever tool he wants to postprocess the
> +generated data.
> +
> +I used Saxonica's XQuery (saxon) to do the job.
> +Since I'm not an XML expert I shall only explain the way I did my
> +postprocessing.
> +
> +Once the test is finished, use the script xml_collect.sh to create file
> +collection.xml that references all *.xml files in the current directory.
> +This must be done every time you generate new xml outputs.
> +
> +
> +Here is an example of xquery script I used to evaluate the frequency at which
> +appears calculation duration degradation bursts in matrix_mult.c
> +
> +The below outputs data for gnuplot.
> +
> +for $s in collection("collection.xml")//ltp-run[testname="matrix_mult"]
> +let $min := min($s/concurrent/stats[title="data"]/data/e/t)
> +where $s/systeminfo/user-free-id = "comb"
> +return
> + for $e in $s/concurrent/stats[title="data"]/data/e
> + order by $e/t
> + return (
> + (data($e/t) - $min) div 1e8 ," ",data($e/y),"
> + "
> + )
> +
> +The plot command is: plot "comb.dat" with impulses
> +The graph shows a comb.
> +
> diff --git a/testcases/realtime/scripts/xml_collect.xml b/testcases/realtime/scripts/xml_collect.xml
> new file mode 100755
> index 0000000..96c2809
> --- /dev/null
> +++ b/testcases/realtime/scripts/xml_collect.xml
> @@ -0,0 +1,15 @@
> +#!/bin/sh
> +
> +COLL=collection.xml
> +COLL_TMP=$COLL.tmp
> +rm -f $COLL $COLL_TMP
> +
> +echo > $COLL_TMP "<collection>"
> +ls -1 *.xml | while read file
> +do
> + [ "$file" == $COLL ] && continue
> + echo >> $COLL_TMP "<doc href=\"$file\"/>"
> +done
> +echo >> $COLL_TMP "</collection>"
> +
> +mv $COLL_TMP $COLL
--
Darren Hart
IBM Linux Technology Center
Real-Time Linux Team
|