[Vxl-maintainers] Dart2

[Wheeler, F. W \(G. Research\) <wh...@cr...>]

[resent to correct vxl-maintainers list]

VXL Maintainers and Users:

We propose to migrate the VXL dashboard server from Dart1 to Dart2.
This is an outline of our plan and a call for comments/discussion.

At first we thought Dart1 should be dropped in favor of Dart2 fairly
soon.  Now, due to some recently discovered email issues with Dart2,
we suggest waiting several months.  However, this email was already
drafted, and a Dart2 server has been started, so we're still sending
this out now.  This early notice will give people plenty of time to
migrate dashboard submissions to ctest.

Dart2 has many new features, including:
 - Nicer look and feel
 - Easier to setup and maintain the server (current Dart1 server needs
frequent manual interaction)
 - Dynamic page generation - more layout options - negligible roll-up
delay
 - Performance plots showing test CPU usage over time
 - User accounts used to map your SourceForge ID to an email address for
nag mail (current Dart2 bug send multiple copies of broken dashboard
emails)

Here are the main transition steps:

1. Run Dart1 and Dart2 dashboards simultaneously. (done)
2. Stop supporting Dart1/ftp submissions, require ctest/xmlrpc
submissions and remove Dart1 dashboard. (probably early 2007 based on
funding, after Dart2 release that fixes email issue)

Step 1 has already begun.  This has no impact on submissions and
allows us to see what the Dart2 server is like.  Amitha has rigged the
system so that all submissions will to go to both dashboards.  The
brand new Dart2 dashboard (up for a couple of weeks now) is here:

  http://vxl.vision.cs.rpi.edu:8081/VXL/Dashboard/

The reason for Step 2 in the plan is that the current Dart1 ftp
submission process is fragile and our interim connection of ftp
submissions to the Dart2 dashboard is a bit of a kludge.  CTest xmlrpc
submissions go directly to the Dart2 server, and require nothing more
than http.

Step 2 should be done in early 2007, when Dart2 has been fixed so that
the broken dashboard emails are only sent once, and to give people a
chance to migrate to ctest.  There is a long delay before we expect
this email bug to be fixed due to funding issues.

Step 2 will require those submitting dashboard builds to use CTest.
CTest comes with CMake and is much easier to use than the Dart1 TCL
method.  We have switched our internal "GEVXL" project from Dart1/TCL
to Dart2/ctest and are happy with the results and the simplification
of the whole process.

All VXL dashboard builds from GE have been switched from Dart1/TCL to
ctest using ctest scripts.  (CTest also does ftp-based submits for the
VXL Dart1 dashboard.)  VXL dashboard builds can be switched to CTest
at any time, and then the transition from Dart1/TCL submits to
Dart2/xmlrpc submits will be seamless and happen automatically.  The
build notes for GE builds on the VXL dashboard (click the icon next to
the Build Name) show the script used for each build so you can use
those as examples if needed.  Running and submitting the build is then
as simple as this:

  nice -10 ctest -V -S vxl_nightly_gccmem_red.cmake

or even just this

  ctest -S vxl_nightly_gccmem_red.cmake

Below is a link to a specific Build Note with a CTest script.  Note
that the CTest script is entirely self-contained.  Environment
settings and CMake cache settings are all in one place in this file.
This CTest script was actually created by yet another script; that is
why there are unnecessary quotes in several places.  There are other
examples on the CTest scripting docs page (see link below).

=20
http://www.cs.rpi.edu/research/groups/vxl/Testing/Sites/mobile/FreeBSD-6
.1_gcc-3.4.4_-Wall_-O_shared/20060911-0300-Nightly/Notes.html

Step 2 will also require the elimination of DartConfig*.cmake files
and the addition of a new CTestConfig.cmake file in the VXL top level
dir in CVS.  The new CTestConfig.cmake file will be:

  SET (CTEST_PROJECT_NAME "VXL")
  SET (CTEST_NIGHTLY_START_TIME "22:00:00 EST")
  SET (CTEST_DROP_METHOD "xmlrpc")
  SET (CTEST_DROP_SITE "http://vxl.vision.cs.rpi.edu:8081")
  SET (CTEST_DROP_LOCATION "VXL")

Some additional links that may be of interest:

Summary of Dart2:
  http://www.na-mic.org/Wiki/index.php/Dart2Summary

Example Dart2 dashboards with a lot of submissions
  http://na-mic.org:8081/Insight/Dashboard/
  http://na-mic.org:8081/CMake/Dashboard/

CTest Wiki
  http://www.cmake.org/Wiki/CMake_Testing_With_CTest

CTest scripting docs
  http://www.cmake.org/HTML/TestingSetup.html

Fred and Amitha