#73 Technical design for test coverage reporting

[Andreas Mangold]

For the implementation of [bugs:#30]:

Please write a technical design for the integration of test coverage reports into SASUnit: Consider logging of coverage information during execution phase, formatting of a coloured source code listing and placement of a coverage indicator and the source code listing into the SASUnit reporting environment. The solution can be dependant on SAS 9.3 and higher, using the new macro trace options.

[Patrick Warnat]

The basic principle for the integration of test coverage assessment into SASUnit is the utilization of the MCOVERAGE and MCOVERAGELOC system options available in SAS 9.3. If the MCOVERAGE system option is activated, a coverage analysis text file is generated that logs which lines of macro code are executed while the SAS system executes source code. The MCOVERAGELOC option determines where the coverage analysis text file is generated.

Using these system options, test coverage (more precisely statement coverage) reports can be integrated in SASUnit with the following modifications:

• initsasunit.sas

  • New macro parameter i_testcoverage with possible values:
    • 0 (default) .. no assessment of test coverage
    • 1 .. assessment of test coverage is activated
  • If i_testcoverage has an other value than 1, set it to 0
  • If i_testcoverage equals 1, check whether the running SAS version is 9.3. If it is not 9.3: Error WrongTcVer, “Invalid SAS version for test coverage assessment - only 9.3 supported”, exit initsasunit
  • Introduce a new column to target.tsu with name tsu_testcoverage containing the value of i_testcoverage

• _sasunit_loadEnvironment.sas

  • Introduce global macro variable g_testcoverage that is filled with value of column tsu_testcoverage of table target.tsu

• runSASUnit.sas

  • if g_testcoverage equals 1:
    • for every scenario file executed, generate a coverage analysis text file in the log sub dir:
      • generate a local macro variable containing the absolute file path to the coverage analysis text file to be generated. The file should be named like its SAS log file, only with the file extension “tcg”, e. g. “001.tcg”.
      • modifiy the String of the sas call (_sCmdString):
      • set the MCOVERAGE system option
      • set MCOVERAGELOC to the value of the macro variable with the file path to the coverage analysis text file to be generated

• Implement New macro saspgm\sasunit_sasunit_reporttcghtml.sas (prototype implementation available as an attachment, please see below):
  The macro takes a macro source code file and a coverage analysis text file generated with the MCOVERAGE option. Using the data of the coverage analysis text file, a html file is generated out of the macro source code file, in which the covered lines of code and the non-covered lines of code are highlighted with bright colours and the non contributing lines are shown in grey. In addition, the percentage of covered lines is written in a headline of the html file. Further implementation details are described in the available prototype implementation file.

• _sasunit_reportAutonHTML.sas

  • if g_testcoverage equals 1:
    • in the log subdir: append all *.tcg files to one file named 000.tcg
      This is done in order to get one file containing coverage data of all calls to the macros under test
    • for every unit under test (see ‘target’ database ): call new macro _sasunit_reporttcghtml.sas once in order to get a html file showing test coverage for the given unit under test. For every call, use the 000.tcg file as coverage analysis text file
    • for every unit under test: read test coverage percentage out of test coverage html file and put it in the report table of the units under test (that is generated by _sasunit_reportAutonHTML) in a new table column. Generate the percentage figure as a html link to the testcoveragehtml file.