#4 Number of tests succeeded reported incorrectly by framework

[mgrollins]

The last line of output "XXX test(s) succeeded." will
count as successful a test that failed an assert
statement. The shUnit framework will increment the
number of successful tests unles the last assert
statement in a test function fails.

This is because in shuRunOneTest, the test function is
run, and then SHU_TOTAL_NR_SUCCEEDED is incremented
based on the variable SHU_TEST_SUCCEDED being equal to
SHU_TRUE. SHU_TEST_SUCCEDED, however, is written on
every shuAssert and shuDeny, so it only reflects the
status of the last assert statement within a test
function. From bobbynations' comment on bug 1185475,
it appers that you do expect/allow multiple asserts
within a single test function.

SHU_STR_FAILED will be the empty string if no asserts
failed during the execution of a single test function,
as it is initialized empty in the for loop of shuStart
before calling shuRunOneTest.
I believe testing SHU_STR_FAILED empty before
incrementing SHU_TOTAL_NR_SUCCEEDED would be a good
fix, as shown below:

if type ${strTestToRun} > /dev/null 2>&1
then
${strTestToRun}
# if [ ${SHU_TEST_SUCCEEDED} -eq ${SHU_TRUE} ]
if [ "${SHU_STR_FAILED}X" = X ]
then
SHU_TOTAL_NR_SUCCEEDED=`expr
$SHU_TOTAL_NR_SUCCEEDED + 1`
fi

This change does make shuUnitTest report that only 7
tests succeeded, because of the intentional failures in
the first test function.

[Nobody/Anonymous]

Logged In: NO

This can be corrected by the following patch :

===================================================================
--- shUnit (revision 85)
+++ shUnit (working copy)
@@ -49,7 +49,7 @@

shuRunOneTest() {
strTestToRun=${1}
- SHU_TEST_SUCCEEDED=${SHU_FALSE}
+ SHU_TEST_SUCCEEDED=${SHU_TRUE}

if type shuSetUp > /dev/null 2>&1
then
@@ -88,6 +88,7 @@
fi
}

+SHU_RETURN_VALUE=0
shuStart() {
strInitFunction="${1}"

@@ -107,6 +108,7 @@
shuTestNbr=`expr ${shuTestNbr} + 1`
printf " Test %i: %s " ${shuTestNbr} "${STR_TEST}"
shuRunOneTest ${STR_TEST}
+ [ ${SHU_TEST_SUCCEEDED} = ${SHU_FALSE} ] && SHU_RETURN_VALUE=1
printf "\n"

if test -n "${SHU_STR_FAILED}"
@@ -152,7 +154,6 @@
shuRegisterFailedTest "${strMessage}"
else
printf "."
- SHU_TEST_SUCCEEDED=${SHU_TRUE}
fi
}

@@ -166,7 +167,6 @@
shuRegisterFailedTest "${strMessage}"
else
printf "."
- SHU_TEST_SUCCEEDED=${SHU_TRUE}
fi
}

[Nobody/Anonymous]

Logged In: NO

This can be corrected by the following patch :

===================================================================
--- shUnit (revision 85)
+++ shUnit (working copy)
@@ -49,7 +49,7 @@

shuRunOneTest() {
strTestToRun=${1}
- SHU_TEST_SUCCEEDED=${SHU_FALSE}
+ SHU_TEST_SUCCEEDED=${SHU_TRUE}

if type shuSetUp > /dev/null 2>&1
then
@@ -88,6 +88,7 @@
fi
}

+SHU_RETURN_VALUE=0
shuStart() {
strInitFunction="${1}"

@@ -107,6 +108,7 @@
shuTestNbr=`expr ${shuTestNbr} + 1`
printf " Test %i: %s " ${shuTestNbr} "${STR_TEST}"
shuRunOneTest ${STR_TEST}
+ [ ${SHU_TEST_SUCCEEDED} = ${SHU_FALSE} ] && SHU_RETURN_VALUE=1
printf "\n"

if test -n "${SHU_STR_FAILED}"
@@ -152,7 +154,6 @@
shuRegisterFailedTest "${strMessage}"
else
printf "."
- SHU_TEST_SUCCEEDED=${SHU_TRUE}
fi
}

@@ -166,7 +167,6 @@
shuRegisterFailedTest "${strMessage}"
else
printf "."
- SHU_TEST_SUCCEEDED=${SHU_TRUE}
fi
}