We have implemented a new feature in the Mantis product and we wonder if
this feature could interest enough Mantis users to justify an integration of
this feature in the official Mantis delivery.
Here is in a few words a description of this feature we may call "test sheet
User's need description:
We develop software products and we need to be able to compute the quality
of these products, mainly to prove to our customers that the delivered
software fulfils the required quality criteria (described in our quality
plan). In order to measure this software quality, we define a number of
tests that must be passed with the considered software version. Once these
tests have been passed, the quality of the software can be computed in the
Software quality (%) = (Number of "OK" tests / Number of passed tests) * 100
A test is said to be "OK" if no (major) problem has been encountered during
this test's execution. In order to compute the test status (OK, Not OK,
...), we must be able to link a bug to a test sheet. Therefore, we need a
new field in the bug report to identify the related test sheet.
We can also easily compute the test progress:
Test progress (%) = (Number of passed tests / Total number of defined tests)
To implement this feature, we have added the following functions:
- new item in Mantis main menu: "Create test sheet"
This item allows the administrator to create test sheets.
- new field in the bug report form: "Test name"
This field allows to link a bug to a test sheet. The test status will be
deduced from the related bug(s) status.
- new item in Mantis main menu: "View test sheet"
This item allows to view the list of the test sheets and to edit one of them
(looks alike the bug list displayed by the "View bugs" item). When a tester
has passed a test, he must fill the corresponding test sheet, i.e. enter his
name and the name of the test site where he passed the test.
The different possible values for the test status are the following:
NP (Not passed)
NOK (Not OK)
- new item in the "Quality" sub-menu: "Quality criteria"
This item allows to compute and display our main quality indicators (for
reporting purpose): test progress, global software quality, number of opened
blocking or major problems, ...
If you wish to get more information about this new feature we implemented, I
can send to you a detailed specification (only in French for the moment,
sorry, translation in progress). I have also updated Mantis 0.17.2 user
documentation for our future users (English version this time).
As an indication of the amount of modifications needed to implement the
"test sheet management" feature, we have added three tables (one for test
sheets, one for test sites, the last one for test history) and modified one
table (mantis_bug_table as you can guess) in Mantis database. In terms of
software code, we have added 14 php files and modified about 25 existing
Note that we have made our modifications starting from a non official Mantis
0.18 version, i.e. a snapshot from Mantis from 13/11/2002. In fact, it
seemed to us that it would be easier that way to merge our evolutions with
the first 0.18 official delivery.
If you are a Mantis user or contributor, are you interested in such a
If you are a Mantis administrator or developer, do you think that the
integration of this test management feature in the Mantis bug tracker can be
considered ? Or maybe it could be the purpose of a separate package to be
applied on a Mantis installation ?
PS: Julian, sorry if I did not have the time to check your corrections in
bug_api.php, now you know why :)