utPLSQL / Feature Request Archive (Use GitHub now) / #1 Add overall summary for test suite

Paul Walker - 2013-07-30

The OUnit front-end gives you a nice big green tick if everything passes (or a big red cross if one or more fail).

This question was obviously asked a long time ago, so I don't know if OUnit existed then!

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2013-10-12

Status: open --> Open

Priority: 5 --> Medium
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2013-10-12

Fixed in version: --> [none]
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2013-10-25

It'd be good if we could get the actual output from running a test suite added to this Ticket for reference so we can see what the current output looks like...

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Svyatoslav Chatchenko - 2014-04-06

I think it is useful. I'd like to try implement this. In fact, I already have a brief implementation for Output reporter, using utoutput.extract and utoutput.replace.

Attachements:
testsuite_current_output.txt - actual output from Output Reporter.
testsuite_output_with_overall.txt - output of mine brief implementation. Difference is "suite success" of "suite failure" in the beginning.

testsuite_current_output.txt

testsuite_output_with_overall.txt

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2014-04-06

That looks good. As well as having the SUITE SUCCESS in large block caps, could you also put it as a normal line underneath it, the same as it does for the normal tests? (I've attached a modified version of your attachment showing what I mean at line 12).

What would you show if there were one or more failures and/or errors? Would you list the package names?

(By the way, I've added you as a developer on this project and assigned the ticket to you)

Last edit: Paul Walker 2014-04-06

testsuite_output_with_overall_2.txt

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2014-04-06

assigned_to: Svyatoslav Chatchenko
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2014-04-06

status: Open --> Verified
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Svyatoslav Chatchenko - 2014-04-18

Changes description:

Added new method to reporter interface before_suite_results, called only for suite execution.

Added before_suite_results implementation for OUTPUT reporter. Display overall banner and suite execution statistics.

File reporter also display suite overall banner and execution statistics, as it uses OUTPUT reporter's method.

Added before_suite_results implementation for HTML reporter. Display overall banner. (Without statistics for now)

Fixed bug: missed call of 'utreport.open' As result, HTML reporter generated file without opening '' tag, '' and '<title>' sections and opening '<BODY>' tag.</li> <li>Fixed bug: missed call of <code>utreport.close</code> for suite execution. As result, FILE reporter and HTML reporter couldn't close file.</li> </ul> Couple things to go: <ul> <li>Bug to fix: HTML reporter always show package name from last test execution. I will create a bug ticket.</li> <li>Enhancement: suite execution statistics should be collected outside of OUTPUT reporter. This will make it accessible for others reporters.</li> <li>To think: Current implementation uses ut_package.last_run_id to show packages results. It means, that we suppose that all last runs were done during current suite execution. Normally it is true.</li> <li>Enhancement: when we call utreport.show without parameter, OUTPUT reporter shows package name. when we call utreport.show passing run_id, OUTPUT reporter shows <code>RUNID</code> instead of package name. I think reporter should always show package name or package name + RUNID.</li> <li>Enchancement: implement order for package results. Same as in <a href="https://sourceforge.net/p/utplsql/bugs/33/">ticket #33</a></li> </ul> Attachements: source.patch - patch outputreporter_success.log, outputreporter_failure.log - result log for output reporter filereporter_success.log, filereporter_failure.log - result log for file reporter htmlreporter_success.html, htmlreporter_failure.html - result log for html reporter</title>

Last edit: Svyatoslav Chatchenko 2014-04-18

filereporter_failure.log

filereporter_success.log

htmlreporter_failure.html

htmlreporter_success.html

outputreporter_failure.log

outputreporter_success.log

source.patch
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Svyatoslav Chatchenko - 2014-05-30

To think: Current implementation uses ut_package.last_run_id to show packages results. It means, that we suppose that all last runs were done during current suite execution. Normally it is true.

I found a case when it isn't true. It happens when testing package cannot be found. Error PLS-00201: identifier '...' must be declared
I'm gonna think about it and fix it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2014-05-31

I've made a branch in SVN and applied the "source.patch" from above as [r174]

Related

Commit: [r174]

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Svyatoslav Chatchenko - 2014-07-23

I did some changes.
Chagelog

There was a bug that I mentioned in previous post. Steps to reproduce: run suite that will succeed; delete ut_teardown from test package; run suite again. As result you will get "success'. The reason for that is when ut_teardown raise exception there is no update of ut_package. Fix in utplsql package.

Moved statistic compute from outputreporter to utresult.get_suite_stats. Now it can be used by other reporters.

Some enhancements in utoutputreporter:
3.1. Always shows packagename and runID. To implement this I added function name_from_last_run_id to utpackage.
3.2. Always shows owner of package. It is very usefull as suite can have packages from different owners or even packages with same name from different owners (ut_package.add(... owner_in ), ut_package.owner). To implement this i added function owner_from_last_run_id to utpackage.

Now showsuite shows individual packages results in same order as packages were executed.

Attachements:
patch_to_revision_175.patch - patch
result_success.log - example of log produced by outputreporter
result_failure.log - example of log produced by outputreporter

Other:

As i already mentioned, current implementation is based on usage of ut_package.last_run_id. Well, i think that normally it is totally fine. So i don't plan change this.

Similar changes as I made to outputreporter can be done to other reporters. But it should be done in others tickets. Reason for that is to reduce scope of changes to release it earlier.

Paul, should I do something else on this ticket or for now my work here is done?

patch_to_revision_175.patch

result_failure.log

result_success.log
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Svyatoslav Chatchenko - 2014-07-24

3.1. Always shows packagename and runID. To implement this I added function name_from_last_run_id to utpackage.

Pardon, this is a little bit changed implementation of feature-requests-7.

Should I leave it here, or extract from this patch and attach in feature-requests-7 ?

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2014-07-24

Thanks for your work on this. Leave it with me for now and I'll have a look over it.

Will this fully address [feature-requests:#7]?

Related

Feature Request Archive (Use GitHub now): #7

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2014-07-24

assigned_to: Svyatoslav Chatchenko --> Paul Walker
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Svyatoslav Chatchenko - 2014-07-25

Will this fully address [feature-requests:#7]?

Yes.

Related

Feature Request Archive (Use GitHub now): #7

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2015-02-08

I've updated the documentation to reflect the new before_suite_results procedure. Committed to the branch as [r181]

Related

Commit: [r181]

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2015-02-08

Merged into trunk in [r182]
Branch deleted in [r183]

Related

Commit: [r182]
Commit: [r183]

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Paul Walker - 2015-02-08

status: Verified --> Closed-Fixed

Fixed in version: [none] --> [next]
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:

Add overall summary for test suite

Fixed in version

Searches

Help

#1 Add overall summary for test suite

Related

Discussion

Related

Related

Related

Related

Related