We develop software for medical devices and typically use the QNX operating system. For those who are unfamiliar with QNX, it supports multi-process functionality using secure message passing to allow one process to provide data to another process.
I've only been looking at cppUnit for a short time, so this question may be premature. So far, I haven't seen how cppUnit allows one to create tests when part of the design requires messages to be sent or received. When it comes to self-contained logic, the capabilities of cppUnit are clear.
So, is there an example somewhere that discusses testing of larger systems with multi-process / multi-tasking architectures? Automation is the way we want to go, but not at the expense of having to create addition testing framework no currently provided.
Any hints/help would be appreciated.
Chris
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
That's the beauty and fun with testing- you need to make it self-contained logic (as I understand it, anyway). Basically you'd "Mock Up" a couple of systems, then send a message from one to the other and make sure that it arrived. If you're testing the actual network layer, then you'd probably need to put in reflection or put in two NICs with two different IP addresses...
I don't have any "real" info on this for you, but the good people on the comp.software.extreme-programming newsgroup likely could answer a question or two.
Good luck. Getting a testing harness in place has made my life so much easier. :)
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We use cppunit to test modules that make up real-time C++ multithreaded CORBA based software components. We've found a number of things that help us build an effective unit test framework:
- building our application out of many smaller source libraries and creating isolatable unit test libraries that correspond to each of these source libraries, and aggregating these into top level executables that run all unit tests.
- learning (experience ... experience ... experience ...) how to design for unit testing, and how to employ techniques in the application code (e.g. event/observation, singleton patterns; higher level partitions and namespaces, etc), and techniques in the unit test code (e.g. framework helpers, inherited unit test cases, etc); that make unit testing easier.
- deciding not to apply cppunit retrospectively to older unit tests (a waste of development time), but either (a) leaving old unit tests as they were, or (b) wrapping old unit tests [sometimes not entirely elegantly] into cpplib unit tests. we mandate that all new code committed into our engineering source base have working unit tests with appropriate coverage.
- mandating that packages in our common library space conform to exporting unit test suite handlers and runners so that new unit test libraries can automatically be composed into higher level unit test libraries.
- developing our NT and Solaris build environemnts to have test targets and executable unit tests (made up out of all appropriate unit test libraries corresponding to a particular executable).
- understanding our entire testing strategy (unit tests, component tests, system tests) and how each of these contribute to our overall coverage and to ensure that we spend our efforts wisely.
- we have developers using different testing strategies (e.g. XP style unit test first), but we ensure that our common engineering department policy does not prohibit different styles and approaches: it only mandates what should happen when code becomes integrated into the overall system.
- we using omniORB for threading & distributed (CORBA) services. we have abstractions (e.g. C++ scheduler classes) that allow us to quickly build app and test code - and our unit tests include multithead, CORBA and stress tests as well (e.g. , creating mock CORBA objects, creating multithreaded unit tests that run simultaneously to stress interfaces, and sometimes using probablistic activities)
- we create a myriad of unit tests for various purposes including those that measure execution time/space performance - especially for some of our debugging/instrumentation code; and we find it useful to execute these unit tests manually with an analysis tool (e.g. Rational Quantity) to help understand areas to optimise if need be.
Hope that helps!
Matthew.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
We develop software for medical devices and typically use the QNX operating system. For those who are unfamiliar with QNX, it supports multi-process functionality using secure message passing to allow one process to provide data to another process.
I've only been looking at cppUnit for a short time, so this question may be premature. So far, I haven't seen how cppUnit allows one to create tests when part of the design requires messages to be sent or received. When it comes to self-contained logic, the capabilities of cppUnit are clear.
So, is there an example somewhere that discusses testing of larger systems with multi-process / multi-tasking architectures? Automation is the way we want to go, but not at the expense of having to create addition testing framework no currently provided.
Any hints/help would be appreciated.
Chris
That's the beauty and fun with testing- you need to make it self-contained logic (as I understand it, anyway). Basically you'd "Mock Up" a couple of systems, then send a message from one to the other and make sure that it arrived. If you're testing the actual network layer, then you'd probably need to put in reflection or put in two NICs with two different IP addresses...
I don't have any "real" info on this for you, but the good people on the comp.software.extreme-programming newsgroup likely could answer a question or two.
Good luck. Getting a testing harness in place has made my life so much easier. :)
We use cppunit to test modules that make up real-time C++ multithreaded CORBA based software components. We've found a number of things that help us build an effective unit test framework:
- building our application out of many smaller source libraries and creating isolatable unit test libraries that correspond to each of these source libraries, and aggregating these into top level executables that run all unit tests.
- learning (experience ... experience ... experience ...) how to design for unit testing, and how to employ techniques in the application code (e.g. event/observation, singleton patterns; higher level partitions and namespaces, etc), and techniques in the unit test code (e.g. framework helpers, inherited unit test cases, etc); that make unit testing easier.
- deciding not to apply cppunit retrospectively to older unit tests (a waste of development time), but either (a) leaving old unit tests as they were, or (b) wrapping old unit tests [sometimes not entirely elegantly] into cpplib unit tests. we mandate that all new code committed into our engineering source base have working unit tests with appropriate coverage.
- mandating that packages in our common library space conform to exporting unit test suite handlers and runners so that new unit test libraries can automatically be composed into higher level unit test libraries.
- developing our NT and Solaris build environemnts to have test targets and executable unit tests (made up out of all appropriate unit test libraries corresponding to a particular executable).
- understanding our entire testing strategy (unit tests, component tests, system tests) and how each of these contribute to our overall coverage and to ensure that we spend our efforts wisely.
- we have developers using different testing strategies (e.g. XP style unit test first), but we ensure that our common engineering department policy does not prohibit different styles and approaches: it only mandates what should happen when code becomes integrated into the overall system.
- we using omniORB for threading & distributed (CORBA) services. we have abstractions (e.g. C++ scheduler classes) that allow us to quickly build app and test code - and our unit tests include multithead, CORBA and stress tests as well (e.g. , creating mock CORBA objects, creating multithreaded unit tests that run simultaneously to stress interfaces, and sometimes using probablistic activities)
- we create a myriad of unit tests for various purposes including those that measure execution time/space performance - especially for some of our debugging/instrumentation code; and we find it useful to execute these unit tests manually with an analysis tool (e.g. Rational Quantity) to help understand areas to optimise if need be.
Hope that helps!
Matthew.