MfcTestRunner with xmlOutputter

2009-07-17
2013-04-22
  • Reimann Daniel
    Reimann Daniel
    2009-07-17

    I try to setup an cppunit appliation using the mfc testsuite and the xmlOutputter.
    Everything works fine, but the results are not counted in the xml output. I get only:
    <Statistics>
        <Tests>0</Tests>
        <FailuresTotal>0</FailuresTotal>
        <Errors>0</Errors>
        <Failures>0</Failures>
      </Statistics>

    My main() looks as follows:

    {...
        CppUnit::TestResult controller;
        CppUnit::TestResultCollector result;
        controller.addListener(&result);
        CppUnit::MfcTestRunner runner;
        CppUnit::XmlOutputter xmlOutputter(&result, xmlout);
        xmlOutputter.setStyleSheet("report.xsl");
        runner.addTest(CppUnit::TestFactoryRegistry::getRegistry().makeTest());
        runner.run();
        xmlOutputter.write();
        return TRUE;
    }

    Has anybody some experience with this?
    Thanks a lot for every help!

     
    • Timo
      Timo
      2009-07-22

      Hi Daniel,
      I'm not sure if this is possible with the MFCTestRunner.
      In you current implementation you do nothing with the TestResult and TestResultCollector object to connect it with the TestRunner, that's why it is still empty at the end.

      In theory, this line is wrong:
      CppUnit::XmlOutputter xmlOutputter(&result, xmlout);
      and should be changed to
      CppUnit::XmlOutputter xmlOutputter(&runner.result(), xmlout);
      At least this is needed for the TextTestRunner, but as far I can see, there is no result() method in the MFCTestRunner.

      Cheers
      Timo

       
    • Reimann Daniel
      Reimann Daniel
      2009-07-22

      Hi Timo
      I can't change the line as you suggested. My MfcTestRunner class does not have the method result().
      But anyway - Thanks a lot for your remarks.
      I hope, that there is somebody else who can help.
      I can't imagine, that this combination (suite and xml recording) is so unusual.

      Kind regards,
      Daniel

       
      • Timo
        Timo
        2009-07-22

        Hi Daniel,

        that's exactly what I said. :-)
        This won't work with the current version of MfcTestRunner.
        You could of course extend this.

        Anyway: Why do you need that?
        Normally the XML output is needed for continuous integration. Therefore you don't want to have a GUI that needs user interaction. Commandline is everything what you need.

        Cheers
        Timo

         
    • Reimann Daniel
      Reimann Daniel
      2009-07-23

      Hi Timo

      So, now I understand you.
      Probably the best idea to write my own methods.

      Why I need this - I have following situation.

      The project I write tests for it is pretty huge.
      The goal is to have hundreds of testcases which are structured in various hierarchies.
      These testcases must get executed from different peoples from different departments.
      (Sometimes some releases does not need all testcases.)
      The people who run tests don't want to change the code.
      Therefore, the cppunit is exactly the right thing.

      The xml logging is not essential, you can just look at the progress bar and say OK if it is green.
      But it would be a nice tool for documentation ((silly, but non technical people like this)) and for information exchange in case of failed tests.

      Cheers,
      Daniel

       
      • Timo
        Timo
        2009-07-23

        Hi Daniel,

        okay, now I get it. We had a similar problem. IMHO I think execution of unit tests by non-developers is bullshit.
        Unit tests are tests written by developers to verify the stability/functionality of  their own work.
        Please point your management also to http://en.wikipedia.org/wiki/Unit_testing
        The first sentence says already enough ("...in which a programmer tests..."). :-)

        Let's face it: If one of 500 unit test fails, what does this say to a person that has no knowledge about the code and the specific test? - This might be a very basic test which - if it fails - heavily influences the functionality of a software system or on the other hand it might be a feature that is currently under development (test-driven development) and not in use.

        Just my 2 cents
        Timo

         
    • Reimann Daniel
      Reimann Daniel
      2009-07-27

      Hi Timo

      Yes, I am aware of this and I'm happy to hear from you, that other people had such situations before.
      But I still think that cppunit may get used from somebody which is outside the development. (I think the cppunit mfc gui is already a step in this direction.)

      It's not a problem when an non-developer gets an error. This person must forward the error log to the development and thats it.

      Meanwhile I think it is still the best idea to extend my MfcTestRunner to write the results to the xml output.

      Thanks again for your help.
      Daniel

       
  • Conrad Braam
    Conrad Braam
    2009-11-11

    I'm also in agreement, MFCtestrunner is only useful for people who are going to make changes to the code and have version control etc. I've not used it, and found the XML files just enough for the Continuous Integration trick.
    Your users might want to look at using Cruise Control.NET as a GUI, you get to automate everything, users get to see just the results - no visibility of the code needed at all. I used it with CPPUnit, just needed o do an xml transformation for the XML to tel Cruise Control understand the wierd xmllogger doc.