Work at SourceForge, help us to make it a better place! We have an immediate need for a Support Technician in our San Francisco or Denver office.

Close

#29 add ignore Known_problems to build options

closed
nobody
None
5
2010-08-15
2010-07-17
Chris Marshall
No

To improve development, I would like to see bug reports
accompanied by new tests in PDL/t as much as possible.
However, that results in failed builds by default until the
bugs are fixed.

I propose an option (environment variable and/or command
line flag) that would skip tests corresponding to ones in the
Known_problems list. That would allow the default to be
testing for the problems but enable easy full builds and
installs of development versions without manual intervention.

Discussion

  • David Mertens
    David Mertens
    2010-07-17

    Or, you could just require such tests to be marked as TODO instead of actual tests. Maybe I should do that with the failing FlexRaw tests...

     
  • Chris Marshall
    Chris Marshall
    2010-07-19

    Setting the SKIP_KNOWN_PROBLEMS environment variable
    before running tests can be used to signal PDL tests in the
    suite corresponding to sf.net bugs in Known_problems. Nothing
    is automated but the skips are in place for the relevant tests
    in the PDL git development code.

    Eventually, we could organize this better and even extract the
    Known_problems tests from the file or sf.net. An easy way
    to add support for these cases would be nice. We'll use a
    manual approach in the meantime with documentation to users
    and developers about the goal of having a test for every bug
    report ticket on sf.net.

     
  • Chris Marshall
    Chris Marshall
    2010-07-19

    • status: open --> pending
     
  • Chris Marshall
    Chris Marshall
    2010-08-15

    • status: pending --> closed
     
  • Chris Marshall
    Chris Marshall
    2010-08-20

    Hmm, setting the problems as TODO would be a partial solution. However, it is not explicit enough that it is a "Known Problem" as on sf.net with a bug report and all. I think it is important that tests fail when things are broken. TODO means that the tests don't fail so you never actually have a hard pass/fail condition.