[Jython-bugs] [issue2419] List expected failures by OS platform in regrtest.py

[Jeff A. <re...@bu...>]

New submission from Jeff Allen:

We'd like to have clean-running regression tests but have never (?) managed it.

We like using CPython tests by default, with necessary exclusions and variations. With those aims, we have added to regrtest.py a 'java' platform, extending the catalogue (_expectations) of tests it expects to raise unittest.SkipTest, and adding a catalogue (_failures) of tests it expects to fail.

But, the catalogues tagged simply 'java' in regrtest.py do not capture the variation in test success amongst OS platforms. In this scheme, if we want clean-running regression tests for everyone, we have to add to _failures, every test that fails on any OS platform. A recent shot in issue #2393 at getting clean regression tests on Windows ended by making these lists only correct for Windows (maybe just Cygwin). Some code in regrtest.py tries to accommodate OS variation, but it is hard to read and may be CPython-specific. This cannot be maintained reliably from any one OS.

I propose we add optional sections with keys like 'java.nt', where after the dot is taken from os._name, to catalogue those things applicable to that OS (taken from os._name). The key 'java' contains those things applicable to all Jython OSs. It is perhaps only necessary to do this to _failures. The OS-specific sections ought to be short, or something is wrong.

Good idea?

----------
components: Library
keywords: test failure causes
messages: 10417
nosy: adamburke, fwierzbicki, jeff.allen, stefan.richthofer, zyasoft
priority: normal
severity: normal
status: open
title: List expected failures by OS platform in regrtest.py
type: behaviour
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<http://bugs.jython.org/issue2419>
_______________________________________