[Jython-dev] Failing tests on GitHub

[Jeff A. <ja...@fa...>]

A while ago our GitHub repo 
https://github.com/jythontools/jython/commits/master sprouted continuous 
integration tests, thanks (I think) to Darjus. However, these tests 
always fail.

This seems to me to remove their primary value, which is as a tripwire 
to tell us when something has broken, or when a PR would break it. 
That's particularly valuable where tests run on a different platform 
from the developer's regression test.

The tests failing at GitHub (over the various tools and test 
conditions), that don't fail on the (= my Windows) desktop, are:

test_socket test_sort test_ssl

I'm tweaking the expected failures in regrtest to get it to run cleanly 
again under ant on the desktop, as we encourage users to run that. But 
perhaps the target should be to have it run cleanly at GitHub too, 
adding things that don't to the "list of shame".

Is it sensible to add as expected failures, those things that routinely 
fail in the GitHub tests? Or do they fail for a reason we could 
immediately fix by configuration there?

Jeff

-- 
Jeff Allen