Re: [Jython-dev] Failing tests on GitHub

[Jeff A. <ja...@fa...>]

I have managed to achieve two green bots. So we have a canary in some 
parts of the mine.

All three Circle bots appear to run exactly the same JVM. Presumably the 
identical configuration is a mistake. (I was suspicious of this, so I 
added a tell to regrtest.py.) Yet one out of three tests passes by chance:

CircleCI 0: (fail test_gc)
      [exec] == 2.7.1rc1 (, Mar 20 2017, 00:16:00)
      [exec] == [OpenJDK 64-Bit Server VM (Oracle Corporation)]
      [exec] == platform: java1.7.0_95
      [exec] == encodings: stdin=UTF-8, stdout=UTF-8, FS=None
      [exec] == locale: default=('en_US', 'UTF-8'), actual=(None, None)
CircleCI 1: (pass)
      [exec] == 2.7.1rc1 (, Mar 20 2017, 00:16:00)
      [exec] == [OpenJDK 64-Bit Server VM (Oracle Corporation)]
      [exec] == platform: java1.7.0_95
      [exec] == encodings: stdin=UTF-8, stdout=UTF-8, FS=None
      [exec] == locale: default=('en_US', 'UTF-8'), actual=(None, None)
CircleCI 2: (fail test_gc)
      [exec] == 2.7.1rc1 (, Mar 20 2017, 00:17:20)
      [exec] == [OpenJDK 64-Bit Server VM (Oracle Corporation)]
      [exec] == platform: java1.7.0_95
      [exec] == encodings: stdin=UTF-8, stdout=UTF-8, FS=None
      [exec] == locale: default=('en_US', 'UTF-8'), actual=(None, None)

Travis is configured to work with 3 genuinely different JVMs:

Travis 1: (pass)
      [exec] == 2.7.1rc1 (, Mar 20 2017, 00:17:49)
      [exec] == [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)]
      [exec] == platform: java1.7.0_80
      [exec] == encodings: stdin=UTF-8, stdout=UTF-8, FS=None
      [exec] == locale: default=('en_US', 'UTF-8'), actual=(None, None)
Travis 2: (fail test_builtin with machine stack dump)
      [exec] == 2.7.1rc1 (, Mar 20 2017, 00:17:48)
      [exec] == [OpenJDK 64-Bit Server VM (Oracle Corporation)]
      [exec] == platform: java1.7.0_121
      [exec] == encodings: stdin=UTF-8, stdout=UTF-8, FS=None
      [exec] == locale: default=('en_US', 'UTF-8'), actual=(None, None)
Travis 3: (fail test_sort)
      [exec] == 2.7.1rc1 (, Mar 20 2017, 00:18:59)
      [exec] == [Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)]
      [exec] == platform: java1.8.0_92
      [exec] == encodings: stdin=UTF-8, stdout=UTF-8, FS=None
      [exec] == locale: default=('en_US', 'UTF-8'), actual=(None, None)

I left test_sort enabled because a fix seemed imminent. However, if I 
were wholly consistent with what I preach here, I'd have made it an 
expected failure in regrtest.py.

Jeff

Jeff Allen