jython-bugs — Private bug notification list. Do not post to this list.

[Jython-bugs] [issue2826] Unicode hex string decode fails with Jython 2.7.2.b3

[Werner F. <re...@bu...>]

New submission from Werner Fouché <wer...@ac...>:

Jython 2.7.2b3 at change https://hg.python.org/jython/rev/159c277c4a80, fails to execute the following code:

import base64

JMSMessageID = u'38303762646637342d386137632d346637632d393364632d'
JMSMessageID_hex = JMSMessageID .decode("hex")
JMSMessageID_b64 = base64.b64encode(JMSMessageID_hex)

print JMSMessageID 
print JMSMessageID_hex
print JMSMessageID_b64

It fails with:

Traceback (most recent call last):
  File "test.py", line 4, in <module>
    JMSMessageID_hex = JMSMessageID .decode("hex")
  File "C:\jython27\jython-standalone.jar\Lib\encodings\hex_codec.py", line 42, in hex_decode
TypeError: a2b_hex() argument 1 must bytes or unicode, not unicode

Jython 2.5.4.rc1 and Python 2.7.16 produces the following output:

38303762646637342d386137632d346637632d393364632d
807bdf74-8a7c-4f7c-93dc-
ODA3YmRmNzQtOGE3Yy00ZjdjLTkzZGMt

----------
components: Core
messages: 12762
nosy: wfouche2
severity: normal
status: open
title: Unicode hex string decode fails with Jython 2.7.2.b3
versions: Jython 2.7.2

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2826>
_______________________________________

[Jython-bugs] [issue2824] `os.path.abspath()` doesn't add drive letter to absolute paths on Windows (regression)

[Pekka K. <re...@bu...>]

New submission from Pekka Klärck <pe...@ik...>:

E:\>c:\jython2.7.2b2\bin\jython.exe
Jython 2.7.2b2 (v2.7.2b2:b9b60766cabe, Nov 1 2019, 07:46:45)
[Java HotSpot(TM) Client VM (Oracle Corporation)] on java1.8.0_231
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.abspath(r'foo\bar')
'E:\\foo\\bar'
>>> os.path.abspath(r'\foo\bar')
'\\foo\\bar'
>>>

With CPython and with Jython 2.7.0 also the latter result has the drive letter.

----------
messages: 12752
nosy: pekka.klarck
severity: normal
status: open
title: `os.path.abspath()` doesn't add drive letter to absolute paths on Windows (regression)

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2824>
_______________________________________

[Jython-bugs] [issue2822] Add support for Java 11

[Roberto E. <re...@bu...>]

New submission from Roberto Espenica <rob...@gm...>:

When running with Java 11 (OpenJDK Runtime Environment Corretto-11.0.5.10.1) and Jython 2.7.1, 2.7.2 Beta 1 and Beta 2, the below warning is shown:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.python.core.PySystemState (file:/robotframework-3.1.2.jar) to method java.io.Console.encoding()
WARNING: Please consider reporting this to the maintainers of org.python.core.PySystemState
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

----------
components: Core
messages: 12750
nosy: respenica
severity: major
status: open
title: Add support for Java 11
type: behaviour
versions: Jython 2.7.1

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2822>
_______________________________________

[Jython-bugs] [issue2820] Importing non-existing module fails with UnicodeDecodeError if sys.path contains non-ASCII characters

[Pekka K. <re...@bu...>]

New submission from Pekka Klärck <pe...@ik...>:

Noticed this regression when testing Jython 2.7.2b2 on Linux. See the example below for a demonstration. Works fine with Jython 2.7.0 but seems to fail also with Jython 2.7.1.


Jython 2.7.2b2 (v2.7.2b2:b9b60766cabe, Nov 1 2019, 07:46:45) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_201
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append('hyv\xe4')
>>> import re       # existing modules can be imported fine
>>> import nonex    # this should fail with ImportError
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe4 in position 3: unexpected end of data

----------
messages: 12742
nosy: pekka.klarck
severity: normal
status: open
title: Importing non-existing module fails with UnicodeDecodeError if sys.path contains non-ASCII characters

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2820>
_______________________________________

[Jython-bugs] [issue2818] Support for Redis Module for PythonInterpreter

[RK <re...@bu...>]

New submission from RK <ram...@gm...>:

We are using PythonInterpreter to interact of Redis script data which was invoked through Python script. And we are not able to import redis module and getting the folowing error. If the PythonInterpreter is not compatible with Redis module can you please check and give us this compatability .

Traceback (most recent call last):
  File "<string>", line 4, in <module>
ImportError: No module named redis

Sample piece of code which give the above error:
public static void main(String[] args) {
        try {
            Properties properties = new Properties();
            properties.setProperty("python.home", "/Users/ranumola/jython2.7.1/");
            properties.setProperty("python.path", "/Users/ranumola/jython2.7.1/Lib");
            properties.setProperty("python.import.site", "false");
            PythonInterpreter.initialize(System.getProperties(), properties, new String[] {""});
            PythonInterpreter python = new PythonInterpreter();
            String script = "# Python3 program to add two numbers \n" + "  \n" + "num1 = 15\n" + "num2 = 12\n" + "  \n" + "# Adding two nos \n"
                    + "sum = num1 + num2 \n" + "  \n" + "# printing values \n" + "print(\"Sum of {0} and {1} is {2}\" .format(num1, num2, sum)) ";
            String script1 = "import sys\n" + "sys.path.append(\"/Users/ranumola/jython2.7.1/Lib\")\n" + "\n" + "import redis\n"
                    + "r = redis.Redis(host='localhost', port=6379)\n" + "r.get('foo')";
            python.set("script", new PyString(script));
            python.exec(script);
            python.close();
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

----------
components: Core, Library
messages: 12728
nosy: rkanumola
severity: normal
status: open
title: Support for Redis Module for PythonInterpreter
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2818>
_______________________________________

[Jython-bugs] [issue2816] Python Interpreter which uses Python and Redis script gives an error

[RK <re...@bu...>]

New submission from RK <ram...@gm...>:

Hi Team,

We are trying to use Jython with Redis script within confluence using a Jython macro.
We are able to execute the action using Jython from the command line and able to read the variable from redis database and get the result, but when we try to use the PythonInterpreter class from within Java application we get the below error. It would be great can suggest ,how to handle this error.

Traceback (most recent call last):
line 4, in
File “/opt/jython/Lib/site-packages/redis/ init .py”, line 1, in
from redis.client import Redis, StrictRedis
SyntaxError: (“no viable alternative at input ‘’’‘“, (‘/opt/jython/Lib/site-packages/redis/client.py’, 28, 13, ‘’))

Please let us know if any thing is not clear or need more information to give a solution to this problem.

----------
components: Jythonc compiler
messages: 12714
nosy: rkanumola
severity: normal
status: open
title: Python Interpreter which uses Python and Redis script gives an error
type: behaviour
versions: Jython 2.5

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2816>
_______________________________________

[Jython-bugs] [issue2814] maven/build.xml does not PGP-sign the publication

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

In the build, since the pgp command is not necessarily available, and this breaks the build, we have removed the commands that use it to sign the artefacts. However, that's necessary for Sonatype to publish (or an equivalent signing process?).

We should restore the pgp steps, either making them optional or fail-soft.

----------
components: Any
messages: 12708
nosy: fwierzbicki, jeff.allen
priority: normal
severity: normal
status: open
title: maven/build.xml does not PGP-sign the publication
type: behaviour

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2814>
_______________________________________

[Jython-bugs] [issue2812] Failure in test_sort.TestBase.testStressfully on Java 11

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

This is showing up intermittently in the CI on Java 11:

    [exec] test test_sort failed -- Traceback (most recent call last):
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/test_sort.py", line 123, in testStressfully
     [exec]     s.sort()
     [exec]   File "/home/travis/build/jythontools/jython/dist/Lib/test/test_sort.py", line 123, in testStressfully
     [exec]     s.sort()
     [exec] IllegalArgumentException: java.lang.IllegalArgumentException: Comparison method violates its general contract!

I am raising the issue and will reference it in a skip as I don't want to divert from the beta release. We made an attempt to handle this by modifying the test, but it was not fully effective and I don't think we've understood it correctly. (I'll revert that.)

The test fails where we supply a pathological comparison function. I *think* all that is being tested is that in those cases, any error is handled and leaves the list unsorted. However, Java (in later versions) detects that the comparator is badly behaved and raises its own error that is not handled.

See #2399 (and https://github.com/jythontools/jython/pull/67) which goes a long way towards solving a related problem in the context of test_list by storing the exception and rethrowing it. An extension to (certain) Java exceptions may be what we need. But also, what about proxied Java lists doing the same?

I don't find the test to be well-designed: it tests too many things at once and I got confused reading it. OTOH it comes to us from CPython and it's nice to use the same source.

----------
components: Core
keywords: test failure causes
messages: 12678
nosy: jeff.allen
priority: normal
severity: normal
status: open
title: Failure in test_sort.TestBase.testStressfully on Java 11
type: crash

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2812>
_______________________________________

[Jython-bugs] [issue2810] NoSuchMethodError in test_jython_initializer Java 10+

[Adam B. <re...@bu...>]

New submission from Adam Burke <ada...@gm...>:

C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e -v test_jython_initializer
== 2.7.2a1+ (uncontrolled:+, Oct. 2 2019, 11:47:29)
== [Java HotSpot(TM) 64-Bit Server VM ("Oracle Corporation")]
== platform: java10.0.2
== encodings: stdin=cp850, stdout=cp850, FS=utf-8
== locale: default=('en_AU', 'windows-1252'), actual=(None, None)
test_jython_initializer
test_syspath_initializer (test.test_jython_initializer.TestUsingInitializer) ... Exception in thread "main" java.lang.NoSuchMethodError: java.nio.ByteBuffer.clear()Ljava/nio/ByteBuffer;
        at org.python.core.io.BufferedReader.clear(BufferedReader.java:147)
        at org.python.core.io.BufferedReader.<init>(BufferedReader.java:27)
        at org.python.core.PyFile.createBuffer(PyFile.java:227)
        at org.python.core.PyFile.file___init__(PyFile.java:185)
        at org.python.core.PyFile.file___init__(PyFile.java:178)
        at org.python.core.PyFile.<init>(PyFile.java:101)
        at org.python.core.PySystemState.<init>(PySystemState.java:237)
        at org.python.core.PySystemState.doInitialize(PySystemState.java:1222)
        at SyspathAppendingInitializer.initialize(SyspathAppendingInitializer.java:15)
        at org.python.core.PySystemState.initialize(PySystemState.java:1171)
        at org.python.core.PySystemState.initialize(PySystemState.java:1102)
        at org.python.core.PySystemState.initialize(PySystemState.java:1085)
        at org.python.core.PySystemState.initialize(PySystemState.java:1080)
        at org.python.core.PySystemState.initialize(PySystemState.java:1075)
        at org.python.util.jython.main(jython.java:531)
FAIL

Current trunk.

Seems to be due to a removed, previously deprecated, method in Java 10.

Currently this is the only regrtest failure on Windows under Java 10+.

----------
components: Core
messages: 12668
nosy: adamburke
severity: normal
status: open
title: NoSuchMethodError in test_jython_initializer Java 10+

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2810>
_______________________________________

[Jython-bugs] [issue2808] lib2to3 test failures on Windows JDK 11

[Adam B. <re...@bu...>]

New submission from Adam Burke <ada...@gm...>:

ib2to3 failure on windows is caused by diffs between CR and CRLF under Java 11


C:\Users\Adam\jython\jython2>dist\bin\jython.exe -m test.regrtest -e test_lib2to3
test_lib2to3
test test_lib2to3 failed -- Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\tests\test_parser.py", line 175, in test_all_project_files
    self.fail("Idempotency failed: {} using {} encoding\n{}".
AssertionError: Idempotency failed: C:\Users\Adam\jython\jython2\dist\Lib\lib2to3\btm_matcher.py using utf-8 encoding
original line 1:
"""A bottom-up tree matching algorithm implementation meant to speed

 differs from:
"""A bottom-up tree matching algorithm implementation meant to speed

Lines differ at char 68: 0d vs 0a (of 70 vs 69)

1 test failed:
    test_lib2to3
1 fail unexpected:
    test_lib2to3


With extra diff debug in Lib/lib2to3/tests/test_parser.py

def diff(fn, result, encoding):
    "A diff the result and original file content independent of OS."
    r = iter(result.encode(encoding).splitlines(True))
    lineNumber = 0
    with open(fn, "rb") as f:
        for line in f:
            lineNumber += 1
            rline = next(r)
            if rline != line:
                return "original line {}:\n{}\n differs from:\n{}\n{}". \
                       format(lineNumber,line,rline,diffLine(line,rline) )
    return False

def diffLine(orig,result):
    charNumber = 0
    for c in orig:
        if c != result[charNumber]:
            return "Lines differ at char {}: {} vs {} (of {} vs {})".format(
                    charNumber,
                    binascii.hexlify(c),
                    binascii.hexlify(result[charNumber]),
                    len(orig),
                    len(result) )
        charNumber += 1

Have a candidate fix but checking a few things. The test code that fails is Jython specific. Python 2.x doesn't run this on Windows, Python 3.x relies on improved string equality (no need for unicode() function).

----------
components: Core
messages: 12664
nosy: adamburke
severity: normal
status: open
title: lib2to3 test failures on Windows JDK 11
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2808>
_______________________________________

[Jython-bugs] [issue2806] Installer contains dependencies unshaded (Java 8)

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

When built with Java 8, although the developer build checks out ok, the installed version of Jython (in fact, the installation itself) fails with the message:

PS 272a-trial> java -jar kit\jython-installer.jar
Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/collect/MapMaker
        at org.python.core.ThreadStateMapping.<clinit>(ThreadStateMapping.java:31)
        at org.python.core.Py.<clinit>(Py.java:1734)
        at org.python.core.PySystemState.<clinit>(PySystemState.java:72)
        at org.python.util.jython.main(jython.java:531)
Caused by: java.lang.ClassNotFoundException: com.google.common.collect.MapMaker
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:583)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:178)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
        ... 4 more

Investigation shows that the call is there are described:

PS jython-trunk> javap -c -cp dist/jython.jar org.python.core.ThreadStateMapping
...
 static {};
   Code:
      0: new           #40                 // class com/google/common/collect/MapMaker

The cause is in the jarjar task in our Ant build that is responsible for renaming dependencies we bundle in the JAR.

When the target JVM is 1.7, the all is well. (I'm still actually building with Oracle Java version 1.8.0_211, but jython.java.version = "1.7".)

PS jython-trunk> ant -D"jython.java.version=1.7" clean installer

(A clean is needed to force recompilation.) In this case, the same investigation shows the call has been renamed correctly:

PS 272a-trial> javap -c -cp inst/jython.jar org.python.core.ThreadStateMapping
...
  static {};
    Code:
       0: new           #179                // class org/python/google/common/collect/MapMaker

Actually, it is only necessary to build the jython.jar (jar-complete target) to see the effect in dist/jython.jar.

I wonder if this is simply a consequence of jarjar 1.4 using an old version of ASM. However, it is not completely obvious where to get the legitimate successor version of jarjar.

Maybe we compile with 8 for this release but target 7 in the generated code?

----------
components: Installer
messages: 12662
milestone: Jython 2.7.2
nosy: jeff.allen
priority: high
severity: major
status: open
title: Installer contains dependencies unshaded (Java 8)
type: crash
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2806>
_______________________________________

[Jython-bugs] [issue2804] Missing class com/google/common/base/CharMatcher

[Werner F. <re...@bu...>]

New submission from Werner Fouché <wer...@ac...>:

jython-standalone.jar build from source at change https://hg.python.org/jython/rev/b180cce20e85 displays the following error at startup:

            Jython 2.7.2a1+
               Exception in thread "main" java.lang.NoClassDefFoundError: com/google/common/base/CharMatcher
                       at org.python.core.Py.fileSystemEncode(Py.java:744)
                       at org.python.core.PySystemState.initRegistry(PySystemState.java:963)
                       at org.python.core.PySystemState.doInitialize(PySystemState.java:1201)
                       at org.python.core.PySystemState.initialize(PySystemState.java:1130)
                       at org.python.core.PySystemState.initialize(PySystemState.java:1085)
                       at org.python.core.PySystemState.initialize(PySystemState.java:1080)
                       at org.python.core.PySystemState.initialize(PySystemState.java:1075)
                       at org.python.util.jython.main(jython.java:531)
               Caused by: java.lang.ClassNotFoundException: com.google.common.base.CharMatcher
                       at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
                       at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
                       at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
                       at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
                       ... 8 more

----------
components: Core
messages: 12660
milestone: Jython 2.7.2
nosy: wfouche2
severity: critical
status: open
title: Missing class com/google/common/base/CharMatcher
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2804>
_______________________________________

[Jython-bugs] [issue2802] PEP 0263 magic coding comment rejected in UTF-8

[Randall W. <re...@bu...>]

New submission from Randall Wood <ran...@al...>:

src/org/python/core/ParserFacade.java rejects any PEP 0263 magic coding comment in a UTF-8 input (lines 314-316). In that case, should it not accept the magic coding comment that indicates the input is UTF-8 (i.e. should the comment "# -*- coding: utf-8 -*-" not be rejected)?

----------
components: Core
messages: 12658
nosy: rhwood
severity: normal
status: open
title: PEP 0263 magic coding comment rejected in UTF-8
type: behaviour
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2802>
_______________________________________

[Jython-bugs] [issue2800] Jython script engine hangs on "genericpath.isfile" during its initialization

[Fabien V. <re...@bu...>]

New submission from Fabien Viale <fab...@ac...>:

>From time to time, the Jython script engine hangs during initialization in our environment.

The attached java stack shows where the jython engine is blocked:

"main" #1 prio=5 os_prio=0 tid=0x00007fc9b400e800 nid=0x116bb runnable [0x00007fc9b8e80000]
   java.lang.Thread.State: RUNNABLE
	at org.python.modules.posix.PosixModule$StatFunction.__call__(PosixModule.java:1273)
	at org.python.core.PyObject.__call__(PyObject.java:465)
	at genericpath$py.isfile$2(/tibco/proactive/dist/lib/jython-standalone-2.7.0.jar/Lib/genericpath.py:32)
	at genericpath$py.call_function(/tibco/proactive/dist/lib/jython-standalone-2.7.0.jar/Lib/genericpath.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:167)
	at org.python.core.PyBaseCode.call(PyBaseCode.java:138)
	at org.python.core.PyFunction.__call__(PyFunction.java:413)
	at sysconfig$py.is_python_build$2(/tibco/proactive/dist/lib/jython-standalone-2.7.0.jar/Lib/sysconfig.py:143)
	at sysconfig$py.call_function(/tibco/proactive/dist/lib/jython-standalone-2.7.0.jar/Lib/sysconfig.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:167)
	at org.python.core.PyBaseCode.call(PyBaseCode.java:124)
	at org.python.core.PyFunction.__call__(PyFunction.java:403)
	at sysconfig$py.f$0(/tibco/proactive/dist/lib/jython-standalone-2.7.0.jar/Lib/sysconfig.py:719)
	at sysconfig$py.call_function(/tibco/proactive/dist/lib/jython-standalone-2.7.0.jar/Lib/sysconfig.py)
	at org.python.core.PyTableCode.call(PyTableCode.java:167)
	at org.python.core.PyCode.call(PyCode.java:18)
	at org.python.core.imp.createFromCode(imp.java:436)
	at org.python.core.util.importer.importer_load_module(importer.java:109)
	at org.python.modules.zipimport.zipimporter.zipimporter_load_module(zipimporter.java:163)
	at org.python.modules.zipimport.zipimporter$zipimporter_load_module_exposer.__call__(Unknown Source)
	at org.python.core.PyBuiltinMethodNarrow.__call__(PyBuiltinMethodNarrow.java:46)
	at org.python.core.imp.loadFromLoader(imp.java:587)
	at org.python.core.imp.find_module(imp.java:537)
	at org.python.core.imp.import_next(imp.java:840)
	at org.python.core.imp.import_module_level(imp.java:959)
	at org.python.core.imp.importName(imp.java:1062)
	at org.python.core.ImportFunction.__call__(__builtin__.java:1280)
	at org.python.core.PyObject.__call__(PyObject.java:431)
	at org.python.core.__builtin__.__import__(__builtin__.java:1232)
	at org.python.core.imp.importFromAs(imp.java:1156)
	at org.python.core.imp.importFrom(imp.java:1132)
...


Additionnally, the standard error of the process shows:
Error in `java': malloc(): memory corruption: 0x00007fcb68beb690

Have you ever encountered before this issue and do you know what could cause it?

----------
components: Core
files: 71348_jstack.log
messages: 12648
nosy: fviale
severity: normal
status: open
title: Jython script engine hangs on "genericpath.isfile" during its initialization
type: crash
versions: Jython 2.7
Added file: https://bugs.jython.org/file1675/71348_jstack.log

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2800>
_______________________________________

[Jython-bugs] [issue2798] OpenJDK 1.7.0_75-b13 string regrtest OutOfMemory on windows

[Adam B. <re...@bu...>]

New submission from Adam Burke <ada...@gm...>:

The regrtest for Jython 2.7.2a+ will show the following failures under OpenJDK 1.7.0_75-b13 on Windows.

Tests are test_bytes test_codecencodings_tw test_lib2to3 test_shadowstr_jy test_str test_string test_sys test_unicode test_userstring

Shows as an OutOfMemoryError:

======================================================================
ERROR: test_replace_overflow (test.test_userstring.MutableStringTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Adam\jython\jython4\dist\Lib\test\string_tests.py", line 893, in test_replace_overflow
    self.checkraises(OverflowError, A2_16, "replace", "", A2_16)
  File "C:\Users\Adam\jython\jython4\dist\Lib\test\test_userstring.py", line 35, in checkraises
    self.assertRaises(
  File "C:\Users\Adam\jython\jython4\dist\Lib\unittest\case.py", line 476, in assertRaises
    callableObj(*args, **kwargs)
  File "C:\Users\Adam\jython\jython4\dist\Lib\UserString.py", line 108, in replace
    return self.__class__(self.data.replace(old, new, maxsplit))
OutOfMemoryError: java.lang.OutOfMemoryError: Java heap space

Found during this work https://github.com/jythontools/jython/pull/132 which also shows passing on Oracle 1.7.0_80-b15 and other versions.

Given the current out of life state of Java 7, we may not necessarily choose to fix this, but I thought it worth documenting here regardless.

----------
components: Core
messages: 12644
nosy: adamburke
severity: normal
status: open
title: OpenJDK 1.7.0_75-b13 string regrtest OutOfMemory on windows
type: crash
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2798>
_______________________________________

[Jython-bugs] [issue2796] Request for oversize arrays not handled in BaseBytes and PyByteArray

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

I noticed this failure in test_bytes, running with 2.7.2a1 standalone JAR:

======================================================================
ERROR: test_repeat (__main__.ByteArrayTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\Users\Jeff\Documents\Jython\272b-trial\Lib\test\test_bytes.py", line 233, in test_repeat
    lambda: b * sys.maxsize)
  File "C:\Users\Jeff\Documents\Jython\272b-trial\kit\jython-standalone.jar\Lib\unittest\case.py", line 476, in assertRaises
    callableObj(*args, **kwargs)
  File "C:\Users\Jeff\Documents\Jython\272b-trial\Lib\test\test_bytes.py", line 233, in <lambda>
    lambda: b * sys.maxsize)
ArrayIndexOutOfBoundsException: java.lang.ArrayIndexOutOfBoundsException: arraycopy: last destination index 2147483646 out of bounds for byte[2147483645]

----------------------------------------------------------------------

Note that the apparent array size 2147483645 is the result of b*sys.maxsize, where len(b) is 3, and a lossy long to int conversion, taking place in org.python.core.BaseBytes.repeatImpl(int). We should be comparing first with sys.maxsize.

In our tests this is usually masked by a MemoryError, but for some reason not in standalone.

Also look elsewhere in BaseBytes and PyByteArray for other places we allocate an array without checking the size.

----------
components: Core
keywords: test failure causes
messages: 12642
nosy: jeff.allen
priority: normal
severity: normal
status: open
title: Request for oversize arrays not handled in BaseBytes and PyByteArray
type: crash
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2796>
_______________________________________

[Jython-bugs] [issue2794] invocation_tests.py missing from modjy tests

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

The target "ant test" fails for several reasons, but they include the tests it performs on our modjy support. I can fix these causes easily except for one, so I'm raising an issue and skipping the failing tests for now.

~/tests/modjy/java/com/xhaus/ModjyTestAppInvocation.java is looking for ~/tests/modjy/test_apps_dir/invocation_tests.py but it has never existed in our codebase. Presumably, it was overlooked at check it in. Definitely, we have not run these tests successfully since https://hg.python.org/jython/rev/76f8e597de04 .

You can run the modjy tests directly (it's a lot quicker) by running ant in ~/tests/modjy. but you have to define (Powershell):

PS modjy> $env:JYTHON_HOME="..\..\dist"
PS modjy> $env:MOCKRUNNER_HOME="..\..\extlibs\mockrunner-0.4.1"

or equivalent in your favourite shell.

----------
components: Library
keywords: test failure causes
messages: 12628
nosy: amak, jeff.allen
priority: normal
severity: normal
status: open
title: invocation_tests.py missing from modjy tests
type: crash
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2794>
_______________________________________

[Jython-bugs] [issue2792] Failures in test_import_jy from Java 13

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

Observed on Linux (my own job on Travis CI) we see two failures in test_import_jy when running on Java 13 or 14:

     [exec] ======================================================================
     [exec] FAIL: test_dunder_init (test.test_import_jy.MislabeledImportTestCase)
     [exec] ----------------------------------------------------------------------
     [exec] Traceback (most recent call last):
     [exec]   File "/home/travis/build/jeff5/jython/dist/Lib/test/test_import_jy.py", line 65, in test_dunder_init
     [exec]     self.assertEquals(module_obj.__file__, init)
     [exec] AssertionError: '/home/travis/build/jeff5/jython/dunder_init_test/__init__.py' != 'dunder_init_test/__init__.py'
     [exec] 
     [exec] ======================================================================
     [exec] FAIL: test_import_star (test.test_import_jy.ImpTestCase)
     [exec] ----------------------------------------------------------------------
     [exec] Traceback (most recent call last):
     [exec]   File "/home/travis/build/jeff5/jython/dist/Lib/test/test_import_jy.py", line 177, in test_import_star
     [exec]     self.assertEquals(0, subprocess.call(
     [exec] AssertionError: 0 != 1
     [exec] 
     [exec] ----------------------------------------------------------------------

Before the second of these, we see:
     [exec] Traceback (most recent call last):
     [exec]   File "/home/travis/build/jeff5/jython/dist/Lib/test/import_star_from_java.py", line 4, in <module>
     [exec]     p = Pattern.compile("foo")
     [exec] NameError: name 'Pattern' is not defined
     [exec] test_import_star (test.test_import_jy.ImpTestCase) ... FAIL

This suggests the import has failed here:
from java.util.regex import *   # Module: java.base
p = Pattern.compile("foo")

I'm pretty sure Pattern is there, so I think this may be the cache playing up again. One line of investigation is that, about Java 13, Oracle note:

Reminder of a change in b24 - A jrt URI can only encode paths to files in /modules tree (JDK-8224946)(https://bugs.openjdk.java.net/browse/JDK-8224946)

A jrt URL is a hierarchical URI with syntax jrt:/[$MODULE[/$PATH]]. When using the jrt file system, a java.net.URI object can be created with the java.nio.file.Path::toUri method to encode a normalized path to a file in the /modules tree. A jrt URL cannot encode a path to a file in the /packages tree. The jrt file system provider has changed in this release so that toUri fails with IOError when it is not possible to encode the file path as a jrt URI. This change may impact tools have been making use of URLs that are not compliant with the syntax. Tools with paths to files in /packages can use the toRealPath() method to obtain the real path (in /modules) before attempting to convert the file path to a URI.

We may as a result be bug-compatible with Java 9 to 12 in the work done on #2362 and #2734. Or it may be something quite different. Or both, as there are 2 failures.

----------
components: Core
messages: 12616
nosy: jeff.allen
priority: normal
severity: normal
status: open
title: Failures in test_import_jy from Java 13
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2792>
_______________________________________

[Jython-bugs] [issue2790] Equivalent signal handlers do not test equal using ==

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

test_signal.BasicSignalTests.test_getsignal has been failing for a while, so I have looked into the reason why two values that look the same are not reported as equal:

     [exec] FAIL: test_getsignal (test.test_signal.BasicSignalTests)
     [exec] ----------------------------------------------------------------------
     [exec] Traceback (most recent call last):
     [exec]   File "/home/travis/build/jeff5/jython/dist/Lib/test/test_signal.py", line 216, in test_getsignal
     [exec]     self.assertEquals(signal.getsignal(signal.SIGHUP), hup)
     [exec] AssertionError: java.lang.Terminator$1@3ce1ca45 != java.lang.Terminator$1@3ce1ca45

The test that produces this runs:

    def test_getsignal(self):
        hup = signal.signal(signal.SIGHUP, self.trivial_signal_handler)
        self.assertEquals(signal.getsignal(signal.SIGHUP),
                          self.trivial_signal_handler)
        signal.signal(signal.SIGHUP, hup)
        self.assertEquals(signal.getsignal(signal.SIGHUP), hup)

and it is the second assertEquals that fails.

The objects in question are different Java objects, both Signal$SunMiscHandler, and proxied by a PyObjectDerived. __eq__ delegates to the proxied objects' comparison, which is Object.equals, and therefore we end up testing object identity. The second call to signal.signal, to re-install the original handler hup, creates a distinct Signal$SunMiscHandler, hence the problem. Its (private) fields are the same, so it has the intended effect, and its toString looks just like the original. But == still yields False.

I will skip this test for now, with a reference to this issue.

Our implementation of signal (see #1074) is somewhat incomplete, and a lot of test_signal is disabled anyway. The present issue is bounded to the __eq__ issue. A general overhaul of signal may be in order, but a point fix would do. (Not marking this for 2.7.2.)

----------
components: Library
keywords: test failure causes
messages: 12614
nosy: jeff.allen
priority: normal
severity: normal
status: open
title: Equivalent signal handlers do not test equal using ==
type: behaviour
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2790>
_______________________________________

[Jython-bugs] [issue2788] utf_8_decode doesn't support memoryview

[Bernie H. <re...@bu...>]

New submission from Bernie Hackett <be...@10...>:

Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_202
Type "help", "copyright", "credits" or "license" for more information.
>>> mm = memoryview("foo")
>>> from codecs import utf_8_decode
>>> utf_8_decode(mm)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: utf_8_decode(): 1st arg can't be coerced to String

Python 2.7.16 (default, Jul  5 2019, 09:06:49) 
[GCC 9.1.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> mm = memoryview("foo")
>>> from codecs import utf_8_decode
>>> utf_8_decode(mm)
(u'foo', 3)

----------
components: Core
messages: 12588
nosy: behackett
severity: normal
status: open
title: utf_8_decode doesn't support memoryview
type: behaviour
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2788>
_______________________________________

[Jython-bugs] [issue2786] Subtracting a Long from None produces TypeError: xxx

[Christopher C. <re...@bu...>]

New submission from Christopher Cooper <chc...@us...>:

>>> 3L - None
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'long' and 'NoneType'
>>> None - 3L
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: xxx
>>> None - 3.0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: unsupported operand type(s) for -: 'NoneType' and 'float'

----------
components: Core
messages: 12580
nosy: chcooper
severity: normal
status: open
title: Subtracting a Long from None produces TypeError: xxx
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2786>
_______________________________________

[Jython-bugs] [issue2784] memoryview accepts unicode strings as input

[Bernie H. <re...@bu...>]

New submission from Bernie Hackett <be...@10...>:

In CPython 2 and 3 and PyPy 2 and 3 memoryview only accepts objects that implement the buffer protocol. unicode / python3 str don't implement the buffer protocol so memoryview doesn't accept them, but memoryview in Jython does.

Jython 2.7.0 (default:9987c746f838, Apr 29 2015, 02:25:11) 
[Java HotSpot(TM) 64-Bit Server VM (Oracle Corporation)] on java1.8.0_202
Type "help", "copyright", "credits" or "license" for more information.
>>> memoryview(u"foobar").tobytes()
'foobar'

Python 3.6.9 (default, Jul 17 2019, 12:02:47) 
[GCC 9.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> memoryview(u"foobar").tobytes()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: memoryview: a bytes-like object is required, not 'str'
>>> 

Python 2.7.16 (default, Mar  6 2019, 09:58:35) 
[GCC 8.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> memoryview(u"foobar").tobytes()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: cannot make memory view because object does not have the buffer interface

----------
components: Core
messages: 12578
nosy: behackett
severity: normal
status: open
title: memoryview accepts unicode strings as input
type: behaviour
versions: Jython 2.7

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2784>
_______________________________________

[Jython-bugs] [issue2782] LFI

[Tugcan O. <re...@bu...>]

New submission from Tugcan Ozel <tug...@gm...>:

while I download jython 2.7 for my burp plugin i see filepath paramater in GET requests and I tried LFI payload and i get picture that in attachments 
payload descp: double encoding and null paramater
payload:https://search.maven.org/remotecontent?filepath=%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%25%5c..%  25%5c..%25%5c..%00

----------
components: website
files: Screenshot from 2019-07-01 11-08-30.jpg
messages: 12570
nosy: forsa41
severity: critical
status: open
title: LFI
type: security
Added file: https://bugs.jython.org/file1673/Screenshot from 2019-07-01 11-08-30.jpg

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2782>
_______________________________________

[Jython-bugs] [issue2780] Update ctypes/jffi to Python 2.7 and test

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

Our implementation of ctypes sits on top of jffi. We upgrade the JARs occasionally, but unfortunately, the code on our side is 10 years old. Except in a few of the source files, I have not found a shred of documentation.

ctypes doesn't get exercised in the regression tests, because test_ctypes does not find a module _ctypes. If I dig out ctypes.tests.runtests specially from lib-python, it fails in a way that suggests ctypes moved on from Python 2.5 and ours didn't. So I think the fix is to enable test_ctypes for Jython, then work off the failures.

I bring this up because of https://github.com/jythontools/jython/issues/147, which we should fix at the same time. If it had been tested, I would have noticed the bug when futzing around in my Chinese persona.

As this is a late arrival, and we've managed so far, I'm inclined to leave it to 2.7.3.

----------
components: Library
messages: 12562
nosy: jeff.allen
priority: normal
severity: normal
status: open
title: Update ctypes/jffi to Python 2.7 and test
type: behaviour

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2780>
_______________________________________

[Jython-bugs] [issue2778] Use Java logging framework

[Jeff A. <re...@bu...>]

New submission from Jeff Allen <ja...@fa...>:

We should use a standard way to issue log messages, preferably plain Java logging.

Motivation:

1. In several pieces of work over the last year or so I have wanted to insert logging instructions (Py.writeDebug() etc.) but have been deterred from doing so, or from leaving them in place for production, because the cost of creating the messages is borne whether they be issued or no. Modern frameworks are lazy: the message is formatted only if it will be emitted.

2. Start-up time, and performance generally, is a worry, and I think it would help us get a grip on it to be able to issue messages with a time stamp, probably at millisecond resolution. We can easily turn this on using Java logging.

3. Using a standard approach gives our users a control mechanism they (probably) already understand if their application uses logging at all.

Implementation:

I propose to use the plain Java logging framework since it does not add another JAR, appears adequate for our needs, and messages can be diverted by application builders into whatever enterprise framework they use.

We can introduce this inside the current logging methods (Py.writeComment() etc.) with minimal code change, then progressively introduce Java logging in place of such calls to get the laziness advantage. I believe we can arrange to produce messages by default only when we currently produce them. There would be differences in detail in the format and names for levels, and that they would emerge by default on syserr.

----------
components: Core
messages: 12560
nosy: jeff.allen
priority: normal
severity: normal
status: open
title: Use Java logging framework
type: rfe

_______________________________________
Jython tracker <re...@bu...>
<https://bugs.jython.org/issue2778>
_______________________________________