jython-dev

[Jython-dev] Unicode user and file names (and v2.7.1)

[Jeff A. <ja...@fa...>]

I've been working on http://bugs.jython.org/issue2356 which I'd like to 
get in 2.7.1 -- it seems rather poor that Jython simply does not run for 
users whose names have an un-American character ;). I know this issue is 
not a blocker in most minds.

I've made pretty good progress by allowing file names to be unicode 
objects more often than they would be in CPython 2, which usually 
returns them as bytes in some encoding that we may not know. I've got 
the launcher to work properly, and straightened the logic in our 
printing of trace-backs and exceptions from Java. Unicode file names 
seems the way to go for Jython because:

 1. Java gives us competently decoded unicode file names, from
    java.io.File, etc.. Re-encoding the result will be a pain (and
    overlooked).
 2. We appear not to have the codec we need ('mbcs'), that CPython
    reports on Windows via sys.getfilesystemencoding().
 3. We do this already. In 2.7.0, os.getcwd() returns unicode if necessary.

Most regression tests pass. However, I'm struggling with test_doctest. 
Problems arise when mixing unicode and bytes when one byte is 128 and 
over. This happens in ''.join(list) and formatted output like "%s %s" % 
(ustr, bstr). The behaviour of these is identical with CPython: they 
raise UnicodeDecodeError because the bytes are promoted to characters 
with a strict ascii interpretation. This happens a lot in doctest.py and 
traceback.py, for example, where file paths and stack dumps that include 
them, are now frequently unicode, while other inputs are byte data 
containing file paths presented in the console encoding.

I can beat this into submission with enough customisation of the stdlib 
modules, but that always makes me uncomfortable. I usually see that as a 
hint that user code might also need to change. This may be unfounded. I 
can probably ensure no impact to users of only ascii paths, and the 
others seem unable to run Jython at all (in the scope of this issue). 
However, I'm seriously wondering if I should pursue the approach where 
file names from Java are re-encoded to bytes (maybe as utf-8 
everywhere), but that's grim.

Thoughts?

-- 
Jeff Allen

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jeff A. <ja...@fa...>]

I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty 
well. Rather than just push directly I have published to here:

https://bitbucket.org/tournesol/jython-utf8

I write to ask for a second or third pair of eyes on it. Please tell me 
you can see it and whether it breaks things you care about.

I touched a lot of files in the core and import system: quite a lot of 
tricky stuff with loaders and search paths has been adjusted. I think it 
a good sign that I changed hardly anything in the standard library we 
inherit from CPython, that we hadn't already specialised.

By "works pretty well" above, I mean that the regression tests run 
cleanly for me when my user name is "Épreuve", where previously Jython 
died horribly. The launcher works from a Chinese user name too, as long 
as I localise Windows to China (CPython 2.7 feature). I can use the 
prompt and runs some tests with that setup, but I can't run the 
regression test yet, and printing a stack dump is fatal, so there's a 
bit more to do for Chinese.

I think this means we have solid support for "latin-1" languages, but 
there are still places where we fatally assume bytes are Unicode code 
points.

Jeff Allen

On 05/04/2017 08:57, Jeff Allen wrote:
> I've been working on http://bugs.jython.org/issue2356 which I'd like to
> get in 2.7.1 -- it seems rather poor that Jython simply does not run for
> users whose names have an un-American character ;). I know this issue is
> not a blocker in most minds.
>
> I've made pretty good progress by allowing file names to be unicode
> objects more often than they would be in CPython 2, which usually
> returns them as bytes in some encoding that we may not know. I've got
> the launcher to work properly, and straightened the logic in our
> printing of trace-backs and exceptions from Java. Unicode file names
> seems the way to go for Jython because:
>
>   1. Java gives us competently decoded unicode file names, from
>      java.io.File, etc.. Re-encoding the result will be a pain (and
>      overlooked).
>   2. We appear not to have the codec we need ('mbcs'), that CPython
>      reports on Windows via sys.getfilesystemencoding().
>   3. We do this already. In 2.7.0, os.getcwd() returns unicode if necessary.
>
> Most regression tests pass. However, I'm struggling with test_doctest.
> Problems arise when mixing unicode and bytes when one byte is 128 and
> over. This happens in ''.join(list) and formatted output like "%s %s" %
> (ustr, bstr). The behaviour of these is identical with CPython: they
> raise UnicodeDecodeError because the bytes are promoted to characters
> with a strict ascii interpretation. This happens a lot in doctest.py and
> traceback.py, for example, where file paths and stack dumps that include
> them, are now frequently unicode, while other inputs are byte data
> containing file paths presented in the console encoding.
>
> I can beat this into submission with enough customisation of the stdlib
> modules, but that always makes me uncomfortable. I usually see that as a
> hint that user code might also need to change. This may be unfounded. I
> can probably ensure no impact to users of only ascii paths, and the
> others seem unable to run Jython at all (in the scope of this issue).
> However, I'm seriously wondering if I should pursue the approach where
> file names from Java are re-encoded to bytes (maybe as utf-8
> everywhere), but that's grim.
>
> Thoughts?
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Darjus L. <da...@gm...>]

Hey Jeff,

It seems your last commit to this branch is of three days ago. Is this
ready for review? BTW, your changes look good to me.
I'm a little hesitant to merge this since we've had an RC and REALLY have
to release 2.7.1 It's miles better than 2.7.0.

Cheers,
Darjus

On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:

> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
> well. Rather than just push directly I have published to here:
>
> https://bitbucket.org/tournesol/jython-utf8
>
> I write to ask for a second or third pair of eyes on it. Please tell me
> you can see it and whether it breaks things you care about.
>
> I touched a lot of files in the core and import system: quite a lot of
> tricky stuff with loaders and search paths has been adjusted. I think it
> a good sign that I changed hardly anything in the standard library we
> inherit from CPython, that we hadn't already specialised.
>
> By "works pretty well" above, I mean that the regression tests run
> cleanly for me when my user name is "Épreuve", where previously Jython
> died horribly. The launcher works from a Chinese user name too, as long
> as I localise Windows to China (CPython 2.7 feature). I can use the
> prompt and runs some tests with that setup, but I can't run the
> regression test yet, and printing a stack dump is fatal, so there's a
> bit more to do for Chinese.
>
> I think this means we have solid support for "latin-1" languages, but
> there are still places where we fatally assume bytes are Unicode code
> points.
>
> Jeff Allen
>
> On 05/04/2017 08:57, Jeff Allen wrote:
> > I've been working on http://bugs.jython.org/issue2356 which I'd like to
> > get in 2.7.1 -- it seems rather poor that Jython simply does not run for
> > users whose names have an un-American character ;). I know this issue is
> > not a blocker in most minds.
> >
> > I've made pretty good progress by allowing file names to be unicode
> > objects more often than they would be in CPython 2, which usually
> > returns them as bytes in some encoding that we may not know. I've got
> > the launcher to work properly, and straightened the logic in our
> > printing of trace-backs and exceptions from Java. Unicode file names
> > seems the way to go for Jython because:
> >
> >   1. Java gives us competently decoded unicode file names, from
> >      java.io.File, etc.. Re-encoding the result will be a pain (and
> >      overlooked).
> >   2. We appear not to have the codec we need ('mbcs'), that CPython
> >      reports on Windows via sys.getfilesystemencoding().
> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
> necessary.
> >
> > Most regression tests pass. However, I'm struggling with test_doctest.
> > Problems arise when mixing unicode and bytes when one byte is 128 and
> > over. This happens in ''.join(list) and formatted output like "%s %s" %
> > (ustr, bstr). The behaviour of these is identical with CPython: they
> > raise UnicodeDecodeError because the bytes are promoted to characters
> > with a strict ascii interpretation. This happens a lot in doctest.py and
> > traceback.py, for example, where file paths and stack dumps that include
> > them, are now frequently unicode, while other inputs are byte data
> > containing file paths presented in the console encoding.
> >
> > I can beat this into submission with enough customisation of the stdlib
> > modules, but that always makes me uncomfortable. I usually see that as a
> > hint that user code might also need to change. This may be unfounded. I
> > can probably ensure no impact to users of only ascii paths, and the
> > others seem unable to run Jython at all (in the scope of this issue).
> > However, I'm seriously wondering if I should pursue the approach where
> > file names from Java are re-encoded to bytes (maybe as utf-8
> > everywhere), but that's grim.
> >
> > Thoughts?
> >
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jeff A. <ja...@fa...>]

Hi Darjus.

On inclusion, I'm happy to go with the community view, as always. On one 
of the related tickets (http://bugs.jython.org/issue1839), Jim said we'd 
get it in if timing allowed and there was some user support.

I'm very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful, 
and we're still making changes, so I assume we're talking about another 
RC first rather than a release?

The UTF-8 work is nearly there, but not quite: one Linux defect to fix, 
as noted on the same issue by James against the "latin-1" version. After 
all the additions in the last couple of weeks (to get full BMP support), 
I'm happy to find from my Linux laptop that it is still the only thing I 
have to do. It looks trivial. I've been unable code at all for a few 
days, so haven't looked into a solution, but now I'm back I expect to 
nail it for us today or tomorrow.

I can, of course, merge all this myself and will. I shared your 
hesitancy initially, hence the fork repository, but it's turned out so 
well I feel it's now low risk, as long as we still have a few days.

I will now dive under the desk and wire up my Linux dev box.

Jeff Allen

On 16/05/2017 21:46, Darjus Loktevic wrote:
> Hey Jeff,
>
> It seems your last commit to this branch is of three days ago. Is this 
> ready for review? BTW, your changes look good to me.
> I'm a little hesitant to merge this since we've had an RC and REALLY 
> have to release 2.7.1 It's miles better than 2.7.0.
>
> Cheers,
> Darjus
>
> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa... 
> <mailto:ja...@fa...>> wrote:
>
>     I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>     well. Rather than just push directly I have published to here:
>
>     https://bitbucket.org/tournesol/jython-utf8
>
>     I write to ask for a second or third pair of eyes on it. Please
>     tell me
>     you can see it and whether it breaks things you care about.
>
>     I touched a lot of files in the core and import system: quite a lot of
>     tricky stuff with loaders and search paths has been adjusted. I
>     think it
>     a good sign that I changed hardly anything in the standard library we
>     inherit from CPython, that we hadn't already specialised.
>
>     By "works pretty well" above, I mean that the regression tests run
>     cleanly for me when my user name is "Épreuve", where previously Jython
>     died horribly. The launcher works from a Chinese user name too, as
>     long
>     as I localise Windows to China (CPython 2.7 feature). I can use the
>     prompt and runs some tests with that setup, but I can't run the
>     regression test yet, and printing a stack dump is fatal, so there's a
>     bit more to do for Chinese.
>
>     I think this means we have solid support for "latin-1" languages, but
>     there are still places where we fatally assume bytes are Unicode code
>     points.
>
>     Jeff Allen
>
>     On 05/04/2017 08:57, Jeff Allen wrote:
>     > I've been working on http://bugs.jython.org/issue2356 which I'd
>     like to
>     > get in 2.7.1 -- it seems rather poor that Jython simply does not
>     run for
>     > users whose names have an un-American character ;). I know this
>     issue is
>     > not a blocker in most minds.
>     >
>     > I've made pretty good progress by allowing file names to be unicode
>     > objects more often than they would be in CPython 2, which usually
>     > returns them as bytes in some encoding that we may not know.
>     I've got
>     > the launcher to work properly, and straightened the logic in our
>     > printing of trace-backs and exceptions from Java. Unicode file names
>     > seems the way to go for Jython because:
>     >
>     >   1. Java gives us competently decoded unicode file names, from
>     >      java.io.File, etc.. Re-encoding the result will be a pain (and
>     >      overlooked).
>     >   2. We appear not to have the codec we need ('mbcs'), that CPython
>     >      reports on Windows via sys.getfilesystemencoding().
>     >   3. We do this already. In 2.7.0, os.getcwd() returns unicode
>     if necessary.
>     >
>     > Most regression tests pass. However, I'm struggling with
>     test_doctest.
>     > Problems arise when mixing unicode and bytes when one byte is
>     128 and
>     > over. This happens in ''.join(list) and formatted output like
>     "%s %s" %
>     > (ustr, bstr). The behaviour of these is identical with CPython: they
>     > raise UnicodeDecodeError because the bytes are promoted to
>     characters
>     > with a strict ascii interpretation. This happens a lot in
>     doctest.py and
>     > traceback.py, for example, where file paths and stack dumps that
>     include
>     > them, are now frequently unicode, while other inputs are byte data
>     > containing file paths presented in the console encoding.
>     >
>     > I can beat this into submission with enough customisation of the
>     stdlib
>     > modules, but that always makes me uncomfortable. I usually see
>     that as a
>     > hint that user code might also need to change. This may be
>     unfounded. I
>     > can probably ensure no impact to users of only ascii paths, and the
>     > others seem unable to run Jython at all (in the scope of this
>     issue).
>     > However, I'm seriously wondering if I should pursue the approach
>     where
>     > file names from Java are re-encoded to bytes (maybe as utf-8
>     > everywhere), but that's grim.
>     >
>     > Thoughts?
>     >
>
>
>     ------------------------------------------------------------------------------
>     Check out the vibrant tech community on one of the world's most
>     engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>     _______________________________________________
>     Jython-dev mailing list
>     Jyt...@li...
>     <mailto:Jyt...@li...>
>     https://lists.sourceforge.net/lists/listinfo/jython-dev
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Darjus L. <da...@gm...>]

Hey Jeff,

Sounds good. Let's do another rc but to be honest I'm not even sure the RC
matters much if there aren't people trying it except us.

Thoughts?
Darjus

On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:

> Hi Darjus.
>
> On inclusion, I'm happy to go with the community view, as always. On one
> of the related tickets (http://bugs.jython.org/issue1839), Jim said we'd
> get it in if timing allowed and there was some user support.
>
> I'm very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful, and
> we're still making changes, so I assume we're talking about another RC
> first rather than a release?
>
> The UTF-8 work is nearly there, but not quite: one Linux defect to fix, as
> noted on the same issue by James against the "latin-1" version. After all
> the additions in the last couple of weeks (to get full BMP support), I'm
> happy to find from my Linux laptop that it is still the only thing I have
> to do. It looks trivial. I've been unable code at all for a few days, so
> haven't looked into a solution, but now I'm back I expect to nail it for us
> today or tomorrow.
>
> I can, of course, merge all this myself and will. I shared your hesitancy
> initially, hence the fork repository, but it's turned out so well I feel
> it's now low risk, as long as we still have a few days.
>
> I will now dive under the desk and wire up my Linux dev box.
>
> Jeff Allen
>
> On 16/05/2017 21:46, Darjus Loktevic wrote:
>
> Hey Jeff,
>
> It seems your last commit to this branch is of three days ago. Is this
> ready for review? BTW, your changes look good to me.
> I'm a little hesitant to merge this since we've had an RC and REALLY have
> to release 2.7.1 It's miles better than 2.7.0.
>
> Cheers,
> Darjus
>
> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>
>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>> well. Rather than just push directly I have published to here:
>>
>> https://bitbucket.org/tournesol/jython-utf8
>>
>> I write to ask for a second or third pair of eyes on it. Please tell me
>> you can see it and whether it breaks things you care about.
>>
>> I touched a lot of files in the core and import system: quite a lot of
>> tricky stuff with loaders and search paths has been adjusted. I think it
>> a good sign that I changed hardly anything in the standard library we
>> inherit from CPython, that we hadn't already specialised.
>>
>> By "works pretty well" above, I mean that the regression tests run
>> cleanly for me when my user name is "Épreuve", where previously Jython
>> died horribly. The launcher works from a Chinese user name too, as long
>> as I localise Windows to China (CPython 2.7 feature). I can use the
>> prompt and runs some tests with that setup, but I can't run the
>> regression test yet, and printing a stack dump is fatal, so there's a
>> bit more to do for Chinese.
>>
>> I think this means we have solid support for "latin-1" languages, but
>> there are still places where we fatally assume bytes are Unicode code
>> points.
>>
>> Jeff Allen
>>
>> On 05/04/2017 08:57, Jeff Allen wrote:
>> > I've been working on http://bugs.jython.org/issue2356 which I'd like to
>> > get in 2.7.1 -- it seems rather poor that Jython simply does not run for
>> > users whose names have an un-American character ;). I know this issue is
>> > not a blocker in most minds.
>> >
>> > I've made pretty good progress by allowing file names to be unicode
>> > objects more often than they would be in CPython 2, which usually
>> > returns them as bytes in some encoding that we may not know. I've got
>> > the launcher to work properly, and straightened the logic in our
>> > printing of trace-backs and exceptions from Java. Unicode file names
>> > seems the way to go for Jython because:
>> >
>> >   1. Java gives us competently decoded unicode file names, from
>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>> >      overlooked).
>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>> >      reports on Windows via sys.getfilesystemencoding().
>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>> necessary.
>> >
>> > Most regression tests pass. However, I'm struggling with test_doctest.
>> > Problems arise when mixing unicode and bytes when one byte is 128 and
>> > over. This happens in ''.join(list) and formatted output like "%s %s" %
>> > (ustr, bstr). The behaviour of these is identical with CPython: they
>> > raise UnicodeDecodeError because the bytes are promoted to characters
>> > with a strict ascii interpretation. This happens a lot in doctest.py and
>> > traceback.py, for example, where file paths and stack dumps that include
>> > them, are now frequently unicode, while other inputs are byte data
>> > containing file paths presented in the console encoding.
>> >
>> > I can beat this into submission with enough customisation of the stdlib
>> > modules, but that always makes me uncomfortable. I usually see that as a
>> > hint that user code might also need to change. This may be unfounded. I
>> > can probably ensure no impact to users of only ascii paths, and the
>> > others seem unable to run Jython at all (in the scope of this issue).
>> > However, I'm seriously wondering if I should pursue the approach where
>> > file names from Java are re-encoded to bytes (maybe as utf-8
>> > everywhere), but that's grim.
>> >
>> > Thoughts?
>> >
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Jython-dev mailing list
>> Jyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Stefan R. <Ste...@gm...>]

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>AFAIK every release happens by having a successful RC that is renamed to &#39;release&#39; after a while. So, per definition another RC is inevitable.</div>

<div>That said, I suppose we should get http://bugs.jython.org/issue2487 fixed before we can release. I guess Jeff&#39;s work will be ready until then. At least that decision can be postponed until an RC is actually doable.</div>

<div>
<div>&nbsp;</div>

<div>&nbsp;</div>

<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Gesendet:</b>&nbsp;Samstag, 20. Mai 2017 um 02:35 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Darjus Loktevic&quot; &lt;da...@gm...&gt;<br/>
<b>An:</b>&nbsp;&quot;Jeff Allen&quot; &lt;ja...@fa...&gt;, &quot;Jython Developers&quot; &lt;jyt...@li...&gt;<br/>
<b>Betreff:</b>&nbsp;Re: [Jython-dev] Unicode user and file names (and v2.7.1)</div>

<div name="quoted-content">
<p>Hey Jeff,</p>

<p>Sounds good. Let&#39;s do another rc but to be honest I&#39;m not even sure the RC matters much if there aren&#39;t people trying it except us.</p>

<p>Thoughts?<br/>
Darjus</p>
&nbsp;

<div class="gmail_quote">
<div>On Fri, May 19, 2017, 1:19 AM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<p>Hi Darjus.</p>

<p>On inclusion, I&#39;m happy to go with the community view, as always. On one of the related tickets (<a class="m_1723547812866307292moz-txt-link-freetext" href="http://bugs.jython.org/issue1839" target="_blank">http://bugs.jython.org/issue1839</a>), Jim said we&#39;d get it in if timing allowed and there was some user support.</p>

<p>I&#39;m very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful, and we&#39;re still making changes, so I assume we&#39;re talking about another RC first rather than a release?</p>

<p>The UTF-8 work is nearly there, but not quite: one Linux defect to fix, as noted on the same issue by James against the &quot;latin-1&quot; version. After all the additions in the last couple of weeks (to get full BMP support), I&#39;m happy to find from my Linux laptop that it is still the only thing I have to do. It looks trivial. I&#39;ve been unable code at all for a few days, so haven&#39;t looked into a solution, but now I&#39;m back I expect to nail it for us today or tomorrow.</p>

<p>I can, of course, merge all this myself and will. I shared your hesitancy initially, hence the fork repository, but it&#39;s turned out so well I feel it&#39;s now low risk, as long as we still have a few days.</p>

<p>I will now dive under the desk and wire up my Linux dev box.</p>
</div>

<div>
<pre class="m_1723547812866307292moz-signature">Jeff Allen</pre>
</div>

<div>
<div class="m_1723547812866307292moz-cite-prefix">On 16/05/2017 21:46, Darjus Loktevic wrote:</div>

<blockquote>
<div>
<div>
<div>
<div>Hey Jeff,<br/>
&nbsp;</div>
It seems your last commit to this branch is of three days ago. Is this ready for review? BTW, your changes look good to me.<br/>
I&#39;m a little hesitant to merge this since we&#39;ve had an RC and REALLY have to release 2.7.1 It&#39;s miles better than 2.7.0.<br/>
&nbsp;</div>
Cheers,</div>
Darjus</div>
&nbsp;

<div class="gmail_quote">
<div>On Mon, May 1, 2017 at 6:34 AM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">I went for sys.getfilesystemencoding() == &#39;utf-8&#39; and it works pretty<br/>
well. Rather than just push directly I have published to here:<br/>
<br/>
<a href="https://bitbucket.org/tournesol/jython-utf8" target="_blank">https://bitbucket.org/tournesol/jython-utf8</a><br/>
<br/>
I write to ask for a second or third pair of eyes on it. Please tell me<br/>
you can see it and whether it breaks things you care about.<br/>
<br/>
I touched a lot of files in the core and import system: quite a lot of<br/>
tricky stuff with loaders and search paths has been adjusted. I think it<br/>
a good sign that I changed hardly anything in the standard library we<br/>
inherit from CPython, that we hadn&#39;t already specialised.<br/>
<br/>
By &quot;works pretty well&quot; above, I mean that the regression tests run<br/>
cleanly for me when my user name is &quot;&Eacute;preuve&quot;, where previously Jython<br/>
died horribly. The launcher works from a Chinese user name too, as long<br/>
as I localise Windows to China (CPython 2.7 feature). I can use the<br/>
prompt and runs some tests with that setup, but I can&#39;t run the<br/>
regression test yet, and printing a stack dump is fatal, so there&#39;s a<br/>
bit more to do for Chinese.<br/>
<br/>
I think this means we have solid support for &quot;latin-1&quot; languages, but<br/>
there are still places where we fatally assume bytes are Unicode code<br/>
points.<br/>
<br/>
Jeff Allen<br/>
<br/>
On 05/04/2017 08:57, Jeff Allen wrote:<br/>
&gt; I&#39;ve been working on <a href="http://bugs.jython.org/issue2356" target="_blank">http://bugs.jython.org/issue2356</a> which I&#39;d like to<br/>
&gt; get in 2.7.1 -- it seems rather poor that Jython simply does not run for<br/>
&gt; users whose names have an un-American character ;). I know this issue is<br/>
&gt; not a blocker in most minds.<br/>
&gt;<br/>
&gt; I&#39;ve made pretty good progress by allowing file names to be unicode<br/>
&gt; objects more often than they would be in CPython 2, which usually<br/>
&gt; returns them as bytes in some encoding that we may not know. I&#39;ve got<br/>
&gt; the launcher to work properly, and straightened the logic in our<br/>
&gt; printing of trace-backs and exceptions from Java. Unicode file names<br/>
&gt; seems the way to go for Jython because:<br/>
&gt;<br/>
&gt;&nbsp; &nbsp;1. Java gives us competently decoded unicode file names, from<br/>
&gt;&nbsp; &nbsp; &nbsp; java.io.File, etc.. Re-encoding the result will be a pain (and<br/>
&gt;&nbsp; &nbsp; &nbsp; overlooked).<br/>
&gt;&nbsp; &nbsp;2. We appear not to have the codec we need (&#39;mbcs&#39;), that CPython<br/>
&gt;&nbsp; &nbsp; &nbsp; reports on Windows via sys.getfilesystemencoding().<br/>
&gt;&nbsp; &nbsp;3. We do this already. In 2.7.0, os.getcwd() returns unicode if necessary.<br/>
&gt;<br/>
&gt; Most regression tests pass. However, I&#39;m struggling with test_doctest.<br/>
&gt; Problems arise when mixing unicode and bytes when one byte is 128 and<br/>
&gt; over. This happens in &#39;&#39;.join(list) and formatted output like &quot;%s %s&quot; %<br/>
&gt; (ustr, bstr). The behaviour of these is identical with CPython: they<br/>
&gt; raise UnicodeDecodeError because the bytes are promoted to characters<br/>
&gt; with a strict ascii interpretation. This happens a lot in doctest.py and<br/>
&gt; traceback.py, for example, where file paths and stack dumps that include<br/>
&gt; them, are now frequently unicode, while other inputs are byte data<br/>
&gt; containing file paths presented in the console encoding.<br/>
&gt;<br/>
&gt; I can beat this into submission with enough customisation of the stdlib<br/>
&gt; modules, but that always makes me uncomfortable. I usually see that as a<br/>
&gt; hint that user code might also need to change. This may be unfounded. I<br/>
&gt; can probably ensure no impact to users of only ascii paths, and the<br/>
&gt; others seem unable to run Jython at all (in the scope of this issue).<br/>
&gt; However, I&#39;m seriously wondering if I should pursue the approach where<br/>
&gt; file names from Java are re-encoded to bytes (maybe as utf-8<br/>
&gt; everywhere), but that&#39;s grim.<br/>
&gt;<br/>
&gt; Thoughts?<br/>
&gt;<br/>
<br/>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world&#39;s most engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot_______________________________________________" target="_blank">http://sdm.link/slashdot_______________________________________________</a> Jython-dev mailing list Jyt...@li... <a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></div>
</div>
</div>
</div></div></body></html>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jim B. <jim...@py...>]

I don't necessarily see http://bugs.jython.org/issue2487 as a blocker, but
it would be nice. It just hasn't come up in real usage, unlike the earlier
iteration of the bug which Darjus hacked around by busy waiting. I did
spend some time on trying to get the publication to work without racing,
using the approach I detail in that bug, but no luck yet. (But mostly
because an utter lack of time to spend on the issue.)

Merging in Jeff's recent work on Unicode is important and we should get it
in. I haven't had a chance to test myself, but given Jeff's amazing
attention to detail, I'm sure it's ready.

The blocker for the RC - because we lost OSX support of setuptools support
of installed executables - is fixed, as I just finally confirmed:
http://bugs.jython.org/issue2570


On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <Ste...@gm...
> wrote:

> AFAIK every release happens by having a successful RC that is renamed to
> 'release' after a while. So, per definition another RC is inevitable.
> That said, I suppose we should get http://bugs.jython.org/issue2487 fixed
> before we can release. I guess Jeff's work will be ready until then. At
> least that decision can be postponed until an RC is actually doable.
>
>
> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
> *Von:* "Darjus Loktevic" <da...@gm...>
> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
> jyt...@li...>
> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>
> Hey Jeff,
>
> Sounds good. Let's do another rc but to be honest I'm not even sure the RC
> matters much if there aren't people trying it except us.
>
> Thoughts?
> Darjus
>
> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>
>> Hi Darjus.
>>
>> On inclusion, I'm happy to go with the community view, as always. On one
>> of the related tickets (http://bugs.jython.org/issue1839), Jim said we'd
>> get it in if timing allowed and there was some user support.
>>
>> I'm very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful,
>> and we're still making changes, so I assume we're talking about another RC
>> first rather than a release?
>>
>> The UTF-8 work is nearly there, but not quite: one Linux defect to fix,
>> as noted on the same issue by James against the "latin-1" version. After
>> all the additions in the last couple of weeks (to get full BMP support),
>> I'm happy to find from my Linux laptop that it is still the only thing I
>> have to do. It looks trivial. I've been unable code at all for a few days,
>> so haven't looked into a solution, but now I'm back I expect to nail it for
>> us today or tomorrow.
>>
>> I can, of course, merge all this myself and will. I shared your hesitancy
>> initially, hence the fork repository, but it's turned out so well I feel
>> it's now low risk, as long as we still have a few days.
>>
>> I will now dive under the desk and wire up my Linux dev box.
>>
>> Jeff Allen
>>
>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>
>> Hey Jeff,
>>
>> It seems your last commit to this branch is of three days ago. Is this
>> ready for review? BTW, your changes look good to me.
>> I'm a little hesitant to merge this since we've had an RC and REALLY have
>> to release 2.7.1 It's miles better than 2.7.0.
>>
>> Cheers,
>> Darjus
>>
>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>>
>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>>> well. Rather than just push directly I have published to here:
>>>
>>> https://bitbucket.org/tournesol/jython-utf8
>>>
>>> I write to ask for a second or third pair of eyes on it. Please tell me
>>> you can see it and whether it breaks things you care about.
>>>
>>> I touched a lot of files in the core and import system: quite a lot of
>>> tricky stuff with loaders and search paths has been adjusted. I think it
>>> a good sign that I changed hardly anything in the standard library we
>>> inherit from CPython, that we hadn't already specialised.
>>>
>>> By "works pretty well" above, I mean that the regression tests run
>>> cleanly for me when my user name is "Épreuve", where previously Jython
>>> died horribly. The launcher works from a Chinese user name too, as long
>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>> prompt and runs some tests with that setup, but I can't run the
>>> regression test yet, and printing a stack dump is fatal, so there's a
>>> bit more to do for Chinese.
>>>
>>> I think this means we have solid support for "latin-1" languages, but
>>> there are still places where we fatally assume bytes are Unicode code
>>> points.
>>>
>>> Jeff Allen
>>>
>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>> > I've been working on http://bugs.jython.org/issue2356 which I'd like
>>> to
>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not run
>>> for
>>> > users whose names have an un-American character ;). I know this issue
>>> is
>>> > not a blocker in most minds.
>>> >
>>> > I've made pretty good progress by allowing file names to be unicode
>>> > objects more often than they would be in CPython 2, which usually
>>> > returns them as bytes in some encoding that we may not know. I've got
>>> > the launcher to work properly, and straightened the logic in our
>>> > printing of trace-backs and exceptions from Java. Unicode file names
>>> > seems the way to go for Jython because:
>>> >
>>> >   1. Java gives us competently decoded unicode file names, from
>>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>>> >      overlooked).
>>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>>> >      reports on Windows via sys.getfilesystemencoding().
>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>> necessary.
>>> >
>>> > Most regression tests pass. However, I'm struggling with test_doctest.
>>> > Problems arise when mixing unicode and bytes when one byte is 128 and
>>> > over. This happens in ''.join(list) and formatted output like "%s %s" %
>>> > (ustr, bstr). The behaviour of these is identical with CPython: they
>>> > raise UnicodeDecodeError because the bytes are promoted to characters
>>> > with a strict ascii interpretation. This happens a lot in doctest.py
>>> and
>>> > traceback.py, for example, where file paths and stack dumps that
>>> include
>>> > them, are now frequently unicode, while other inputs are byte data
>>> > containing file paths presented in the console encoding.
>>> >
>>> > I can beat this into submission with enough customisation of the stdlib
>>> > modules, but that always makes me uncomfortable. I usually see that as
>>> a
>>> > hint that user code might also need to change. This may be unfounded. I
>>> > can probably ensure no impact to users of only ascii paths, and the
>>> > others seem unable to run Jython at all (in the scope of this issue).
>>> > However, I'm seriously wondering if I should pursue the approach where
>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>> > everywhere), but that's grim.
>>> >
>>> > Thoughts?
>>> >
>>>
>>>
>>> ------------------------------------------------------------
>>> ------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Jython-dev mailing list
>>> Jyt...@li...
>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most engaging
> tech sites, Slashdot.org! http://sdm.link/slashdot______
> _________________________________________ Jython-dev mailing list
> Jyt...@li... https://lists.sourceforge.net/
> lists/listinfo/jython-dev
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Darjus L. <da...@gm...>]

Agreed regarding not blocking on 2487. That whole area needs a rewrite and
we could potentially utilize libraries available for Java 8.

Let's get Jeff's work in and do an RC?

Darjus

On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...> wrote:

> I don't necessarily see http://bugs.jython.org/issue2487 as a blocker,
> but it would be nice. It just hasn't come up in real usage, unlike the
> earlier iteration of the bug which Darjus hacked around by busy waiting. I
> did spend some time on trying to get the publication to work without
> racing, using the approach I detail in that bug, but no luck yet. (But
> mostly because an utter lack of time to spend on the issue.)
>
> Merging in Jeff's recent work on Unicode is important and we should get it
> in. I haven't had a chance to test myself, but given Jeff's amazing
> attention to detail, I'm sure it's ready.
>
> The blocker for the RC - because we lost OSX support of setuptools support
> of installed executables - is fixed, as I just finally confirmed:
> http://bugs.jython.org/issue2570
>
>
> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
> Ste...@gm...> wrote:
>
>> AFAIK every release happens by having a successful RC that is renamed to
>> 'release' after a while. So, per definition another RC is inevitable.
>> That said, I suppose we should get http://bugs.jython.org/issue2487
>> fixed before we can release. I guess Jeff's work will be ready until then.
>> At least that decision can be postponed until an RC is actually doable.
>>
>>
>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>> *Von:* "Darjus Loktevic" <da...@gm...>
>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>> jyt...@li...>
>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>
>> Hey Jeff,
>>
>> Sounds good. Let's do another rc but to be honest I'm not even sure the
>> RC matters much if there aren't people trying it except us.
>>
>> Thoughts?
>> Darjus
>>
>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>
>>> Hi Darjus.
>>>
>>> On inclusion, I'm happy to go with the community view, as always. On one
>>> of the related tickets (http://bugs.jython.org/issue1839), Jim said
>>> we'd get it in if timing allowed and there was some user support.
>>>
>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful,
>>> and we're still making changes, so I assume we're talking about another RC
>>> first rather than a release?
>>>
>>> The UTF-8 work is nearly there, but not quite: one Linux defect to fix,
>>> as noted on the same issue by James against the "latin-1" version. After
>>> all the additions in the last couple of weeks (to get full BMP support),
>>> I'm happy to find from my Linux laptop that it is still the only thing I
>>> have to do. It looks trivial. I've been unable code at all for a few days,
>>> so haven't looked into a solution, but now I'm back I expect to nail it for
>>> us today or tomorrow.
>>>
>>> I can, of course, merge all this myself and will. I shared your
>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>> I feel it's now low risk, as long as we still have a few days.
>>>
>>> I will now dive under the desk and wire up my Linux dev box.
>>>
>>> Jeff Allen
>>>
>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>
>>> Hey Jeff,
>>>
>>> It seems your last commit to this branch is of three days ago. Is this
>>> ready for review? BTW, your changes look good to me.
>>> I'm a little hesitant to merge this since we've had an RC and REALLY
>>> have to release 2.7.1 It's miles better than 2.7.0.
>>>
>>> Cheers,
>>> Darjus
>>>
>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>>>
>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>>>> well. Rather than just push directly I have published to here:
>>>>
>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>
>>>> I write to ask for a second or third pair of eyes on it. Please tell me
>>>> you can see it and whether it breaks things you care about.
>>>>
>>>> I touched a lot of files in the core and import system: quite a lot of
>>>> tricky stuff with loaders and search paths has been adjusted. I think it
>>>> a good sign that I changed hardly anything in the standard library we
>>>> inherit from CPython, that we hadn't already specialised.
>>>>
>>>> By "works pretty well" above, I mean that the regression tests run
>>>> cleanly for me when my user name is "Épreuve", where previously Jython
>>>> died horribly. The launcher works from a Chinese user name too, as long
>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>> prompt and runs some tests with that setup, but I can't run the
>>>> regression test yet, and printing a stack dump is fatal, so there's a
>>>> bit more to do for Chinese.
>>>>
>>>> I think this means we have solid support for "latin-1" languages, but
>>>> there are still places where we fatally assume bytes are Unicode code
>>>> points.
>>>>
>>>> Jeff Allen
>>>>
>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd like
>>>> to
>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not run
>>>> for
>>>> > users whose names have an un-American character ;). I know this issue
>>>> is
>>>> > not a blocker in most minds.
>>>> >
>>>> > I've made pretty good progress by allowing file names to be unicode
>>>> > objects more often than they would be in CPython 2, which usually
>>>> > returns them as bytes in some encoding that we may not know. I've got
>>>> > the launcher to work properly, and straightened the logic in our
>>>> > printing of trace-backs and exceptions from Java. Unicode file names
>>>> > seems the way to go for Jython because:
>>>> >
>>>> >   1. Java gives us competently decoded unicode file names, from
>>>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>>>> >      overlooked).
>>>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>>> necessary.
>>>> >
>>>> > Most regression tests pass. However, I'm struggling with test_doctest.
>>>> > Problems arise when mixing unicode and bytes when one byte is 128 and
>>>> > over. This happens in ''.join(list) and formatted output like "%s %s"
>>>> %
>>>> > (ustr, bstr). The behaviour of these is identical with CPython: they
>>>> > raise UnicodeDecodeError because the bytes are promoted to characters
>>>> > with a strict ascii interpretation. This happens a lot in doctest.py
>>>> and
>>>> > traceback.py, for example, where file paths and stack dumps that
>>>> include
>>>> > them, are now frequently unicode, while other inputs are byte data
>>>> > containing file paths presented in the console encoding.
>>>> >
>>>> > I can beat this into submission with enough customisation of the
>>>> stdlib
>>>> > modules, but that always makes me uncomfortable. I usually see that
>>>> as a
>>>> > hint that user code might also need to change. This may be unfounded.
>>>> I
>>>> > can probably ensure no impact to users of only ascii paths, and the
>>>> > others seem unable to run Jython at all (in the scope of this issue).
>>>> > However, I'm seriously wondering if I should pursue the approach where
>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>> > everywhere), but that's grim.
>>>> >
>>>> > Thoughts?
>>>> >
>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> Jython-dev mailing list
>>>> Jyt...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>
>>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most engaging
>> tech sites, Slashdot.org!
>> http://sdm.link/slashdot_______________________________________________
>> Jython-dev mailing list Jyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Jython-dev mailing list
>> Jyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jim B. <jim...@py...>]

+100

On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...> wrote:

> Agreed regarding not blocking on 2487. That whole area needs a rewrite and
> we could potentially utilize libraries available for Java 8.
>
> Let's get Jeff's work in and do an RC?
>
> Darjus
>
> On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...> wrote:
>
>> I don't necessarily see http://bugs.jython.org/issue2487 as a blocker,
>> but it would be nice. It just hasn't come up in real usage, unlike the
>> earlier iteration of the bug which Darjus hacked around by busy waiting. I
>> did spend some time on trying to get the publication to work without
>> racing, using the approach I detail in that bug, but no luck yet. (But
>> mostly because an utter lack of time to spend on the issue.)
>>
>> Merging in Jeff's recent work on Unicode is important and we should get
>> it in. I haven't had a chance to test myself, but given Jeff's amazing
>> attention to detail, I'm sure it's ready.
>>
>> The blocker for the RC - because we lost OSX support of setuptools
>> support of installed executables - is fixed, as I just finally confirmed:
>> http://bugs.jython.org/issue2570
>>
>>
>> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
>> Ste...@gm...> wrote:
>>
>>> AFAIK every release happens by having a successful RC that is renamed to
>>> 'release' after a while. So, per definition another RC is inevitable.
>>> That said, I suppose we should get http://bugs.jython.org/issue2487
>>> fixed before we can release. I guess Jeff's work will be ready until then.
>>> At least that decision can be postponed until an RC is actually doable.
>>>
>>>
>>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>>> *Von:* "Darjus Loktevic" <da...@gm...>
>>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>>> jyt...@li...>
>>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>>
>>> Hey Jeff,
>>>
>>> Sounds good. Let's do another rc but to be honest I'm not even sure the
>>> RC matters much if there aren't people trying it except us.
>>>
>>> Thoughts?
>>> Darjus
>>>
>>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>>
>>>> Hi Darjus.
>>>>
>>>> On inclusion, I'm happy to go with the community view, as always. On
>>>> one of the related tickets (http://bugs.jython.org/issue1839), Jim
>>>> said we'd get it in if timing allowed and there was some user support.
>>>>
>>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful,
>>>> and we're still making changes, so I assume we're talking about another RC
>>>> first rather than a release?
>>>>
>>>> The UTF-8 work is nearly there, but not quite: one Linux defect to fix,
>>>> as noted on the same issue by James against the "latin-1" version. After
>>>> all the additions in the last couple of weeks (to get full BMP support),
>>>> I'm happy to find from my Linux laptop that it is still the only thing I
>>>> have to do. It looks trivial. I've been unable code at all for a few days,
>>>> so haven't looked into a solution, but now I'm back I expect to nail it for
>>>> us today or tomorrow.
>>>>
>>>> I can, of course, merge all this myself and will. I shared your
>>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>>> I feel it's now low risk, as long as we still have a few days.
>>>>
>>>> I will now dive under the desk and wire up my Linux dev box.
>>>>
>>>> Jeff Allen
>>>>
>>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>>
>>>> Hey Jeff,
>>>>
>>>> It seems your last commit to this branch is of three days ago. Is this
>>>> ready for review? BTW, your changes look good to me.
>>>> I'm a little hesitant to merge this since we've had an RC and REALLY
>>>> have to release 2.7.1 It's miles better than 2.7.0.
>>>>
>>>> Cheers,
>>>> Darjus
>>>>
>>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>>>>
>>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>>>>> well. Rather than just push directly I have published to here:
>>>>>
>>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>>
>>>>> I write to ask for a second or third pair of eyes on it. Please tell me
>>>>> you can see it and whether it breaks things you care about.
>>>>>
>>>>> I touched a lot of files in the core and import system: quite a lot of
>>>>> tricky stuff with loaders and search paths has been adjusted. I think
>>>>> it
>>>>> a good sign that I changed hardly anything in the standard library we
>>>>> inherit from CPython, that we hadn't already specialised.
>>>>>
>>>>> By "works pretty well" above, I mean that the regression tests run
>>>>> cleanly for me when my user name is "Épreuve", where previously Jython
>>>>> died horribly. The launcher works from a Chinese user name too, as long
>>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>>> prompt and runs some tests with that setup, but I can't run the
>>>>> regression test yet, and printing a stack dump is fatal, so there's a
>>>>> bit more to do for Chinese.
>>>>>
>>>>> I think this means we have solid support for "latin-1" languages, but
>>>>> there are still places where we fatally assume bytes are Unicode code
>>>>> points.
>>>>>
>>>>> Jeff Allen
>>>>>
>>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd
>>>>> like to
>>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not run
>>>>> for
>>>>> > users whose names have an un-American character ;). I know this
>>>>> issue is
>>>>> > not a blocker in most minds.
>>>>> >
>>>>> > I've made pretty good progress by allowing file names to be unicode
>>>>> > objects more often than they would be in CPython 2, which usually
>>>>> > returns them as bytes in some encoding that we may not know. I've got
>>>>> > the launcher to work properly, and straightened the logic in our
>>>>> > printing of trace-backs and exceptions from Java. Unicode file names
>>>>> > seems the way to go for Jython because:
>>>>> >
>>>>> >   1. Java gives us competently decoded unicode file names, from
>>>>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>>>>> >      overlooked).
>>>>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>>>> necessary.
>>>>> >
>>>>> > Most regression tests pass. However, I'm struggling with
>>>>> test_doctest.
>>>>> > Problems arise when mixing unicode and bytes when one byte is 128 and
>>>>> > over. This happens in ''.join(list) and formatted output like "%s
>>>>> %s" %
>>>>> > (ustr, bstr). The behaviour of these is identical with CPython: they
>>>>> > raise UnicodeDecodeError because the bytes are promoted to characters
>>>>> > with a strict ascii interpretation. This happens a lot in doctest.py
>>>>> and
>>>>> > traceback.py, for example, where file paths and stack dumps that
>>>>> include
>>>>> > them, are now frequently unicode, while other inputs are byte data
>>>>> > containing file paths presented in the console encoding.
>>>>> >
>>>>> > I can beat this into submission with enough customisation of the
>>>>> stdlib
>>>>> > modules, but that always makes me uncomfortable. I usually see that
>>>>> as a
>>>>> > hint that user code might also need to change. This may be
>>>>> unfounded. I
>>>>> > can probably ensure no impact to users of only ascii paths, and the
>>>>> > others seem unable to run Jython at all (in the scope of this issue).
>>>>> > However, I'm seriously wondering if I should pursue the approach
>>>>> where
>>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>>> > everywhere), but that's grim.
>>>>> >
>>>>> > Thoughts?
>>>>> >
>>>>>
>>>>>
>>>>> ------------------------------------------------------------
>>>>> ------------------
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> Jython-dev mailing list
>>>>> Jyt...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>
>>>> ------------------------------------------------------------------------------
>>> Check out the vibrant tech community on one of the world's most engaging
>>> tech sites, Slashdot.org! http://sdm.link/slashdot______
>>> _________________________________________ Jython-dev mailing list
>>> Jyt...@li... https://lists.sourceforge.net/
>>> lists/listinfo/jython-dev
>>>
>>> ------------------------------------------------------------
>>> ------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Jython-dev mailing list
>>> Jyt...@li...
>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>
>>>
>>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Stefan R. <Ste...@gm...>]

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>I&#39;d like to have http://bugs.jython.org/issue2536 &quot;fixed&quot; by removing that infinite recursion test; maybe we could even close this annoying issue then. A new Jython release should have reliable test suite. However I don&#39;t know the best way to remove or blacklist a certain test.</div>

<div>Jeff, that&#39;s your domain IIRC...? Suggestions?</div>

<div>&nbsp;</div>

<div>Also, we might want to update (some) extlibs to current versions, e.g. I tested current Guava while working on #2536 two months ago and found no issues with it (at least not regarding regrtests). http://bugs.jython.org/issue2582 requires a current jnr version in order to decide further steps on that issue. There are probably more. I don&#39;t suggest to spend much time on this. Just do updates that cause no issues and leave everything else as it is. Yes the list of extlibs to check is long. Maybe we can split this work between us and maybe more volunteers?</div>

<div>&nbsp;</div>

<div>@Jim: Could you share some more insights about #2487, what you tried and why it failed. I&#39;d like to give it at least anther look before next RC.</div>

<div>&nbsp;</div>

<div>I once tried to update lib-python to CPython 2.7.13 version, but that caused lots of additional regrtest failures, 35 or so. Would probably take another year to resolve this stuff. I guess that time would be better invested in Jython 3. However, maybe we can revisit this for Jython 2.7.2.</div>

<div>&nbsp;</div>

<div>BTW, has anyone thought about a path for porting Jython 2.7.1 features and fixes to Jython 3? Jython 3 was forked shortly after 2.7.0 release and already received some serious amount of work. So we already have a notable divergence between Jython 3 and Jython 2.7.1. IMO this is a high priority topic right after Jython 2.7.1 release, because every further piece of 2.7.x work can make this worse.</div>

<div>&nbsp;</div>

<div>&nbsp;</div>

<div>-Stefan</div>

<div>&nbsp;</div>

<div>&nbsp;
<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Gesendet:</b>&nbsp;Samstag, 20. Mai 2017 um 20:48 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Jim Baker&quot; &lt;jim...@py...&gt;<br/>
<b>An:</b>&nbsp;&quot;Darjus Loktevic&quot; &lt;da...@gm...&gt;<br/>
<b>Cc:</b>&nbsp;&quot;Stefan Richthofer&quot; &lt;Ste...@gm...&gt;, &quot;Jython Developers&quot; &lt;jyt...@li...&gt;<br/>
<b>Betreff:</b>&nbsp;Re: [Jython-dev] Unicode user and file names (and v2.7.1)</div>

<div name="quoted-content">
<div>+100</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <span>&lt;<a href="mailto:da...@gm..." onclick="parent.window.location.href=&#39;da...@gm...&#39;; return false;" target="_blank">da...@gm...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<div>
<div>Agreed regarding not blocking on 2487. That whole area needs a rewrite and we could potentially utilize libraries available for Java 8.<br/>
&nbsp;</div>
Let&#39;s get Jeff&#39;s work in and do an RC?<br/>
&nbsp;</div>
<span class="HOEnZb"><font color="#888888">Darjus</font></span></div>

<div class="HOEnZb">
<div class="h5">&nbsp;
<div class="gmail_quote">
<div>On Fri, May 19, 2017 at 7:51 PM Jim Baker &lt;<a href="mailto:jim...@py..." onclick="parent.window.location.href=&#39;jim...@py...&#39;; return false;" target="_blank">jim...@py...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>I don&#39;t necessarily see <a href="http://bugs.jython.org/issue2487" target="_blank">http://bugs.jython.org/issue2487</a> as a blocker, but it would be nice. It just hasn&#39;t come up in real usage, unlike the earlier iteration of the bug which Darjus hacked around by busy waiting. I did spend some time on trying to get the publication to work without racing, using the approach I detail in that bug, but no luck yet. (But mostly because an utter lack of time to spend on the issue.)

<div>&nbsp;</div>

<div>Merging in Jeff&#39;s recent work on Unicode is important and we should get it in. I haven&#39;t had a chance to test myself, but given Jeff&#39;s amazing attention to detail, I&#39;m sure it&#39;s ready.</div>

<div>&nbsp;</div>

<div>The blocker for the RC - because we lost OSX support of setuptools support of installed executables - is fixed, as I just finally confirmed:&nbsp;<a href="http://bugs.jython.org/issue2570" target="_blank">http://bugs.jython.org/issue2570</a></div>

<div>&nbsp;</div>
</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <span>&lt;<a href="mailto:Ste...@gm..." onclick="parent.window.location.href=&#39;Ste...@gm...&#39;; return false;" target="_blank">Ste...@gm...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<div style="font-family: Verdana;font-size: 12.0px;">
<div>
<div>AFAIK every release happens by having a successful RC that is renamed to &#39;release&#39; after a while. So, per definition another RC is inevitable.</div>

<div>That said, I suppose we should get <a href="http://bugs.jython.org/issue2487" target="_blank">http://bugs.jython.org/issue2487</a> fixed before we can release. I guess Jeff&#39;s work will be ready until then. At least that decision can be postponed until an RC is actually doable.</div>

<div>
<div>&nbsp;</div>

<div>&nbsp;</div>

<div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);">
<div style="margin: 0 0 10.0px 0;"><b>Gesendet:</b>&nbsp;Samstag, 20. Mai 2017 um 02:35 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Darjus Loktevic&quot; &lt;<a href="mailto:da...@gm..." onclick="parent.window.location.href=&#39;da...@gm...&#39;; return false;" target="_blank">da...@gm...</a>&gt;<br/>
<b>An:</b>&nbsp;&quot;Jeff Allen&quot; &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt;, &quot;Jython Developers&quot; &lt;<a href="mailto:jyt...@li..." onclick="parent.window.location.href=&#39;jyt...@li...&#39;; return false;" target="_blank">jyt...@li...</a>&gt;<br/>
<b>Betreff:</b>&nbsp;Re: [Jython-dev] Unicode user and file names (and v2.7.1)</div>

<div>
<div>
<div class="m_8810925058365348314m_-8779070674509468614h5">
<p>Hey Jeff,</p>

<p>Sounds good. Let&#39;s do another rc but to be honest I&#39;m not even sure the RC matters much if there aren&#39;t people trying it except us.</p>

<p>Thoughts?<br/>
Darjus</p>
&nbsp;

<div class="gmail_quote">
<div>On Fri, May 19, 2017, 1:19 AM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<p>Hi Darjus.</p>

<p>On inclusion, I&#39;m happy to go with the community view, as always. On one of the related tickets (<a class="m_8810925058365348314m_-8779070674509468614m_-528727688133917487m_1723547812866307292moz-txt-link-freetext" href="http://bugs.jython.org/issue1839" target="_blank">http://bugs.jython.org/issue1839</a>), Jim said we&#39;d get it in if timing allowed and there was some user support.</p>

<p>I&#39;m very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful, and we&#39;re still making changes, so I assume we&#39;re talking about another RC first rather than a release?</p>

<p>The UTF-8 work is nearly there, but not quite: one Linux defect to fix, as noted on the same issue by James against the &quot;latin-1&quot; version. After all the additions in the last couple of weeks (to get full BMP support), I&#39;m happy to find from my Linux laptop that it is still the only thing I have to do. It looks trivial. I&#39;ve been unable code at all for a few days, so haven&#39;t looked into a solution, but now I&#39;m back I expect to nail it for us today or tomorrow.</p>

<p>I can, of course, merge all this myself and will. I shared your hesitancy initially, hence the fork repository, but it&#39;s turned out so well I feel it&#39;s now low risk, as long as we still have a few days.</p>

<p>I will now dive under the desk and wire up my Linux dev box.</p>
</div>

<div>
<pre class="m_8810925058365348314m_-8779070674509468614m_-528727688133917487m_1723547812866307292moz-signature">Jeff Allen</pre>
</div>

<div>
<div class="m_8810925058365348314m_-8779070674509468614m_-528727688133917487m_1723547812866307292moz-cite-prefix">On 16/05/2017 21:46, Darjus Loktevic wrote:</div>

<blockquote>
<div>
<div>
<div>
<div>Hey Jeff,<br/>
&nbsp;</div>
It seems your last commit to this branch is of three days ago. Is this ready for review? BTW, your changes look good to me.<br/>
I&#39;m a little hesitant to merge this since we&#39;ve had an RC and REALLY have to release 2.7.1 It&#39;s miles better than 2.7.0.<br/>
&nbsp;</div>
Cheers,</div>
Darjus</div>
&nbsp;

<div class="gmail_quote">
<div>On Mon, May 1, 2017 at 6:34 AM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">I went for sys.getfilesystemencoding() == &#39;utf-8&#39; and it works pretty<br/>
well. Rather than just push directly I have published to here:<br/>
<br/>
<a href="https://bitbucket.org/tournesol/jython-utf8" target="_blank">https://bitbucket.org/tournesol/jython-utf8</a><br/>
<br/>
I write to ask for a second or third pair of eyes on it. Please tell me<br/>
you can see it and whether it breaks things you care about.<br/>
<br/>
I touched a lot of files in the core and import system: quite a lot of<br/>
tricky stuff with loaders and search paths has been adjusted. I think it<br/>
a good sign that I changed hardly anything in the standard library we<br/>
inherit from CPython, that we hadn&#39;t already specialised.<br/>
<br/>
By &quot;works pretty well&quot; above, I mean that the regression tests run<br/>
cleanly for me when my user name is &quot;&Eacute;preuve&quot;, where previously Jython<br/>
died horribly. The launcher works from a Chinese user name too, as long<br/>
as I localise Windows to China (CPython 2.7 feature). I can use the<br/>
prompt and runs some tests with that setup, but I can&#39;t run the<br/>
regression test yet, and printing a stack dump is fatal, so there&#39;s a<br/>
bit more to do for Chinese.<br/>
<br/>
I think this means we have solid support for &quot;latin-1&quot; languages, but<br/>
there are still places where we fatally assume bytes are Unicode code<br/>
points.<br/>
<br/>
Jeff Allen<br/>
<br/>
On 05/04/2017 08:57, Jeff Allen wrote:<br/>
&gt; I&#39;ve been working on <a href="http://bugs.jython.org/issue2356" target="_blank">http://bugs.jython.org/issue2356</a> which I&#39;d like to<br/>
&gt; get in 2.7.1 -- it seems rather poor that Jython simply does not run for<br/>
&gt; users whose names have an un-American character ;). I know this issue is<br/>
&gt; not a blocker in most minds.<br/>
&gt;<br/>
&gt; I&#39;ve made pretty good progress by allowing file names to be unicode<br/>
&gt; objects more often than they would be in CPython 2, which usually<br/>
&gt; returns them as bytes in some encoding that we may not know. I&#39;ve got<br/>
&gt; the launcher to work properly, and straightened the logic in our<br/>
&gt; printing of trace-backs and exceptions from Java. Unicode file names<br/>
&gt; seems the way to go for Jython because:<br/>
&gt;<br/>
&gt;&nbsp; &nbsp;1. Java gives us competently decoded unicode file names, from<br/>
&gt;&nbsp; &nbsp; &nbsp; java.io.File, etc.. Re-encoding the result will be a pain (and<br/>
&gt;&nbsp; &nbsp; &nbsp; overlooked).<br/>
&gt;&nbsp; &nbsp;2. We appear not to have the codec we need (&#39;mbcs&#39;), that CPython<br/>
&gt;&nbsp; &nbsp; &nbsp; reports on Windows via sys.getfilesystemencoding().<br/>
&gt;&nbsp; &nbsp;3. We do this already. In 2.7.0, os.getcwd() returns unicode if necessary.<br/>
&gt;<br/>
&gt; Most regression tests pass. However, I&#39;m struggling with test_doctest.<br/>
&gt; Problems arise when mixing unicode and bytes when one byte is 128 and<br/>
&gt; over. This happens in &#39;&#39;.join(list) and formatted output like &quot;%s %s&quot; %<br/>
&gt; (ustr, bstr). The behaviour of these is identical with CPython: they<br/>
&gt; raise UnicodeDecodeError because the bytes are promoted to characters<br/>
&gt; with a strict ascii interpretation. This happens a lot in doctest.py and<br/>
&gt; traceback.py, for example, where file paths and stack dumps that include<br/>
&gt; them, are now frequently unicode, while other inputs are byte data<br/>
&gt; containing file paths presented in the console encoding.<br/>
&gt;<br/>
&gt; I can beat this into submission with enough customisation of the stdlib<br/>
&gt; modules, but that always makes me uncomfortable. I usually see that as a<br/>
&gt; hint that user code might also need to change. This may be unfounded. I<br/>
&gt; can probably ensure no impact to users of only ascii paths, and the<br/>
&gt; others seem unable to run Jython at all (in the scope of this issue).<br/>
&gt; However, I&#39;m seriously wondering if I should pursue the approach where<br/>
&gt; file names from Java are re-encoded to bytes (maybe as utf-8<br/>
&gt; everywhere), but that&#39;s grim.<br/>
&gt;<br/>
&gt; Thoughts?<br/>
&gt;<br/>
<br/>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world&#39;s most engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot_______________________________________________" target="_blank">http://sdm.link/slashdot_______________________________________________</a> Jython-dev mailing list <a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a> <a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></div>
</div>
</div>
</div>
</div>
</div>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a><br/>
&nbsp;</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div></div></body></html>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jeff A. <ja...@fa...>]

Thanks all. +1 on the RC. Nearly there with my bit.

I have fixed the  test_runpy failure James reported. It's not 
Linux-specific, just I had to quieten the unlink() error to see it on 
Windows. Bonus: we now pass the standard CPython test_runpy. The 
regrtest has been running one last time as I typed. I've pushed to 
https://bitbucket.org/tournesol/jython-utf8 just now.

I will next merge into the Jython trunk. That may not be totally smooth 
because of the pervasive change. And now I think about it, it's worth a 
note in NEWS. My time is a little limited today, so it could be much 
later today or tomorrow evening.

Jeff

Jeff Allen

On 20/05/2017 19:48, Jim Baker wrote:
> +100
>
> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm... 
> <mailto:da...@gm...>> wrote:
>
>     Agreed regarding not blocking on 2487. That whole area needs a
>     rewrite and we could potentially utilize libraries available for
>     Java 8.
>
>     Let's get Jeff's work in and do an RC?
>
>     Darjus
>
>     On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...
>     <mailto:jim...@py...>> wrote:
>
>         I don't necessarily see http://bugs.jython.org/issue2487
>         <http://bugs.jython.org/issue2487> as a blocker, but it would
>         be nice. It just hasn't come up in real usage, unlike the
>         earlier iteration of the bug which Darjus hacked around by
>         busy waiting. I did spend some time on trying to get the
>         publication to work without racing, using the approach I
>         detail in that bug, but no luck yet. (But mostly because an
>         utter lack of time to spend on the issue.)
>
>         Merging in Jeff's recent work on Unicode is important and we
>         should get it in. I haven't had a chance to test myself, but
>         given Jeff's amazing attention to detail, I'm sure it's ready.
>
>         The blocker for the RC - because we lost OSX support of
>         setuptools support of installed executables - is fixed, as I
>         just finally confirmed: http://bugs.jython.org/issue2570
>         <http://bugs.jython.org/issue2570>
>
>
>         On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer
>         <Ste...@gm... <mailto:Ste...@gm...>>
>         wrote:
>
>             AFAIK every release happens by having a successful RC that
>             is renamed to 'release' after a while. So, per definition
>             another RC is inevitable.
>             That said, I suppose we should get
>             http://bugs.jython.org/issue2487
>             <http://bugs.jython.org/issue2487> fixed before we can
>             release. I guess Jeff's work will be ready until then. At
>             least that decision can be postponed until an RC is
>             actually doable.
>             *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>             *Von:* "Darjus Loktevic" <da...@gm...
>             <mailto:da...@gm...>>
>             *An:* "Jeff Allen" <ja...@fa...
>             <mailto:ja...@fa...>>, "Jython Developers"
>             <jyt...@li...
>             <mailto:jyt...@li...>>
>             *Betreff:* Re: [Jython-dev] Unicode user and file names
>             (and v2.7.1)
>
>             Hey Jeff,
>
>             Sounds good. Let's do another rc but to be honest I'm not
>             even sure the RC matters much if there aren't people
>             trying it except us.
>
>             Thoughts?
>             Darjus
>
>             On Fri, May 19, 2017, 1:19 AM Jeff Allen
>             <ja...@fa... <mailto:ja...@fa...>> wrote:
>
>                 Hi Darjus.
>
>                 On inclusion, I'm happy to go with the community view,
>                 as always. On one of the related tickets
>                 (http://bugs.jython.org/issue1839
>                 <http://bugs.jython.org/issue1839>), Jim said we'd get
>                 it in if timing allowed and there was some user support.
>
>                 I'm very keen to see a 2.7.1 too. The last (soft) RC
>                 was unsuccessful, and we're still making changes, so I
>                 assume we're talking about another RC first rather
>                 than a release?
>
>                 The UTF-8 work is nearly there, but not quite: one
>                 Linux defect to fix, as noted on the same issue by
>                 James against the "latin-1" version. After all the
>                 additions in the last couple of weeks (to get full BMP
>                 support), I'm happy to find from my Linux laptop that
>                 it is still the only thing I have to do. It looks
>                 trivial. I've been unable code at all for a few days,
>                 so haven't looked into a solution, but now I'm back I
>                 expect to nail it for us today or tomorrow.
>
>                 I can, of course, merge all this myself and will. I
>                 shared your hesitancy initially, hence the fork
>                 repository, but it's turned out so well I feel it's
>                 now low risk, as long as we still have a few days.
>
>                 I will now dive under the desk and wire up my Linux
>                 dev box.
>
>                 Jeff Allen
>
>                 On 16/05/2017 21:46, Darjus Loktevic wrote:
>
>                     Hey Jeff,
>                     It seems your last commit to this branch is of
>                     three days ago. Is this ready for review? BTW,
>                     your changes look good to me.
>                     I'm a little hesitant to merge this since we've
>                     had an RC and REALLY have to release 2.7.1 It's
>                     miles better than 2.7.0.
>                     Cheers,
>                     Darjus
>                     On Mon, May 1, 2017 at 6:34 AM Jeff Allen
>                     <ja...@fa... <mailto:ja...@fa...>>
>                     wrote:
>
>                         I went for sys.getfilesystemencoding() ==
>                         'utf-8' and it works pretty
>                         well. Rather than just push directly I have
>                         published to here:
>
>                         https://bitbucket.org/tournesol/jython-utf8
>                         <https://bitbucket.org/tournesol/jython-utf8>
>
>                         I write to ask for a second or third pair of
>                         eyes on it. Please tell me
>                         you can see it and whether it breaks things
>                         you care about.
>
>                         I touched a lot of files in the core and
>                         import system: quite a lot of
>                         tricky stuff with loaders and search paths has
>                         been adjusted. I think it
>                         a good sign that I changed hardly anything in
>                         the standard library we
>                         inherit from CPython, that we hadn't already
>                         specialised.
>
>                         By "works pretty well" above, I mean that the
>                         regression tests run
>                         cleanly for me when my user name is "Épreuve",
>                         where previously Jython
>                         died horribly. The launcher works from a
>                         Chinese user name too, as long
>                         as I localise Windows to China (CPython 2.7
>                         feature). I can use the
>                         prompt and runs some tests with that setup,
>                         but I can't run the
>                         regression test yet, and printing a stack dump
>                         is fatal, so there's a
>                         bit more to do for Chinese.
>
>                         I think this means we have solid support for
>                         "latin-1" languages, but
>                         there are still places where we fatally assume
>                         bytes are Unicode code
>                         points.
>
>                         Jeff Allen
>
>                         On 05/04/2017 08:57, Jeff Allen wrote:
>                         > I've been working on
>                         http://bugs.jython.org/issue2356
>                         <http://bugs.jython.org/issue2356> which I'd
>                         like to
>                         > get in 2.7.1 -- it seems rather poor that
>                         Jython simply does not run for
>                         > users whose names have an un-American
>                         character ;). I know this issue is
>                         > not a blocker in most minds.
>                         >
>                         > I've made pretty good progress by allowing
>                         file names to be unicode
>                         > objects more often than they would be in
>                         CPython 2, which usually
>                         > returns them as bytes in some encoding that
>                         we may not know. I've got
>                         > the launcher to work properly, and
>                         straightened the logic in our
>                         > printing of trace-backs and exceptions from
>                         Java. Unicode file names
>                         > seems the way to go for Jython because:
>                         >
>                         >   1. Java gives us competently decoded
>                         unicode file names, from
>                         > java.io.File, etc.. Re-encoding the result
>                         will be a pain (and
>                         > overlooked).
>                         >   2. We appear not to have the codec we need
>                         ('mbcs'), that CPython
>                         >      reports on Windows via
>                         sys.getfilesystemencoding().
>                         >   3. We do this already. In 2.7.0,
>                         os.getcwd() returns unicode if necessary.
>                         >
>                         > Most regression tests pass. However, I'm
>                         struggling with test_doctest.
>                         > Problems arise when mixing unicode and bytes
>                         when one byte is 128 and
>                         > over. This happens in ''.join(list) and
>                         formatted output like "%s %s" %
>                         > (ustr, bstr). The behaviour of these is
>                         identical with CPython: they
>                         > raise UnicodeDecodeError because the bytes
>                         are promoted to characters
>                         > with a strict ascii interpretation. This
>                         happens a lot in doctest.py and
>                         > traceback.py, for example, where file paths
>                         and stack dumps that include
>                         > them, are now frequently unicode, while
>                         other inputs are byte data
>                         > containing file paths presented in the
>                         console encoding.
>                         >
>                         > I can beat this into submission with enough
>                         customisation of the stdlib
>                         > modules, but that always makes me
>                         uncomfortable. I usually see that as a
>                         > hint that user code might also need to
>                         change. This may be unfounded. I
>                         > can probably ensure no impact to users of
>                         only ascii paths, and the
>                         > others seem unable to run Jython at all (in
>                         the scope of this issue).
>                         > However, I'm seriously wondering if I should
>                         pursue the approach where
>                         > file names from Java are re-encoded to bytes
>                         (maybe as utf-8
>                         > everywhere), but that's grim.
>                         >
>                         > Thoughts?
>                         >
>
>
>                         ------------------------------------------------------------------------------
>                         Check out the vibrant tech community on one of
>                         the world's most
>                         engaging tech sites, Slashdot.org!
>                         http://sdm.link/slashdot
>                         _______________________________________________
>                         Jython-dev mailing list
>                         Jyt...@li...
>                         <mailto:Jyt...@li...>
>                         https://lists.sourceforge.net/lists/listinfo/jython-dev
>                         <https://lists.sourceforge.net/lists/listinfo/jython-dev>
>
>             ------------------------------------------------------------------------------
>             Check out the vibrant tech community on one of the world's
>             most engaging tech sites, Slashdot.org!
>             http://sdm.link/slashdot_______________________________________________
>             <http://sdm.link/slashdot_______________________________________________>
>             Jython-dev mailing list Jyt...@li...
>             <mailto:Jyt...@li...>
>             https://lists.sourceforge.net/lists/listinfo/jython-dev
>             <https://lists.sourceforge.net/lists/listinfo/jython-dev>
>
>             ------------------------------------------------------------------------------
>             Check out the vibrant tech community on one of the world's
>             most
>             engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>             _______________________________________________
>             Jython-dev mailing list
>             Jyt...@li...
>             <mailto:Jyt...@li...>
>             https://lists.sourceforge.net/lists/listinfo/jython-dev
>             <https://lists.sourceforge.net/lists/listinfo/jython-dev>
>
>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Darjus L. <da...@gm...>]

Hey Guys,

Regarding Jython3, looks like Isaiah has done a ton of work in 2016 (CCd).
Not sure how far he progressed, but indeed merging will be hard and
therefore I'd say we should not diverge further while developing on both
branches, but instead try to finalize 2.7 and switch to Jython3 full-time.

Feel free to disagree, but here's my thinking on it:

   1. Release Jython 2.7.1
   2. Modernize the codebase. I think it's important for the project to
   feel modern for us to attract new contributors.
   1. Java8 as the minimum (may be too much for Jython2).
      2. Github/core-workflow
      3. (Ideally) ANTLR4 for both branches, but worst case, Jython3 only.
      ANTLR3 is not getting much love and ANTLR4 is quite different (does not
      generate AST).
      4. Gradle, directory structure.
   3. Develop Jython3 primarily. Only bugfixes for 2.7 series.
      1. Target 3.6 (really like the typing improvements).
      2. Merge JyNI if possible.

Cheers,
Darjus

On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:

> Thanks all. +1 on the RC. Nearly there with my bit.
>
> I have fixed the  test_runpy failure James reported. It's not
> Linux-specific, just I had to quieten the unlink() error to see it on
> Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest
> has been running one last time as I typed. I've pushed to
> https://bitbucket.org/tournesol/jython-utf8 just now.
> I will next merge into the Jython trunk. That may not be totally smooth
> because of the pervasive change. And now I think about it, it's worth a
> note in NEWS. My time is a little limited today, so it could be much later
> today or tomorrow evening.
>
> Jeff
>
> Jeff Allen
>
> On 20/05/2017 19:48, Jim Baker wrote:
>
> +100
>
> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
> wrote:
>
>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>> and we could potentially utilize libraries available for Java 8.
>>
>> Let's get Jeff's work in and do an RC?
>>
>> Darjus
>>
>> On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...> wrote:
>>
>>> I don't necessarily see http://bugs.jython.org/issue2487 as a blocker,
>>> but it would be nice. It just hasn't come up in real usage, unlike the
>>> earlier iteration of the bug which Darjus hacked around by busy waiting. I
>>> did spend some time on trying to get the publication to work without
>>> racing, using the approach I detail in that bug, but no luck yet. (But
>>> mostly because an utter lack of time to spend on the issue.)
>>>
>>> Merging in Jeff's recent work on Unicode is important and we should get
>>> it in. I haven't had a chance to test myself, but given Jeff's amazing
>>> attention to detail, I'm sure it's ready.
>>>
>>> The blocker for the RC - because we lost OSX support of setuptools
>>> support of installed executables - is fixed, as I just finally confirmed:
>>> http://bugs.jython.org/issue2570
>>>
>>>
>>> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
>>> Ste...@gm...> wrote:
>>>
>>>> AFAIK every release happens by having a successful RC that is renamed
>>>> to 'release' after a while. So, per definition another RC is inevitable.
>>>> That said, I suppose we should get http://bugs.jython.org/issue2487
>>>> fixed before we can release. I guess Jeff's work will be ready until then.
>>>> At least that decision can be postponed until an RC is actually doable.
>>>>
>>>>
>>>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>>>> *Von:* "Darjus Loktevic" <da...@gm...>
>>>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>>>> jyt...@li...>
>>>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>>>
>>>> Hey Jeff,
>>>>
>>>> Sounds good. Let's do another rc but to be honest I'm not even sure the
>>>> RC matters much if there aren't people trying it except us.
>>>>
>>>> Thoughts?
>>>> Darjus
>>>>
>>>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>>>
>>>>> Hi Darjus.
>>>>>
>>>>> On inclusion, I'm happy to go with the community view, as always. On
>>>>> one of the related tickets (http://bugs.jython.org/issue1839), Jim
>>>>> said we'd get it in if timing allowed and there was some user support.
>>>>>
>>>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful,
>>>>> and we're still making changes, so I assume we're talking about another RC
>>>>> first rather than a release?
>>>>>
>>>>> The UTF-8 work is nearly there, but not quite: one Linux defect to
>>>>> fix, as noted on the same issue by James against the "latin-1" version.
>>>>> After all the additions in the last couple of weeks (to get full BMP
>>>>> support), I'm happy to find from my Linux laptop that it is still the only
>>>>> thing I have to do. It looks trivial. I've been unable code at all for a
>>>>> few days, so haven't looked into a solution, but now I'm back I expect to
>>>>> nail it for us today or tomorrow.
>>>>>
>>>>> I can, of course, merge all this myself and will. I shared your
>>>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>>>> I feel it's now low risk, as long as we still have a few days.
>>>>>
>>>>> I will now dive under the desk and wire up my Linux dev box.
>>>>>
>>>>> Jeff Allen
>>>>>
>>>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>>>
>>>>> Hey Jeff,
>>>>>
>>>>> It seems your last commit to this branch is of three days ago. Is this
>>>>> ready for review? BTW, your changes look good to me.
>>>>> I'm a little hesitant to merge this since we've had an RC and REALLY
>>>>> have to release 2.7.1 It's miles better than 2.7.0.
>>>>>
>>>>> Cheers,
>>>>> Darjus
>>>>>
>>>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>>>>>
>>>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>>>>>> well. Rather than just push directly I have published to here:
>>>>>>
>>>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>>>
>>>>>> I write to ask for a second or third pair of eyes on it. Please tell
>>>>>> me
>>>>>> you can see it and whether it breaks things you care about.
>>>>>>
>>>>>> I touched a lot of files in the core and import system: quite a lot of
>>>>>> tricky stuff with loaders and search paths has been adjusted. I think
>>>>>> it
>>>>>> a good sign that I changed hardly anything in the standard library we
>>>>>> inherit from CPython, that we hadn't already specialised.
>>>>>>
>>>>>> By "works pretty well" above, I mean that the regression tests run
>>>>>> cleanly for me when my user name is "Épreuve", where previously Jython
>>>>>> died horribly. The launcher works from a Chinese user name too, as
>>>>>> long
>>>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>>>> prompt and runs some tests with that setup, but I can't run the
>>>>>> regression test yet, and printing a stack dump is fatal, so there's a
>>>>>> bit more to do for Chinese.
>>>>>>
>>>>>> I think this means we have solid support for "latin-1" languages, but
>>>>>> there are still places where we fatally assume bytes are Unicode code
>>>>>> points.
>>>>>>
>>>>>> Jeff Allen
>>>>>>
>>>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd
>>>>>> like to
>>>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not
>>>>>> run for
>>>>>> > users whose names have an un-American character ;). I know this
>>>>>> issue is
>>>>>> > not a blocker in most minds.
>>>>>> >
>>>>>> > I've made pretty good progress by allowing file names to be unicode
>>>>>> > objects more often than they would be in CPython 2, which usually
>>>>>> > returns them as bytes in some encoding that we may not know. I've
>>>>>> got
>>>>>> > the launcher to work properly, and straightened the logic in our
>>>>>> > printing of trace-backs and exceptions from Java. Unicode file names
>>>>>> > seems the way to go for Jython because:
>>>>>> >
>>>>>> >   1. Java gives us competently decoded unicode file names, from
>>>>>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>>>>>> >      overlooked).
>>>>>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>>>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>>>>> necessary.
>>>>>> >
>>>>>> > Most regression tests pass. However, I'm struggling with
>>>>>> test_doctest.
>>>>>> > Problems arise when mixing unicode and bytes when one byte is 128
>>>>>> and
>>>>>> > over. This happens in ''.join(list) and formatted output like "%s
>>>>>> %s" %
>>>>>> > (ustr, bstr). The behaviour of these is identical with CPython: they
>>>>>> > raise UnicodeDecodeError because the bytes are promoted to
>>>>>> characters
>>>>>> > with a strict ascii interpretation. This happens a lot in
>>>>>> doctest.py and
>>>>>> > traceback.py, for example, where file paths and stack dumps that
>>>>>> include
>>>>>> > them, are now frequently unicode, while other inputs are byte data
>>>>>> > containing file paths presented in the console encoding.
>>>>>> >
>>>>>> > I can beat this into submission with enough customisation of the
>>>>>> stdlib
>>>>>> > modules, but that always makes me uncomfortable. I usually see that
>>>>>> as a
>>>>>> > hint that user code might also need to change. This may be
>>>>>> unfounded. I
>>>>>> > can probably ensure no impact to users of only ascii paths, and the
>>>>>> > others seem unable to run Jython at all (in the scope of this
>>>>>> issue).
>>>>>> > However, I'm seriously wondering if I should pursue the approach
>>>>>> where
>>>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>>>> > everywhere), but that's grim.
>>>>>> >
>>>>>> > Thoughts?
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>> _______________________________________________
>>>>>> Jython-dev mailing list
>>>>>> Jyt...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>> Check out the vibrant tech community on one of the world's most engaging
>>>> tech sites, Slashdot.org!
>>>> http://sdm.link/slashdot_______________________________________________
>>>> Jython-dev mailing list Jyt...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>> _______________________________________________
>>>> Jython-dev mailing list
>>>> Jyt...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>
>>>>
>>>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Jython-dev mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/jython-dev
>
>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jim B. <jim...@py...>]

Agreed. Jython 3 is inherently much more interesting to work on, and at
this point we should be more or less done with must have functionality for
2.7.1. Get it out now! :)

I would not worry so much about backporting Antlr changes to Jython 2.7. We
should restrict backports to runtime fixes, which has been the focus of all
recent 2.7 work anyway.

Because of async/await and standard typing support (
https://docs.python.org/3/whatsnew/3.6.html#pep-526-syntax-for-variable-annotations
is particularly relevant), we should focus on 3.6, although 3.7 is not so
different, at least yet (https://docs.python.org/3.7/whatsnew/3.7.html)

Personally I could see two things I might work on:

   - typing, specifically how we can use provide type signatures for
   imports from Java into Python code; and vice versa.
   - async support, taking advantage of Netty to get similar results as
   seen in https://magic.io/blog/uvloop-blazing-fast-python-networking/,
   but no GIL makes this more awesome. Ideally something like
   https://github.com/channelcat/sanic could just run as-is on Jython 3.

- Jim

On Sun, May 21, 2017 at 9:35 AM, Darjus Loktevic <da...@gm...> wrote:

> Hey Guys,
>
> Regarding Jython3, looks like Isaiah has done a ton of work in 2016 (CCd).
> Not sure how far he progressed, but indeed merging will be hard and
> therefore I'd say we should not diverge further while developing on both
> branches, but instead try to finalize 2.7 and switch to Jython3 full-time.
>
> Feel free to disagree, but here's my thinking on it:
>
>    1. Release Jython 2.7.1
>    2. Modernize the codebase. I think it's important for the project to
>    feel modern for us to attract new contributors.
>    1. Java8 as the minimum (may be too much for Jython2).
>       2. Github/core-workflow
>       3. (Ideally) ANTLR4 for both branches, but worst case, Jython3
>       only. ANTLR3 is not getting much love and ANTLR4 is quite different (does
>       not generate AST).
>       4. Gradle, directory structure.
>    3. Develop Jython3 primarily. Only bugfixes for 2.7 series.
>       1. Target 3.6 (really like the typing improvements).
>       2. Merge JyNI if possible.
>
> Cheers,
> Darjus
>
> On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:
>
>> Thanks all. +1 on the RC. Nearly there with my bit.
>>
>> I have fixed the  test_runpy failure James reported. It's not
>> Linux-specific, just I had to quieten the unlink() error to see it on
>> Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest
>> has been running one last time as I typed. I've pushed to
>> https://bitbucket.org/tournesol/jython-utf8 just now.
>> I will next merge into the Jython trunk. That may not be totally smooth
>> because of the pervasive change. And now I think about it, it's worth a
>> note in NEWS. My time is a little limited today, so it could be much later
>> today or tomorrow evening.
>>
>> Jeff
>>
>> Jeff Allen
>>
>> On 20/05/2017 19:48, Jim Baker wrote:
>>
>> +100
>>
>> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
>> wrote:
>>
>>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>>> and we could potentially utilize libraries available for Java 8.
>>>
>>> Let's get Jeff's work in and do an RC?
>>>
>>> Darjus
>>>
>>> On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...> wrote:
>>>
>>>> I don't necessarily see http://bugs.jython.org/issue2487 as a blocker,
>>>> but it would be nice. It just hasn't come up in real usage, unlike the
>>>> earlier iteration of the bug which Darjus hacked around by busy waiting. I
>>>> did spend some time on trying to get the publication to work without
>>>> racing, using the approach I detail in that bug, but no luck yet. (But
>>>> mostly because an utter lack of time to spend on the issue.)
>>>>
>>>> Merging in Jeff's recent work on Unicode is important and we should get
>>>> it in. I haven't had a chance to test myself, but given Jeff's amazing
>>>> attention to detail, I'm sure it's ready.
>>>>
>>>> The blocker for the RC - because we lost OSX support of setuptools
>>>> support of installed executables - is fixed, as I just finally confirmed:
>>>> http://bugs.jython.org/issue2570
>>>>
>>>>
>>>> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
>>>> Ste...@gm...> wrote:
>>>>
>>>>> AFAIK every release happens by having a successful RC that is renamed
>>>>> to 'release' after a while. So, per definition another RC is inevitable.
>>>>> That said, I suppose we should get http://bugs.jython.org/issue2487
>>>>> fixed before we can release. I guess Jeff's work will be ready until then.
>>>>> At least that decision can be postponed until an RC is actually doable.
>>>>>
>>>>>
>>>>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>>>>> *Von:* "Darjus Loktevic" <da...@gm...>
>>>>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>>>>> jyt...@li...>
>>>>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>>>>
>>>>> Hey Jeff,
>>>>>
>>>>> Sounds good. Let's do another rc but to be honest I'm not even sure
>>>>> the RC matters much if there aren't people trying it except us.
>>>>>
>>>>> Thoughts?
>>>>> Darjus
>>>>>
>>>>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>>>>
>>>>>> Hi Darjus.
>>>>>>
>>>>>> On inclusion, I'm happy to go with the community view, as always. On
>>>>>> one of the related tickets (http://bugs.jython.org/issue1839), Jim
>>>>>> said we'd get it in if timing allowed and there was some user support.
>>>>>>
>>>>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was
>>>>>> unsuccessful, and we're still making changes, so I assume we're talking
>>>>>> about another RC first rather than a release?
>>>>>>
>>>>>> The UTF-8 work is nearly there, but not quite: one Linux defect to
>>>>>> fix, as noted on the same issue by James against the "latin-1" version.
>>>>>> After all the additions in the last couple of weeks (to get full BMP
>>>>>> support), I'm happy to find from my Linux laptop that it is still the only
>>>>>> thing I have to do. It looks trivial. I've been unable code at all for a
>>>>>> few days, so haven't looked into a solution, but now I'm back I expect to
>>>>>> nail it for us today or tomorrow.
>>>>>>
>>>>>> I can, of course, merge all this myself and will. I shared your
>>>>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>>>>> I feel it's now low risk, as long as we still have a few days.
>>>>>>
>>>>>> I will now dive under the desk and wire up my Linux dev box.
>>>>>>
>>>>>> Jeff Allen
>>>>>>
>>>>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>>>>
>>>>>> Hey Jeff,
>>>>>>
>>>>>> It seems your last commit to this branch is of three days ago. Is
>>>>>> this ready for review? BTW, your changes look good to me.
>>>>>> I'm a little hesitant to merge this since we've had an RC and REALLY
>>>>>> have to release 2.7.1 It's miles better than 2.7.0.
>>>>>>
>>>>>> Cheers,
>>>>>> Darjus
>>>>>>
>>>>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>>>>>>
>>>>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>>>>>>> well. Rather than just push directly I have published to here:
>>>>>>>
>>>>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>>>>
>>>>>>> I write to ask for a second or third pair of eyes on it. Please tell
>>>>>>> me
>>>>>>> you can see it and whether it breaks things you care about.
>>>>>>>
>>>>>>> I touched a lot of files in the core and import system: quite a lot
>>>>>>> of
>>>>>>> tricky stuff with loaders and search paths has been adjusted. I
>>>>>>> think it
>>>>>>> a good sign that I changed hardly anything in the standard library we
>>>>>>> inherit from CPython, that we hadn't already specialised.
>>>>>>>
>>>>>>> By "works pretty well" above, I mean that the regression tests run
>>>>>>> cleanly for me when my user name is "Épreuve", where previously
>>>>>>> Jython
>>>>>>> died horribly. The launcher works from a Chinese user name too, as
>>>>>>> long
>>>>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>>>>> prompt and runs some tests with that setup, but I can't run the
>>>>>>> regression test yet, and printing a stack dump is fatal, so there's a
>>>>>>> bit more to do for Chinese.
>>>>>>>
>>>>>>> I think this means we have solid support for "latin-1" languages, but
>>>>>>> there are still places where we fatally assume bytes are Unicode code
>>>>>>> points.
>>>>>>>
>>>>>>> Jeff Allen
>>>>>>>
>>>>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd
>>>>>>> like to
>>>>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not
>>>>>>> run for
>>>>>>> > users whose names have an un-American character ;). I know this
>>>>>>> issue is
>>>>>>> > not a blocker in most minds.
>>>>>>> >
>>>>>>> > I've made pretty good progress by allowing file names to be unicode
>>>>>>> > objects more often than they would be in CPython 2, which usually
>>>>>>> > returns them as bytes in some encoding that we may not know. I've
>>>>>>> got
>>>>>>> > the launcher to work properly, and straightened the logic in our
>>>>>>> > printing of trace-backs and exceptions from Java. Unicode file
>>>>>>> names
>>>>>>> > seems the way to go for Jython because:
>>>>>>> >
>>>>>>> >   1. Java gives us competently decoded unicode file names, from
>>>>>>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>>>>>>> >      overlooked).
>>>>>>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>>>>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>>>>>> necessary.
>>>>>>> >
>>>>>>> > Most regression tests pass. However, I'm struggling with
>>>>>>> test_doctest.
>>>>>>> > Problems arise when mixing unicode and bytes when one byte is 128
>>>>>>> and
>>>>>>> > over. This happens in ''.join(list) and formatted output like "%s
>>>>>>> %s" %
>>>>>>> > (ustr, bstr). The behaviour of these is identical with CPython:
>>>>>>> they
>>>>>>> > raise UnicodeDecodeError because the bytes are promoted to
>>>>>>> characters
>>>>>>> > with a strict ascii interpretation. This happens a lot in
>>>>>>> doctest.py and
>>>>>>> > traceback.py, for example, where file paths and stack dumps that
>>>>>>> include
>>>>>>> > them, are now frequently unicode, while other inputs are byte data
>>>>>>> > containing file paths presented in the console encoding.
>>>>>>> >
>>>>>>> > I can beat this into submission with enough customisation of the
>>>>>>> stdlib
>>>>>>> > modules, but that always makes me uncomfortable. I usually see
>>>>>>> that as a
>>>>>>> > hint that user code might also need to change. This may be
>>>>>>> unfounded. I
>>>>>>> > can probably ensure no impact to users of only ascii paths, and the
>>>>>>> > others seem unable to run Jython at all (in the scope of this
>>>>>>> issue).
>>>>>>> > However, I'm seriously wondering if I should pursue the approach
>>>>>>> where
>>>>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>>>>> > everywhere), but that's grim.
>>>>>>> >
>>>>>>> > Thoughts?
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ------------------
>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>> _______________________________________________
>>>>>>> Jython-dev mailing list
>>>>>>> Jyt...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>> Check out the vibrant tech community on one of the world's most engaging
>>>>> tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>>> _________________________________________ Jython-dev mailing list
>>>>> Jyt...@li... https://lists.sourceforge.net/
>>>>> lists/listinfo/jython-dev
>>>>>
>>>>> ------------------------------------------------------------
>>>>> ------------------
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> Jython-dev mailing list
>>>>> Jyt...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>
>>>>>
>>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Jython-dev mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>>
>>

[Jython-dev] Updating extlibs

[Stefan R. <Ste...@gm...>]

Hey all,
I spent some effort to explore feasibility of updating extlibs, especially minor versions (but sometimes even major version, e.g. guava and icu4j).
My results so far, tested with regrtest on Linux and Windows 10 using Java8 ("okay" means I did not observe additional regrtest failures):

ASM 5.0.4             -> 5.2     okay
bouncycastle:
bcpkix-jdk15on-1.54   -> 1.57    okay
bcprov-jdk15on-1.54   -> 1.57    okay

commons-compress-1.12 -> 1.14    okay
guava-20.0            -> 22.0rc1 okay
icu4j-58.1            -> 59_1    okay
Netty 4.1.6           -> 4.1.11  okay
java-sizeof-0.0.5     -- still current
jffi-1.2.13           -> 1.2.15  okay
jnr-ffi-2.1.0         -> 2.1.5   okay
jnr-netdb-1.1.6       -- still current
jnr-posix-3.0.31      -> 3.0.41  okay
jnr-constants-0.9.5   -> 0.9.9   okay
New platforms: jffi-aarch64-Linux.jar, jffi-ppc64le-Linux.jar, can be added...?
Updated various other changed platform specific jars to jffi-1.2.15 (okay as far as tested)
xercesImpl-2.11.0          -- still current
jline-2.14.2               -> 2.14.3      okay (didn't try jline-3 this time)
jarjar-1.4                 -- still current
mysql-connector-java-5.1.6 -> 5.1.42      okay
postgresql-8.3-603.jdbc4   -> 42.1.1-jre7 okay

----------These failed, so leaving them as it is:
Antlr 3.1.3                -> 3.5.2  fails
junit-4.10                 -> 4.12 or 4.11 class file for org.hamcrest.Matcher not found
(staying with 4.10 for now to avoid new dependency on hamcrest-matcher)
javax.servlet-api-2.5      -> 3.1.0  fails
mockrunner      (better don't touch; whole structure changed)
cpptasks        (better don't touch)

- will be able to test with Java 7 on Tuesday, because I left my old laptop in the office.
- will upload a fork containing these updates. Would be good if someone else could also test, especially on OSX.
Maybe some stuff was not covered by regrtests. However the chance that updates solve issues and that they create issues are probably somewhat equal and I'd prefer to focus on fixing issues with current versions rather than older ones.
So I'm strongly for getting this in if regrtests go through on Java 7 and OSX. 2.7.1RC-phase will be a good opportunity to confirm workability of these updates. Any concerns?

-Stefan

[Jython-dev] Updating extlibs

[Stefan R. <Ste...@gm...>]

I uploaded the mentioned updates to
https://github.com/Stewori/jython.
See detailed changes at
https://github.com/Stewori/jython/commit/ee2b1263306f779bf8e9499afc6d02267648ac36

This might not yet work smoothly with Java 7, I will check and adjust that tomorrow.
Some packages were built from source using Java 8 and I'm not sure whether the gradle
scripts always configured Java 7 source compatibility properly.
However if some people could test it, especially on OSX, would be nice.

Best

Stefan


> Gesendet: Montag, 22. Mai 2017 um 04:07 Uhr
> Von: "Stefan Richthofer" <Ste...@gm...>
> An: "Jython Developers" <jyt...@li...>
> Betreff: Updating extlibs
>
> Hey all,
> I spent some effort to explore feasibility of updating extlibs, especially minor versions (but sometimes even major version, e.g. guava and icu4j).
> My results so far, tested with regrtest on Linux and Windows 10 using Java8 ("okay" means I did not observe additional regrtest failures):
> 
> ASM 5.0.4             -> 5.2     okay
> bouncycastle:
> bcpkix-jdk15on-1.54   -> 1.57    okay
> bcprov-jdk15on-1.54   -> 1.57    okay
> 
> commons-compress-1.12 -> 1.14    okay
> guava-20.0            -> 22.0rc1 okay
> icu4j-58.1            -> 59_1    okay
> Netty 4.1.6           -> 4.1.11  okay
> java-sizeof-0.0.5     -- still current
> jffi-1.2.13           -> 1.2.15  okay
> jnr-ffi-2.1.0         -> 2.1.5   okay
> jnr-netdb-1.1.6       -- still current
> jnr-posix-3.0.31      -> 3.0.41  okay
> jnr-constants-0.9.5   -> 0.9.9   okay
> New platforms: jffi-aarch64-Linux.jar, jffi-ppc64le-Linux.jar, can be added...?
> Updated various other changed platform specific jars to jffi-1.2.15 (okay as far as tested)
> xercesImpl-2.11.0          -- still current
> jline-2.14.2               -> 2.14.3      okay (didn't try jline-3 this time)
> jarjar-1.4                 -- still current
> mysql-connector-java-5.1.6 -> 5.1.42      okay
> postgresql-8.3-603.jdbc4   -> 42.1.1-jre7 okay
> 
> ----------These failed, so leaving them as it is:
> Antlr 3.1.3                -> 3.5.2  fails
> junit-4.10                 -> 4.12 or 4.11 class file for org.hamcrest.Matcher not found
> (staying with 4.10 for now to avoid new dependency on hamcrest-matcher)
> javax.servlet-api-2.5      -> 3.1.0  fails
> mockrunner      (better don't touch; whole structure changed)
> cpptasks        (better don't touch)
> 
> - will be able to test with Java 7 on Tuesday, because I left my old laptop in the office.
> - will upload a fork containing these updates. Would be good if someone else could also test, especially on OSX.
> Maybe some stuff was not covered by regrtests. However the chance that updates solve issues and that they create issues are probably somewhat equal and I'd prefer to focus on fixing issues with current versions rather than older ones.
> So I'm strongly for getting this in if regrtests go through on Java 7 and OSX. 2.7.1RC-phase will be a good opportunity to confirm workability of these updates. Any concerns?
> 
> -Stefan

Re: [Jython-dev] Updating extlibs

[Jim B. <jim...@py...>]

These changes look good to me. I will test out your patch, but all of this
is in line with similar updates we have made in the past, usually around
this time in the dev cycle.

I'm glad that moving to Gradle will make this re-pinning to upstream
dependencies much easier going forward!

On Mon, May 22, 2017 at 8:46 AM, Stefan Richthofer <Ste...@gm...
> wrote:

> I uploaded the mentioned updates to
> https://github.com/Stewori/jython.
> See detailed changes at
> https://github.com/Stewori/jython/commit/ee2b1263306f779bf8e9499afc6d02
> 267648ac36
>
> This might not yet work smoothly with Java 7, I will check and adjust that
> tomorrow.
> Some packages were built from source using Java 8 and I'm not sure whether
> the gradle
> scripts always configured Java 7 source compatibility properly.
> However if some people could test it, especially on OSX, would be nice.
>
> Best
>
> Stefan
>
>
> > Gesendet: Montag, 22. Mai 2017 um 04:07 Uhr
> > Von: "Stefan Richthofer" <Ste...@gm...>
> > An: "Jython Developers" <jyt...@li...>
> > Betreff: Updating extlibs
> >
> > Hey all,
> > I spent some effort to explore feasibility of updating extlibs,
> especially minor versions (but sometimes even major version, e.g. guava and
> icu4j).
> > My results so far, tested with regrtest on Linux and Windows 10 using
> Java8 ("okay" means I did not observe additional regrtest failures):
> >
> > ASM 5.0.4             -> 5.2     okay
> > bouncycastle:
> > bcpkix-jdk15on-1.54   -> 1.57    okay
> > bcprov-jdk15on-1.54   -> 1.57    okay
> >
> > commons-compress-1.12 -> 1.14    okay
> > guava-20.0            -> 22.0rc1 okay
> > icu4j-58.1            -> 59_1    okay
> > Netty 4.1.6           -> 4.1.11  okay
> > java-sizeof-0.0.5     -- still current
> > jffi-1.2.13           -> 1.2.15  okay
> > jnr-ffi-2.1.0         -> 2.1.5   okay
> > jnr-netdb-1.1.6       -- still current
> > jnr-posix-3.0.31      -> 3.0.41  okay
> > jnr-constants-0.9.5   -> 0.9.9   okay
> > New platforms: jffi-aarch64-Linux.jar, jffi-ppc64le-Linux.jar, can be
> added...?
> > Updated various other changed platform specific jars to jffi-1.2.15
> (okay as far as tested)
> > xercesImpl-2.11.0          -- still current
> > jline-2.14.2               -> 2.14.3      okay (didn't try jline-3 this
> time)
> > jarjar-1.4                 -- still current
> > mysql-connector-java-5.1.6 -> 5.1.42      okay
> > postgresql-8.3-603.jdbc4   -> 42.1.1-jre7 okay
> >
> > ----------These failed, so leaving them as it is:
> > Antlr 3.1.3                -> 3.5.2  fails
> > junit-4.10                 -> 4.12 or 4.11 class file for
> org.hamcrest.Matcher not found
> > (staying with 4.10 for now to avoid new dependency on hamcrest-matcher)
> > javax.servlet-api-2.5      -> 3.1.0  fails
> > mockrunner      (better don't touch; whole structure changed)
> > cpptasks        (better don't touch)
> >
> > - will be able to test with Java 7 on Tuesday, because I left my old
> laptop in the office.
> > - will upload a fork containing these updates. Would be good if someone
> else could also test, especially on OSX.
> > Maybe some stuff was not covered by regrtests. However the chance that
> updates solve issues and that they create issues are probably somewhat
> equal and I'd prefer to focus on fixing issues with current versions rather
> than older ones.
> > So I'm strongly for getting this in if regrtests go through on Java 7
> and OSX. 2.7.1RC-phase will be a good opportunity to confirm workability of
> these updates. Any concerns?
> >
> > -Stefan
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>

Re: [Jython-dev] Updating extlibs

[Stefan R. <Ste...@gm...>]

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>Just noticed guava &gt; 20.0 requires Java 8.</div>

<div>Strange thing: When building and running Jython on Java 8 with guava 22.0 all regrtests pass fine, but jar-standalone build is bad. Seems like our ant task fails to copy com.google -&gt; org.python.google. Note that I double and triple checked I adjusted build.xml correctly. Is this related to jarjar? Is it maybe not compatible with Java8 jar-files? This might become a problem at some point.</div>

<div>Studying guava 22.0 release notes more properly showed that one shall use guava-22.0-android.jar to target Java 7 also on non-Android platforms. With guava-22.0-android.jar, building Jython jar-standalone works well indeed. regrtests also seem to pass then.</div>

<div>Ideas?</div>

<div>I&#39;ll update the linked repo from last email with guava-22.0-android.jar...</div>

<div>&nbsp;
<div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);">
<div style="margin: 0 0 10.0px 0;"><b>Gesendet:</b>&nbsp;Montag, 22. Mai 2017 um 17:22 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Jim Baker&quot; &lt;jim...@py...&gt;<br/>
<b>An:</b>&nbsp;&quot;Stefan Richthofer&quot; &lt;Ste...@gm...&gt;<br/>
<b>Cc:</b>&nbsp;&quot;Jython Developers&quot; &lt;jyt...@li...&gt;<br/>
<b>Betreff:</b>&nbsp;Re: [Jython-dev] Updating extlibs</div>

<div>
<div>These changes look good to me. I will test out your patch, but all of this is in line with similar updates we have made in the past, usually around this time in the dev cycle.
<div>&nbsp;</div>

<div>I&#39;m glad that moving to Gradle will make this re-pinning to upstream dependencies much easier going forward!</div>
</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Mon, May 22, 2017 at 8:46 AM, Stefan Richthofer <span>&lt;<a href="mailto:Ste...@gm..." onclick="parent.window.location.href=&#39;Ste...@gm...&#39;; return false;" target="_blank">Ste...@gm...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">I uploaded the mentioned updates to<br/>
<a href="https://github.com/Stewori/jython" target="_blank">https://github.com/Stewori/jython</a>.<br/>
See detailed changes at<br/>
<a href="https://github.com/Stewori/jython/commit/ee2b1263306f779bf8e9499afc6d02267648ac36" target="_blank">https://github.com/Stewori/jython/commit/ee2b1263306f779bf8e9499afc6d02267648ac36</a><br/>
<br/>
This might not yet work smoothly with Java 7, I will check and adjust that tomorrow.<br/>
Some packages were built from source using Java 8 and I&#39;m not sure whether the gradle<br/>
scripts always configured Java 7 source compatibility properly.<br/>
However if some people could test it, especially on OSX, would be nice.<br/>
<br/>
Best<br/>
<br/>
Stefan<br/>
<br/>
<br/>
&gt; Gesendet: Montag, 22. Mai 2017 um 04:07 Uhr<br/>
&gt; Von: &quot;Stefan Richthofer&quot; &lt;<a href="mailto:Ste...@gm..." onclick="parent.window.location.href=&#39;Ste...@gm...&#39;; return false;" target="_blank">Ste...@gm...</a>&gt;<br/>
&gt; An: &quot;Jython Developers&quot; &lt;<a href="mailto:jyt...@li..." onclick="parent.window.location.href=&#39;jyt...@li...&#39;; return false;" target="_blank">jyt...@li...</a>&gt;<br/>
&gt; Betreff: Updating extlibs
<div class="HOEnZb">
<div class="h5">&gt;<br/>
&gt; Hey all,<br/>
&gt; I spent some effort to explore feasibility of updating extlibs, especially minor versions (but sometimes even major version, e.g. guava and icu4j).<br/>
&gt; My results so far, tested with regrtest on Linux and Windows 10 using Java8 (&quot;okay&quot; means I did not observe additional regrtest failures):<br/>
&gt;<br/>
&gt; ASM 5.0.4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 5.2&nbsp;&nbsp;&nbsp;&nbsp; okay<br/>
&gt; bouncycastle:<br/>
&gt; bcpkix-jdk15on-1.54&nbsp;&nbsp; -&gt; 1.57&nbsp;&nbsp;&nbsp; okay<br/>
&gt; bcprov-jdk15on-1.54&nbsp;&nbsp; -&gt; 1.57&nbsp;&nbsp;&nbsp; okay<br/>
&gt;<br/>
&gt; commons-compress-1.12 -&gt; 1.14&nbsp;&nbsp;&nbsp; okay<br/>
&gt; guava-20.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 22.0rc1 okay<br/>
&gt; icu4j-58.1&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 59_1&nbsp;&nbsp;&nbsp; okay<br/>
&gt; Netty 4.1.6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 4.1.11&nbsp; okay<br/>
&gt; java-sizeof-0.0.5&nbsp;&nbsp;&nbsp;&nbsp; -- still current<br/>
&gt; jffi-1.2.13&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 1.2.15&nbsp;&nbsp;okay<br/>
&gt; jnr-ffi-2.1.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 2.1.5&nbsp;&nbsp;&nbsp;okay<br/>
&gt; jnr-netdb-1.1.6&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -- still current<br/>
&gt; jnr-posix-3.0.31&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 3.0.41&nbsp; okay<br/>
&gt; jnr-constants-0.9.5&nbsp;&nbsp; -&gt; 0.9.9&nbsp;&nbsp; okay<br/>
&gt; New platforms: jffi-aarch64-Linux.jar, jffi-ppc64le-Linux.jar, can be added...?<br/>
&gt; Updated various other changed platform specific jars to jffi-1.2.15 (okay as far as tested)<br/>
&gt; xercesImpl-2.11.0&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -- still current<br/>
&gt; jline-2.14.2&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 2.14.3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; okay (didn&#39;t try jline-3 this time)<br/>
&gt; jarjar-1.4&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -- still current<br/>
&gt; mysql-connector-java-5.1.6 -&gt; 5.1.42&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; okay<br/>
&gt; postgresql-8.3-603.jdbc4&nbsp;&nbsp; -&gt; 42.1.1-jre7 okay<br/>
&gt;<br/>
&gt; ----------These failed, so leaving them as it is:<br/>
&gt; Antlr 3.1.3&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 3.5.2&nbsp; fails<br/>
&gt; junit-4.10&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 4.12 or 4.11 class file for org.hamcrest.Matcher not found<br/>
&gt; (staying with 4.10 for now to avoid new dependency on hamcrest-matcher)<br/>
&gt; javax.servlet-api-2.5&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; -&gt; 3.1.0&nbsp; fails<br/>
&gt; mockrunner&nbsp;&nbsp;&nbsp;&nbsp;&nbsp; (better don&#39;t touch; whole structure changed)<br/>
&gt; cpptasks&nbsp; &nbsp; &nbsp; &nbsp; (better don&#39;t touch)<br/>
&gt;<br/>
&gt; - will be able to test with Java 7 on Tuesday, because I left my old laptop in the office.<br/>
&gt; - will upload a fork containing these updates. Would be good if someone else could also test, especially on OSX.<br/>
&gt; Maybe some stuff was not covered by regrtests. However the chance that updates solve issues and that they create issues are probably somewhat equal and I&#39;d prefer to focus on fixing issues with current versions rather than older ones.<br/>
&gt; So I&#39;m strongly for getting this in if regrtests go through on Java 7 and OSX. 2.7.1RC-phase will be a good opportunity to confirm workability of these updates. Any concerns?<br/>
&gt;<br/>
&gt; -Stefan<br/>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div></div></body></html>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Alan K. <jyt...@xh...>]

Hi folks,

Great to see a solid 2.7.1 jython, and work begin in earnest on jython 3.

I have only one small suggestion to make: if jython 2.7.1 is going to be
one of the last 2.7 releases, maybe consider numbering it in a way that
indicates it is derived from the latest version of cpython 2.7.12. This
could indicate that it is as up-to-date as it can be, i.e. not derived from
cpython 2.7.1 and then abandoned.

Perception of abandonment is often a problem for jython: I think it's worth
an effort to counter this mis-perception.

Regards,

Alan.


On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <da...@gm...> wrote:

> Hey Guys,
>
> Regarding Jython3, looks like Isaiah has done a ton of work in 2016 (CCd).
> Not sure how far he progressed, but indeed merging will be hard and
> therefore I'd say we should not diverge further while developing on both
> branches, but instead try to finalize 2.7 and switch to Jython3 full-time.
>
> Feel free to disagree, but here's my thinking on it:
>
>    1. Release Jython 2.7.1
>    2. Modernize the codebase. I think it's important for the project to
>    feel modern for us to attract new contributors.
>    1. Java8 as the minimum (may be too much for Jython2).
>       2. Github/core-workflow
>       3. (Ideally) ANTLR4 for both branches, but worst case, Jython3
>       only. ANTLR3 is not getting much love and ANTLR4 is quite different (does
>       not generate AST).
>       4. Gradle, directory structure.
>    3. Develop Jython3 primarily. Only bugfixes for 2.7 series.
>       1. Target 3.6 (really like the typing improvements).
>       2. Merge JyNI if possible.
>
> Cheers,
> Darjus
>
> On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:
>
>> Thanks all. +1 on the RC. Nearly there with my bit.
>>
>> I have fixed the  test_runpy failure James reported. It's not
>> Linux-specific, just I had to quieten the unlink() error to see it on
>> Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest
>> has been running one last time as I typed. I've pushed to
>> https://bitbucket.org/tournesol/jython-utf8 just now.
>> I will next merge into the Jython trunk. That may not be totally smooth
>> because of the pervasive change. And now I think about it, it's worth a
>> note in NEWS. My time is a little limited today, so it could be much later
>> today or tomorrow evening.
>>
>> Jeff
>>
>> Jeff Allen
>>
>> On 20/05/2017 19:48, Jim Baker wrote:
>>
>> +100
>>
>> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
>> wrote:
>>
>>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>>> and we could potentially utilize libraries available for Java 8.
>>>
>>> Let's get Jeff's work in and do an RC?
>>>
>>> Darjus
>>>
>>> On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...> wrote:
>>>
>>>> I don't necessarily see http://bugs.jython.org/issue2487 as a blocker,
>>>> but it would be nice. It just hasn't come up in real usage, unlike the
>>>> earlier iteration of the bug which Darjus hacked around by busy waiting. I
>>>> did spend some time on trying to get the publication to work without
>>>> racing, using the approach I detail in that bug, but no luck yet. (But
>>>> mostly because an utter lack of time to spend on the issue.)
>>>>
>>>> Merging in Jeff's recent work on Unicode is important and we should get
>>>> it in. I haven't had a chance to test myself, but given Jeff's amazing
>>>> attention to detail, I'm sure it's ready.
>>>>
>>>> The blocker for the RC - because we lost OSX support of setuptools
>>>> support of installed executables - is fixed, as I just finally confirmed:
>>>> http://bugs.jython.org/issue2570
>>>>
>>>>
>>>> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
>>>> Ste...@gm...> wrote:
>>>>
>>>>> AFAIK every release happens by having a successful RC that is renamed
>>>>> to 'release' after a while. So, per definition another RC is inevitable.
>>>>> That said, I suppose we should get http://bugs.jython.org/issue2487
>>>>> fixed before we can release. I guess Jeff's work will be ready until then.
>>>>> At least that decision can be postponed until an RC is actually doable.
>>>>>
>>>>>
>>>>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>>>>> *Von:* "Darjus Loktevic" <da...@gm...>
>>>>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>>>>> jyt...@li...>
>>>>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>>>>
>>>>> Hey Jeff,
>>>>>
>>>>> Sounds good. Let's do another rc but to be honest I'm not even sure
>>>>> the RC matters much if there aren't people trying it except us.
>>>>>
>>>>> Thoughts?
>>>>> Darjus
>>>>>
>>>>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>>>>
>>>>>> Hi Darjus.
>>>>>>
>>>>>> On inclusion, I'm happy to go with the community view, as always. On
>>>>>> one of the related tickets (http://bugs.jython.org/issue1839), Jim
>>>>>> said we'd get it in if timing allowed and there was some user support.
>>>>>>
>>>>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was
>>>>>> unsuccessful, and we're still making changes, so I assume we're talking
>>>>>> about another RC first rather than a release?
>>>>>>
>>>>>> The UTF-8 work is nearly there, but not quite: one Linux defect to
>>>>>> fix, as noted on the same issue by James against the "latin-1" version.
>>>>>> After all the additions in the last couple of weeks (to get full BMP
>>>>>> support), I'm happy to find from my Linux laptop that it is still the only
>>>>>> thing I have to do. It looks trivial. I've been unable code at all for a
>>>>>> few days, so haven't looked into a solution, but now I'm back I expect to
>>>>>> nail it for us today or tomorrow.
>>>>>>
>>>>>> I can, of course, merge all this myself and will. I shared your
>>>>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>>>>> I feel it's now low risk, as long as we still have a few days.
>>>>>>
>>>>>> I will now dive under the desk and wire up my Linux dev box.
>>>>>>
>>>>>> Jeff Allen
>>>>>>
>>>>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>>>>
>>>>>> Hey Jeff,
>>>>>>
>>>>>> It seems your last commit to this branch is of three days ago. Is
>>>>>> this ready for review? BTW, your changes look good to me.
>>>>>> I'm a little hesitant to merge this since we've had an RC and REALLY
>>>>>> have to release 2.7.1 It's miles better than 2.7.0.
>>>>>>
>>>>>> Cheers,
>>>>>> Darjus
>>>>>>
>>>>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...> wrote:
>>>>>>
>>>>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works pretty
>>>>>>> well. Rather than just push directly I have published to here:
>>>>>>>
>>>>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>>>>
>>>>>>> I write to ask for a second or third pair of eyes on it. Please tell
>>>>>>> me
>>>>>>> you can see it and whether it breaks things you care about.
>>>>>>>
>>>>>>> I touched a lot of files in the core and import system: quite a lot
>>>>>>> of
>>>>>>> tricky stuff with loaders and search paths has been adjusted. I
>>>>>>> think it
>>>>>>> a good sign that I changed hardly anything in the standard library we
>>>>>>> inherit from CPython, that we hadn't already specialised.
>>>>>>>
>>>>>>> By "works pretty well" above, I mean that the regression tests run
>>>>>>> cleanly for me when my user name is "Épreuve", where previously
>>>>>>> Jython
>>>>>>> died horribly. The launcher works from a Chinese user name too, as
>>>>>>> long
>>>>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>>>>> prompt and runs some tests with that setup, but I can't run the
>>>>>>> regression test yet, and printing a stack dump is fatal, so there's a
>>>>>>> bit more to do for Chinese.
>>>>>>>
>>>>>>> I think this means we have solid support for "latin-1" languages, but
>>>>>>> there are still places where we fatally assume bytes are Unicode code
>>>>>>> points.
>>>>>>>
>>>>>>> Jeff Allen
>>>>>>>
>>>>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd
>>>>>>> like to
>>>>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not
>>>>>>> run for
>>>>>>> > users whose names have an un-American character ;). I know this
>>>>>>> issue is
>>>>>>> > not a blocker in most minds.
>>>>>>> >
>>>>>>> > I've made pretty good progress by allowing file names to be unicode
>>>>>>> > objects more often than they would be in CPython 2, which usually
>>>>>>> > returns them as bytes in some encoding that we may not know. I've
>>>>>>> got
>>>>>>> > the launcher to work properly, and straightened the logic in our
>>>>>>> > printing of trace-backs and exceptions from Java. Unicode file
>>>>>>> names
>>>>>>> > seems the way to go for Jython because:
>>>>>>> >
>>>>>>> >   1. Java gives us competently decoded unicode file names, from
>>>>>>> >      java.io.File, etc.. Re-encoding the result will be a pain (and
>>>>>>> >      overlooked).
>>>>>>> >   2. We appear not to have the codec we need ('mbcs'), that CPython
>>>>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>>>>>> necessary.
>>>>>>> >
>>>>>>> > Most regression tests pass. However, I'm struggling with
>>>>>>> test_doctest.
>>>>>>> > Problems arise when mixing unicode and bytes when one byte is 128
>>>>>>> and
>>>>>>> > over. This happens in ''.join(list) and formatted output like "%s
>>>>>>> %s" %
>>>>>>> > (ustr, bstr). The behaviour of these is identical with CPython:
>>>>>>> they
>>>>>>> > raise UnicodeDecodeError because the bytes are promoted to
>>>>>>> characters
>>>>>>> > with a strict ascii interpretation. This happens a lot in
>>>>>>> doctest.py and
>>>>>>> > traceback.py, for example, where file paths and stack dumps that
>>>>>>> include
>>>>>>> > them, are now frequently unicode, while other inputs are byte data
>>>>>>> > containing file paths presented in the console encoding.
>>>>>>> >
>>>>>>> > I can beat this into submission with enough customisation of the
>>>>>>> stdlib
>>>>>>> > modules, but that always makes me uncomfortable. I usually see
>>>>>>> that as a
>>>>>>> > hint that user code might also need to change. This may be
>>>>>>> unfounded. I
>>>>>>> > can probably ensure no impact to users of only ascii paths, and the
>>>>>>> > others seem unable to run Jython at all (in the scope of this
>>>>>>> issue).
>>>>>>> > However, I'm seriously wondering if I should pursue the approach
>>>>>>> where
>>>>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>>>>> > everywhere), but that's grim.
>>>>>>> >
>>>>>>> > Thoughts?
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ------------------
>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>> _______________________________________________
>>>>>>> Jython-dev mailing list
>>>>>>> Jyt...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>>
>>>>>> ------------------------------------------------------------------------------
>>>>> Check out the vibrant tech community on one of the world's most engaging
>>>>> tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>>> _________________________________________ Jython-dev mailing list
>>>>> Jyt...@li... https://lists.sourceforge.net/
>>>>> lists/listinfo/jython-dev
>>>>>
>>>>> ------------------------------------------------------------
>>>>> ------------------
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> Jython-dev mailing list
>>>>> Jyt...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>
>>>>>
>>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>
>>
>>
>> _______________________________________________
>> Jython-dev mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>>
>>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jim B. <jim...@py...>]

Alan,

That's a great suggestion. 2.7 was specifically chosen to show this
correspondence. In the past, we were not so as focused on compatibility,
but most of the changes — and corresponding delays — in what we have been
planning to call 2.7.1 are because of the continued development on CPython
2.7, by backporting fixes from successive versions of CPython 3.

So calling it 2.7.12 helps illustrate this. Any other thoughts on Alan's
proposal?

- Jim

On Mon, May 22, 2017 at 11:25 AM, Alan Kennedy <jyt...@xh...> wrote:

> Hi folks,
>
> Great to see a solid 2.7.1 jython, and work begin in earnest on jython 3.
>
> I have only one small suggestion to make: if jython 2.7.1 is going to be
> one of the last 2.7 releases, maybe consider numbering it in a way that
> indicates it is derived from the latest version of cpython 2.7.12. This
> could indicate that it is as up-to-date as it can be, i.e. not derived from
> cpython 2.7.1 and then abandoned.
>
> Perception of abandonment is often a problem for jython: I think it's
> worth an effort to counter this mis-perception.
>
> Regards,
>
> Alan.
>
>
> On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <da...@gm...> wrote:
>
>> Hey Guys,
>>
>> Regarding Jython3, looks like Isaiah has done a ton of work in 2016
>> (CCd). Not sure how far he progressed, but indeed merging will be hard and
>> therefore I'd say we should not diverge further while developing on both
>> branches, but instead try to finalize 2.7 and switch to Jython3 full-time.
>>
>> Feel free to disagree, but here's my thinking on it:
>>
>>    1. Release Jython 2.7.1
>>    2. Modernize the codebase. I think it's important for the project to
>>    feel modern for us to attract new contributors.
>>    1. Java8 as the minimum (may be too much for Jython2).
>>       2. Github/core-workflow
>>       3. (Ideally) ANTLR4 for both branches, but worst case, Jython3
>>       only. ANTLR3 is not getting much love and ANTLR4 is quite different (does
>>       not generate AST).
>>       4. Gradle, directory structure.
>>    3. Develop Jython3 primarily. Only bugfixes for 2.7 series.
>>       1. Target 3.6 (really like the typing improvements).
>>       2. Merge JyNI if possible.
>>
>> Cheers,
>> Darjus
>>
>> On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:
>>
>>> Thanks all. +1 on the RC. Nearly there with my bit.
>>>
>>> I have fixed the  test_runpy failure James reported. It's not
>>> Linux-specific, just I had to quieten the unlink() error to see it on
>>> Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest
>>> has been running one last time as I typed. I've pushed to
>>> https://bitbucket.org/tournesol/jython-utf8 just now.
>>> I will next merge into the Jython trunk. That may not be totally smooth
>>> because of the pervasive change. And now I think about it, it's worth a
>>> note in NEWS. My time is a little limited today, so it could be much later
>>> today or tomorrow evening.
>>>
>>> Jeff
>>>
>>> Jeff Allen
>>>
>>> On 20/05/2017 19:48, Jim Baker wrote:
>>>
>>> +100
>>>
>>> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
>>> wrote:
>>>
>>>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>>>> and we could potentially utilize libraries available for Java 8.
>>>>
>>>> Let's get Jeff's work in and do an RC?
>>>>
>>>> Darjus
>>>>
>>>> On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...> wrote:
>>>>
>>>>> I don't necessarily see http://bugs.jython.org/issue2487 as a
>>>>> blocker, but it would be nice. It just hasn't come up in real usage, unlike
>>>>> the earlier iteration of the bug which Darjus hacked around by busy
>>>>> waiting. I did spend some time on trying to get the publication to work
>>>>> without racing, using the approach I detail in that bug, but no luck yet.
>>>>> (But mostly because an utter lack of time to spend on the issue.)
>>>>>
>>>>> Merging in Jeff's recent work on Unicode is important and we should
>>>>> get it in. I haven't had a chance to test myself, but given Jeff's amazing
>>>>> attention to detail, I'm sure it's ready.
>>>>>
>>>>> The blocker for the RC - because we lost OSX support of setuptools
>>>>> support of installed executables - is fixed, as I just finally confirmed:
>>>>> http://bugs.jython.org/issue2570
>>>>>
>>>>>
>>>>> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
>>>>> Ste...@gm...> wrote:
>>>>>
>>>>>> AFAIK every release happens by having a successful RC that is renamed
>>>>>> to 'release' after a while. So, per definition another RC is inevitable.
>>>>>> That said, I suppose we should get http://bugs.jython.org/issue2487
>>>>>> fixed before we can release. I guess Jeff's work will be ready until then.
>>>>>> At least that decision can be postponed until an RC is actually doable.
>>>>>>
>>>>>>
>>>>>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>>>>>> *Von:* "Darjus Loktevic" <da...@gm...>
>>>>>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>>>>>> jyt...@li...>
>>>>>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>>>>>
>>>>>> Hey Jeff,
>>>>>>
>>>>>> Sounds good. Let's do another rc but to be honest I'm not even sure
>>>>>> the RC matters much if there aren't people trying it except us.
>>>>>>
>>>>>> Thoughts?
>>>>>> Darjus
>>>>>>
>>>>>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>>>>>
>>>>>>> Hi Darjus.
>>>>>>>
>>>>>>> On inclusion, I'm happy to go with the community view, as always. On
>>>>>>> one of the related tickets (http://bugs.jython.org/issue1839), Jim
>>>>>>> said we'd get it in if timing allowed and there was some user support.
>>>>>>>
>>>>>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was
>>>>>>> unsuccessful, and we're still making changes, so I assume we're talking
>>>>>>> about another RC first rather than a release?
>>>>>>>
>>>>>>> The UTF-8 work is nearly there, but not quite: one Linux defect to
>>>>>>> fix, as noted on the same issue by James against the "latin-1" version.
>>>>>>> After all the additions in the last couple of weeks (to get full BMP
>>>>>>> support), I'm happy to find from my Linux laptop that it is still the only
>>>>>>> thing I have to do. It looks trivial. I've been unable code at all for a
>>>>>>> few days, so haven't looked into a solution, but now I'm back I expect to
>>>>>>> nail it for us today or tomorrow.
>>>>>>>
>>>>>>> I can, of course, merge all this myself and will. I shared your
>>>>>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>>>>>> I feel it's now low risk, as long as we still have a few days.
>>>>>>>
>>>>>>> I will now dive under the desk and wire up my Linux dev box.
>>>>>>>
>>>>>>> Jeff Allen
>>>>>>>
>>>>>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>>>>>
>>>>>>> Hey Jeff,
>>>>>>>
>>>>>>> It seems your last commit to this branch is of three days ago. Is
>>>>>>> this ready for review? BTW, your changes look good to me.
>>>>>>> I'm a little hesitant to merge this since we've had an RC and REALLY
>>>>>>> have to release 2.7.1 It's miles better than 2.7.0.
>>>>>>>
>>>>>>> Cheers,
>>>>>>> Darjus
>>>>>>>
>>>>>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works
>>>>>>>> pretty
>>>>>>>> well. Rather than just push directly I have published to here:
>>>>>>>>
>>>>>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>>>>>
>>>>>>>> I write to ask for a second or third pair of eyes on it. Please
>>>>>>>> tell me
>>>>>>>> you can see it and whether it breaks things you care about.
>>>>>>>>
>>>>>>>> I touched a lot of files in the core and import system: quite a lot
>>>>>>>> of
>>>>>>>> tricky stuff with loaders and search paths has been adjusted. I
>>>>>>>> think it
>>>>>>>> a good sign that I changed hardly anything in the standard library
>>>>>>>> we
>>>>>>>> inherit from CPython, that we hadn't already specialised.
>>>>>>>>
>>>>>>>> By "works pretty well" above, I mean that the regression tests run
>>>>>>>> cleanly for me when my user name is "Épreuve", where previously
>>>>>>>> Jython
>>>>>>>> died horribly. The launcher works from a Chinese user name too, as
>>>>>>>> long
>>>>>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>>>>>> prompt and runs some tests with that setup, but I can't run the
>>>>>>>> regression test yet, and printing a stack dump is fatal, so there's
>>>>>>>> a
>>>>>>>> bit more to do for Chinese.
>>>>>>>>
>>>>>>>> I think this means we have solid support for "latin-1" languages,
>>>>>>>> but
>>>>>>>> there are still places where we fatally assume bytes are Unicode
>>>>>>>> code
>>>>>>>> points.
>>>>>>>>
>>>>>>>> Jeff Allen
>>>>>>>>
>>>>>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>>>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd
>>>>>>>> like to
>>>>>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not
>>>>>>>> run for
>>>>>>>> > users whose names have an un-American character ;). I know this
>>>>>>>> issue is
>>>>>>>> > not a blocker in most minds.
>>>>>>>> >
>>>>>>>> > I've made pretty good progress by allowing file names to be
>>>>>>>> unicode
>>>>>>>> > objects more often than they would be in CPython 2, which usually
>>>>>>>> > returns them as bytes in some encoding that we may not know. I've
>>>>>>>> got
>>>>>>>> > the launcher to work properly, and straightened the logic in our
>>>>>>>> > printing of trace-backs and exceptions from Java. Unicode file
>>>>>>>> names
>>>>>>>> > seems the way to go for Jython because:
>>>>>>>> >
>>>>>>>> >   1. Java gives us competently decoded unicode file names, from
>>>>>>>> >      java.io.File, etc.. Re-encoding the result will be a pain
>>>>>>>> (and
>>>>>>>> >      overlooked).
>>>>>>>> >   2. We appear not to have the codec we need ('mbcs'), that
>>>>>>>> CPython
>>>>>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>>>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode if
>>>>>>>> necessary.
>>>>>>>> >
>>>>>>>> > Most regression tests pass. However, I'm struggling with
>>>>>>>> test_doctest.
>>>>>>>> > Problems arise when mixing unicode and bytes when one byte is 128
>>>>>>>> and
>>>>>>>> > over. This happens in ''.join(list) and formatted output like "%s
>>>>>>>> %s" %
>>>>>>>> > (ustr, bstr). The behaviour of these is identical with CPython:
>>>>>>>> they
>>>>>>>> > raise UnicodeDecodeError because the bytes are promoted to
>>>>>>>> characters
>>>>>>>> > with a strict ascii interpretation. This happens a lot in
>>>>>>>> doctest.py and
>>>>>>>> > traceback.py, for example, where file paths and stack dumps that
>>>>>>>> include
>>>>>>>> > them, are now frequently unicode, while other inputs are byte data
>>>>>>>> > containing file paths presented in the console encoding.
>>>>>>>> >
>>>>>>>> > I can beat this into submission with enough customisation of the
>>>>>>>> stdlib
>>>>>>>> > modules, but that always makes me uncomfortable. I usually see
>>>>>>>> that as a
>>>>>>>> > hint that user code might also need to change. This may be
>>>>>>>> unfounded. I
>>>>>>>> > can probably ensure no impact to users of only ascii paths, and
>>>>>>>> the
>>>>>>>> > others seem unable to run Jython at all (in the scope of this
>>>>>>>> issue).
>>>>>>>> > However, I'm seriously wondering if I should pursue the approach
>>>>>>>> where
>>>>>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>>>>>> > everywhere), but that's grim.
>>>>>>>> >
>>>>>>>> > Thoughts?
>>>>>>>> >
>>>>>>>>
>>>>>>>>
>>>>>>>> ------------------------------------------------------------
>>>>>>>> ------------------
>>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>>> _______________________________________________
>>>>>>>> Jython-dev mailing list
>>>>>>>> Jyt...@li...
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>>>
>>>>>>> ------------------------------------------------------------------------------
>>>>>> Check out the vibrant tech community on one of the world's most engaging
>>>>>> tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>>>> _________________________________________ Jython-dev mailing list
>>>>>> Jyt...@li... https://lists.sourceforge.net/
>>>>>> lists/listinfo/jython-dev
>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ------------------
>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>> _______________________________________________
>>>>>> Jython-dev mailing list
>>>>>> Jyt...@li...
>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>>
>>>>>>
>>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>
>>>
>>>
>>> _______________________________________________
>>> Jython-dev mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>
>>>
>>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Jython-dev mailing list
>> Jyt...@li...
>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>
>>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Javen O'N. <jav...@gm...>]

+0.5

Calling it 2.7.12 implicitly claims that it is 100% compatible with CPython
2.7.12. If you need to make a small security fix or bug fix, would it be
released as 2.7.12.1 or 2.7.13?

On May 22, 2017 10:51, "Jim Baker" <jim...@py...> wrote:

> Alan,
>
> That's a great suggestion. 2.7 was specifically chosen to show this
> correspondence. In the past, we were not so as focused on compatibility,
> but most of the changes — and corresponding delays — in what we have been
> planning to call 2.7.1 are because of the continued development on CPython
> 2.7, by backporting fixes from successive versions of CPython 3.
>
> So calling it 2.7.12 helps illustrate this. Any other thoughts on Alan's
> proposal?
>
> - Jim
>
> On Mon, May 22, 2017 at 11:25 AM, Alan Kennedy <jyt...@xh...>
> wrote:
>
>> Hi folks,
>>
>> Great to see a solid 2.7.1 jython, and work begin in earnest on jython 3.
>>
>> I have only one small suggestion to make: if jython 2.7.1 is going to be
>> one of the last 2.7 releases, maybe consider numbering it in a way that
>> indicates it is derived from the latest version of cpython 2.7.12. This
>> could indicate that it is as up-to-date as it can be, i.e. not derived from
>> cpython 2.7.1 and then abandoned.
>>
>> Perception of abandonment is often a problem for jython: I think it's
>> worth an effort to counter this mis-perception.
>>
>> Regards,
>>
>> Alan.
>>
>>
>> On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <da...@gm...>
>> wrote:
>>
>>> Hey Guys,
>>>
>>> Regarding Jython3, looks like Isaiah has done a ton of work in 2016
>>> (CCd). Not sure how far he progressed, but indeed merging will be hard and
>>> therefore I'd say we should not diverge further while developing on both
>>> branches, but instead try to finalize 2.7 and switch to Jython3 full-time.
>>>
>>> Feel free to disagree, but here's my thinking on it:
>>>
>>>    1. Release Jython 2.7.1
>>>    2. Modernize the codebase. I think it's important for the project to
>>>    feel modern for us to attract new contributors.
>>>    1. Java8 as the minimum (may be too much for Jython2).
>>>       2. Github/core-workflow
>>>       3. (Ideally) ANTLR4 for both branches, but worst case, Jython3
>>>       only. ANTLR3 is not getting much love and ANTLR4 is quite different (does
>>>       not generate AST).
>>>       4. Gradle, directory structure.
>>>    3. Develop Jython3 primarily. Only bugfixes for 2.7 series.
>>>       1. Target 3.6 (really like the typing improvements).
>>>       2. Merge JyNI if possible.
>>>
>>> Cheers,
>>> Darjus
>>>
>>> On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:
>>>
>>>> Thanks all. +1 on the RC. Nearly there with my bit.
>>>>
>>>> I have fixed the  test_runpy failure James reported. It's not
>>>> Linux-specific, just I had to quieten the unlink() error to see it on
>>>> Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest
>>>> has been running one last time as I typed. I've pushed to
>>>> https://bitbucket.org/tournesol/jython-utf8 just now.
>>>> I will next merge into the Jython trunk. That may not be totally smooth
>>>> because of the pervasive change. And now I think about it, it's worth a
>>>> note in NEWS. My time is a little limited today, so it could be much later
>>>> today or tomorrow evening.
>>>>
>>>> Jeff
>>>>
>>>> Jeff Allen
>>>>
>>>> On 20/05/2017 19:48, Jim Baker wrote:
>>>>
>>>> +100
>>>>
>>>> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
>>>> wrote:
>>>>
>>>>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>>>>> and we could potentially utilize libraries available for Java 8.
>>>>>
>>>>> Let's get Jeff's work in and do an RC?
>>>>>
>>>>> Darjus
>>>>>
>>>>> On Fri, May 19, 2017 at 7:51 PM Jim Baker <jim...@py...>
>>>>> wrote:
>>>>>
>>>>>> I don't necessarily see http://bugs.jython.org/issue2487 as a
>>>>>> blocker, but it would be nice. It just hasn't come up in real usage, unlike
>>>>>> the earlier iteration of the bug which Darjus hacked around by busy
>>>>>> waiting. I did spend some time on trying to get the publication to work
>>>>>> without racing, using the approach I detail in that bug, but no luck yet.
>>>>>> (But mostly because an utter lack of time to spend on the issue.)
>>>>>>
>>>>>> Merging in Jeff's recent work on Unicode is important and we should
>>>>>> get it in. I haven't had a chance to test myself, but given Jeff's amazing
>>>>>> attention to detail, I'm sure it's ready.
>>>>>>
>>>>>> The blocker for the RC - because we lost OSX support of setuptools
>>>>>> support of installed executables - is fixed, as I just finally confirmed:
>>>>>> http://bugs.jython.org/issue2570
>>>>>>
>>>>>>
>>>>>> On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <
>>>>>> Ste...@gm...> wrote:
>>>>>>
>>>>>>> AFAIK every release happens by having a successful RC that is
>>>>>>> renamed to 'release' after a while. So, per definition another RC is
>>>>>>> inevitable.
>>>>>>> That said, I suppose we should get http://bugs.jython.org/issue2487
>>>>>>> fixed before we can release. I guess Jeff's work will be ready until then.
>>>>>>> At least that decision can be postponed until an RC is actually doable.
>>>>>>>
>>>>>>>
>>>>>>> *Gesendet:* Samstag, 20. Mai 2017 um 02:35 Uhr
>>>>>>> *Von:* "Darjus Loktevic" <da...@gm...>
>>>>>>> *An:* "Jeff Allen" <ja...@fa...>, "Jython Developers" <
>>>>>>> jyt...@li...>
>>>>>>> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
>>>>>>>
>>>>>>> Hey Jeff,
>>>>>>>
>>>>>>> Sounds good. Let's do another rc but to be honest I'm not even sure
>>>>>>> the RC matters much if there aren't people trying it except us.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>> Darjus
>>>>>>>
>>>>>>> On Fri, May 19, 2017, 1:19 AM Jeff Allen <ja...@fa...> wrote:
>>>>>>>
>>>>>>>> Hi Darjus.
>>>>>>>>
>>>>>>>> On inclusion, I'm happy to go with the community view, as always.
>>>>>>>> On one of the related tickets (http://bugs.jython.org/issue1839),
>>>>>>>> Jim said we'd get it in if timing allowed and there was some user support.
>>>>>>>>
>>>>>>>> I'm very keen to see a 2.7.1 too. The last (soft) RC was
>>>>>>>> unsuccessful, and we're still making changes, so I assume we're talking
>>>>>>>> about another RC first rather than a release?
>>>>>>>>
>>>>>>>> The UTF-8 work is nearly there, but not quite: one Linux defect to
>>>>>>>> fix, as noted on the same issue by James against the "latin-1" version.
>>>>>>>> After all the additions in the last couple of weeks (to get full BMP
>>>>>>>> support), I'm happy to find from my Linux laptop that it is still the only
>>>>>>>> thing I have to do. It looks trivial. I've been unable code at all for a
>>>>>>>> few days, so haven't looked into a solution, but now I'm back I expect to
>>>>>>>> nail it for us today or tomorrow.
>>>>>>>>
>>>>>>>> I can, of course, merge all this myself and will. I shared your
>>>>>>>> hesitancy initially, hence the fork repository, but it's turned out so well
>>>>>>>> I feel it's now low risk, as long as we still have a few days.
>>>>>>>>
>>>>>>>> I will now dive under the desk and wire up my Linux dev box.
>>>>>>>>
>>>>>>>> Jeff Allen
>>>>>>>>
>>>>>>>> On 16/05/2017 21:46, Darjus Loktevic wrote:
>>>>>>>>
>>>>>>>> Hey Jeff,
>>>>>>>>
>>>>>>>> It seems your last commit to this branch is of three days ago. Is
>>>>>>>> this ready for review? BTW, your changes look good to me.
>>>>>>>> I'm a little hesitant to merge this since we've had an RC and
>>>>>>>> REALLY have to release 2.7.1 It's miles better than 2.7.0.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Darjus
>>>>>>>>
>>>>>>>> On Mon, May 1, 2017 at 6:34 AM Jeff Allen <ja...@fa...>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I went for sys.getfilesystemencoding() == 'utf-8' and it works
>>>>>>>>> pretty
>>>>>>>>> well. Rather than just push directly I have published to here:
>>>>>>>>>
>>>>>>>>> https://bitbucket.org/tournesol/jython-utf8
>>>>>>>>>
>>>>>>>>> I write to ask for a second or third pair of eyes on it. Please
>>>>>>>>> tell me
>>>>>>>>> you can see it and whether it breaks things you care about.
>>>>>>>>>
>>>>>>>>> I touched a lot of files in the core and import system: quite a
>>>>>>>>> lot of
>>>>>>>>> tricky stuff with loaders and search paths has been adjusted. I
>>>>>>>>> think it
>>>>>>>>> a good sign that I changed hardly anything in the standard library
>>>>>>>>> we
>>>>>>>>> inherit from CPython, that we hadn't already specialised.
>>>>>>>>>
>>>>>>>>> By "works pretty well" above, I mean that the regression tests run
>>>>>>>>> cleanly for me when my user name is "Épreuve", where previously
>>>>>>>>> Jython
>>>>>>>>> died horribly. The launcher works from a Chinese user name too, as
>>>>>>>>> long
>>>>>>>>> as I localise Windows to China (CPython 2.7 feature). I can use the
>>>>>>>>> prompt and runs some tests with that setup, but I can't run the
>>>>>>>>> regression test yet, and printing a stack dump is fatal, so
>>>>>>>>> there's a
>>>>>>>>> bit more to do for Chinese.
>>>>>>>>>
>>>>>>>>> I think this means we have solid support for "latin-1" languages,
>>>>>>>>> but
>>>>>>>>> there are still places where we fatally assume bytes are Unicode
>>>>>>>>> code
>>>>>>>>> points.
>>>>>>>>>
>>>>>>>>> Jeff Allen
>>>>>>>>>
>>>>>>>>> On 05/04/2017 08:57, Jeff Allen wrote:
>>>>>>>>> > I've been working on http://bugs.jython.org/issue2356 which I'd
>>>>>>>>> like to
>>>>>>>>> > get in 2.7.1 -- it seems rather poor that Jython simply does not
>>>>>>>>> run for
>>>>>>>>> > users whose names have an un-American character ;). I know this
>>>>>>>>> issue is
>>>>>>>>> > not a blocker in most minds.
>>>>>>>>> >
>>>>>>>>> > I've made pretty good progress by allowing file names to be
>>>>>>>>> unicode
>>>>>>>>> > objects more often than they would be in CPython 2, which usually
>>>>>>>>> > returns them as bytes in some encoding that we may not know.
>>>>>>>>> I've got
>>>>>>>>> > the launcher to work properly, and straightened the logic in our
>>>>>>>>> > printing of trace-backs and exceptions from Java. Unicode file
>>>>>>>>> names
>>>>>>>>> > seems the way to go for Jython because:
>>>>>>>>> >
>>>>>>>>> >   1. Java gives us competently decoded unicode file names, from
>>>>>>>>> >      java.io.File, etc.. Re-encoding the result will be a pain
>>>>>>>>> (and
>>>>>>>>> >      overlooked).
>>>>>>>>> >   2. We appear not to have the codec we need ('mbcs'), that
>>>>>>>>> CPython
>>>>>>>>> >      reports on Windows via sys.getfilesystemencoding().
>>>>>>>>> >   3. We do this already. In 2.7.0, os.getcwd() returns unicode
>>>>>>>>> if necessary.
>>>>>>>>> >
>>>>>>>>> > Most regression tests pass. However, I'm struggling with
>>>>>>>>> test_doctest.
>>>>>>>>> > Problems arise when mixing unicode and bytes when one byte is
>>>>>>>>> 128 and
>>>>>>>>> > over. This happens in ''.join(list) and formatted output like
>>>>>>>>> "%s %s" %
>>>>>>>>> > (ustr, bstr). The behaviour of these is identical with CPython:
>>>>>>>>> they
>>>>>>>>> > raise UnicodeDecodeError because the bytes are promoted to
>>>>>>>>> characters
>>>>>>>>> > with a strict ascii interpretation. This happens a lot in
>>>>>>>>> doctest.py and
>>>>>>>>> > traceback.py, for example, where file paths and stack dumps that
>>>>>>>>> include
>>>>>>>>> > them, are now frequently unicode, while other inputs are byte
>>>>>>>>> data
>>>>>>>>> > containing file paths presented in the console encoding.
>>>>>>>>> >
>>>>>>>>> > I can beat this into submission with enough customisation of the
>>>>>>>>> stdlib
>>>>>>>>> > modules, but that always makes me uncomfortable. I usually see
>>>>>>>>> that as a
>>>>>>>>> > hint that user code might also need to change. This may be
>>>>>>>>> unfounded. I
>>>>>>>>> > can probably ensure no impact to users of only ascii paths, and
>>>>>>>>> the
>>>>>>>>> > others seem unable to run Jython at all (in the scope of this
>>>>>>>>> issue).
>>>>>>>>> > However, I'm seriously wondering if I should pursue the approach
>>>>>>>>> where
>>>>>>>>> > file names from Java are re-encoded to bytes (maybe as utf-8
>>>>>>>>> > everywhere), but that's grim.
>>>>>>>>> >
>>>>>>>>> > Thoughts?
>>>>>>>>> >
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ------------------
>>>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>>>> _______________________________________________
>>>>>>>>> Jython-dev mailing list
>>>>>>>>> Jyt...@li...
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>>>>
>>>>>>>> ------------------------------------------------------------------------------
>>>>>>> Check out the vibrant tech community on one of the world's most engaging
>>>>>>> tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>>>>> _________________________________________ Jython-dev mailing list
>>>>>>> Jyt...@li... https://lists.sourceforge.net/
>>>>>>> lists/listinfo/jython-dev
>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ------------------
>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>> _______________________________________________
>>>>>>> Jython-dev mailing list
>>>>>>> Jyt...@li...
>>>>>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Jython-dev mailing lis...@li...https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>>
>>>>
>>>>
>>> ------------------------------------------------------------
>>> ------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Jython-dev mailing list
>>> Jyt...@li...
>>> https://lists.sourceforge.net/lists/listinfo/jython-dev
>>>
>>>
>>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Stefan R. <Ste...@gm...>]

<html><head></head><body><div style="font-family: Verdana;font-size: 12.0px;"><div>
<div>If we move away from 2.7.1 at all, the micro version should match the point where we updated python-lib last time IMO. I doubt it is 2.7.12. Maybe it&#39;s 2.7.6 or so I suspect (sorry if I should be wrong). Does someone now an efficient way to look it up?</div>

<div>A different numbering scheme for marketing purposes would be misleading and might disappoint users even more.</div>

<div>&nbsp;</div>

<div>Also. I don&#39;t find Jython 2.7.1 should be the last Jython 2.7 or likewise. There will continue to be (maybe minor) progress and we should release this based on time intervals (6months was the plan, wasn&#39;t it?). IMO it&#39;s not so important that huge progress happens from version to version. Progress from 2.7.0 to 2.7.1 is actually far too large. Much more important is that there is progress at all and that it&#39;s displayed to the community by frequent releases.</div>

<div>&nbsp;</div>

<div>-Stefan</div>

<div>&nbsp;
<div name="quote" style="margin:10px 5px 5px 10px; padding: 10px 0 10px 10px; border-left:2px solid #C3D9E5; word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;">
<div style="margin:0 0 10px 0;"><b>Gesendet:</b>&nbsp;Montag, 22. Mai 2017 um 19:50 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Jim Baker&quot; &lt;jim...@py...&gt;<br/>
<b>An:</b>&nbsp;&quot;Alan Kennedy&quot; &lt;jyt...@xh...&gt;<br/>
<b>Cc:</b>&nbsp;&quot;Darjus Loktevic&quot; &lt;da...@gm...&gt;, &quot;Jeff Allen&quot; &lt;ja...@fa...&gt;, &quot;Stefan Richthofer&quot; &lt;Ste...@gm...&gt;, &quot;Jython Developers&quot; &lt;jyt...@li...&gt;<br/>
<b>Betreff:</b>&nbsp;Re: [Jython-dev] Unicode user and file names (and v2.7.1)</div>

<div name="quoted-content">
<div>Alan,
<div>&nbsp;</div>

<div>That&#39;s a great suggestion. 2.7 was specifically chosen to show this correspondence. In the past, we were not so as focused on compatibility, but most of the changes &mdash; and corresponding delays &mdash; in what we have been planning to call 2.7.1 are because of the continued development on CPython 2.7, by backporting fixes from successive versions of CPython 3.</div>

<div>&nbsp;</div>

<div>So calling it 2.7.12 helps illustrate this. Any other thoughts on Alan&#39;s proposal?</div>

<div>&nbsp;</div>

<div>- Jim</div>
</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Mon, May 22, 2017 at 11:25 AM, Alan Kennedy <span>&lt;<a href="mailto:jyt...@xh..." onclick="parent.window.location.href=&#39;jyt...@xh...&#39;; return false;" target="_blank">jyt...@xh...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>Hi folks,
<div>&nbsp;</div>

<div>Great to see a solid 2.7.1 jython, and work begin in earnest on jython 3.</div>

<div>&nbsp;</div>

<div>I have only one small suggestion to make: if jython 2.7.1 is going to be one of the last 2.7 releases, maybe consider numbering it in a way that indicates it is derived from the latest version of cpython 2.7.12. This could indicate that it is as up-to-date as it can be, i.e. not derived from cpython 2.7.1 and then abandoned.</div>

<div>&nbsp;</div>

<div>Perception of abandonment is often a problem for jython: I think it&#39;s worth an effort to counter this mis-perception.</div>

<div>&nbsp;</div>

<div>Regards,</div>

<div>&nbsp;</div>

<div>Alan.</div>

<div>&nbsp;</div>
</div>

<div class="HOEnZb">
<div class="h5">
<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <span>&lt;<a href="mailto:da...@gm..." onclick="parent.window.location.href=&#39;da...@gm...&#39;; return false;" target="_blank">da...@gm...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<div>
<div>Hey Guys,<br/>
&nbsp;</div>
Regarding Jython3, looks like Isaiah has done a ton of work in 2016 (CCd). Not sure how far he progressed, but indeed merging will be hard and therefore I&#39;d say we should not diverge further while developing on both branches, but instead try to finalize 2.7 and switch to Jython3 full-time.<br/>
&nbsp;</div>
Feel free to disagree, but here&#39;s my thinking on it:

<ol>
	<li>Release Jython 2.7.1</li>
	<li>Modernize the codebase. I think it&#39;s important for the project to feel modern for us to attract new contributors.
	<ol>
		<li>Java8 as the minimum (may be too much for Jython2).</li>
		<li>Github/core-workflow</li>
		<li>(Ideally) ANTLR4 for both branches, but worst case, Jython3 only. ANTLR3 is not getting much love and ANTLR4 is quite different (does not generate AST).</li>
		<li>Gradle, directory structure.</li>
	</ol>
	</li>
	<li>Develop Jython3 primarily. Only bugfixes for 2.7 series.
	<ol>
		<li>Target 3.6 (really like the typing improvements).</li>
		<li>Merge JyNI if possible.</li>
	</ol>
	</li>
</ol>

<div>
<div>Cheers,</div>

<div>Darjus</div>

<div>
<div class="m_6125419342958386497h5">
<div>&nbsp;
<div class="gmail_quote">
<div>On Sat, May 20, 2017 at 11:45 PM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<p>Thanks all. +1 on the RC. Nearly there with my bit.</p>

<p>I have fixed the&nbsp; test_runpy failure James reported. It&#39;s not Linux-specific, just I had to quieten the unlink() error to see it on Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest has been running one last time as I typed. I&#39;ve pushed to <a class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046moz-txt-link-freetext" href="https://bitbucket.org/tournesol/jython-utf8" target="_blank">https://bitbucket.org/tournesol/jython-utf8</a> just now.</p>
I will next merge into the Jython trunk. That may not be totally smooth because of the pervasive change. And now I think about it, it&#39;s worth a note in NEWS. My time is a little limited today, so it could be much later today or tomorrow evening.<br/>
<br/>
Jeff</div>

<div>&nbsp;
<pre class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046moz-signature">Jeff Allen</pre>
</div>

<div>
<div class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046moz-cite-prefix">On 20/05/2017 19:48, Jim Baker wrote:</div>

<blockquote>
<div>+100</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <span>&lt;<a href="mailto:da...@gm..." onclick="parent.window.location.href=&#39;da...@gm...&#39;; return false;" target="_blank">da...@gm...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<div>
<div>Agreed regarding not blocking on 2487. That whole area needs a rewrite and we could potentially utilize libraries available for Java 8.<br/>
&nbsp;</div>
Let&#39;s get Jeff&#39;s work in and do an RC?<br/>
&nbsp;</div>
<span class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046HOEnZb"><font color="#888888">Darjus</font></span></div>

<div class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046HOEnZb">
<div class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046h5">&nbsp;
<div class="gmail_quote">
<div>On Fri, May 19, 2017 at 7:51 PM Jim Baker &lt;<a href="mailto:jim...@py..." onclick="parent.window.location.href=&#39;jim...@py...&#39;; return false;" target="_blank">jim...@py...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>I don&#39;t necessarily see <a href="http://bugs.jython.org/issue2487" target="_blank">http://bugs.jython.org/issue2487</a> as a blocker, but it would be nice. It just hasn&#39;t come up in real usage, unlike the earlier iteration of the bug which Darjus hacked around by busy waiting. I did spend some time on trying to get the publication to work without racing, using the approach I detail in that bug, but no luck yet. (But mostly because an utter lack of time to spend on the issue.)

<div>&nbsp;</div>

<div>Merging in Jeff&#39;s recent work on Unicode is important and we should get it in. I haven&#39;t had a chance to test myself, but given Jeff&#39;s amazing attention to detail, I&#39;m sure it&#39;s ready.</div>

<div>&nbsp;</div>

<div>The blocker for the RC - because we lost OSX support of setuptools support of installed executables - is fixed, as I just finally confirmed:&nbsp;<a href="http://bugs.jython.org/issue2570" target="_blank">http://bugs.jython.org/issue2570</a></div>

<div>&nbsp;</div>
</div>

<div class="gmail_extra">&nbsp;
<div class="gmail_quote">On Fri, May 19, 2017 at 8:26 PM, Stefan Richthofer <span>&lt;<a href="mailto:Ste...@gm..." onclick="parent.window.location.href=&#39;Ste...@gm...&#39;; return false;" target="_blank">Ste...@gm...</a>&gt;</span> wrote:

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<div style="font-family: Verdana;font-size: 12.0px;">
<div>
<div>AFAIK every release happens by having a successful RC that is renamed to &#39;release&#39; after a while. So, per definition another RC is inevitable.</div>

<div>That said, I suppose we should get <a href="http://bugs.jython.org/issue2487" target="_blank">http://bugs.jython.org/issue2487</a> fixed before we can release. I guess Jeff&#39;s work will be ready until then. At least that decision can be postponed until an RC is actually doable.</div>

<div>
<div>&nbsp;</div>

<div>&nbsp;</div>

<div style="margin: 10.0px 5.0px 5.0px 10.0px;padding: 10.0px 0 10.0px 10.0px;border-left: 2.0px solid rgb(195,217,229);">
<div style="margin: 0 0 10.0px 0;"><b>Gesendet:</b>&nbsp;Samstag, 20. Mai 2017 um 02:35 Uhr<br/>
<b>Von:</b>&nbsp;&quot;Darjus Loktevic&quot; &lt;<a href="mailto:da...@gm..." onclick="parent.window.location.href=&#39;da...@gm...&#39;; return false;" target="_blank">da...@gm...</a>&gt;<br/>
<b>An:</b>&nbsp;&quot;Jeff Allen&quot; &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt;, &quot;Jython Developers&quot; &lt;<a href="mailto:jyt...@li..." onclick="parent.window.location.href=&#39;jyt...@li...&#39;; return false;" target="_blank">jyt...@li...</a>&gt;<br/>
<b>Betreff:</b>&nbsp;Re: [Jython-dev] Unicode user and file names (and v2.7.1)</div>

<div>
<div>
<div class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046m_8810925058365348314m_-8779070674509468614h5">
<p>Hey Jeff,</p>

<p>Sounds good. Let&#39;s do another rc but to be honest I&#39;m not even sure the RC matters much if there aren&#39;t people trying it except us.</p>

<p>Thoughts?<br/>
Darjus</p>
&nbsp;

<div class="gmail_quote">
<div>On Fri, May 19, 2017, 1:19 AM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">
<div>
<p>Hi Darjus.</p>

<p>On inclusion, I&#39;m happy to go with the community view, as always. On one of the related tickets (<a class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046m_8810925058365348314m_-8779070674509468614m_-528727688133917487m_1723547812866307292moz-txt-link-freetext" href="http://bugs.jython.org/issue1839" target="_blank">http://bugs.jython.org/issue1839</a>), Jim said we&#39;d get it in if timing allowed and there was some user support.</p>

<p>I&#39;m very keen to see a 2.7.1 too. The last (soft) RC was unsuccessful, and we&#39;re still making changes, so I assume we&#39;re talking about another RC first rather than a release?</p>

<p>The UTF-8 work is nearly there, but not quite: one Linux defect to fix, as noted on the same issue by James against the &quot;latin-1&quot; version. After all the additions in the last couple of weeks (to get full BMP support), I&#39;m happy to find from my Linux laptop that it is still the only thing I have to do. It looks trivial. I&#39;ve been unable code at all for a few days, so haven&#39;t looked into a solution, but now I&#39;m back I expect to nail it for us today or tomorrow.</p>

<p>I can, of course, merge all this myself and will. I shared your hesitancy initially, hence the fork repository, but it&#39;s turned out so well I feel it&#39;s now low risk, as long as we still have a few days.</p>

<p>I will now dive under the desk and wire up my Linux dev box.</p>
</div>

<div>
<pre class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046m_8810925058365348314m_-8779070674509468614m_-528727688133917487m_1723547812866307292moz-signature">Jeff Allen</pre>
</div>

<div>
<div class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046m_8810925058365348314m_-8779070674509468614m_-528727688133917487m_1723547812866307292moz-cite-prefix">On 16/05/2017 21:46, Darjus Loktevic wrote:</div>

<blockquote>
<div>
<div>
<div>
<div>Hey Jeff,<br/>
&nbsp;</div>
It seems your last commit to this branch is of three days ago. Is this ready for review? BTW, your changes look good to me.<br/>
I&#39;m a little hesitant to merge this since we&#39;ve had an RC and REALLY have to release 2.7.1 It&#39;s miles better than 2.7.0.<br/>
&nbsp;</div>
Cheers,</div>
Darjus</div>
&nbsp;

<div class="gmail_quote">
<div>On Mon, May 1, 2017 at 6:34 AM Jeff Allen &lt;<a href="mailto:ja...@fa..." onclick="parent.window.location.href=&#39;ja...@fa...&#39;; return false;" target="_blank">ja...@fa...</a>&gt; wrote:</div>

<blockquote class="gmail_quote" style="margin: 0 0 0 0.8ex;border-left: 1.0px rgb(204,204,204) solid;padding-left: 1.0ex;">I went for sys.getfilesystemencoding() == &#39;utf-8&#39; and it works pretty<br/>
well. Rather than just push directly I have published to here:<br/>
<br/>
<a href="https://bitbucket.org/tournesol/jython-utf8" target="_blank">https://bitbucket.org/tournesol/jython-utf8</a><br/>
<br/>
I write to ask for a second or third pair of eyes on it. Please tell me<br/>
you can see it and whether it breaks things you care about.<br/>
<br/>
I touched a lot of files in the core and import system: quite a lot of<br/>
tricky stuff with loaders and search paths has been adjusted. I think it<br/>
a good sign that I changed hardly anything in the standard library we<br/>
inherit from CPython, that we hadn&#39;t already specialised.<br/>
<br/>
By &quot;works pretty well&quot; above, I mean that the regression tests run<br/>
cleanly for me when my user name is &quot;&Eacute;preuve&quot;, where previously Jython<br/>
died horribly. The launcher works from a Chinese user name too, as long<br/>
as I localise Windows to China (CPython 2.7 feature). I can use the<br/>
prompt and runs some tests with that setup, but I can&#39;t run the<br/>
regression test yet, and printing a stack dump is fatal, so there&#39;s a<br/>
bit more to do for Chinese.<br/>
<br/>
I think this means we have solid support for &quot;latin-1&quot; languages, but<br/>
there are still places where we fatally assume bytes are Unicode code<br/>
points.<br/>
<br/>
Jeff Allen<br/>
<br/>
On 05/04/2017 08:57, Jeff Allen wrote:<br/>
&gt; I&#39;ve been working on <a href="http://bugs.jython.org/issue2356" target="_blank">http://bugs.jython.org/issue2356</a> which I&#39;d like to<br/>
&gt; get in 2.7.1 -- it seems rather poor that Jython simply does not run for<br/>
&gt; users whose names have an un-American character ;). I know this issue is<br/>
&gt; not a blocker in most minds.<br/>
&gt;<br/>
&gt; I&#39;ve made pretty good progress by allowing file names to be unicode<br/>
&gt; objects more often than they would be in CPython 2, which usually<br/>
&gt; returns them as bytes in some encoding that we may not know. I&#39;ve got<br/>
&gt; the launcher to work properly, and straightened the logic in our<br/>
&gt; printing of trace-backs and exceptions from Java. Unicode file names<br/>
&gt; seems the way to go for Jython because:<br/>
&gt;<br/>
&gt;&nbsp; &nbsp;1. Java gives us competently decoded unicode file names, from<br/>
&gt;&nbsp; &nbsp; &nbsp; java.io.File, etc.. Re-encoding the result will be a pain (and<br/>
&gt;&nbsp; &nbsp; &nbsp; overlooked).<br/>
&gt;&nbsp; &nbsp;2. We appear not to have the codec we need (&#39;mbcs&#39;), that CPython<br/>
&gt;&nbsp; &nbsp; &nbsp; reports on Windows via sys.getfilesystemencoding().<br/>
&gt;&nbsp; &nbsp;3. We do this already. In 2.7.0, os.getcwd() returns unicode if necessary.<br/>
&gt;<br/>
&gt; Most regression tests pass. However, I&#39;m struggling with test_doctest.<br/>
&gt; Problems arise when mixing unicode and bytes when one byte is 128 and<br/>
&gt; over. This happens in &#39;&#39;.join(list) and formatted output like &quot;%s %s&quot; %<br/>
&gt; (ustr, bstr). The behaviour of these is identical with CPython: they<br/>
&gt; raise UnicodeDecodeError because the bytes are promoted to characters<br/>
&gt; with a strict ascii interpretation. This happens a lot in doctest.py and<br/>
&gt; traceback.py, for example, where file paths and stack dumps that include<br/>
&gt; them, are now frequently unicode, while other inputs are byte data<br/>
&gt; containing file paths presented in the console encoding.<br/>
&gt;<br/>
&gt; I can beat this into submission with enough customisation of the stdlib<br/>
&gt; modules, but that always makes me uncomfortable. I usually see that as a<br/>
&gt; hint that user code might also need to change. This may be unfounded. I<br/>
&gt; can probably ensure no impact to users of only ascii paths, and the<br/>
&gt; others seem unable to run Jython at all (in the scope of this issue).<br/>
&gt; However, I&#39;m seriously wondering if I should pursue the approach where<br/>
&gt; file names from Java are re-encoded to bytes (maybe as utf-8<br/>
&gt; everywhere), but that&#39;s grim.<br/>
&gt;<br/>
&gt; Thoughts?<br/>
&gt;<br/>
<br/>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></blockquote>
</div>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
------------------------------------------------------------------------------ Check out the vibrant tech community on one of the world&#39;s most engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot_______________________________________________" target="_blank">http://sdm.link/slashdot_______________________________________________</a> Jython-dev mailing list <a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a> <a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a></div>
</div>
</div>
</div>
</div>
</div>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a><br/>
&nbsp;</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
&nbsp;

<fieldset class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046mimeAttachmentHeader">&nbsp;</fieldset>
&nbsp;

<pre>------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world&#39;s most
engaging tech sites, Slashdot.org! <a class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046moz-txt-link-freetext" href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a></pre>
&nbsp;

<fieldset class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046mimeAttachmentHeader">&nbsp;</fieldset>
&nbsp;

<pre>_______________________________________________
Jython-dev mailing list
<a class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046moz-txt-link-abbreviated" href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a>
<a class="m_6125419342958386497m_274999079042873323m_8002637137386272236m_9159863954202074352m_-2554705276724151046moz-txt-link-freetext" href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a>
</pre>
</blockquote>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div>
<br/>
------------------------------------------------------------------------------<br/>
Check out the vibrant tech community on one of the world&#39;s most<br/>
engaging tech sites, Slashdot.org! <a href="http://sdm.link/slashdot" target="_blank">http://sdm.link/slashdot</a><br/>
_______________________________________________<br/>
Jython-dev mailing list<br/>
<a href="mailto:Jyt...@li..." onclick="parent.window.location.href=&#39;Jyt...@li...&#39;; return false;" target="_blank">Jyt...@li...</a><br/>
<a href="https://lists.sourceforge.net/lists/listinfo/jython-dev" target="_blank">https://lists.sourceforge.net/lists/listinfo/jython-dev</a><br/>
&nbsp;</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</div>
</div></div></body></html>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Jeff A. <ja...@fa...>]

The numbering scheme for Python versions is quite well known.

https://docs.python.org/3/faq/general.html#how-does-the-python-version-numbering-scheme-work

One ought not to interpret the micro-version as more than bug-fix 
numbering. While I don't see a prohibition against skipping micro 
numbers I think it never seemed sensible to CPython when fear of double 
digits was an issue. So I start out opposed to the idea.

However, I recognise that 2.7, because there will never be a 2.8, has 
become a bit of a special case, and the micro-version might be read as a 
maturity claim. I like the logic of using the provenance of  
~/lib-python/2.7/test. So for "clarity in marketing" I'm not against a 
modest vorsprung.

I don't think Jython 2.7.1 is our last release in that version of the 
language. I'm sure we'll do more bug-fixes, and some back-ports from 3. 
(Like others, I've told myself that after this release, I'll give some 
time to 3.)

... so now, I find I've agreed with everything Stefan said!

To answer Stefan's question, the last mass update of lib-python was 
2013/03/09 (https://hg.python.org/jython/rev/f763cd15ee2b). I don't 
believe the CPython change set it cites (README says its a v3.4), while 
CPython 2.7.4 was in beta at that date. We've pulled in specific parts 
of the CPython library since then to meet specific needs, but made no 
mass update. This alone seems a good reason for a further release.

Jeff Allen

On 22/05/2017 19:03, Stefan Richthofer wrote:
> If we move away from 2.7.1 at all, the micro version should match the 
> point where we updated python-lib last time IMO. I doubt it is 2.7.12. 
> Maybe it's 2.7.6 or so I suspect (sorry if I should be wrong). Does 
> someone now an efficient way to look it up?
> A different numbering scheme for marketing purposes would be 
> misleading and might disappoint users even more.
> Also. I don't find Jython 2.7.1 should be the last Jython 2.7 or 
> likewise. There will continue to be (maybe minor) progress and we 
> should release this based on time intervals (6months was the plan, 
> wasn't it?). IMO it's not so important that huge progress happens from 
> version to version. Progress from 2.7.0 to 2.7.1 is actually far too 
> large. Much more important is that there is progress at all and that 
> it's displayed to the community by frequent releases.
> -Stefan
> *Gesendet:* Montag, 22. Mai 2017 um 19:50 Uhr
> *Von:* "Jim Baker" <jim...@py...>
> *An:* "Alan Kennedy" <jyt...@xh...>
> *Cc:* "Darjus Loktevic" <da...@gm...>, "Jeff Allen" 
> <ja...@fa...>, "Stefan Richthofer" <Ste...@gm...>, 
> "Jython Developers" <jyt...@li...>
> *Betreff:* Re: [Jython-dev] Unicode user and file names (and v2.7.1)
> Alan,
> That's a great suggestion. 2.7 was specifically chosen to show this 
> correspondence. In the past, we were not so as focused on 
> compatibility, but most of the changes — and corresponding delays — in 
> what we have been planning to call 2.7.1 are because of the continued 
> development on CPython 2.7, by backporting fixes from successive 
> versions of CPython 3.
> So calling it 2.7.12 helps illustrate this. Any other thoughts on 
> Alan's proposal?
> - Jim
> On Mon, May 22, 2017 at 11:25 AM, Alan Kennedy <jyt...@xh... 
> <mailto:jyt...@xh...>> wrote:
>
>     Hi folks,
>     Great to see a solid 2.7.1 jython, and work begin in earnest on
>     jython 3.
>     I have only one small suggestion to make: if jython 2.7.1 is going
>     to be one of the last 2.7 releases, maybe consider numbering it in
>     a way that indicates it is derived from the latest version of
>     cpython 2.7.12. This could indicate that it is as up-to-date as it
>     can be, i.e. not derived from cpython 2.7.1 and then abandoned.
>     Perception of abandonment is often a problem for jython: I think
>     it's worth an effort to counter this mis-perception.
>     Regards,
>     Alan.
>     On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <da...@gm...
>     <mailto:da...@gm...>> wrote:
>
>         Hey Guys,
>         Regarding Jython3, looks like Isaiah has done a ton of work in
>         2016 (CCd). Not sure how far he progressed, but indeed merging
>         will be hard and therefore I'd say we should not diverge
>         further while developing on both branches, but instead try to
>         finalize 2.7 and switch to Jython3 full-time.
>         Feel free to disagree, but here's my thinking on it:
>
>          1. Release Jython 2.7.1
>          2. Modernize the codebase. I think it's important for the
>             project to feel modern for us to attract new contributors.
>              1. Java8 as the minimum (may be too much for Jython2).
>              2. Github/core-workflow
>              3. (Ideally) ANTLR4 for both branches, but worst case,
>                 Jython3 only. ANTLR3 is not getting much love and
>                 ANTLR4 is quite different (does not generate AST).
>              4. Gradle, directory structure.
>          3. Develop Jython3 primarily. Only bugfixes for 2.7 series.
>              1. Target 3.6 (really like the typing improvements).
>              2. Merge JyNI if possible.
>
>         Cheers,
>         Darjus
>         On Sat, May 20, 2017 at 11:45 PM Jeff Allen
>         <ja...@fa... <mailto:ja...@fa...>> wrote:
>
>             Thanks all. +1 on the RC. Nearly there with my bit.
>
>             I have fixed the test_runpy failure James reported. It's
>             not Linux-specific, just I had to quieten the unlink()
>             error to see it on Windows. Bonus: we now pass the
>             standard CPython test_runpy. The regrtest has been running
>             one last time as I typed. I've pushed to
>             https://bitbucket.org/tournesol/jython-utf8 just now.
>
>             I will next merge into the Jython trunk. That may not be
>             totally smooth because of the pervasive change. And now I
>             think about it, it's worth a note in NEWS. My time is a
>             little limited today, so it could be much later today or
>             tomorrow evening.
>
>             Jeff
>
>             Jeff Allen
>
>             On 20/05/2017 19:48, Jim Baker wrote:
>
>                 +100
>                 On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic
>                 <da...@gm... <mailto:da...@gm...>> wrote:
>
>                     Agreed regarding not blocking on 2487. That whole
>                     area needs a rewrite and we could potentially
>                     utilize libraries available for Java 8.
>                     Let's get Jeff's work in and do an RC?
>                     Darjus
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[<fwi...@gm...>]

I am in agreement with Stefan as well. I think making 2.7.x
exceptional in that the numbering could convey the version of CPython
since so much extra work went into CPython 2.7.x. I'd be against doing
that for 3.x.x. Though we'd want to be honest and if the reality is
that it is based on (say) 2.7.6 at time of release, that should be the
number until we actually do get up to 2.7.12 (or later).

On Mon, May 22, 2017 at 1:15 PM, Jeff Allen <ja...@fa...> wrote:
> The numbering scheme for Python versions is quite well known.
>
> https://docs.python.org/3/faq/general.html#how-does-the-python-version-numbering-scheme-work
>
> One ought not to interpret the micro-version as more than bug-fix numbering.
> While I don't see a prohibition against skipping micro numbers I think it
> never seemed sensible to CPython when fear of double digits was an issue. So
> I start out opposed to the idea.
>
> However, I recognise that 2.7, because there will never be a 2.8, has become
> a bit of a special case, and the micro-version might be read as a maturity
> claim. I like the logic of using the provenance of  ~/lib-python/2.7/test.
> So for "clarity in marketing" I'm not against a modest vorsprung.
>
> I don't think Jython 2.7.1 is our last release in that version of the
> language. I'm sure we'll do more bug-fixes, and some back-ports from 3.
> (Like others, I've told myself that after this release, I'll give some time
> to 3.)
>
> ... so now, I find I've agreed with everything Stefan said!
>
> To answer Stefan's question, the last mass update of lib-python was
> 2013/03/09 (https://hg.python.org/jython/rev/f763cd15ee2b). I don't believe
> the CPython change set it cites (README says its a v3.4), while CPython
> 2.7.4 was in beta at that date. We've pulled in specific parts of the
> CPython library since then to meet specific needs, but made no mass update.
> This alone seems a good reason for a further release.
>
> Jeff Allen
>
> On 22/05/2017 19:03, Stefan Richthofer wrote:
>
> If we move away from 2.7.1 at all, the micro version should match the point
> where we updated python-lib last time IMO. I doubt it is 2.7.12. Maybe it's
> 2.7.6 or so I suspect (sorry if I should be wrong). Does someone now an
> efficient way to look it up?
> A different numbering scheme for marketing purposes would be misleading and
> might disappoint users even more.
>
> Also. I don't find Jython 2.7.1 should be the last Jython 2.7 or likewise.
> There will continue to be (maybe minor) progress and we should release this
> based on time intervals (6months was the plan, wasn't it?). IMO it's not so
> important that huge progress happens from version to version. Progress from
> 2.7.0 to 2.7.1 is actually far too large. Much more important is that there
> is progress at all and that it's displayed to the community by frequent
> releases.
>
> -Stefan
>
> Gesendet: Montag, 22. Mai 2017 um 19:50 Uhr
> Von: "Jim Baker" <jim...@py...>
> An: "Alan Kennedy" <jyt...@xh...>
> Cc: "Darjus Loktevic" <da...@gm...>, "Jeff Allen" <ja...@fa...>,
> "Stefan Richthofer" <Ste...@gm...>, "Jython Developers"
> <jyt...@li...>
> Betreff: Re: [Jython-dev] Unicode user and file names (and v2.7.1)
> Alan,
>
> That's a great suggestion. 2.7 was specifically chosen to show this
> correspondence. In the past, we were not so as focused on compatibility, but
> most of the changes — and corresponding delays — in what we have been
> planning to call 2.7.1 are because of the continued development on CPython
> 2.7, by backporting fixes from successive versions of CPython 3.
>
> So calling it 2.7.12 helps illustrate this. Any other thoughts on Alan's
> proposal?
>
> - Jim
>
> On Mon, May 22, 2017 at 11:25 AM, Alan Kennedy <jyt...@xh...> wrote:
>>
>> Hi folks,
>>
>> Great to see a solid 2.7.1 jython, and work begin in earnest on jython 3.
>>
>> I have only one small suggestion to make: if jython 2.7.1 is going to be
>> one of the last 2.7 releases, maybe consider numbering it in a way that
>> indicates it is derived from the latest version of cpython 2.7.12. This
>> could indicate that it is as up-to-date as it can be, i.e. not derived from
>> cpython 2.7.1 and then abandoned.
>>
>> Perception of abandonment is often a problem for jython: I think it's
>> worth an effort to counter this mis-perception.
>>
>> Regards,
>>
>> Alan.
>>
>>
>> On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <da...@gm...> wrote:
>>>
>>> Hey Guys,
>>>
>>> Regarding Jython3, looks like Isaiah has done a ton of work in 2016
>>> (CCd). Not sure how far he progressed, but indeed merging will be hard and
>>> therefore I'd say we should not diverge further while developing on both
>>> branches, but instead try to finalize 2.7 and switch to Jython3 full-time.
>>>
>>> Feel free to disagree, but here's my thinking on it:
>>>
>>> Release Jython 2.7.1
>>> Modernize the codebase. I think it's important for the project to feel
>>> modern for us to attract new contributors.
>>>
>>> Java8 as the minimum (may be too much for Jython2).
>>> Github/core-workflow
>>> (Ideally) ANTLR4 for both branches, but worst case, Jython3 only. ANTLR3
>>> is not getting much love and ANTLR4 is quite different (does not generate
>>> AST).
>>> Gradle, directory structure.
>>>
>>> Develop Jython3 primarily. Only bugfixes for 2.7 series.
>>>
>>> Target 3.6 (really like the typing improvements).
>>> Merge JyNI if possible.
>>>
>>> Cheers,
>>> Darjus
>>>
>>> On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:
>>>>
>>>> Thanks all. +1 on the RC. Nearly there with my bit.
>>>>
>>>> I have fixed the  test_runpy failure James reported. It's not
>>>> Linux-specific, just I had to quieten the unlink() error to see it on
>>>> Windows. Bonus: we now pass the standard CPython test_runpy. The regrtest
>>>> has been running one last time as I typed. I've pushed to
>>>> https://bitbucket.org/tournesol/jython-utf8 just now.
>>>>
>>>> I will next merge into the Jython trunk. That may not be totally smooth
>>>> because of the pervasive change. And now I think about it, it's worth a note
>>>> in NEWS. My time is a little limited today, so it could be much later today
>>>> or tomorrow evening.
>>>>
>>>> Jeff
>>>>
>>>>
>>>> Jeff Allen
>>>>
>>>> On 20/05/2017 19:48, Jim Baker wrote:
>>>>
>>>> +100
>>>>
>>>> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
>>>> wrote:
>>>>>
>>>>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>>>>> and we could potentially utilize libraries available for Java 8.
>>>>>
>>>>> Let's get Jeff's work in and do an RC?
>>>>>
>>>>> Darjus
>>>>>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[Javen O'N. <jav...@gm...>]

Based on the discussion, I'm now against a 2.7.12, since the bugfix/micro
part of the number shouldn't need to match CPython.

Furthermore, sys.version and sys.version_info wouldn't be able to handle a
4th number.

On May 22, 2017 17:59, "fwi...@gm..." <fwi...@gm...>
wrote:

I am in agreement with Stefan as well. I think making 2.7.x
exceptional in that the numbering could convey the version of CPython
since so much extra work went into CPython 2.7.x. I'd be against doing
that for 3.x.x. Though we'd want to be honest and if the reality is
that it is based on (say) 2.7.6 at time of release, that should be the
number until we actually do get up to 2.7.12 (or later).

On Mon, May 22, 2017 at 1:15 PM, Jeff Allen <ja...@fa...> wrote:
> The numbering scheme for Python versions is quite well known.
>
> https://docs.python.org/3/faq/general.html#how-does-the-
python-version-numbering-scheme-work
>
> One ought not to interpret the micro-version as more than bug-fix
numbering.
> While I don't see a prohibition against skipping micro numbers I think it
> never seemed sensible to CPython when fear of double digits was an issue.
So
> I start out opposed to the idea.
>
> However, I recognise that 2.7, because there will never be a 2.8, has
become
> a bit of a special case, and the micro-version might be read as a maturity
> claim. I like the logic of using the provenance of  ~/lib-python/2.7/test.
> So for "clarity in marketing" I'm not against a modest vorsprung.
>
> I don't think Jython 2.7.1 is our last release in that version of the
> language. I'm sure we'll do more bug-fixes, and some back-ports from 3.
> (Like others, I've told myself that after this release, I'll give some
time
> to 3.)
>
> ... so now, I find I've agreed with everything Stefan said!
>
> To answer Stefan's question, the last mass update of lib-python was
> 2013/03/09 (https://hg.python.org/jython/rev/f763cd15ee2b). I don't
believe
> the CPython change set it cites (README says its a v3.4), while CPython
> 2.7.4 was in beta at that date. We've pulled in specific parts of the
> CPython library since then to meet specific needs, but made no mass
update.
> This alone seems a good reason for a further release.
>
> Jeff Allen
>
> On 22/05/2017 19:03, Stefan Richthofer wrote:
>
> If we move away from 2.7.1 at all, the micro version should match the
point
> where we updated python-lib last time IMO. I doubt it is 2.7.12. Maybe
it's
> 2.7.6 or so I suspect (sorry if I should be wrong). Does someone now an
> efficient way to look it up?
> A different numbering scheme for marketing purposes would be misleading
and
> might disappoint users even more.
>
> Also. I don't find Jython 2.7.1 should be the last Jython 2.7 or likewise.
> There will continue to be (maybe minor) progress and we should release
this
> based on time intervals (6months was the plan, wasn't it?). IMO it's not
so
> important that huge progress happens from version to version. Progress
from
> 2.7.0 to 2.7.1 is actually far too large. Much more important is that
there
> is progress at all and that it's displayed to the community by frequent
> releases.
>
> -Stefan
>
> Gesendet: Montag, 22. Mai 2017 um 19:50 Uhr
> Von: "Jim Baker" <jim...@py...>
> An: "Alan Kennedy" <jyt...@xh...>
> Cc: "Darjus Loktevic" <da...@gm...>, "Jeff Allen" <ja...@fa...
>,
> "Stefan Richthofer" <Ste...@gm...>, "Jython Developers"
> <jyt...@li...>
> Betreff: Re: [Jython-dev] Unicode user and file names (and v2.7.1)
> Alan,
>
> That's a great suggestion. 2.7 was specifically chosen to show this
> correspondence. In the past, we were not so as focused on compatibility,
but
> most of the changes — and corresponding delays — in what we have been
> planning to call 2.7.1 are because of the continued development on CPython
> 2.7, by backporting fixes from successive versions of CPython 3.
>
> So calling it 2.7.12 helps illustrate this. Any other thoughts on Alan's
> proposal?
>
> - Jim
>
> On Mon, May 22, 2017 at 11:25 AM, Alan Kennedy <jyt...@xh...>
wrote:
>>
>> Hi folks,
>>
>> Great to see a solid 2.7.1 jython, and work begin in earnest on jython 3.
>>
>> I have only one small suggestion to make: if jython 2.7.1 is going to be
>> one of the last 2.7 releases, maybe consider numbering it in a way that
>> indicates it is derived from the latest version of cpython 2.7.12. This
>> could indicate that it is as up-to-date as it can be, i.e. not derived
from
>> cpython 2.7.1 and then abandoned.
>>
>> Perception of abandonment is often a problem for jython: I think it's
>> worth an effort to counter this mis-perception.
>>
>> Regards,
>>
>> Alan.
>>
>>
>> On Sun, May 21, 2017 at 4:35 PM, Darjus Loktevic <da...@gm...>
wrote:
>>>
>>> Hey Guys,
>>>
>>> Regarding Jython3, looks like Isaiah has done a ton of work in 2016
>>> (CCd). Not sure how far he progressed, but indeed merging will be hard
and
>>> therefore I'd say we should not diverge further while developing on both
>>> branches, but instead try to finalize 2.7 and switch to Jython3
full-time.
>>>
>>> Feel free to disagree, but here's my thinking on it:
>>>
>>> Release Jython 2.7.1
>>> Modernize the codebase. I think it's important for the project to feel
>>> modern for us to attract new contributors.
>>>
>>> Java8 as the minimum (may be too much for Jython2).
>>> Github/core-workflow
>>> (Ideally) ANTLR4 for both branches, but worst case, Jython3 only. ANTLR3
>>> is not getting much love and ANTLR4 is quite different (does not
generate
>>> AST).
>>> Gradle, directory structure.
>>>
>>> Develop Jython3 primarily. Only bugfixes for 2.7 series.
>>>
>>> Target 3.6 (really like the typing improvements).
>>> Merge JyNI if possible.
>>>
>>> Cheers,
>>> Darjus
>>>
>>> On Sat, May 20, 2017 at 11:45 PM Jeff Allen <ja...@fa...> wrote:
>>>>
>>>> Thanks all. +1 on the RC. Nearly there with my bit.
>>>>
>>>> I have fixed the  test_runpy failure James reported. It's not
>>>> Linux-specific, just I had to quieten the unlink() error to see it on
>>>> Windows. Bonus: we now pass the standard CPython test_runpy. The
regrtest
>>>> has been running one last time as I typed. I've pushed to
>>>> https://bitbucket.org/tournesol/jython-utf8 just now.
>>>>
>>>> I will next merge into the Jython trunk. That may not be totally smooth
>>>> because of the pervasive change. And now I think about it, it's worth
a note
>>>> in NEWS. My time is a little limited today, so it could be much later
today
>>>> or tomorrow evening.
>>>>
>>>> Jeff
>>>>
>>>>
>>>> Jeff Allen
>>>>
>>>> On 20/05/2017 19:48, Jim Baker wrote:
>>>>
>>>> +100
>>>>
>>>> On Sat, May 20, 2017 at 12:33 PM, Darjus Loktevic <da...@gm...>
>>>> wrote:
>>>>>
>>>>> Agreed regarding not blocking on 2487. That whole area needs a rewrite
>>>>> and we could potentially utilize libraries available for Java 8.
>>>>>
>>>>> Let's get Jeff's work in and do an RC?
>>>>>
>>>>> Darjus
>>>>>
>
>
>
> ------------------------------------------------------------
------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Jython-dev mailing list
> Jyt...@li...
> https://lists.sourceforge.net/lists/listinfo/jython-dev
>

------------------------------------------------------------
------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Jython-dev mailing list
Jyt...@li...
https://lists.sourceforge.net/lists/listinfo/jython-dev

Re: [Jython-dev] Unicode user and file names (and v2.7.1)

[<fwi...@gm...>]

On Sat, May 20, 2017 at 5:00 PM, Stefan Richthofer
<Ste...@gm...> wrote:
> I'd like to have http://bugs.jython.org/issue2536 "fixed" by removing that
> infinite recursion test; maybe we could even close this annoying issue then.
> A new Jython release should have reliable test suite. However I don't know
> the best way to remove or blacklist a certain test.
I don't think I saw anyone answer this (but the thread is long so I
may have missed it) but if you look in regrtest.py you'll see a list
starting around line 880, with one section called 'java'. That's a
list of tests that are deliberately skipped. That list hasn't been
looked at in a long time, and I bet there are tests in there that
actually shouldn't get skipped anymore... but that's a whole different
can of worms :)

-Frank