Re: [Jython-dev] This may interest you. (wrt Jython 3)

[Jeff A. <ja...@fa...>]

Jonathan, Vinay:

thanks for thinking of us. I played with JavaCC a long time ago and 
thought it useful.

I'm working on some fundamental things in the core of Jython 3 at the 
moment. Having (as it were) the engine in parts scattered over the 
garage floor, a compiler and "jump starting" anything, seems a distant 
gleam. :) But obviously we will need a compiler as the whole goes back 
together.

It is true that maintaining a grammar of Python, as the language 
evolves, is work. In that past, this has meant evolving an ANTLR 
grammar. Vinay would be familiar with PEP 617 I guess. The PSF will 
maintain a grammar of Python directly in the form the PEG parser reads. 
My working assumption is that the compiler for Jython 3 will be 
generated by the PEP 617 PEG parser, adapted to generate Java. Our work 
then is in the action routines, when the grammar needs new ones. In fact 
there has been some promising work, not mine, in just this direction 
(private correspondance).

Of course, there are two layers to this. The parser in Python of the PEG 
language needs to emit a parser for Python in Java. And that parser then 
has to emit AST nodes that are objects in Java, but also in Python.

To use JavaCC21 one presumably has to transform the PSF's grammar to 
LL(k) in JavaCC21 notation. I know one can do this, removing left 
recursion at the expense of additional rules. But it might be worse. The 
PEG parser allows unbounded backtracking. One motivation for PEG is to 
be free of the constraints the previous relaxed LL(1) parser imposes. Do 
you not think that in the future, the grammar of Python will develop in 
ways that *only* the PEG parser can deal with, and no LL(k) parser could?

Jeff Allen


On 16/10/2021 14:19, Jonathan Revusky wrote:
> ...
> First of all, JavaCC 21 <https://javacc.com/> is a continuation of 
> development of the venerable JavaCC parser generator released by Sun 
> Microsystems back in the 90's. One could call it a "fork" but in 
> reality, at this point, it is basically a complete rewrite. For the 
> last few months, my main collaborator on the project has been Vinay 
> Sajip, who is a committer on the CPython project. His main focus has 
> been on reworking the JavaCC 21 codebase to be able to support 
> generating parsers in Python -- a PythonCC if you will. And, actually, 
> that subproject is in quite an advanced state and Vinay blogged about 
> it here:
>
> https://blog.red-dove.com/posts/parsing-in-python/ 
> <https://blog.red-dove.com/posts/parsing-in-python/>
>
> ...

> Or, in other words, you guys could just rely on us to handle the 
> parsing side and you can concentrate on the actual run-time internals 
> and bridging the Java and Python object models basically.
>