From: Daniel <dan...@gm...> - 2009-05-30 22:50:58
|
Just to chime in, IntelliJ IDEA is based on ANTLR[1]. AFAIK they wrote custom parser for every language they support out of the box, to have tight integration with the error recovery and to support quick fixes, etc. Cheers, Daniel [1] http://antlr.org/showcase/list On Sun, May 31, 2009 at 5:33 AM, Jim Baker <jb...@zy...> wrote: > Yin, > I'd suggest deriving this from our Antlr-based parser, since Antlr has > ostensibly the best error recovery support for generated parsers. Frank > Wierzbicki knows much more about this, but in looking at Python.g, we just > use this error recovery to better match CPython's syntax errors. > > But Antlr can do much more, specifically around add/delete a single token > to support resynchronization. > http://www.antlr.org/wiki/pages/viewpage.action?pageId=11567208 > Terrence Parr explains his rationale a bit here: > http://www.antlr2.org/blog/antlr3/error.handling.tml > > NetBeans support for Python (CPython/Jython) uses our Antlr grammar, so > there may be some ideas there you can borrow. > > Good luck, this sounds like a fun project! > > - Jim > > On Sat, May 30, 2009 at 11:08 PM, Yin Wang <yin...@gm...> wrote: > >> Thanks a lot! My original purpose is to have a fault-tolerant parser >> to use in an IDE-like environment where code can be checked to the end >> even syntax errors exist, so I'd like to have a parser that has some >> recovery abilities. Could you suggest some parsers that can do that? I >> know Pydev's parsers can recover from syntax errors but they seem to >> be tightly connected with eclipse. Thanks. >> >> >> -- yin >> >> >> >> On Fri, May 29, 2009 at 4:50 PM, Jim Baker <jb...@zy...> wrote: >> > Yin, >> > PythonPartial is not meant to return an AST. Its sole purpose is to >> > determine if the input presented at the console is complete or not. For >> the >> > Python REPL, we need a separate grammar because it's OK if the partial >> parse >> > contains a fragment, such as a multiline string that's not been >> terminated >> > with a """. >> > PythonPartial is generated from PythonPartial.g, exposed by >> ParserFacade, >> > and documented in terms of its usage in the comments for >> > InteractiveInterpreter#runsource. >> > If you compare PythonPartial.g and Python.g, you will see the specifics >> of >> > this (I would not really look at the generated source in >> > PythonPartial.java). PythonPartial.g is just a parser grammar, with some >> > sharing against Python.g for tokens (tokenVocab). There is no AST >> > construction performed. In contrast, Python.g contains the tree >> construction >> > as needed for subsequent compilation, and is consequently a much larger >> > file. >> > - Jim >> > >> > On Sat, May 30, 2009 at 4:28 AM, Yin Wang <yin...@gm...> wrote: >> >> >> >> Hi, >> >> >> >> I'm trying to figure out how to use the partial parser >> >> org.python.antlr.PythonPartial, but I haven't found how to get an AST >> >> from it. It doesn't even refer to any type in the PythonTree >> >> hierarchy. Strange. Any ideas or code examples? The file >> >> org.python.antlr.PythonPartialTester simple succeeds without giving >> >> out any value. It calls single_input() but that method's return type >> >> is void. Thanks. >> >> >> >> >> >> -- yin >> >> >> >> >> >> >> ------------------------------------------------------------------------------ >> >> Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT >> >> is a gathering of tech-side developers & brand creativity >> professionals. >> >> Meet >> >> the minds behind Google Creative Lab, Visual Complexity, Processing, & >> >> iPhoneDevCamp as they present alongside digital heavyweights like >> >> Barbarian >> >> Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com >> >> _______________________________________________ >> >> Jython-dev mailing list >> >> Jyt...@li... >> >> https://lists.sourceforge.net/lists/listinfo/jython-dev >> > >> > >> > >> > -- >> > Jim Baker >> > jb...@zy... >> > >> > > > > -- > Jim Baker > jb...@zy... > > > ------------------------------------------------------------------------------ > Register Now for Creativity and Technology (CaT), June 3rd, NYC. CaT > is a gathering of tech-side developers & brand creativity professionals. > Meet > the minds behind Google Creative Lab, Visual Complexity, Processing, & > iPhoneDevCamp as they present alongside digital heavyweights like Barbarian > Group, R/GA, & Big Spaceship. http://p.sf.net/sfu/creativitycat-com > _______________________________________________ > Jython-dev mailing list > Jyt...@li... > https://lists.sourceforge.net/lists/listinfo/jython-dev > > |