> Thanks for sending this in! It looks like a nice improvement. I've
> stuck it in jython.org/patches/1555182 until someone can retrofit it
> to work with a Reader.
> How far has your parser diverged from jython's? Have you made any
> other significant improvements or bugfixes I could pull over to
> jython's version? Should we look to your version to get
> implementations for Python 2.3-2.5 syntax? I imagine you've already
> added that for PyDev.
I've recently (yesterday) finished adding the python 2.5 grammar, but it
still needs more testing... I did a number of changes and it is showing some
conflicts on the javacc compiling which I still have to sort out.
The only thing is that the parser for pydev had to be changed because pydev
needs info that jython does not need. E.g.: when doing code analysis, the
ImportFrom needed the exact position of the module and the ClassDef needed
the exact position of the name, so that I can show it highlighted later, I
need to know where a suite ends at some times and I need the comments for
doing pretty printing (still only half done)...
So, in short, the parser changed a lot in pydev and I'm not sure how
feasible it would be to use the same implementation, because I have
different needs (usually stuff that jython does not need), so, even my AST
in slightly different to gather this additional info...
Still, when the jython grammar is updated, I'm sure it would be worth for
you to take a look at the grammar at pydev to help you sort some things
Also, I fixed jython.org/bugs/1465476 recently which might still
> affect your version. I can't see the error popping up very often, but
> I imagine it isn't too pleasant for PyDev if the parser completely
> blows up if the user makes a typo. If you can use bugfixes like that
> I can ping you as they occur.
I tried the example in the bug in the grammar and pydev and had no problems,
so, I think I had already fixed that on some other change I did to the
grammar (the TreeBuilder changed a also changed a lot)
-- But yes, that would be nice ;-)
> On 9/7/06, Fabio Zadrozny <fabiofz@...> wrote:
> > Hi all,
> > Some time ago I decided to improve the parser for pydev (which was
> > based on the same parser used for jython), and one of the changes to
> > the performance was changing the CharStream implementation (in this case
> > ReaderCharStream.java).
> > The problem is that it is highly inneficient for large files, so I
> created a
> > FastCharStream and it has been noticeable faster than the old version
> > is actually the default created for JavaCC, but is surely not optimal
> > because it does several reallocations, which is not efficient for large
> > files).
> > I'm sending the new version attached (it should be just a matter of
> > replacing one with the other... there is only one important difference:
> > version attached works with an in-memory String, while the current
> > version takes a java.io.Reader... still, it should be easy to change my
> > version to use a Reader -- or to pass a String instead of the reader).
> > One of the tests I did included parsing a 3MB file, which took about 2
> > seconds with that implementation (while the older implementation took
> > 3 minutes).
> > Cheers,
> > Fabio
> > Using Tomcat but need to do more? Need to support web services,
> > Get stuff done quickly with pre-integrated technology to make your job
> > easier
> > Download IBM WebSphere Application Server v.1.0.1 based on Apache
> > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Jython-dev mailing list
> > Jython-dev@...
> > https://lists.sourceforge.net/lists/listinfo/jython-dev