Re: [Jython-dev] new static analyzer from Google

[Oti <oh...@gm...>]

Hi Steve,

sorry about that - no idea what was hindering you in the bug tracker.
I added your comment, and the .zip file.

best wishes,
Oti.

On Wed, Jan 13, 2010 at 9:59 PM, Steve Yegge <st...@go...> wrote:
> Bummer - I tried it and it said:  "You do not have permission to create
> file".
> Here's what I'd added to the comments field:
> ---------------------------------------------------------
> The attached patchfile is a zip of 2 files:  modified files and new files.
>  To apply it:
>   - unzip google-indexer.zip into .../jython (parent of src/ directory)
>   - tar zxf google-indexer-newfiles.tar.gz
>   - patch -p0 < google-indexer-patch.diff
> You can run the unit tests with "ant idxtest".  Instructions for running the
> demo app are in the doc header of
> src/org/python/indexer/demos/HtmlDemo.java.
> ---------------------------------------------------------
> I've attached google-indexer.zip to this email.
> I'm happy to make changes as needed until you feel it's ready to put in the
> trunk.  We'll probably also want
> a lightweight process for getting periodic batches of fixes in from my team.
>  We just launched it internally
> at Google today, so we're likely to see lots of bug reports trickle in.
> Cheers,
> -steve
> On Wed, Jan 13, 2010 at 6:56 AM, Oti <oh...@gm...> wrote:
>>
>> Steve,
>>
>> this is great news!
>> Please feel free to attach your patch to http://bugs.jython.org/issue1541.
>>
>> Many thanks,
>> Oti.
>>
>> On Wed, Jan 13, 2010 at 1:42 AM, Steve Yegge <st...@go...> wrote:
>> > I've got a patch to submit for everyone's consideration.  It's a static
>> > analyzer and indexer for Python
>> > source, built atop the Jython antlr parser.  It was primarily designed
>> > to
>> > produce an index that can
>> > support IDE functionality, but it could potentially also be used to
>> > provide
>> > type information to the
>> > interpreter for optimization purposes.
>> > Some notes about the indexer:
>> >   * it was an intern project from last summer, written from scratch by
>> > our
>> > intern Yin Wang
>> >      - I've been tinkering with it for 4 or 5 months, getting it to
>> > index
>> > Google's entire Python code base
>> >   * the foundation is a traditional bottom-up/top-down type-inferencer.
>> >  Yin
>> > did some really great work here.
>> >     It could be better (what inferencer couldn't?), but it does a nice
>> > job
>> > in the absence of type annotations.
>> >     - Jim Baker's suggestion for using decorators for annotations seems
>> > like
>> > a great way to feed it type hints
>> >     - I'm avoiding going that route until the Python community gains
>> > consensus around standardized type hints
>> >   * it builds an index of uniquely qualified global names, prefixing
>> > them
>> > with package paths as necessary.
>> >     This makes it possible to build a merged index across many projects
>> > without fear of name collisions.
>> >   * it statically models a large portion of the standard library -- some
>> > 50
>> > modules and perhaps 1000 definitions.
>> >      - this would be easier and more effective if Python had a way to
>> > annotate return types, but it works OK.
>> >   * it provides some error/warning lint diagnostics, although we haven't
>> > done too much here yet.
>> >     It probably has enough information today to replicate the
>> > diagnostics
>> > from every other Python lint tool.
>> >   * it has a decent unit-test suite, with around 500 assertions.  could
>> > be
>> > better, but it's a good start.
>> >   * it comes with a quick-and-dirty demo app I wrote last week.  You
>> > point
>> > the app at a file or directory,
>> >     and it recursively analyzes/indexes all the source and creates
>> > static
>> > html pages with LXR-like links
>> >     between identifiers, per-file structural outlines, and syntax and
>> > (some)
>> > semantic highlighting.
>> >     - the demo app doesn't show off some of the fancier features (e.g.
>> > inferred types, xrefs, diagnostics)
>> > There are bugs, of course, and there are many ways it can be improved.
>> >  But
>> > it works pretty well.
>> > If nothing else, it's one of the most ass-kicking intern projects I've
>> > seen
>> > in the last twenty years.
>> > The indexer is mostly self-contained, residing in a new package
>> > org.python.indexer.  We had
>> > to make some tweaks to Python.g and the support classes, but we didn't
>> > modify the AST; instead
>> > Yin designed a new AST format for the indexer, and wrote a conversion
>> > pass
>> > before the analysis.
>> > (We could use the current AST, but we wanted to minimize the diffs on
>> > the
>> > existing code for now.)
>> > Yin is back in his Ph.D. program, but he has expressed interest in
>> > continuing work on the analyzer.
>> > My team at Google also plans to continue work on it on a part-time
>> > basis, as
>> > it's an important part
>> > of a larger project we're doing in this space.
>> > I'd like to submit a patch for review .  I'm happy to make changes as
>> > needed, provided they're
>> > not earth-shatteringly large feature requests.
>> > The patch is in two parts:  files we needed to change in Jython, and a
>> > zip
>> > of new files and directories.
>> > I can't easily put them all in one patch because I can't svn add them to
>> > my
>> > working copy of head.
>> > If someone creates a bug or feature request at bugs.jython.org, I should
>> > be
>> > able to attach the
>> > patch to it.  Or I can email it directly to one of the maintainers and
>> > they
>> > can do it; either way.
>> > Let me know how you'd like to proceed.
>> > Thanks,
>> > -steve
>> >
>> > ------------------------------------------------------------------------------
>> > This SF.Net email is sponsored by the Verizon Developer Community
>> > Take advantage of Verizon's best-in-class app development support
>> > A streamlined, 14 day to market process makes app distribution fast and
>> > easy
>> > Join now and get one step closer to millions of Verizon customers
>> > http://p.sf.net/sfu/verizon-dev2dev
>> > _______________________________________________
>> > Jython-dev mailing list
>> > Jyt...@li...
>> > https://lists.sourceforge.net/lists/listinfo/jython-dev
>> >
>> >
>
>