Home / Browse / CEDET / Mail / Archive

CEDET

Email Archive: cedet-devel (read-only)

Re: [CEDET-devel] python parser now generates valid semantic tokens

From: David PONCE <David.Ponce@wa...> - 2003-01-24 10:51

Hi Richard,

> I just checked in new wisent-python.el and wisent-python.wy.
> 
> The grammar now generates proper semanatic tokens for many
> of the non-terminals.  Further work is required though.

You really did great work :-)  Thanks!

> I would like to ask one question before I proceed further.
> 
> What name should be given to expressions that do not define any names=3F=

> 
> Unlike java, python's top-level forms can be virtually any expression
> that do not define any names.
> For example, a simple python code is shown below:
> 
>     s and t
>     q | r
>     o ^ p
> 
> This results in the following semantic tokens:
> 
>    ("s and t" expr nil nil nil #<overlay from 467 to 474 in test.py>)
>    ("q | r" expr nil nil nil #<overlay from 475 to 480 in test.py>)
>    ("o ^ p" expr nil nil nil #<overlay from 481 to 486 in test.py>)
> 
> All three lines match the "expr" non-terminal.
> 
> What name (i.e., the first element of the semantic tokens) should
> be given to such forms=3F

Maybe you could use something like:

(wisent-token "expr" 'code nil $N nil)

Where $N is the string representation of the expression.

I used a such form for setup code in semantic-grammar.wy.

Maybe also it would be better to use the token class `code' instead
of `expr' to be consistent with tokens produced by both
semantic-el.el and semantic-grammar.el for general expressions=3F
Eric=3F

Finally, it would be nice if you could add something in the NEWS file
about python into the "* Languages." topic ;-)

David

Thread View

Thread	Author	Date
Re: [CEDET-devel] python parser now generates valid semantic tokens	David PONCE <David.Ponce@wa...>

Re: [CEDET-devel] python parser now generates valid semantic tokens

From: <ryk@ds...> - 2003-01-25 04:32

>>>>> "DP" == David PONCE <David.Ponce@...> writes:
       [...]
    DP> 
    DP> You really did great work :-)  Thanks!
    DP> 

It was mostly a shameless lifting of code from wisent-java-*
files ...

    DP> Maybe you could use something like:
    DP> 
    DP> (wisent-token "expr" 'code nil $N nil)
    DP> 
    DP> Where $N is the string representation of the expression.

I did this for all expressions that do not have a keyword.
Also I did not include $N part, because I feared that it
might make the tokens really bloated.  Is that a valid concern?

    DP> I used a such form for setup code in semantic-grammar.wy.
    DP> 
    DP> Maybe also it would be better to use the token class `code' instead
    DP> of `expr' to be consistent with tokens produced by both
    DP> semantic-el.el and semantic-grammar.el for general expressions?
    DP> Eric?

As I said in an earlier email, I used 'code' in most cases.

    DP> Finally, it would be nice if you could add something
    DP> in the NEWS file
    DP> about python into the "* Languages." topic ;-)
 
I would be happy to do that except that I was waiting to see
if I could get python parser to a decent state first.
I wouldn't want to announce something if it is not too
useful.  I'll try hard to fix as much lose ends as possible
before 2.0 beta whenever that is.

Thanks for all your inputs.

Re[2]: [CEDET-devel] python parser now generates valid semantic tokens

From: Eric M. Ludlam <eric@si...> - 2003-01-27 19:20

>>> ryk@... seems to think that:
  [ ... ]
>    DP> Finally, it would be nice if you could add something
>    DP> in the NEWS file
>    DP> about python into the "* Languages." topic ;-)
> 
>I would be happy to do that except that I was waiting to see
>if I could get python parser to a decent state first.
>I wouldn't want to announce something if it is not too
>useful.  I'll try hard to fix as much lose ends as possible
>before 2.0 beta whenever that is.
  [ ... ]

You should download semantic 1.0 sometime.  It really wanted to be
useful, but was not. ;)

I often fall victim of thinking "just fix a few more bugs first",
which is a habit I need to get out of for my Emacs Projects.  I have
found that I meet really cool people when I release buggy Emacs
code. ;)

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Re[2]: [CEDET-devel] python parser now generates valid semantic tokens

From: <ryk@ds...> - 2003-01-29 03:24

Hi Eric,

I'll mention python in NEWS file in the next few days.
The python parser seems to be able to generate proper tokens
although I would like the tokens for functions and classes
to include more information such as function arguments.

For the past few days, I tried to add code to generate
tokens for function arguments, but I have not been successful.
Most likely I don't understand how to use EXPANDFULL.
I'll plug away for a few more days.  Either I'll be
successful or I'll cry for help. We'll see.

>>>>> "EL" == Eric M Ludlam <eric@...> writes:
    EL> 
    >>>> ryk@... seems to think that:
    EL>   [ ... ]
    DP> Finally, it would be nice if you could add something
    DP> in the NEWS file
    DP> about python into the "* Languages." topic ;-)
    >> 
    >> I would be happy to do that except that I was waiting to see
    >> if I could get python parser to a decent state first.
    >> I wouldn't want to announce something if it is not too
    >> useful.  I'll try hard to fix as much lose ends as possible
    >> before 2.0 beta whenever that is.
    EL>   [ ... ]
    EL> 
    EL> You should download semantic 1.0 sometime.  It really wanted to be
    EL> useful, but was not. ;)
    EL> 
    EL> I often fall victim of thinking "just fix a few more bugs first",
    EL> which is a habit I need to get out of for my Emacs Projects.  I have
    EL> found that I meet really cool people when I release buggy Emacs
    EL> code. ;)
    EL> 
    EL> Eric