[CEDET-devel] another update of wisent-python.wy
Brought to you by:
zappo
From: Richard Y. K. <ry...@ds...> - 2002-06-13 08:37:46
|
David and Erid, I made lots of little fixes. Some of them were quite embarssing ones, so I had to post another version to prevent one or both of you from spending time on already-fixed things. I've been keeping this track of versions on my own RCS archive. Whenever this things gets to be actually half-way useful, we can worry about checking this into sf.net CVS database along with other semantic files. ;;; wisent-python.wy -- LALR grammar for Python ;; ;; Copyright (C) 2002 Richard Kim ;; ;; Author: Richard Kim <ry...@ds...> ;; Maintainer: Richard Kim <ry...@ds...> ;; Created: June 2002 ;; Keywords: syntax ;; X-RCS: $Id: wisent-python.wy,v 1.104 2002/06/13 08:29:03 ryk Exp $ ;; ;; This file is not part of GNU Emacs. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2, or (at ;; your option) any later version. ;; ;; This software is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with GNU Emacs; see the file COPYING. If not, write to the ;; Free Software Foundation, Inc., 59 Temple Place - Suite 330, ;; Boston, MA 02111-1307, USA. ;;; Commentary: ;; ;; This is still a work in progress. ;; The official Grammar file from the Python source code distribution ;; was the starting point of this wisent version. ;; -------- ;; Settings ;; -------- %{ (setq ;; How `semantic-flex' will setup the lexer input stream. semantic-flex-depth nil ;; python grammar requires BOL tokens to compute indent/dedent semantic-flex-enable-bol t ;; python grammar requires NEWLINE tokens! semantic-flex-enable-newlines t ;; Tell `semantic-flex' to handle Python numbers semantic-number-expression semantic-java-number-regexp ;; Character used to separation a parent/child relationship semantic-type-relation-separator-character '(".") semantic-command-separation-character ";" ;; Init indentation stack wisent-python-lexer-indent-stack nil ) %} %outputfile "wisent-python.el" %parsetable wisent-python-parser-tables %keywordtable wisent-python-keywords %tokentable wisent-python-tokens %languagemode python-mode %setupfunction wisent-python-default-setup %start goal %token <newline> NEWLINE %token <punctuation> LTLTEQ "<<=" %token <punctuation> GTGTEQ ">>=" %token <punctuation> EXPEQ "**=" %token <punctuation> DIVDIVEQ "//=" %token <punctuation> DIVDIV "//" %token <punctuation> LTLT "<<" %token <punctuation> GTGT ">>" %token <punctuation> EXPONENT "**" %token <punctuation> EQ "==" %token <punctuation> GE ">=" %token <punctuation> LE "<=" %token <punctuation> PLUSEQ "+=" %token <punctuation> MINUSEQ "-=" %token <punctuation> MULTEQ "*=" %token <punctuation> DIVEQ "/=" %token <punctuation> MODEQ "%=" %token <punctuation> AMPEQ "&=" %token <punctuation> OREQ "|=" %token <punctuation> HATEQ "^=" %token <punctuation> LTGT "<>" %token <punctuation> NE "!=" %token <punctuation> HAT "^" %token <punctuation> LT "<" %token <punctuation> GT ">" %token <punctuation> AMP "&" %token <punctuation> MULT "*" %token <punctuation> DIV "/" %token <punctuation> MOD "%" %token <punctuation> PLUS "+" %token <punctuation> MINUS "-" %token <punctuation> PERIOD "." %token <punctuation> TILDE "~" %token <punctuation> BAR "|" %token <punctuation> COLON ":" %token <punctuation> SEMICOLON ";" %token <punctuation> COMMA "," %token <punctuation> POPEN "(" %token <punctuation> PCLOSE ")" %token <punctuation> BOPEN "[" %token <punctuation> BCLOSE "]" %token <punctuation> BRACEOPEN "{" %token <punctuation> BRACECLOSE "}" %token <punctuation> ASSIGN "=" %token <punctuation> BACKQUOTE "`" %token <string> STRING_LITERAL %token <number> NUMBER_LITERAL %token <symbol> NAME ;; Customize `wisent-flex' match algorithm ;; - Use string comparison for: ;; - An operator can be made of multiple successive punctuations %put punctuation {string t multiple t} ;; Handle 'bol tokens via function `wisent-python-lex-bol' %token <bol> INDENT DEDENT %put bol handler wisent-python-lex-bol ;; ----------------- ;; Literal terminals ;; ----------------- ;; ----------------- ;; Keyword terminals ;; ----------------- %token AND "and" %put AND summary " ... " %token ASSERT "assert" %put ASSERT summary " ... " %token BREAK "break" %put BREAK summary " ... " %token CLASS "class" %put CLASS summary " ... " %token CONTINUE "continue" %put CONTINUE summary " ... " %token DEF "def" %put DEF summary " ... " %token DEL "del" %put DEL summary " ... " %token ELIF "elif" %put ELIF summary " ... " %token ELSE "else" %put ELSE summary " ... " %token EXCEPT "except" %put EXCEPT summary " ... " %token EXEC "exec" %put EXEC summary " ... " %token FINALLY "finally" %put FINALLY summary " ... " %token FOR "for" %put FOR summary " ... " %token FROM "from" %put FROM summary " ... " %token GLOBAL "global" %put GLOBAL summary " ... " %token IF "if" %put IF summary " ... " %token IMPORT "import" %put IMPORT summary " ... " %token IN "in" %put IN summary " ... " %token IS "is" %put IS summary " ... " %token LAMBDA "lambda" %put LAMDA summary "..." %token NOT "not" %put NOT summary "..." %token OR "or" %put OR summary "..." %token PASS "pass" %put PASS summary "..." %token PRINT "print" %put PRINT summary "..." %token RAISE "raise" %put RAISE summary "..." %token RETURN "return" %put RETURN summary "..." %token TRY "try" %put TRY summary "..." %token WHILE "while" %put WHILE summary "..." %token YIELD "yield" %put YIELD summary "..." %% goal : single_input ;; | file_input ;; | eval_input ; ;; single_input: NEWLINE | simple_stmt | compound_stmt NEWLINE single_input : NEWLINE | simple_stmt | compound_stmt NEWLINE ; ;; file_input: (NEWLINE | stmt)* ENDMARKER file_input : stmt_list ENDMARKER (wisent-token $1 'stmt_list nil nil) ; ;; What is ENDMARKER? ;; Make it empty non-terminal until I figure out what it is. -ryk 6/11/02 10:10pm. ENDMARKER : ;;EMPTY ; ;; (NEWLINE | stmt)* stmt_list : ;;EMPTY | stmt_list newline_or_stmt (wisent-token (concat $1 $2) 'stmt_list nil nil) ; ;; NEWLINE | stmt newline_or_stmt : NEWLINE | stmt (wisent-token $1 'stmt nil nil) ; ;; eval_input: testlist NEWLINE* ENDMARKER eval_input : testlist newline_list ENDMARKER (wisent-token $1 'testlist nil nil) ; ;; NEWLINE* newline_list : ;;EMPTY | newline_list NEWLINE ; ;; funcdef: 'def' NAME parameters ':' suite funcdef : DEF NAME parameters COLON suite ; ;; parameters: '(' [varargslist] ')' parameters : POPEN varargslist_opt PCLOSE ; ;; [varargslist] varargslist_opt : ;;EMPTY | varargslist ; ;; varargslist: (fpdef ['=' test] ',')* ('*' NAME [',' '**' NAME] | '**' NAME) | fpdef ['=' test] (',' fpdef ['=' test])* [','] varargslist : fpdef_opt_test_zom mult_name | fpdef eq_test_opt fpdef_opt_test_trailer_zom comma_opt ; ;; (fpdef ['=' test] ',')* fpdef_opt_test_zom : ;;EMPTY | fpdef_opt_test_zom fpdef eq_test_opt COMMA ; ;; ['=' test] eq_test_opt : ;;EMPTY | ASSIGN test ; ;; ('*' NAME [',' '**' NAME] | '**' NAME) mult_name : MULT NAME multmult_name_opt | EXPONENT NAME ; ;; [',' '**' NAME] multmult_name_opt : ;;EMPTY | COMMA EXPONENT NAME ; ;; (',' fpdef ['=' test])* fpdef_opt_test_trailer_zom : ;;EMPTY | fpdef_opt_test_trailer_zom COMMA fpdef eq_test_opt ; ;; fpdef: NAME | '(' fplist ')' fpdef : NAME | POPEN fplist PCLOSE ; ;; fplist: fpdef (',' fpdef)* [','] fplist : fpdef fpdef_list comma_opt ; ;; (',' fpdef)* fpdef_list : ;;EMPTY | fpdef_list COMMA fpdef ; ;; stmt: simple_stmt | compound_stmt stmt : simple_stmt | compound_stmt (wisent-token $1 'compound_stmt nil nil) ; ;; simple_stmt: small_stmt (';' small_stmt)* [';'] NEWLINE simple_stmt : small_stmt_list semicolon_opt NEWLINE (wisent-token $1 'small_stmt_list nil nil) ; ;; small_stmt (';' small_stmt)* small_stmt_list : small_stmt | small_stmt_list SEMICOLON small_stmt (format "%s; %s" $1 $3) ; ;; [';'] semicolon_opt : ;;EMPTY | SEMICOLON ; ;; small_stmt: expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt ;; | import_stmt | global_stmt | exec_stmt | assert_stmt small_stmt : expr_stmt | print_stmt | del_stmt | pass_stmt | flow_stmt | import_stmt | global_stmt | exec_stmt | assert_stmt ; ;; expr_stmt: testlist (augassign testlist | ('=' testlist)*) expr_stmt : testlist expr_stmt_trailer (if $2 (format "%s %s" $1 $2) (format "%s" $1)) ; ;; (augassign testlist | ('=' testlist)*) expr_stmt_trailer : augassign testlist (format "%s %s" $1 $2) | eq_testlist_zom ; ;; ('=' testlist)* eq_testlist_zom : ;;EMPTY | eq_testlist_zom ASSIGN testlist (format "%s %s %s" $1 $2 $3) ; ;; augassign: '+=' | '-=' | '*=' | '/=' | '%=' | '&=' | '|=' | '^=' ;; | '<<=' | '>>=' | '**=' | '//=' augassign : PLUSEQ | MINUSEQ | MULTEQ | DIVEQ | MODEQ | AMPEQ | OREQ | HATEQ | LTLTEQ | GTGTEQ | EXPEQ | DIVDIVEQ ; ;; # For normal assignments, additional restrictions enforced by the interpreter ;; print_stmt: 'print' ( [ test (',' test)* [','] ] | '>>' test [ (',' test)+ [','] ] ) print_stmt : PRINT print_stmt_trailer (if $2 (format "%s %s" $1 $2) (format "%s" $1)) ; ;; ( [ test (',' test)* [','] ] | '>>' test [ (',' test)+ [','] ] ) print_stmt_trailer : test_list_with_opt_comma_opt | GTGT test trailing_test_list_with_opt_comma_opt (if $3 (format "%s %s %s" $1 $2 $3) (format "%s %s" $1 $2)) ; ;; [ (',' test)+ [','] ] trailing_test_list_with_opt_comma_opt : ;;EMPTY | trailing_test_list comma_opt ; ;; (',' test)+ trailing_test_list : COMMA test (format ", %s" $2) | trailing_test_list COMMA test (format "%s, %s" $1 $3) ; ;; [ test (',' test)* [','] ] test_list_with_opt_comma_opt : ;;EMPTY | testlist ; ;; del_stmt: 'del' exprlist del_stmt : DEL exprlist (format "del %s" $2) ; ;; pass_stmt: 'pass' pass_stmt : PASS ; ;; flow_stmt: break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt flow_stmt : break_stmt | continue_stmt | return_stmt | raise_stmt | yield_stmt ; ;; break_stmt: 'break' break_stmt : BREAK ; ;; continue_stmt: 'continue' continue_stmt : CONTINUE ; ;; return_stmt: 'return' [testlist] return_stmt : RETURN testlist_opt (if $2 (format "return %s" $2) "return") ; ;; [testlist] testlist_opt : ;;EMPTY | testlist ; ;; yield_stmt: 'yield' testlist yield_stmt : YIELD testlist (format "%s %s" $1 $2) ; ;; raise_stmt: 'raise' [test [',' test [',' test]]] raise_stmt : RAISE zero_one_two_or_three_tests ; ;; [test [',' test [',' test]]] zero_one_two_or_three_tests : ;;EMPTY | test zero_one_or_two_tests ; ;; [',' test [',' test]] zero_one_or_two_tests : ;;EMPTY | COMMA test zero_or_one_comma_test ; ;; [',' test] zero_or_one_comma_test : ;;EMPTY | COMMA test ; ;; import_stmt : 'import' dotted_as_name (',' dotted_as_name)* ;; | 'from' dotted_name 'import' ('*' | import_as_name (',' import_as_name)*) import_stmt : IMPORT dotted_as_name_list | FROM dotted_name IMPORT star_or_import_as_name_list ; ;; dotted_as_name (',' dotted_as_name)* dotted_as_name_list : dotted_as_name | dotted_as_name_list COMMA dotted_as_name ; ;; ('*' | import_as_name (',' import_as_name)*) star_or_import_as_name_list : MULT | import_as_name_list ; ;; import_as_name (',' import_as_name)* import_as_name_list : import_as_name | import_as_name_list COMMA import_as_name ; ;; import_as_name: NAME [NAME NAME] import_as_name : NAME name_name_opt ; ;; dotted_as_name: dotted_name [NAME NAME] dotted_as_name : dotted_name name_name_opt ; ;; [NAME NAME] name_name_opt : ;;EMPTY | NAME NAME (format "%s %s" $1 $2) ; ;; dotted_name: NAME ('.' NAME)* dotted_name : NAME | dotted_name PERIOD NAME (format "%s %s %s" $1 $2 $3) ; ;; global_stmt: 'global' NAME (',' NAME)* global_stmt : GLOBAL comma_sep_name_list ; ;; NAME (',' NAME)* comma_sep_name_list : NAME | comma_sep_name_list COMMA NAME (format "%s %s %s" $1 $2 $3) ; ;; exec_stmt: 'exec' expr ['in' test [',' test]] exec_stmt : EXEC expr exec_trailer ; ;; ['in' test [',' test]] exec_trailer : ;;EMPTY | IN test comma_test_opt (format "%s %s" $1 $2) ; ;; [',' test] comma_test_opt : ;;EMPTY | COMMA test ; ;; assert_stmt: 'assert' test [',' test] assert_stmt : ASSERT test comma_test_opt (format "%s %s" $1 $2) ; ;; compound_stmt: if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef compound_stmt : if_stmt | while_stmt | for_stmt | try_stmt | funcdef | classdef ; ;; if_stmt: 'if' test ':' suite ('elif' test ':' suite)* ['else' ':' suite] if_stmt : IF test COLON suite elif_suite_pair_list else_suite_pair_opt ; ;; ('elif' test ':' suite)* elif_suite_pair_list : ;;EMPTY | elif_suite_pair_list ELIF test COLON suite ; ;; while_stmt: 'while' test ':' suite ['else' ':' suite] while_stmt : WHILE test COLON suite else_suite_pair_opt ; ;; for_stmt: 'for' exprlist 'in' testlist ':' suite ['else' ':' suite] for_stmt : FOR exprlist IN testlist COLON suite else_suite_pair_opt ; ;; try_stmt: ('try' ':' suite (except_clause ':' suite)+ #diagram:break ;; ['else' ':' suite] | 'try' ':' suite 'finally' ':' suite) try_stmt : TRY COLON suite except_clause_suite_pair_list else_suite_pair_opt | TRY COLON suite FINALLY COLON suite ; ;; (except_clause ':' suite)+ except_clause_suite_pair_list : except_clause COLON suite | except_clause_suite_pair_list except_clause COLON suite ; ;; ['else' ':' suite] else_suite_pair_opt : ;;EMPTY | ELSE COLON suite ; ;; # NB compile.c makes sure that the default except clause is last ;; except_clause: 'except' [test [',' test]] except_clause : EXCEPT zero_one_or_two_test ; ;; [test [',' test]] zero_one_or_two_test : ;;EMPTY | test zero_or_one_comma_test ; ;; suite: simple_stmt | NEWLINE INDENT stmt+ DEDENT suite : simple_stmt | NEWLINE INDENT stmt_oom DEDENT ; ;; stmt+ stmt_oom : stmt | stmt_oom stmt ; ;; test: and_test ('or' and_test)* | lambdef test : and_test test_trailer (if $2 (format "%s %s" $1 $2) (format "%s" $1)) | lambdef ; ;; ('or' and_test)* test_trailer : ;;EMPTY | test_trailer OR and_test (if $1 (format "%s %s %s" $1 $2 $3) (format "%s %s" $2 $3)) ; ;; and_test: not_test ('and' not_test)* and_test : not_test | and_test AND not_test (format "%s %s %s" $1 $2 $3) ; ;; not_test: 'not' not_test | comparison not_test : NOT not_test (format "%s %s" $1 $2) | comparison ; ;; comparison: expr (comp_op expr)* comparison : expr | comparison comp_op expr (format "%s %s %s" $1 $2 $3) ; ;; comp_op: '<'|'>'|'=='|'>='|'<='|'<>'|'!='|'in'|'not' 'in'|'is'|'is' 'not' comp_op : LT | GT | EQ | GE | LE | LTGT | NE | IN | NOT IN | IS | IS NOT ; ;; expr: xor_expr ('|' xor_expr)* expr : xor_expr | expr BAR xor_expr (format "%s %s %s" $1 $2 $3) ; ;; xor_expr: and_expr ('^' and_expr)* xor_expr : and_expr | xor_expr HAT and_expr (format "%s %s %s" $1 $2 $3) ; ;; and_expr: shift_expr ('&' shift_expr)* and_expr : shift_expr | and_expr AMP shift_expr (format "%s %s %s" $1 $2 $3) ; ;; shift_expr: arith_expr (('<<'|'>>') arith_expr)* shift_expr : arith_expr | shift_expr shift_expr_operators arith_expr (format "%s %s %s" $1 $2 $3) ; ;; ('<<'|'>>') shift_expr_operators : LTLT | GTGT ; ;; arith_expr: term (('+'|'-') term)* arith_expr : term | arith_expr plus_or_minus term (format "%s %s %s" $1 $2 $3) ; ;; ('+'|'-') plus_or_minus : PLUS | MINUS ; ;; term: factor (('*'|'/'|'%'|'//') factor)* term : factor | term term_operator factor (format "%s %s %s" $1 $2 $3) ; term_operator : MULT | DIV | MOD | DIVDIV ; ;; factor: ('+'|'-'|'~') factor | power factor : prefix_operators factor (format "%s %s" $1 $2) | power ; ;; ('+'|'-'|'~') prefix_operators : PLUS | MINUS | TILDE ; ;; power: atom trailer* ('**' factor)* power : atom trailer_opt exponent_opt (if $3 (if $2 (format "%s $2 %s" $1 $2 $3) (format "%s ** %s" $1 $3)) (if $2 (format "%s %s" $1 $2) (format "%s" $1))) ; trailer_opt : ;;EMPTY | trailer ; exponent_opt : ;;EMPTY | EXPONENT factor (format "** %s" $2) ; ;; atom: '(' [testlist] ')' | '[' [listmaker] ']' | '{' [dictmaker] '}' | '`' testlist '`' | NAME | NUMBER | STRING+ atom : POPEN testlist_opt PCLOSE | BOPEN listmaker_opt BCLOSE | BRACEOPEN dictmaker_opt BRACECLOSE | BACKQUOTE testlist BACKQUOTE | NAME | NUMBER_LITERAL | one_or_more_string ; listmaker_opt : ;;EMPTY | listmaker ; dictmaker_opt : ;;EMPTY | dictmaker ; ;; Use (read $1) to peel away the double quotes. one_or_more_string : STRING_LITERAL (read $1) | one_or_more_string STRING_LITERAL (format "%s %s" $1 (read $2)) ; ;; listmaker: test ( list_for | (',' test)* [','] ) listmaker : test listmaker_trailer ; ;; ( list_for | (',' test)* [','] ) listmaker_trailer : list_for | testlist_trailer comma_opt ; ;; (',' test)* testlist_trailer : ;;EMPTY | testlist_trailer COMMA test (format "%s %s %s" $1 $2 $3) ; ;; lambdef: 'lambda' [varargslist] ':' test lambdef : LAMBDA COLON test (format "%s %s %s" $1 $2 $3) ; ;; trailer: '(' [arglist] ')' | '[' subscriptlist ']' | '.' NAME trailer : POPEN arglist_opt PCLOSE | BOPEN subscriptlist BCLOSE | PERIOD NAME ; ;; [arglist] arglist_opt : ;;EMPTY | arglist ; ;; subscriptlist: subscript (',' subscript)* [','] subscriptlist : comma_sep_subscript_list comma_opt ; ;; subscript (',' subscript)* comma_sep_subscript_list : subscript | comma_sep_subscript_list COMMA subscript ; ;; subscript: '.' '.' '.' | test | [test] ':' [test] [sliceop] subscript : PERIOD PERIOD PERIOD | test | zero_or_one_test COLON zero_or_one_test zero_or_one_sliceop ; ;; [sliceop] zero_or_one_sliceop : ;;EMPTY | sliceop ; ;; sliceop: ':' [test] sliceop : COLON zero_or_one_test ; ;; [test] zero_or_one_test : ;;EMPTY | test ; ;; exprlist: expr (',' expr)* [','] exprlist : expr_list comma_opt ; ;; expr (',' expr)* expr_list : expr | expr_list COMMA expr ; ;; testlist: test (',' test)* [','] testlist : comma_sep_test_list comma_opt (if $2 (format "%s %s" $1 $2) (format "%s" $1)) ; ;; test (',' test)* comma_sep_test_list : test | comma_sep_test_list COMMA test (format "%s %s %s" $1 $2 $3) ; ;; [','] comma_opt : ;;EMPTY | COMMA ; ;; How is testlist_safe different with testlist??? -ryk 6/13/02 ;; testlist_safe: test [(',' test)+ [',']] testlist_safe : test testlist_safe_trailer_opt (if $2 (format "%s %s" $1 $2) (format "%s" $1)) ; ;; [(',' test)+ [',']] testlist_safe_trailer_opt : ;; EMPTY | testlist_safe_term comma_opt (if $2 (format "%s %s" $1 $2) (format "%s" $1)) ; ;; (',' test)+ testlist_safe_term : COMMA test (format "%s %s" $1 $2) | testlist_safe_term COMMA test (format "%s %s %s" $1 $2 $3) ; ;; dictmaker: test ':' test (',' test ':' test)* [','] dictmaker : test COLON test colon_sep_test comma_opt ; ;; (',' test ':' test)* colon_sep_test : ;;EMPTY | colon_sep_test COMMA test COLON test ; ;; classdef: 'class' NAME ['(' testlist ')'] ':' suite classdef : CLASS NAME paren_testlist_opt COLON suite ; ;; ['(' testlist ')'] paren_testlist_opt : ;;EMPTY | POPEN testlist PCLOSE ; ;; arglist: (argument ',')* (argument [',']| '*' test [',' '**' test] | '**' test) arglist : argument_comma_zom arglist_trailer ; ;; (argument ',')* argument_comma_zom : ;;EMPTY | argument_comma_zom argument COMMA ; ;; (argument [',']| '*' test [',' '**' test] | '**' test) arglist_trailer : argument comma_opt | MULT test comma_mult_mult_test_opt | EXPONENT test ; ;; [',' '**' test] comma_mult_mult_test_opt : ;;EMPTY | COMMA EXPONENT test ; ;; argument: [test '='] test # Really [keyword '='] test argument : test_eq_opt test ; ;; [test '='] test_eq_opt : ;;EMPTY | test ASSIGN ; ;; list_iter: list_for | list_if list_iter : list_for | list_if ; ;; list_for: 'for' exprlist 'in' testlist_safe [list_iter] list_for : FOR exprlist IN testlist_safe list_iter_opt ; ;; list_if: 'if' test [list_iter] list_if : IF test list_iter_opt ; ;; [list_iter] list_iter_opt : ;;EMPTY | list_iter ; ;;; wisent-python.wy ends here |