Thread: Re: [cedet-semantic] using semantic/wisent as a parser
Brought to you by:
zappo
From: David P. <dav...@wa...> - 2004-04-15 09:09:20
|
Hi Joe, > OK - I said I would get back to you. Here we go. > > I want to use *wisent* to parse a simple language. > > But I am having some trouble with this. > > Specifically, in my example file, "example.simple", this is what I > see: > > M-: (semantic-parse-region (point-min) (point-max) nil nil t) > ;=> (void-function semantic-simple-parse-region) That looks right, `semantic-simple-parse-region' isn't defined anywhere in your code. See also my comments below. > > M-: (semantic-debug) > ;=> (error "This major mode does not support parser debugging") That's true, for now wisent doesn't support semantic-debug. [...] > ;;=============== > ;; System Setup > ;;=============== > > ;; The needed files. Load these once. > (add-to-list 'load-path "~/cedet-1.0beta2a/semantic/") > (add-to-list 'load-path "~/cedet-1.0beta2a/common/") > (add-to-list 'load-path "~/cedet-1.0beta2a/eieio/") > ;(setq semantic-load-turn-everything-on t) > (load "~/cedet-1.0beta2a/semantic/semantic-load.el") > (load "/Users/arided/cedet-1.0beta2a/common/cedet-load.el") > (load "~/cedet-1.0beta2a/eieio/eieio-load.el") Why not just: (load "~/cedet-1.0beta2a/common/cedet.el") It will setup the load-path, and other needed things, for you. [...] > ;;========================= > ;;========================= > ;; FILE: semantic-simple.el > ;;========================= > ;;========================= > > (require 'simple-wy) ;; The parser > > (define-mode-overload-implementation > semantic-parse-region simple-mode > (start end &optional nonterminal depth returnonerror) > "Over-ride in order to initialize some variables." > (semantic-parse-region-default > start end nonterminal depth returnonerror)) > > ;; As instructed by the info node Semantic Overload Mechanism, we look at > ;; examples. The examples seem to indicate that we should do something like > ;; this: > > ;;;###autoload > (defun semantic-default-simple-setup () > "Set up a buffer for semantic parsing of the SIMPLE language." > (semantic-install-function-overrides > '((parse-region . semantic-simple-parse-region) > (parse-changes . semantic-simple-parse-changes))) [...] In fact your problem is in the above code. First you define a override to semantic-parse-region in simple-mode (looks correct), but then you override it again locally in the buffer with the undefined function `semantic-simple-parse-region'. IMO, you should just remove the above call to `semantic-install-function-overrides'. Good luck! David |
From: David P. <dav...@wa...> - 2004-04-16 11:04:57
|
Hi Joe, [...] > Thanks. I revised that function so that it looks like this: > > (defun semantic-default-simple-setup () > "Set up a buffer for semantic parsing of a SIMPLE language." > (setq semantic--parse-table t > document-comment-start "/*" > document-comment-end " */" > )) > > But upon running > > M-: (semantic-parse-region (point-min) (point-max)) > > or > > M-: (semantic-parse-region (point-min) (point-max) 'expr) > > where `expr' is my start symbol, the following error is triggered. > > Debugger entered--Lisp error: (wrong-type-argument listp t) [...] This is because you specified that your `semantic--parse-table' is `t' instead of a list (for the bovinator) or a vector (for wisent). As you didn't install a specific parser in your setup function, the default bovinator is used. Obviously, it fails with a "wrong-type-argument" on parser table value `t', which is not of the expected format: 'listp'. Your setup function above must look like: (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." (simple-wy--install-parser) (setq document-comment-start "/*" document-comment-end " */" )) where `simple-wy--install-parser' is a function generated from your simple.wy grammar, that setups for you the wisent parser, based on what is generated in the simple-wy.el library. Probably you will also need to add (require 'simple-wy) in your semantic-simple library. Good luck! David |
From: Joe C. <jco...@ma...> - 2004-04-26 18:41:09
|
(defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." (simple-wy--install-parser) (setq document-comment-start "/*" document-comment-end " */" )) where `simple-wy--install-parser' is a function generated from your simple.wy grammar, that setups for you the wisent parser, based on what is generated in the simple-wy.el library. Probably you will also need to add (require 'simple-wy) in your semantic-simple library. I have set it up as you suggested, but this time upon running (semantic-parse-region (point-min) (point-max)) I just get `nil'. I'd appreciate it if you could take a look. I seem to have gotten down to the bare bones here, and I think I've followed all your instructions, and it still doesn't seem to be working as expected. Everything I'm working with, including the auto-generated parser is below. I hope I'm just making a simple error. /* example.simple -- about the simplest possible example */ abc + xyz /* example.simple ends here */ ;;; semantic-hacks.el -- actually just a simple setup (load "~/cedet-1.0beta2a/common/cedet.el") ;; Make my working directory available to Emacs (add-to-list 'load-path "~/Parser/simple/") ;; my parser-generator code to be "compiled" with keyboard command C-c C-c (find-file "/Users/arided/Parser/simple/simple.wy") ;; the elisp parser automatically pops up and loads but just to be sure... (load "/Users/arided/Parser/simple/simple-wy.el") ;; this is code to set up the parser. In particular, it contains the code ;; (add-hook 'simple-mode-hook 'semantic-default-simple-setup) (load "/Users/arided/Parser/simple/semantic-simple.el") ;; we might want to take a look at this file! (find-file "/Users/arided/Parser/simple/semantic-simple.el") ;; tiny major mode that works on files with suffix ".simple" (load "~/site-lisp/simple-mode.el") ;;; semantic-hacks.el ends here ;;; simple.wy -- LALR grammar for (much simplified) Tiger %package simple-wy ;; apparently not useful for wisent? %start expr %type <punctuation> %token <punctuation> PLUS "+" %type <symbol> %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*" %% expr : ;; empty | symbol (quote $1) | symbol PLUS symbol (quote (+ $1 $3)) ; ;;; simple.wy ends here ;;; semantic-simple.el -- parser support for a simple language. (require 'simple-wy) ;; The parser (define-mode-overload-implementation semantic-parse-region simple-mode (start end &optional nonterminal depth returnonerror) "Over-ride in order to initialize some variables." (semantic-parse-region-default start end nonterminal depth returnonerror)) ;; As instructed by the info node Semantic Overload Mechanism, ;; looking at examples ;;;###autoload (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." (simple-wy--install-parser) (setq document-comment-start "/*" document-comment-end " */" )) ;;;###autoload (add-hook 'simple-mode-hook 'semantic-default-simple-setup) (provide 'semantic-simple) ;;; semantic-simple.el ends here ;;; simple-mode.el --- simple mode to get my parser set up (defvar simple-mode-hook nil) (define-derived-mode simple-mode text-mode "simple" "" (run-hooks 'simple-mode-hook)) (provide 'simple-mode) ;;; simple-mode.el ends here ;;; simple-wy.el --- Generated parser support file ;; Copyright (C) 2004 arided ;; Author: arided <arided@hope-of-a-stone.local> ;; Created: 2004-04-26 13:25:35-0500 ;; Keywords: syntax ;; X-RCS: $Id$ ;; This file is not part of GNU Emacs. ;; ;; This program is free software; you can redistribute it and/or ;; modify it under the terms of the GNU General Public License as ;; published by the Free Software Foundation; either version 2, or (at ;; your option) any later version. ;; ;; This software is distributed in the hope that it will be useful, ;; but WITHOUT ANY WARRANTY; without even the implied warranty of ;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU ;; General Public License for more details. ;; ;; You should have received a copy of the GNU General Public License ;; along with GNU Emacs; see the file COPYING. If not, write to the ;; Free Software Foundation, Inc., 59 Temple Place - Suite 330, ;; Boston, MA 02111-1307, USA. ;;; Commentary: ;; ;; PLEASE DO NOT MANUALLY EDIT THIS FILE! It is automatically ;; generated from the grammar file simple.wy. ;;; History: ;; ;;; Code: ;;; Prologue ;; ;;; Declarations ;; (defconst simple-wy--keyword-table (semantic-lex-make-keyword-table 'nil 'nil) "Table of language keywords.") (defconst simple-wy--token-table (semantic-lex-make-type-table '(("symbol" (symbol . "[A-Za-z][_A-Za-z0-9]*")) ("punctuation" (PLUS . "+"))) '(("symbol" :declared t) ("punctuation" :declared t))) "Table of lexical tokens.") (defconst simple-wy--parse-table (progn (eval-when-compile (require 'wisent-comp)) (wisent-compile-grammar '((PLUS symbol) nil (expr (nil) ((symbol) '$1) ((symbol PLUS symbol) '(+ $1 $3)))) '(expr))) "Parser table.") (defun simple-wy--install-parser () "Setup the Semantic Parser." (semantic-install-function-overrides '((parse-stream . wisent-parse-stream))) (setq semantic-parser-name "LALR" semantic--parse-table simple-wy--parse-table semantic-debug-parser-source "simple.wy" semantic-flex-keywords-obarray simple-wy--keyword-table semantic-lex-types-obarray simple-wy--token-table) ;; Collect unmatched syntax lexical tokens (semantic-make-local-hook 'wisent-discarding-token-functions) (add-hook 'wisent-discarding-token-functions 'wisent-collect-unmatched-syntax nil t)) ;;; Analyzers ;; (require 'semantic-lex) (define-lex-regex-type-analyzer simple-wy--<symbol>-regexp-analyzer "regexp analyzer for <symbol> tokens." "\\(\\sw\\|\\s_\\)+" '((symbol . "[A-Za-z][_A-Za-z0-9]*")) 'symbol) (define-lex-string-type-analyzer simple-wy--<punctuation>-string-analyzer "string analyzer for <punctuation> tokens." "\\(\\s.\\|\\s$\\|\\s'\\)+" '((PLUS . "+")) 'punctuation) ;;; Epilogue ;; (provide 'simple-wy) ;;; simple-wy.el ends here |
From: David P. <dav...@wa...> - 2004-04-27 08:36:33
|
Hi Joe, [...] > I have set it up as you suggested, but this time upon running > (semantic-parse-region (point-min) (point-max)) I just get `nil'. >=20 > I'd appreciate it if you could take a look. I seem to have gotten > down to the bare bones here, and I think I've followed all your > instructions, and it still doesn't seem to be working as expected. >=20 > Everything I'm working with, including the auto-generated parser > is below. I hope I'm just making a simple error. In fact, you're missing a lexer definition, based on lexical rule analyzers auto-generated from the %type statements in your grammar. Also there is no need to override `semantic-parse-region' in simple-mode, because you use the default! I hacked a little bit you grammar & simple-mode support code, and the parser works well now with this example.simple file: ---------- cut here abc + xyz xxx a + b ---------- cut here Here is my hacked simple-mode.el which merge your semantic-simple and simple-mode files (please don't use the `semantic-' prefix for your libraries to avoid confusion with semantic's ones). ;;; simple-mode.el -- major mode for a simple language ;;; Semantic parsing support ;; (require 'semantic-wisent) (require 'simple-wy) ;;;###autoload (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." ;; Install the parser (simple-wy--install-parser) ;; Setup the lexer (setq semantic-lex-analyzer 'simple-lexer ;; Do a full depth lexical analysis. semantic-lex-depth nil) ;; Other useful things. (setq document-comment-start "/*" document-comment-end " */")) ;;;###autoload (add-hook 'simple-mode-hook 'semantic-default-simple-setup) =0C ;;; simple major mode ;; (defvar simple-mode-syntax-table (let ((table (make-syntax-table (standard-syntax-table)))) (modify-syntax-entry ?\+ "." table) ;; Operator PLUS (modify-syntax-entry ?\- "." table) ;; Operator MINUS (modify-syntax-entry ?\* "." table) ;; Operator MULT (modify-syntax-entry ?\/ "." table) ;; Operator DIV table) "Syntax table used in simple mode buffers. Define operators as punctuations.") ;;;###autoload (define-derived-mode simple-mode fundamental-mode "simple") ;;;###autoload (add-to-list 'auto-mode-alist '("\\.simple\\'" . simple-mode)) (provide 'simple-mode) ;;; simple-mode.el ends here Here is the simple.wy grammar which now include the lexical analyzer definition: ;;; simple.wy -- LALR grammar for (much simplified) Tiger %package simple-wy ;; No really necessary, as it is the default start symbol %start expr %type <punctuation> %token <punctuation> PLUS "+" %type <symbol> %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*" %% ;; For use with Semantic, must return valid semantic tags! expr : ;; empty | symbol (TAG "expr" 'expr :value $1) | symbol PLUS symbol (TAG "expr" 'expr :value (concat $1 $2 $3)) ; %% (define-lex simple-lexer "Simple lexical analyzer." semantic-lex-ignore-whitespace semantic-lex-ignore-newline semantic-lex-ignore-comments ;;;; Auto-generated analyzers. simple-wy--<symbol>-regexp-analyzer simple-wy--<punctuation>-string-analyzer ;;;; semantic-lex-default-action) ;;; simple.wy ends here To use that semantic-enabled simple-mode, put the above files somewhere in your load-path. Edit the grammar and generate the simple-wy.el file with C-c C-c. Save it. Then do (require 'simple-mode) and open the example.simple file. 'M-x bovinate' should show you the tags obtained from the simple expressions found in the example. Enjoy! David |
From: Joe C. <jco...@ma...> - 2004-04-27 17:21:59
|
In fact, you're missing a lexer definition, based on lexical rule analyzers auto-generated from the %type statements in your grammar. For some reason I thought it would be totally auto-generated - as in, no need to write anything extra. Oops! Also there is no need to override `semantic-parse-region' in simple-mode, because you use the default! I hacked a little bit you grammar & simple-mode support code, and the parser works well now with this example.simple file: Thanks a bunch - is indeed working just fine for me now. If it seems to you (as it does to me) that these files might help other people who want to see a bare-bones set up, you might then include them in the semantic distribution. I have a few questions at this point. The wisent-calc.wy doesn't produce the same kind of tags as you suggested here: ;; For use with Semantic, must return valid semantic tags! expr : ;; empty | symbol (TAG "expr" 'expr :value $1) | symbol PLUS symbol (TAG "expr" 'expr :value (concat $1 $2 $3)) ; Rather, like Many of the examples in the documentation for wisent produce things like `(cons $1 $2)' or `(+ $1 $3)'. Since I want to "grow" this parser into an interpreter for a simple programming language (Tiger), it seems like it would be nice to get some usable LISP code generated at this phase. What do you suggest for this? Suppose that, following the pattern established in wisent-calc.wy, I define expr : ;; empty | symbol (string-to-number $1) | symbol PLUS symbol (+ $1 $3) ; Then I feed in the example input 10 + 11 with bovinate and get `nil'. (I naively might expect to get something like `21' or `(+ 10 11)' instead.) Even the kluge expr : ;; empty | symbol (concat $1) | symbol PLUS symbol (concat "(+ " $1 " " $3 " )") ; doesn't work, though this seems to be very much like the code that appears in wisent-java.wy. Of course, I can write expr : ;; empty | symbol (TAG "expr" 'expr :value $1) | symbol PLUS symbol (TAG "expr" 'expr :value (concat "(+ " $1 " " $3 ")")) ; and *this* will work with bovinate, but then I'd be curious to know what the prefered method for doing operations on the output of bovinate is. My next question is this - even with the document-comment-start and document-comment-end as set up here -- (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." ;; Install the parser (simple-wy--install-parser) ;; Setup the lexer (setq semantic-lex-analyzer 'simple-lexer ;; Do a full depth lexical analysis. semantic-lex-depth nil) ;; Other useful things. (setq document-comment-start "/*" document-comment-end " */")) and the apparent instruction to ignore comments as given here (define-lex simple-lexer "Simple lexical analyzer." semantic-lex-ignore-whitespace semantic-lex-ignore-newline semantic-lex-ignore-comments ;;;; Auto-generated analyzers. simple-wy--<symbol>-regexp-analyzer simple-wy--<punctuation>-string-analyzer ;;;; semantic-lex-default-action) Parsing the following input /* this is a comment */ 10 + 11 1000 + 1 produces this as output: (("expr" expr (:value "this") nil #<overlay from 4 to 8 in example.simple>) ("expr" expr (:value "is") nil #<overlay from 9 to 11 in example.simple>) ("expr" expr (:value "a") nil #<overlay from 12 to 13 in example.simple>) ("expr" expr (:value "comment") nil #<overlay from 14 to 21 in example.simple>) ("expr" expr (:value "(+ 10 11)") nil #<overlay from 26 to 33 in example.simple>) ("expr" expr (:value "(+ 1000 1)") nil #<overlay from 35 to 43 in example.simple>)) How to ensure that things between /* and */ are ignored? (Is that space character in the definition of document-comment-end relevant?) |
From: Eric M. L. <er...@si...> - 2004-04-28 15:02:54
|
Joe, After David's answer, do you recall what lead to your confusion on how wisent worked? Is there something obvious we could do to prevent this confusion for the next person? Thanks Eric >>> David PONCE <dav...@wa...> seems to think that: >Hi Joe, > >[...] >> I have set it up as you suggested, but this time upon running >> (semantic-parse-region (point-min) (point-max)) I just get `nil'. >> >> I'd appreciate it if you could take a look. I seem to have gotten >> down to the bare bones here, and I think I've followed all your >> instructions, and it still doesn't seem to be working as expected. >> >> Everything I'm working with, including the auto-generated parser >> is below. I hope I'm just making a simple error. > >In fact, you're missing a lexer definition, based on lexical rule >analyzers auto-generated from the %type statements in your grammar. > >Also there is no need to override `semantic-parse-region' in >simple-mode, because you use the default! > >I hacked a little bit you grammar & simple-mode support code, and the >parser works well now with this example.simple file: [ ... ] -- Eric Ludlam: za...@gn..., er...@si... Home: http://www.ludlam.net Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net GNU: www.gnu.org |
From: Joe C. <jco...@ma...> - 2004-04-28 21:45:02
|
After David's answer, do you recall what lead to your confusion on how wisent worked? Is there something obvious we could do to prevent this confusion for the next person? Well, as I recently noted there is at least one thing that I'm still confused about, and that is: what is the correct way to produce output from the parsing system. David suggested writing my wisent input in the following style: ;; For use with Semantic, must return valid semantic tags! expr : ;; empty | symbol (TAG "expr" 'expr :value $1) | symbol PLUS symbol (TAG "expr" 'expr :value (concat $1 $2 $3)) ; But in wisent-java.wy I see things like type_parameters : LT type_parameter_list_1 (progn $2) ; type_parameter_list : type_parameter_list COMMA type_parameter (cons $3 $1) | type_parameter (list $1) ; Similarly, but more dramatically, wisent-calc.wy actually does the computations that the user types in! So my question is - what is the correct protocol to use for producing output from the parser? Recall that my goal is to use wisent to transform code for another programming language (Tiger) into Elisp. (It might be fun to make the language work with semantic, but that is not a high priority goal for me at this time -- not for this particular language anyway.) Other than this question, I think my first point of confusion was that I thought that if the user (me) didn't supply a lexer, then some default lexer would be used. That apparently was just incorrect. Another question would be, why is wisent better to use as a parser than bovine? (Maybe the full answer is more technical than I really need to know, but I'm still interested to know something about this question.) I think my confusion would be massively reduced, if not prevented altogether, by documentation that contains a step-by-step guide (tutorial, maybe) for such things as writing a working wisent parsing system and getting this system to work with semantic. I would definitely be willing to help write this documentation if you like -- but since I'm still riding the learning curve, I'm not sure if that would be the best thing to do. Still, I'd be happy to contribute my code (once it is working), and that might be helpful to others. To sum up: as always, more thorough documentation almost always reduces confusion! |
From: Joe C. <jco...@ma...> - 2004-05-07 07:16:15
|
After David's answer, do you recall what lead to your confusion on how wisent worked? Is there something obvious we could do to prevent this confusion for the next person? More advice: This is taken from the GNU coding standards: The only good way to use documentation strings in writing a good manual is to use them as a source of information for writing good text. Bearing this in mind, you might take another look at File: semantic-appdev.info, Node: Tag Query File: semantic-appdev.info, Node: Breadth Search File: grammar-fw.info, Node: Specialized Implementation etc. from the point of view of a novice user. Such sections, and indeed all of your documentation, could be made much more friendly if the following principle, also from the GNU coding standards, was adhered to throughout: At every level, from the sentences in a paragraph to the grouping of topics into separate manuals, the right way to structure documentation is according to the concepts and questions that a user will have in mind when reading it. I have a lot of questions that I'm not sure are actually answered in the documentation at all. Not that I could find easily anyway. I would be happy to help identify more of these user-level questions for you - and I hope my previous postings have highlighted a couple of the key ones. I'm now stuck on what seem to be a couple of relatively minor problems: 1. How to deal with comments? 2. Why does the rule | LPAREN WHILE RPAREN (TAG "expr" 'expr :value (concat "(" "foo" ")")) not work? (This is drawn from a slightly more complex parser-generator file than the one I posted earlier... the new file is below.) This should work with the simple-mode.el file you already have. ;;; simple.wy -- LALR grammar for (simplified) Tiger %package simple-wy ;; Not really necessary, as it is the default start symbol %start expr %type <punctuation> %token <punctuation> PLUS "+" %type <symbol> %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*" %type <string> %token <string> STRING_LITERAL %type <number> %token <number> NUMBER_LITERAL %type <keyword> %keyword ARRAY "array" %keyword break "break" %keyword DO "do" %keyword ELSE "else" %keyword END "end" %keyword FOR "for" %keyword FUNCTION "function" %keyword IF "if" %keyword IN "in" %keyword LET "let" %keyword NIL "nil" %keyword OF "of" %keyword THEN "then" %keyword TO "to" %keyword TYPE "type" %keyword VAR "var" %keyword WHILE "while" %token <open-paren> LPAREN "(" %token <open-paren> LBRACE "{" %token <open-paren> LBRACK "[" %token <close-paren> RPAREN ")" %token <close-paren> RBRACE "}" %token <close-paren> RBRACK "]" %% ;; For use with Semantic, must return valid semantic tags! expr : ;; empty | STRING_LITERAL (TAG "string" 'expr :value $1) | NUMBER_LITERAL (TAG "number" 'expr :value $1) | symbol (TAG "expr" 'expr :value $1) | symbol PLUS symbol (TAG "expr" 'expr :value (concat "(+ " $1 " " $3 ")")) | NIL (TAG "expr" 'expr :value $1) | LPAREN WHILE RPAREN (TAG "expr" 'expr :value (concat "(" "foo" ")")) ; %% (define-lex simple-lexer "Simple lexical analyzer." semantic-lex-ignore-whitespace semantic-lex-ignore-newline semantic-lex-ignore-comments semantic-lex-open-paren semantic-lex-close-paren ;;;; Auto-generated analyzers. simple-wy--<punctuation>-string-analyzer simple-wy--<block>-block-analyzer simple-wy--<symbol>-regexp-analyzer simple-wy--<string>-sexp-analyzer simple-wy--<number>-regexp-analyzer simple-wy--<keyword>-keyword-analyzer ;;;; semantic-lex-default-action) ;;; simple.wy ends here |
From: Eric M. L. <er...@si...> - 2004-05-07 12:55:49
|
Hi, >>> Joe Corneli <jco...@ma...> seems to think that: > After David's answer, do you recall what lead to your confusion > on how wisent worked? Is there something obvious we could do to > prevent this confusion for the next person? > >More advice: > >This is taken from the GNU coding standards: > > The only good way to use documentation strings in writing a good > manual is to use them as a source of information for writing good > text. > >Bearing this in mind, you might take another look at > > File: semantic-appdev.info, Node: Tag Query > File: semantic-appdev.info, Node: Breadth Search > File: grammar-fw.info, Node: Specialized Implementation > >etc. from the point of view of a novice user. Such sections, and >indeed all of your documentation, could be made much more friendly >if the following principle, also from the GNU coding standards, was >adhered to throughout: > > At every level, from the sentences in a paragraph to the grouping > of topics into separate manuals, the right way to structure > documentation is according to the concepts and questions that a > user will have in mind when reading it. > >I have a lot of questions that I'm not sure are actually answered in >the documentation at all. Not that I could find easily anyway. > >I would be happy to help identify more of these user-level questions >for you - and I hope my previous postings have highlighted a couple >of the key ones. [ ... ] I think you will find everyone here contributing to the doc is an engineer these days, though Richard Kim did help a great deal. For me, at least, documentation is an engineering challenge. How does it work, how to avoid duplication of effort, etc. Building a reference manual automatically from sources seems like the way to go for me. I then spend documentation authoring time (when I have some) trying to improve the doc strings of my lisp functions. Much of the manuals are just catalogs of things. Complicated bits tend to go into auto-generated code which David has been working on, or a skeleton (-skel.*) which can be copied as a starting point for a new language. Over the years it is apparent to me that I am not a good documentation writer. If anyone out there wishes to help resolve these issues via the contribution of text and structure, please let me know. Eric -- Eric Ludlam: za...@gn..., er...@si... Home: http://www.ludlam.net Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net GNU: www.gnu.org |
From: Joe C. <jco...@ma...> - 2004-05-07 15:35:44
|
Over the years it is apparent to me that I am not a good documentation writer. If anyone out there wishes to help resolve these issues via the contribution of text and structure, please let me know. Your email responses to my questions have generally been quite helpful. Maybe looking in the mailing list archives, you'll see other answers to user's questions that could go into the documentation. I think that, if possible, its good to take a look and see what questions people are actually asking and make these the central focus for documentation. |
From: Eric M. L. <er...@si...> - 2004-05-07 18:10:48
|
>>> Joe Corneli <jco...@ma...> seems to think that: > > Over the years it is apparent to me that I am not a good > documentation writer. If anyone out there wishes to help resolve > these issues via the contribution of text and structure, please > let me know. > >Your email responses to my questions have generally been quite >helpful. Maybe looking in the mailing list archives, you'll see >other answers to user's questions that could go into the >documentation. I think that, if possible, its good to take a look >and see what questions people are actually asking and make these the >central focus for documentation. If you look in the .texi file, you will find some of those email messages in comment blocks, waiting to be better described there. This had been graciously done for us by Richard Kim. Eric -- Eric Ludlam: za...@gn..., er...@si... Home: http://www.ludlam.net Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net GNU: www.gnu.org |
From: Joe C. <jco...@ma...> - 2004-05-08 21:48:09
|
Here are a couple more simple questions. I noticed that when I (1) change the grammar file, (2) run `semantic-grammar-create-package' (C-c C-c) and load the resulting generated parser support file, (3) bovinate the example source file that I'm working with, the changes to the grammar are not always reflected in the *Parser Output* buffer. When this happens, (4) I have to kill the source code buffer and (5) open it again, then (6) re-bovinate. Is this the expected behavior? Is there a shortcut I can use to make sure that the current parser is being used when I bovinate? Second question is that after repeating (1)-(3) and/or (1)-(6), my whole system (X window manager and Emacs) begins to handle much more sluggishly than usual. Has anyone seen anything like this happen before? Where might the slow-down be coming from? |
From: Joe C. <jco...@ma...> - 2004-05-09 00:29:32
|
Things are beginning to come along with this code for me. But here is another problem that seems somewhat weird. As part of my definition of expression (`expr') I have the following rule: | expr EQ expr (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")")) | expr NEQ expr (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))")) But this rule does not seem to be dealt with properly by the parser. For example, neither of these two inputs is recognized: foo <> bar "foo" <> "bar" If I modify the rule to say | STRING_LITERAL EQ STRING_LITERAL (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")")) | STRING_LITERAL NEQ STRING_LITERAL (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))")) then "foo" <> "bar" is recognized just fine. Running semantic-lex-debug seems to indicate that foo <> bar is lexically OK. I'm including the revised file below. Perhaps someone can tell me why this particular bit of code is causing trouble? ;;; simple.wy -- LALR grammar for (simplified) Tiger %package simple-wy ;; Not really necessary, as it is the default start symbol %start expr %start function_parameters %type <punctuation> %token <punctuation> PLUS "+" %token <punctuation> MINUS "-" %token <punctuation> TIMES "*" %token <punctuation> DIVIDE "/" %type <symbol> %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*" %type <string> %token <string> STRING_LITERAL %type <number> %token <number> NUMBER_LITERAL %type <keyword> %keyword WHILE "while" %keyword FOR "for" %keyword NIL "nil" %keyword GREATER_THAN ">" %keyword LESS_THAN "<" ;; these seem to need to be handled as keywords (not punctuation) %keyword GREATER_THAN_OR_EQUAL ">=" %keyword LESS_THAN_OR_EQUAL "<=" %keyword EQ "=" %keyword NEQ "<>" %left UMINUS %left PLUS %left MINUS %left TIMES %left DIVIDE %type <open-paren> %token <open-paren> LPAREN "(" %token <open-paren> LBRACE "{" %token <open-paren> LBRACK "[" %type <close-paren> %token <close-paren> RPAREN ")" %token <close-paren> RBRACE "}" %token <close-paren> RBRACK "]" %type <block> %token <block> PAREN_BLOCK "(LPAREN RPAREN)" %token <block> BRACE_BLOCK "(LBRACE RBRACE)" %token <block> BRACK_BLOCK "(LBRACK RBRACK)" %% ;; For use with Semantic, must return valid semantic tags! expr : ;; Empty | PAREN_BLOCK (EXPANDFULL $1 function_parameters) | FOR (TAG "expr" 'expr :value $1) | STRING_LITERAL (TAG "string" 'expr :value $1) | NUMBER_LITERAL (TAG "number" 'expr :value $1) | MINUS NUMBER_LITERAL %prec UMINUS (TAG "expr" 'expr :value (concat "-" $2)) | NUMBER_LITERAL binop NUMBER_LITERAL (TAG "binop result" 'expr :value (concat "(" $2 " " $1 " " $3 ")")) | string_comparison | expr EQ expr (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")")) | expr NEQ expr (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))")) ;; | STRING_LITERAL EQ STRING_LITERAL ;; (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")")) ;; | STRING_LITERAL NEQ STRING_LITERAL ;; (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))")) | symbol (TAG "expr" 'expr :value $1) | NIL (TAG "expr" 'expr :value $1) ; ;; parameters: '(' [varargslist] ')' function_parameters : LPAREN () | RPAREN () ;; | function_parameter COMMA | function_parameter RPAREN ; function_parameter : WHILE (TAG "function_parameter" 'function_parameter :value $1) ; binop : PLUS | MINUS | TIMES | DIVIDE | GREATER_THAN | LESS_THAN | GREATER_THAN_OR_EQUAL | LESS_THAN_OR_EQUAL ; string_comparison : STRING_LITERAL GREATER_THAN STRING_LITERAL (TAG "sum" 'expr :value (concat "(string> " $1 " " $3 ")")) | STRING_LITERAL LESS_THAN STRING_LITERAL (TAG "sum" 'expr :value (concat "(string< " $1 " " $3 ")")) | STRING_LITERAL LESS_THAN_OR_EQUAL STRING_LITERAL (TAG "sum" 'expr :value (concat "(or (string< " $1 " " $3 ") " "(string= " $1 " " $3 "))")) | STRING_LITERAL GREATER_THAN_OR_EQUAL STRING_LITERAL (TAG "sum" 'expr :value (concat "(or (string> " $1 " " $3 ") " "(string= " $1 " " $3 "))")) ; ;; expr_comparison ;; : expr EQ expr ;; (TAG "sum" 'expr :value (concat "(equal" $1 " " $3 ")")) ;; | expr NEQ expr ;; (TAG "sum" 'expr :value (concat "(not (equal" $1 " " $3 "))")) ;; ; %% ;; this turns out not to be needed. ;; (define-lex-regex-type-analyzer simple-jac--<symbol>-regexp-analyzer ;; "regexp analyzer for <symbol> tokens." ;; "\\([A-Za-z][_A-Za-z0-9]\\)+" ;; '((symbol . "[A-Za-z][_A-Za-z0-9]*")) ;; 'symbol) (define-lex simple-lexer "Simple lexical analyzer." semantic-lex-ignore-whitespace semantic-lex-ignore-newline semantic-lex-ignore-comments ;;;; Auto-generated analyzers. simple-wy--<block>-block-analyzer simple-wy--<keyword>-keyword-analyzer simple-wy--<punctuation>-string-analyzer simple-wy--<number>-regexp-analyzer ;; From the documentation string for `semantic-lex-tokens': ;; "Always add this analyzer *after* `semantic-lex-number', or other ;; analyzers that match its regular expression." (my emphasis) simple-wy--<symbol>-regexp-analyzer simple-wy--<string>-sexp-analyzer ;;;; semantic-lex-default-action) ;;; simple.wy ends here Just in case its relevant, here's the other file too. ;;; simple-mode.el -- major mode for a simple language ;;; Semantic parsing support (require 'semantic-wisent) (require 'simple-wy) ;;;###autoload (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." ;; Do some useful things. (setq semantic-ignore-comments t) ;; Install the parser (simple-wy--install-parser) ;; Setup the lexer (setq semantic-lex-analyzer 'simple-lexer ;; Do a full depth lexical analysis. semantic-lex-depth nil)) ;;;###autoload (add-hook 'simple-mode-hook 'semantic-default-simple-setup) ;;; post parser processing ;; I'm not sure why Bovine is not getting rid of comments -- I suppose ;; I haven't dealt with them properly. I was sort of under the ;; impression that this would be handled automatically after setting ;; the document comment start and end -- but that doesn't seem to be ;; the case. Maybe I just need to remove comments as part of my own ;; preprocessing phase. Wouldn't be hard to do. (defun clean-up-parser-output () (interactive) (set-buffer (get-buffer "*Parser Output*")) (flush-lines "overlay") (flush-lines "expr") ;; (flush-lines "\"expr\" expr") (goto-char (point-min)) (replace-regexp " (:value \"\\|\")$" "") (goto-char (point-min)) (replace-regexp "\\\\\"" "\"")) (defun novinate () (interactive) (bovinate) (clean-up-parser-output)) ;;; simple major mode (defvar simple-mode-syntax-table (let ((table (make-syntax-table (standard-syntax-table)))) (modify-syntax-entry ?\+ "." table) ;; Operator PLUS (modify-syntax-entry ?\- "." table) ;; Operator MINUS ;; also deals with comments (modify-syntax-entry ?\* ". 23" table) ;; Operator MULT (modify-syntax-entry ?\/ ". 14" table) ;; Operator DIV ;; deal with brackets (modify-syntax-entry ?\( "()" table) (modify-syntax-entry ?\{ "(}" table) (modify-syntax-entry ?\[ "(]" table) (modify-syntax-entry ?\) ")(" table) (modify-syntax-entry ?\} "){" table) (modify-syntax-entry ?\] ")[" table) table) "Syntax table used in simple mode buffers. Define operators as punctuations.") ;;;###autoload (define-derived-mode simple-mode fundamental-mode "simple" (setq comment-start-skip "/\\*+ *")) ;;;###autoload (add-to-list 'auto-mode-alist '("\\.simple\\'" . simple-mode)) ;(defvar simple-mode-hook nil) (provide 'simple-mode) ;;; simple-mode.el ends here |
From: Eric M. L. <er...@si...> - 2004-05-09 12:55:30
|
>>> Joe Corneli <jco...@ma...> seems to think that: >Here are a couple more simple questions. I noticed that when I (1) >change the grammar file, (2) run `semantic-grammar-create-package' >(C-c C-c) and load the resulting generated parser support file, (3) >bovinate the example source file that I'm working with, the changes >to the grammar are not always reflected in the *Parser Output* >buffer. When this happens, (4) I have to kill the source code >buffer and (5) open it again, then (6) re-bovinate. Is this the >expected behavior? Is there a shortcut I can use to make sure that >the current parser is being used when I bovinate? My expectation (meaning, what it did when I last was using and fixing bugs in it) is the following: 1) I edit the grammar 2) I hit C-c C-c to compile the grammar. 3) The grammar generator looks at the %languagemode declaration, finds all buffers matching those major modes and reinitializes them with the major-mode function. 4) Wait a moment, and the auto-parse mechanism will refresh the tags table on those buffers. >Second question is that after repeating (1)-(3) and/or (1)-(6), my >whole system (X window manager and Emacs) begins to handle much more >sluggishly than usual. Has anyone seen anything like this happen >before? Where might the slow-down be coming from? [ ... ] I have not seen this problem, though I almost always work on a dual processor system. (Either two processors, or Emacs on one machine displaying to a second machine.) When Emacs becomes unresponsive, I often start turning off all the different semantic modes until I'm done debugging whatever feature started breaking things. I don't recall language modification doing that to me though. Eric -- Eric Ludlam: za...@gn..., er...@si... Home: http://www.ludlam.net Siege: www.siege-engine.com Emacs: http://cedet.sourceforge.net GNU: www.gnu.org |
From: David P. <dav...@wa...> - 2004-05-07 13:30:12
|
Hi Joe, I am sorry for this late reply, but I am very busy these days! [...] > I'm now stuck on what seem to be a couple of relatively > minor problems: > > 1. How to deal with comments? semantic-lex uses the current Emacs syntax table to recognize language comments. So you must set your major mode syntax table accordingly. In your case, it looks that your language comments are like C ones, so you could change the `simple-mode-syntax-table' into your simple-mode.el file like this: (defvar simple-mode-syntax-table (let ((table (make-syntax-table (standard-syntax-table)))) (modify-syntax-entry ?\+ "." table) ;; Operator PLUS (modify-syntax-entry ?\- "." table) ;; Operator MINUS ;; Operator MULT & 2nd char of "/*" comment-start sequence & 1st ;; char of "*/" comment-end sequence (modify-syntax-entry ?\* ". 23" table) ;; Operator DIV & 1st char of "/*" comment-start sequence & 2nd ;; char of "*/" comment-end sequence (modify-syntax-entry ?\/ ". 14" table) table) "Syntax table used in simple mode buffers. Define operators as punctuations.") I recommend you to read the chapter on syntax tables on the "Emacs Lisp Reference Manual", to understand the meaning of the `1', `2', `3', `4' syntax flags. > > 2. Why does the rule > > | LPAREN WHILE RPAREN > (TAG "expr" 'expr :value (concat "(" "foo" ")")) > > not work? (This is drawn from a slightly more complex > parser-generator file than the one I posted earlier... the new file > is below.) This should work with the simple-mode.el file you > already have. > [...] Parenthesis-like tokens are better handled by an auto-generated <block> analyzer. To declare it you should add: %type <block> %token <block> PAREN_BLOCK "(LPAREN RPAREN)" %token <block> BRACE_BLOCK "(LBRACE RBRACE)" %token <block> BRACK_BLOCK "(LBRACK RBRACK)" to indicate the possible blocks made of these parenthesis delimiters. > %token <open-paren> LPAREN "(" > %token <open-paren> LBRACE "{" > %token <open-paren> LBRACK "[" > > %token <close-paren> RPAREN ")" > %token <close-paren> RBRACE "}" > %token <close-paren> RBRACK "]" Unfortunately, the grammar framework documentation is not yet complete, particularly the chapters on auto-generated lexical analyzers :( For now, you can read the doc string of `define-lex-block-type-analyzer', for more details. > [...] > (define-lex simple-lexer > "Simple lexical analyzer." > semantic-lex-ignore-whitespace > semantic-lex-ignore-newline > semantic-lex-ignore-comments These two analyzers aren't necessary because parenthesis-like tokens are handled by the <block> analyzer (see above). > semantic-lex-open-paren > semantic-lex-close-paren > ;;;; Auto-generated analyzers. > simple-wy--<punctuation>-string-analyzer > simple-wy--<block>-block-analyzer [...] Hope this will help. Good luck! David |
From: Joe C. <jco...@ma...> - 2004-05-07 15:22:19
|
semantic-lex uses the current Emacs syntax table to recognize language comments. I recommend you to read the chapter on syntax tables on the "Emacs Lisp Reference Manual", to understand the meaning of the `1', `2', `3', `4' syntax flags. Ok, thanks - I had been looking at the documentation string for modify-syntax-entry and for some reason thought that the different properties were to be added sequentially. However, while that didn't work, neither did the revised syntax table that you suggested, which follows the pattern set forth in the ELISP manual. Parenthesis-like tokens are better handled by an auto-generated <block> analyzer. To declare it you should add: %type <block> %token <block> PAREN_BLOCK "(LPAREN RPAREN)" %token <block> BRACE_BLOCK "(LBRACE RBRACE)" %token <block> BRACK_BLOCK "(LBRACK RBRACK)" to indicate the possible blocks made of these parenthesis delimiters. OK, this (and grammer elements in the style of wisent-python) appear to be working for me. Thanks. Anyone willing to take a look and try to understand why comments aren't being ignored properly? Here are my new files. ;;; simple-mode.el -- major mode for a simple language ;;; Semantic parsing support (require 'semantic-wisent) (require 'simple-wy) ;;;###autoload (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." ;; Do some useful things. (setq semantic-ignore-comments t document-comment-start "/\\*" document-comment-end "\\*/") ;; Install the parser (simple-wy--install-parser) ;; Setup the lexer (setq semantic-lex-analyzer 'simple-lexer ;; Do a full depth lexical analysis. semantic-lex-depth nil)) ;;;###autoload (add-hook 'simple-mode-hook 'semantic-default-simple-setup) ;;; post parser processing ;; I'm not sure why Bovine is not getting rid of comments -- I suppose ;; I haven't dealt with them properly. I was sort of under the ;; impression that this would be handled automatically after setting ;; the document comment start and end -- but that doesn't seem to be ;; the case. Maybe I just need to remove comments as part of my own ;; preprocessing phase. Wouldn't be hard to do. (defun clean-up-parser-output () (interactive) (set-buffer (get-buffer "*Parser Output*")) (flush-lines "overlay") (flush-lines "\"expr\" expr") (goto-char (point-min)) (replace-regexp " (:value \"\\|\")$" "")) (defun novinate () (interactive) (bovinate) (clean-up-parser-output)) ;;; simple major mode (defvar simple-mode-syntax-table (let ((table (make-syntax-table (standard-syntax-table)))) (modify-syntax-entry ?\+ "." table) ;; Operator PLUS (modify-syntax-entry ?\- "." table) ;; Operator MINUS ;; also deals with comments (modify-syntax-entry ?\* ". 23" table) ;; Operator MULT (modify-syntax-entry ?\/ ". 14" table) ;; Operator DIV ;; deal with brackets (modify-syntax-entry ?\( "()" table) (modify-syntax-entry ?\{ "(}" table) (modify-syntax-entry ?\[ "(]" table) (modify-syntax-entry ?\) ")(" table) (modify-syntax-entry ?\} "){" table) (modify-syntax-entry ?\] ")[" table) table) "Syntax table used in simple mode buffers. Define operators as punctuations.") ;;;###autoload (define-derived-mode simple-mode fundamental-mode "simple") ;;;###autoload (add-to-list 'auto-mode-alist '("\\.simple\\'" . simple-mode)) ;(defvar simple-mode-hook nil) (provide 'simple-mode) ;;; simple-mode.el ends here ;;; simple.wy -- LALR grammar for (simplified) Tiger %package simple-wy ;; Not really necessary, as it is the default start symbol %start expr %start function_parameters ;; %type <symbol> ;; %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*" %type <keyword> %keyword WHILE "while" %keyword FOR "for" %type <open-paren> %token <open-paren> LPAREN "(" %token <open-paren> LBRACE "{" %token <open-paren> LBRACK "[" %type <close-paren> %token <close-paren> RPAREN ")" %token <close-paren> RBRACE "}" %token <close-paren> RBRACK "]" %type <block> %token <block> PAREN_BLOCK "(LPAREN RPAREN)" %token <block> BRACE_BLOCK "(LBRACE RBRACE)" %token <block> BRACK_BLOCK "(LBRACK RBRACK)" %% ;; For use with Semantic, must return valid semantic tags! expr : ;; Empty | PAREN_BLOCK ;;(TAG "expr" 'expr :value $2) (EXPANDFULL $1 function_parameters) | FOR (TAG "expr" 'expr :value $1) ; ;; parameters: '(' [varargslist] ')' function_parameters : LPAREN () | RPAREN () ;; | function_parameter COMMA | function_parameter RPAREN ; function_parameter : WHILE (TAG "function_parameter" 'function_parameter :value $1) ; %% (define-lex simple-lexer "Simple lexical analyzer." semantic-lex-ignore-whitespace semantic-lex-ignore-newline semantic-lex-ignore-comments ;;;; Auto-generated analyzers. simple-wy--<block>-block-analyzer simple-wy--<keyword>-keyword-analyzer ;;;; semantic-lex-default-action) ;;; simple.wy ends here |
From: Joe C. <jco...@ma...> - 2004-04-16 00:13:35
|
> ;;;###autoload > (defun semantic-default-simple-setup () > "Set up a buffer for semantic parsing of the SIMPLE language." > (semantic-install-function-overrides > '((parse-region . semantic-simple-parse-region) > (parse-changes . semantic-simple-parse-changes))) [...] In fact your problem is in the above code. First you define a override to semantic-parse-region in simple-mode (looks correct), but then you override it again locally in the buffer with the undefined function `semantic-simple-parse-region'. IMO, you should just remove the above call to `semantic-install-function-overrides'. Thanks. I revised that function so that it looks like this: (defun semantic-default-simple-setup () "Set up a buffer for semantic parsing of a SIMPLE language." (setq semantic--parse-table t document-comment-start "/*" document-comment-end " */" )) But upon running M-: (semantic-parse-region (point-min) (point-max)) or M-: (semantic-parse-region (point-min) (point-max) 'expr) where `expr' is my start symbol, the following error is triggered. Debugger entered--Lisp error: (wrong-type-argument listp t) apply(debug error (wrong-type-argument listp t)) edebug(error (wrong-type-argument listp t)) signal(wrong-type-argument (listp t)) edebug-signal(wrong-type-argument (listp t)) semantic-parse-stream-default(((symbol 1 . 4) (symbol 5 . 6) (symbol 7 . 10)) nil) semantic-parse-stream(((symbol 1 . 4) (symbol 5 . 6) (symbol 7 . 10)) nil) semantic-repeat-parse-whole-stream(((symbol 1 . 4) (symbol 5 . 6) (symbol 7 . 10)) nil nil) semantic-parse-region-default(1 10 nil nil nil) semantic-parse-region-simple-mode(1 10 nil nil nil) semantic-parse-region(1 10) (edebug-after (edebug-before 0) 5 (semantic-parse-region (edebug-after ... 2 ...) (edebug-after ... 4 ...))) (lambda nil (edebug-after (edebug-before 0) 5 (semantic-parse-region ... ...)))() edebug-enter(ok-parser nil (lambda nil (edebug-after (edebug-before 0) 5 (semantic-parse-region ... ...)))) edebug-enter(ok-parser nil (lambda nil (edebug-after (edebug-before 0) 5 (semantic-parse-region ... ...)))) ok-parser() I ran the function call through edebug, and I was able to isolate the occurance of the error to the last bit of this piece of code: (defun semantic-repeat-parse-whole-stream (stream nonterm &optional returnonerror) "Iteratively parse the entire stream STREAM starting with NONTERM. Optional argument RETURNONERROR indicates that the parser should exit with the current results on a parse error. This function returns semantic tags without overlays." (let ((result nil) (case-fold-search semantic-case-fold) nontermsym tag) (while stream (setq nontermsym (semantic-parse-stream stream nonterm) ;; *snip* ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ error appears when this is evaluated But `semantic-parse-stream' is defined with a `define-overload', so I didn't think I could edebug it. But then I *did* edebug it, and was able to push on a little bit further into the code. I got the same error as above, as well as the message Parse error @ 1 The edebug process now runs through ;; *pins* tag (car (cdr nontermsym))) (if (not nontermsym) (error "Parse error @ %d" (car (cdr (car stream))))) ;; *snip* The problem seems to be that nontermsym comes up as nil. I'm not sure how to control this or where to turn next for further debugging. The only change in my setup was the change in `semantic-default-simple-setup' documented above. Maybe you can help me figure out why nontermsym is coming up nil? For the record, here is my wisent file again. ;;; simple.wy -- LALR grammar for (much simplified) Tiger %package simple-wy ;; apparently not useful for wisent? %start expr %type <punctuation> %token <punctuation> PLUS "+" %type <symbol> %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*" %% expr : ;; empty | symbol (quote $1) | symbol PLUS symbol (quote (+ $1 $3)) ; |