CEDET

> ;;;###autoload
   > (defun semantic-default-simple-setup ()
   >   "Set up a buffer for semantic parsing of the SIMPLE language."
   >   (semantic-install-function-overrides
   >    '((parse-region . semantic-simple-parse-region)
   >      (parse-changes . semantic-simple-parse-changes)))
   [...]

   In fact your problem is in the above code.  First you define a
   override to semantic-parse-region in simple-mode (looks correct), but
   then you override it again locally in the buffer with the undefined
   function `semantic-simple-parse-region'.

   IMO, you should just remove the above call to
   `semantic-install-function-overrides'.

Thanks.  I revised that function so that it looks like this:

(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
  (setq semantic--parse-table t
        document-comment-start "/*"
        document-comment-end " */"
	))

But upon running 

M-: (semantic-parse-region (point-min) (point-max))

or

M-: (semantic-parse-region (point-min) (point-max) 'expr)

where `expr' is my start symbol, the following error is triggered.

Debugger entered--Lisp error: (wrong-type-argument listp t)
  apply(debug error (wrong-type-argument listp t))
  edebug(error (wrong-type-argument listp t))
  signal(wrong-type-argument (listp t))
  edebug-signal(wrong-type-argument (listp t))
  semantic-parse-stream-default(((symbol 1 . 4) (symbol 5 . 6) (symbol 7 . 10)) nil)
  semantic-parse-stream(((symbol 1 . 4) (symbol 5 . 6) (symbol 7 . 10)) nil)
  semantic-repeat-parse-whole-stream(((symbol 1 . 4) (symbol 5 . 6) (symbol 7 . 10)) nil nil)
  semantic-parse-region-default(1 10 nil nil nil)
  semantic-parse-region-simple-mode(1 10 nil nil nil)
  semantic-parse-region(1 10)
  (edebug-after (edebug-before 0) 5 (semantic-parse-region (edebug-after ... 2 ...) (edebug-after ... 4 ...)))
  (lambda nil (edebug-after (edebug-before 0) 5 (semantic-parse-region ... ...)))()
  edebug-enter(ok-parser nil (lambda nil (edebug-after (edebug-before 0) 5 (semantic-parse-region ... ...))))
  edebug-enter(ok-parser nil (lambda nil (edebug-after (edebug-before 0) 5 (semantic-parse-region ... ...))))
  ok-parser()


I ran the function call through edebug, and I was able to isolate
the occurance of the error to the last bit of this piece of code:

(defun semantic-repeat-parse-whole-stream
  (stream nonterm &optional returnonerror)
  "Iteratively parse the entire stream STREAM starting with NONTERM.
Optional argument RETURNONERROR indicates that the parser should exit
with the current results on a parse error.
This function returns semantic tags without overlays."
  (let ((result nil)
        (case-fold-search semantic-case-fold)
        nontermsym tag)
    (while stream
      (setq nontermsym (semantic-parse-stream stream nonterm)

;; *snip*              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                        error appears when this is evaluated

But `semantic-parse-stream' is defined with a `define-overload', so
I didn't think I could edebug it.  But then I *did* edebug it, and
was able to push on a little bit further into the code.

I got the same error as above, as well as the message

Parse error @ 1

The edebug process now runs through

;; *pins*

            tag (car (cdr nontermsym)))
      (if (not nontermsym)
          (error "Parse error @ %d" (car (cdr (car stream)))))

;; *snip*

The problem seems to be that nontermsym comes up as nil.

I'm not sure how to control this or where to turn next for further
debugging.  The only change in my setup was the change in
`semantic-default-simple-setup' documented above.

Maybe you can help me figure out why nontermsym is coming up nil?

For the record, here is my wisent file again.

;;; simple.wy -- LALR grammar for (much simplified) Tiger

%package simple-wy

;; apparently not useful for wisent?
%start expr

%type  <punctuation>
%token <punctuation>    PLUS	"+"

%type  <symbol>
%token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*"

%%

expr
  : ;; empty
  | symbol
    (quote $1)
  | symbol PLUS symbol
    (quote (+ $1 $3))
  ;

Hi Joe,

[...]
> Thanks.  I revised that function so that it looks like this:
> 
> (defun semantic-default-simple-setup ()
>   "Set up a buffer for semantic parsing of a SIMPLE language."
>   (setq semantic--parse-table t
>         document-comment-start "/*"
>         document-comment-end " */"
> 	))
> 
> But upon running 
> 
> M-: (semantic-parse-region (point-min) (point-max))
> 
> or
> 
> M-: (semantic-parse-region (point-min) (point-max) 'expr)
> 
> where `expr' is my start symbol, the following error is triggered.
> 
> Debugger entered--Lisp error: (wrong-type-argument listp t)
[...]

This is because you specified that your `semantic--parse-table' is `t'
instead of a list (for the bovinator) or a vector (for wisent).

As you didn't install a specific parser in your setup function, the
default bovinator is used.  Obviously, it fails with a
"wrong-type-argument" on parser table value `t', which is not of the
expected format: 'listp'.

Your setup function above must look like:

(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
  (simple-wy--install-parser)
  (setq document-comment-start "/*"
        document-comment-end " */"
 	))

where `simple-wy--install-parser' is a function generated from your
simple.wy grammar, that setups for you the  wisent parser, based on
what is generated in the simple-wy.el library.

Probably you will also need to add (require 'simple-wy) in your
semantic-simple library.

Good luck!
David

(defun semantic-default-simple-setup ()
     "Set up a buffer for semantic parsing of a SIMPLE language."
     (simple-wy--install-parser)
     (setq document-comment-start "/*"
           document-comment-end " */"
    	))
   
   where `simple-wy--install-parser' is a function generated from your
   simple.wy grammar, that setups for you the  wisent parser, based on
   what is generated in the simple-wy.el library.
   
   Probably you will also need to add (require 'simple-wy) in your
   semantic-simple library.


I have set it up as you suggested, but this time upon running
(semantic-parse-region (point-min) (point-max)) I just get `nil'.

I'd appreciate it if you could take a look.  I seem to have gotten
down to the bare bones here, and I think I've followed all your
instructions, and it still doesn't seem to be working as expected.

Everything I'm working with, including the auto-generated parser
is below.  I hope I'm just making a simple error.

/* example.simple -- about the simplest possible example */

abc + xyz

/* example.simple ends here */

;;; semantic-hacks.el -- actually just a simple setup

(load "~/cedet-1.0beta2a/common/cedet.el") 
;; Make my working directory available to Emacs
(add-to-list 'load-path "~/Parser/simple/")
;; my parser-generator code to be "compiled" with keyboard command C-c C-c
(find-file "/Users/arided/Parser/simple/simple.wy")
;; the elisp parser automatically pops up and loads but just to be sure...
(load "/Users/arided/Parser/simple/simple-wy.el")
;; this is code to set up the parser.  In particular, it contains the code
;; (add-hook 'simple-mode-hook 'semantic-default-simple-setup)
(load "/Users/arided/Parser/simple/semantic-simple.el")
;; we might want to take a look at this file!
(find-file "/Users/arided/Parser/simple/semantic-simple.el")
;; tiny major mode that works on files with suffix ".simple"
(load "~/site-lisp/simple-mode.el")

;;; semantic-hacks.el ends here

;;; simple.wy -- LALR grammar for (much simplified) Tiger

%package simple-wy

;; apparently not useful for wisent?
%start expr

%type  <punctuation>
%token <punctuation>    PLUS	"+"

%type  <symbol>
%token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*"

%%

expr
  : ;; empty
  | symbol
    (quote $1)
  | symbol PLUS symbol
    (quote (+ $1 $3))
  ;

;;; simple.wy ends here

;;; semantic-simple.el -- parser support for a simple language.

(require 'simple-wy)  ;; The parser

(define-mode-overload-implementation
  semantic-parse-region simple-mode
  (start end &optional nonterminal depth returnonerror)
  "Over-ride in order to initialize some variables."
    (semantic-parse-region-default
     start end nonterminal depth returnonerror))

;; As instructed by the info node Semantic Overload Mechanism,
;; looking at examples
;;;###autoload
(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
  (simple-wy--install-parser) 
  (setq document-comment-start "/*"
        document-comment-end " */" ))

;;;###autoload
(add-hook 'simple-mode-hook 'semantic-default-simple-setup)

(provide 'semantic-simple)

;;; semantic-simple.el ends here

;;; simple-mode.el --- simple mode to get my parser set up

(defvar simple-mode-hook nil)

(define-derived-mode simple-mode text-mode "simple"
  ""
  (run-hooks 'simple-mode-hook))

(provide 'simple-mode)

;;; simple-mode.el ends here

;;; simple-wy.el --- Generated parser support file

;; Copyright (C) 2004 arided

;; Author: arided <arided@...>
;; Created: 2004-04-26 13:25:35-0500
;; Keywords: syntax
;; X-RCS: $Id$

;; This file is not part of GNU Emacs.
;;
;; This program is free software; you can redistribute it and/or
;; modify it under the terms of the GNU General Public License as
;; published by the Free Software Foundation; either version 2, or (at
;; your option) any later version.
;;
;; This software is distributed in the hope that it will be useful,
;; but WITHOUT ANY WARRANTY; without even the implied warranty of
;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
;; General Public License for more details.
;;
;; You should have received a copy of the GNU General Public License
;; along with GNU Emacs; see the file COPYING.  If not, write to the
;; Free Software Foundation, Inc., 59 Temple Place - Suite 330,
;; Boston, MA 02111-1307, USA.

;;; Commentary:
;;
;; PLEASE DO NOT MANUALLY EDIT THIS FILE!  It is automatically
;; generated from the grammar file simple.wy.

;;; History:
;;

;;; Code:

;;; Prologue
;;

;;; Declarations
;;
(defconst simple-wy--keyword-table
  (semantic-lex-make-keyword-table 'nil 'nil)
  "Table of language keywords.")

(defconst simple-wy--token-table
  (semantic-lex-make-type-table
   '(("symbol"
      (symbol . "[A-Za-z][_A-Za-z0-9]*"))
     ("punctuation"
      (PLUS . "+")))
   '(("symbol" :declared t)
     ("punctuation" :declared t)))
  "Table of lexical tokens.")

(defconst simple-wy--parse-table
  (progn
    (eval-when-compile
      (require 'wisent-comp))
    (wisent-compile-grammar
     '((PLUS symbol)
       nil
       (expr
        (nil)
        ((symbol)
         '$1)
        ((symbol PLUS symbol)
         '(+ $1 $3))))
     '(expr)))
  "Parser table.")

(defun simple-wy--install-parser ()
  "Setup the Semantic Parser."
  (semantic-install-function-overrides
   '((parse-stream . wisent-parse-stream)))
  (setq semantic-parser-name "LALR"
        semantic--parse-table simple-wy--parse-table
        semantic-debug-parser-source "simple.wy"
        semantic-flex-keywords-obarray simple-wy--keyword-table
        semantic-lex-types-obarray simple-wy--token-table)
  ;; Collect unmatched syntax lexical tokens
  (semantic-make-local-hook 'wisent-discarding-token-functions)
  (add-hook 'wisent-discarding-token-functions
            'wisent-collect-unmatched-syntax nil t))


;;; Analyzers
;;
(require 'semantic-lex)

(define-lex-regex-type-analyzer simple-wy--<symbol>-regexp-analyzer
  "regexp analyzer for <symbol> tokens."
  "\\(\\sw\\|\\s_\\)+"
  '((symbol . "[A-Za-z][_A-Za-z0-9]*"))
  'symbol)

(define-lex-string-type-analyzer simple-wy--<punctuation>-string-analyzer
  "string analyzer for <punctuation> tokens."
  "\\(\\s.\\|\\s$\\|\\s'\\)+"
  '((PLUS . "+"))
  'punctuation)


;;; Epilogue
;;

(provide 'simple-wy)

;;; simple-wy.el ends here

Hi Joe,

[...]
> I have set it up as you suggested, but this time upon running
> (semantic-parse-region (point-min) (point-max)) I just get `nil'.
>=20
> I'd appreciate it if you could take a look.  I seem to have gotten
> down to the bare bones here, and I think I've followed all your
> instructions, and it still doesn't seem to be working as expected.
>=20
> Everything I'm working with, including the auto-generated parser
> is below.  I hope I'm just making a simple error.

In fact, you're missing a lexer definition, based on lexical rule
analyzers auto-generated from the %type statements in your grammar.

Also there is no need to override `semantic-parse-region' in
simple-mode, because you use the default!

I hacked a little bit you grammar & simple-mode support code, and the
parser works well now with this example.simple file:

---------- cut here
abc + xyz

xxx

a + b
---------- cut here

Here is my hacked simple-mode.el which merge your semantic-simple and
simple-mode files (please don't use the `semantic-' prefix for your
libraries to avoid confusion with semantic's ones).

;;; simple-mode.el -- major mode for a simple language

;;; Semantic parsing support
;;
(require 'semantic-wisent)
(require 'simple-wy)

;;;###autoload
(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
  ;; Install the parser
  (simple-wy--install-parser)
  ;; Setup the lexer
  (setq semantic-lex-analyzer 'simple-lexer
        ;; Do a full depth lexical analysis.
        semantic-lex-depth nil)
  ;; Other useful things.
  (setq document-comment-start "/*"
        document-comment-end   " */"))

;;;###autoload
(add-hook 'simple-mode-hook 'semantic-default-simple-setup)
=0C
;;; simple major mode
;;
(defvar simple-mode-syntax-table
  (let ((table (make-syntax-table (standard-syntax-table))))
    (modify-syntax-entry ?\+ "."     table) ;; Operator PLUS
    (modify-syntax-entry ?\- "."     table) ;; Operator MINUS
    (modify-syntax-entry ?\* "."     table) ;; Operator MULT
    (modify-syntax-entry ?\/ "."     table) ;; Operator DIV
    table)
  "Syntax table used in simple mode buffers.
Define operators as punctuations.")

;;;###autoload
(define-derived-mode simple-mode fundamental-mode "simple")

;;;###autoload
(add-to-list 'auto-mode-alist '("\\.simple\\'" . simple-mode))

(provide 'simple-mode)

;;; simple-mode.el ends here

Here is the simple.wy grammar which now include the lexical analyzer
definition:

;;; simple.wy -- LALR grammar for (much simplified) Tiger

%package simple-wy

;; No really necessary, as it is the default start symbol
%start expr

%type  <punctuation>
%token <punctuation> PLUS "+"

%type  <symbol>
%token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*"

%%

;; For use with Semantic, must return valid semantic tags!
expr
  : ;; empty
  | symbol
    (TAG "expr" 'expr :value $1)
  | symbol PLUS symbol
    (TAG "expr" 'expr :value (concat $1 $2 $3))
  ;

%%

(define-lex simple-lexer
  "Simple lexical analyzer."
  semantic-lex-ignore-whitespace
  semantic-lex-ignore-newline
  semantic-lex-ignore-comments
  ;;;; Auto-generated analyzers.
  simple-wy--<symbol>-regexp-analyzer
  simple-wy--<punctuation>-string-analyzer
  ;;;;
  semantic-lex-default-action)

;;; simple.wy ends here

To use that semantic-enabled simple-mode, put the above files
somewhere in your load-path.  Edit the grammar and
generate the simple-wy.el file with C-c C-c.  Save it.

Then do (require 'simple-mode) and open the example.simple file.
'M-x bovinate' should show you the tags obtained from the simple
expressions found in the example.

Enjoy!
David

In fact, you're missing a lexer definition, based on lexical rule
   analyzers auto-generated from the %type statements in your grammar.

For some reason I thought it would be totally auto-generated - as
in, no need to write anything extra. Oops!

   Also there is no need to override `semantic-parse-region' in
   simple-mode, because you use the default!

   I hacked a little bit you grammar & simple-mode support code, and
   the parser works well now with this example.simple file:

Thanks a bunch - is indeed working just fine for me now.

If it seems to you (as it does to me) that these files might help
other people who want to see a bare-bones set up, you might then
include them in the semantic distribution.

I have a few questions at this point.  The wisent-calc.wy doesn't
produce the same kind of tags as you suggested here:

   ;; For use with Semantic, must return valid semantic tags!
   expr
     : ;; empty
     | symbol
       (TAG "expr" 'expr :value $1)
     | symbol PLUS symbol
       (TAG "expr" 'expr :value (concat $1 $2 $3))
     ;

Rather, like Many of the examples in the documentation for wisent
produce things like `(cons $1 $2)' or `(+ $1 $3)'.

Since I want to "grow" this parser into an interpreter for a simple
programming language (Tiger), it seems like it would be nice to get
some usable LISP code generated at this phase.  What do you suggest
for this?  

Suppose that, following the pattern established in wisent-calc.wy, I
define

   expr
     : ;; empty
     | symbol
       (string-to-number $1)
     | symbol PLUS symbol
       (+ $1 $3)
     ;

Then I feed in the example input

   10 + 11

with bovinate and get `nil'.  (I naively might expect to get
something like `21' or `(+ 10 11)' instead.)

Even the kluge

   expr
     : ;; empty
     | symbol
       (concat $1)
     | symbol PLUS symbol
       (concat "(+ " $1 " " $3 " )")
     ;

doesn't work, though this seems to be very much like the code that
appears in wisent-java.wy.

Of course, I can write

   expr
     : ;; empty
     | symbol
       (TAG "expr" 'expr :value $1)
     | symbol PLUS symbol
       (TAG "expr" 'expr :value (concat "(+ " $1 " " $3 ")"))
     ;

and *this* will work with bovinate, but then I'd be curious to know
what the prefered method for doing operations on the output of
bovinate is.

My next question is this - even with the document-comment-start and
document-comment-end as set up here --

(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
  ;; Install the parser
  (simple-wy--install-parser)
  ;; Setup the lexer
  (setq semantic-lex-analyzer 'simple-lexer
        ;; Do a full depth lexical analysis.
        semantic-lex-depth nil)
  ;; Other useful things.
  (setq document-comment-start "/*"
        document-comment-end   " */"))

and the apparent instruction to ignore comments as given here

(define-lex simple-lexer
  "Simple lexical analyzer."
  semantic-lex-ignore-whitespace
  semantic-lex-ignore-newline
  semantic-lex-ignore-comments
  ;;;; Auto-generated analyzers.
  simple-wy--<symbol>-regexp-analyzer
  simple-wy--<punctuation>-string-analyzer
  ;;;;
  semantic-lex-default-action)

Parsing the following input 

  /* this is a comment */

  10 + 11

  1000 + 1

produces this as output:

(("expr" expr
  (:value "this")
  nil #<overlay from 4 to 8 in example.simple>)
 ("expr" expr
  (:value "is")
  nil #<overlay from 9 to 11 in example.simple>)
 ("expr" expr
  (:value "a")
  nil #<overlay from 12 to 13 in example.simple>)
 ("expr" expr
  (:value "comment")
  nil #<overlay from 14 to 21 in example.simple>)
 ("expr" expr
  (:value "(+ 10 11)")
  nil #<overlay from 26 to 33 in example.simple>)
 ("expr" expr
  (:value "(+ 1000 1)")
  nil #<overlay from 35 to 43 in example.simple>))

How to ensure that things between /* and */ are ignored?  (Is that
space character in the definition of document-comment-end relevant?)

Joe,

  After David's answer, do you recall what lead to your confusion on
how wisent worked?  Is there something obvious we could do to prevent
this confusion for the next person?

Thanks
Eric

>>> David PONCE <david.ponce@...> seems to think that:
>Hi Joe,
>
>[...]
>> I have set it up as you suggested, but this time upon running
>> (semantic-parse-region (point-min) (point-max)) I just get `nil'.
>> 
>> I'd appreciate it if you could take a look.  I seem to have gotten
>> down to the bare bones here, and I think I've followed all your
>> instructions, and it still doesn't seem to be working as expected.
>> 
>> Everything I'm working with, including the auto-generated parser
>> is below.  I hope I'm just making a simple error.
>
>In fact, you're missing a lexer definition, based on lexical rule
>analyzers auto-generated from the %type statements in your grammar.
>
>Also there is no need to override `semantic-parse-region' in
>simple-mode, because you use the default!
>
>I hacked a little bit you grammar & simple-mode support code, and the
>parser works well now with this example.simple file:
  [ ... ]

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

After David's answer, do you recall what lead to your confusion on
   how wisent worked?  Is there something obvious we could do to prevent
   this confusion for the next person?

Well, as I recently noted there is at least one thing that I'm still
confused about, and that is: what is the correct way to produce
output from the parsing system.

David suggested writing my wisent input in the following style:

   ;; For use with Semantic, must return valid semantic tags!
   expr
     : ;; empty
     | symbol
       (TAG "expr" 'expr :value $1)
     | symbol PLUS symbol
       (TAG "expr" 'expr :value (concat $1 $2 $3))
     ;

But in wisent-java.wy I see things like

type_parameters
  : LT type_parameter_list_1
    (progn $2)
  ;

type_parameter_list
  : type_parameter_list COMMA type_parameter
    (cons $3 $1)
  | type_parameter
    (list $1)
  ;

Similarly, but more dramatically, wisent-calc.wy actually does the
computations that the user types in!

So my question is - what is the correct protocol to use for
producing output from the parser?  Recall that my goal is to use
wisent to transform code for another programming language (Tiger)
into Elisp.  (It might be fun to make the language work with
semantic, but that is not a high priority goal for me at this time
-- not for this particular language anyway.)

Other than this question, I think my first point of confusion was
that I thought that if the user (me) didn't supply a lexer, then
some default lexer would be used.  That apparently was just
incorrect.  Another question would be, why is wisent better to use
as a parser than bovine?  (Maybe the full answer is more technical
than I really need to know, but I'm still interested to know
something about this question.)

I think my confusion would be massively reduced, if not prevented
altogether, by documentation that contains a step-by-step guide
(tutorial, maybe) for such things as writing a working wisent
parsing system and getting this system to work with semantic.

I would definitely be willing to help write this documentation if
you like -- but since I'm still riding the learning curve, I'm not
sure if that would be the best thing to do.  Still, I'd be happy
to contribute my code (once it is working), and that might be helpful
to others.

To sum up: as always, more thorough documentation almost always
reduces confusion!

After David's answer, do you recall what lead to your confusion
   on how wisent worked?  Is there something obvious we could do to
   prevent this confusion for the next person?

More advice:

This is taken from the GNU coding standards:

   The only good way to use documentation strings in writing a good
   manual is to use them as a source of information for writing good
   text.

Bearing this in mind, you might take another look at

  File: semantic-appdev.info,  Node: Tag Query
  File: semantic-appdev.info,  Node: Breadth Search
  File: grammar-fw.info,  Node: Specialized Implementation

etc. from the point of view of a novice user.  Such sections, and
indeed all of your documentation, could be made much more friendly
if the following principle, also from the GNU coding standards, was
adhered to throughout:

   At every level, from the sentences in a paragraph to the grouping
   of topics into separate manuals, the right way to structure
   documentation is according to the concepts and questions that a
   user will have in mind when reading it.

I have a lot of questions that I'm not sure are actually answered in
the documentation at all.  Not that I could find easily anyway.

I would be happy to help identify more of these user-level questions
for you - and I hope my previous postings have highlighted a couple
of the key ones.

I'm now stuck on what seem to be a couple of relatively
minor problems:

1. How to deal with comments?

2. Why does the rule

  | LPAREN WHILE RPAREN
    (TAG "expr" 'expr :value (concat "(" "foo" ")"))

not work?  (This is drawn from a slightly more complex
parser-generator file than the one I posted earlier... the new file
is below.)  This should work with the simple-mode.el file you
already have.


;;; simple.wy -- LALR grammar for (simplified) Tiger

%package simple-wy

;; Not really necessary, as it is the default start symbol
%start expr

%type  <punctuation>
%token <punctuation> PLUS "+"

%type  <symbol>
%token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*"

%type  <string>
%token <string> STRING_LITERAL

%type  <number>
%token <number> NUMBER_LITERAL

%type  <keyword>
%keyword ARRAY "array" 
%keyword break "break"
%keyword DO "do"
%keyword ELSE "else"
%keyword END "end"
%keyword FOR "for"
%keyword FUNCTION "function"
%keyword IF "if"
%keyword IN "in"
%keyword LET "let"
%keyword NIL "nil"
%keyword OF "of"
%keyword THEN "then"
%keyword TO "to"
%keyword TYPE "type"
%keyword VAR "var"
%keyword WHILE "while"

%token <open-paren>  LPAREN      "("
%token <open-paren>  LBRACE      "{"
%token <open-paren>  LBRACK      "["

%token <close-paren> RPAREN      ")"
%token <close-paren> RBRACE      "}"
%token <close-paren> RBRACK      "]"

%%

;; For use with Semantic, must return valid semantic tags!
expr
  : ;; empty
  | STRING_LITERAL
    (TAG "string" 'expr :value $1)
  | NUMBER_LITERAL
    (TAG "number" 'expr :value $1)
  | symbol
    (TAG "expr" 'expr :value $1)
  | symbol PLUS symbol
    (TAG "expr" 'expr :value (concat "(+ " $1 " " $3 ")"))
  | NIL
    (TAG "expr" 'expr :value $1)
  | LPAREN WHILE RPAREN
    (TAG "expr" 'expr :value (concat "(" "foo" ")"))
  ;

%%

(define-lex simple-lexer
  "Simple lexical analyzer."
  semantic-lex-ignore-whitespace
  semantic-lex-ignore-newline
  semantic-lex-ignore-comments
  semantic-lex-open-paren
  semantic-lex-close-paren
  ;;;; Auto-generated analyzers.
  simple-wy--<punctuation>-string-analyzer
  simple-wy--<block>-block-analyzer
  simple-wy--<symbol>-regexp-analyzer
  simple-wy--<string>-sexp-analyzer
  simple-wy--<number>-regexp-analyzer
  simple-wy--<keyword>-keyword-analyzer
  ;;;;
  semantic-lex-default-action)

;;; simple.wy ends here

Hi,

>>> Joe Corneli <jcorneli@...> seems to think that:
>     After David's answer, do you recall what lead to your confusion
>   on how wisent worked?  Is there something obvious we could do to
>   prevent this confusion for the next person?
>
>More advice:
>
>This is taken from the GNU coding standards:
>
>   The only good way to use documentation strings in writing a good
>   manual is to use them as a source of information for writing good
>   text.
>
>Bearing this in mind, you might take another look at
>
>  File: semantic-appdev.info,  Node: Tag Query
>  File: semantic-appdev.info,  Node: Breadth Search
>  File: grammar-fw.info,  Node: Specialized Implementation
>
>etc. from the point of view of a novice user.  Such sections, and
>indeed all of your documentation, could be made much more friendly
>if the following principle, also from the GNU coding standards, was
>adhered to throughout:
>
>   At every level, from the sentences in a paragraph to the grouping
>   of topics into separate manuals, the right way to structure
>   documentation is according to the concepts and questions that a
>   user will have in mind when reading it.
>
>I have a lot of questions that I'm not sure are actually answered in
>the documentation at all.  Not that I could find easily anyway.
>
>I would be happy to help identify more of these user-level questions
>for you - and I hope my previous postings have highlighted a couple
>of the key ones.
  [ ... ]

I think you will find everyone here contributing to the doc is an
engineer these days, though Richard Kim did help a great deal.

For me, at least, documentation is an engineering challenge.  How
does it work, how to avoid duplication of effort, etc.  Building a
reference manual automatically from sources seems like the way to go
for me.  I then spend documentation authoring time (when I have some)
trying to improve the doc strings of my lisp functions.

Much of the manuals are just catalogs of things.  Complicated bits
tend to go into auto-generated code which David has been working on,
or a skeleton (-skel.*) which can be copied as a starting point for a
new language.

Over the years it is apparent to me that I am not a good documentation
writer.  If anyone out there wishes to help resolve these issues via
the contribution of text and structure, please let me know.

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Hi Joe,

I am sorry for this late reply, but I am very busy these days!

[...]
> I'm now stuck on what seem to be a couple of relatively
> minor problems:
> 
> 1. How to deal with comments?

semantic-lex uses the current Emacs syntax table to recognize language
comments.  So you must set your major mode syntax table accordingly.
In your case, it looks that your language comments are like C ones, so
you could change the `simple-mode-syntax-table' into your
simple-mode.el file like this:

(defvar simple-mode-syntax-table
  (let ((table (make-syntax-table (standard-syntax-table))))
    (modify-syntax-entry ?\+ "."     table) ;; Operator PLUS
    (modify-syntax-entry ?\- "."     table) ;; Operator MINUS
    ;; Operator MULT & 2nd char of "/*" comment-start sequence & 1st
    ;; char of "*/" comment-end sequence
    (modify-syntax-entry ?\* ". 23"  table)
    ;; Operator DIV & 1st char of "/*" comment-start sequence & 2nd
    ;; char of "*/" comment-end sequence
    (modify-syntax-entry ?\/ ". 14" table)
    table)
  "Syntax table used in simple mode buffers.
Define operators as punctuations.")

I recommend you to read the chapter on syntax tables on the "Emacs
Lisp Reference Manual", to understand the meaning of the `1', `2',
`3', `4' syntax flags.

> 
> 2. Why does the rule
> 
>   | LPAREN WHILE RPAREN
>     (TAG "expr" 'expr :value (concat "(" "foo" ")"))
> 
> not work?  (This is drawn from a slightly more complex
> parser-generator file than the one I posted earlier... the new file
> is below.)  This should work with the simple-mode.el file you
> already have.
> 
[...]

Parenthesis-like tokens are better handled by an auto-generated
<block> analyzer.  To declare it you should add:

%type  <block>
%token <block> PAREN_BLOCK "(LPAREN RPAREN)"
%token <block> BRACE_BLOCK "(LBRACE RBRACE)"
%token <block> BRACK_BLOCK "(LBRACK RBRACK)"

to indicate the possible blocks made of these parenthesis
delimiters.

> %token <open-paren>  LPAREN      "("
> %token <open-paren>  LBRACE      "{"
> %token <open-paren>  LBRACK      "["
> 
> %token <close-paren> RPAREN      ")"
> %token <close-paren> RBRACE      "}"
> %token <close-paren> RBRACK      "]"

Unfortunately, the grammar framework documentation is not yet
complete, particularly the chapters on auto-generated lexical
analyzers :(  For now, you can read the doc string of
`define-lex-block-type-analyzer', for more details.

> 
[...]
> (define-lex simple-lexer
>   "Simple lexical analyzer."
>   semantic-lex-ignore-whitespace
>   semantic-lex-ignore-newline
>   semantic-lex-ignore-comments

These two analyzers aren't necessary because parenthesis-like tokens
are handled by the <block> analyzer (see above).

>   semantic-lex-open-paren
>   semantic-lex-close-paren

>   ;;;; Auto-generated analyzers.
>   simple-wy--<punctuation>-string-analyzer
>   simple-wy--<block>-block-analyzer
[...]

Hope this will help.

Good luck!
David

semantic-lex uses the current Emacs syntax table to recognize
   language comments.

   I recommend you to read the chapter on syntax tables on the "Emacs
   Lisp Reference Manual", to understand the meaning of the `1', `2',
   `3', `4' syntax flags.

Ok, thanks - I had been looking at the documentation string for
modify-syntax-entry and for some reason thought that the different
properties were to be added sequentially.  However, while that
didn't work, neither did the revised syntax table that you
suggested, which follows the pattern set forth in the ELISP manual.

   Parenthesis-like tokens are better handled by an auto-generated
   <block> analyzer.  To declare it you should add:

   %type  <block>
   %token <block> PAREN_BLOCK "(LPAREN RPAREN)"
   %token <block> BRACE_BLOCK "(LBRACE RBRACE)"
   %token <block> BRACK_BLOCK "(LBRACK RBRACK)"

   to indicate the possible blocks made of these parenthesis
   delimiters.

OK, this (and grammer elements in the style of wisent-python) appear
to be working for me.  Thanks.

Anyone willing to take a look and try to understand why comments
aren't being ignored properly?  Here are my new files.


;;; simple-mode.el -- major mode for a simple language

;;; Semantic parsing support

(require 'semantic-wisent)
(require 'simple-wy)

;;;###autoload
(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
    ;; Do some useful things.
  (setq 
   semantic-ignore-comments t
   document-comment-start "/\\*"
   document-comment-end   "\\*/")
  ;; Install the parser
  (simple-wy--install-parser)
  ;; Setup the lexer
  (setq semantic-lex-analyzer 'simple-lexer
        ;; Do a full depth lexical analysis.
        semantic-lex-depth nil))

;;;###autoload
(add-hook 'simple-mode-hook 'semantic-default-simple-setup)

;;; post parser processing

;; I'm not sure why Bovine is not getting rid of comments -- I suppose
;; I haven't dealt with them properly.  I was sort of under the
;; impression that this would be handled automatically after setting
;; the document comment start and end -- but that doesn't seem to be
;; the case.  Maybe I just need to remove comments as part of my own
;; preprocessing phase.  Wouldn't be hard to do.

(defun clean-up-parser-output ()
  (interactive)
  (set-buffer (get-buffer "*Parser Output*"))
  (flush-lines "overlay")
  (flush-lines "\"expr\" expr")
  (goto-char (point-min))
  (replace-regexp "  (:value \"\\|\")$" ""))

(defun novinate ()
  (interactive)
  (bovinate)
  (clean-up-parser-output))

;;; simple major mode

(defvar simple-mode-syntax-table
  (let ((table (make-syntax-table (standard-syntax-table))))
    (modify-syntax-entry ?\+ "."     table) ;; Operator PLUS
    (modify-syntax-entry ?\- "."     table) ;; Operator MINUS
    ;; also deals with comments
    (modify-syntax-entry ?\* ". 23"  table) ;; Operator MULT
    (modify-syntax-entry ?\/ ". 14"  table) ;; Operator DIV
    ;; deal with brackets
    (modify-syntax-entry ?\( "()"    table)
    (modify-syntax-entry ?\{ "(}"    table)
    (modify-syntax-entry ?\[ "(]"    table)
    (modify-syntax-entry ?\) ")("    table)
    (modify-syntax-entry ?\} "){"    table)
    (modify-syntax-entry ?\] ")["    table)
    table)
  "Syntax table used in simple mode buffers.
Define operators as punctuations.")

;;;###autoload
(define-derived-mode simple-mode fundamental-mode "simple")

;;;###autoload
(add-to-list 'auto-mode-alist '("\\.simple\\'" . simple-mode))

;(defvar simple-mode-hook nil)

(provide 'simple-mode)

;;; simple-mode.el ends here


;;; simple.wy -- LALR grammar for (simplified) Tiger

%package simple-wy

;; Not really necessary, as it is the default start symbol
%start expr
%start function_parameters

;; %type  <symbol>
;; %token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*"

%type  <keyword>
%keyword WHILE "while"
%keyword FOR   "for"

%type  <open-paren>
%token <open-paren>  LPAREN      "("
%token <open-paren>  LBRACE      "{"
%token <open-paren>  LBRACK      "["

%type  <close-paren>
%token <close-paren> RPAREN      ")"
%token <close-paren> RBRACE      "}"
%token <close-paren> RBRACK      "]"

%type  <block>
%token <block> PAREN_BLOCK "(LPAREN RPAREN)"
%token <block> BRACE_BLOCK "(LBRACE RBRACE)"
%token <block> BRACK_BLOCK "(LBRACK RBRACK)"

%%

;; For use with Semantic, must return valid semantic tags!
expr
  : ;; Empty
  | PAREN_BLOCK
 ;;(TAG "expr" 'expr :value $2)
    (EXPANDFULL $1 function_parameters)
  | FOR
    (TAG "expr" 'expr :value $1)
  ;

;; parameters: '(' [varargslist] ')'
function_parameters
  : LPAREN
    ()
  | RPAREN
    ()
 ;;  | function_parameter COMMA
  | function_parameter RPAREN
  ;

function_parameter
  : WHILE
    (TAG "function_parameter" 'function_parameter :value $1)
  ;

%%

(define-lex simple-lexer
  "Simple lexical analyzer."
  semantic-lex-ignore-whitespace
  semantic-lex-ignore-newline
  semantic-lex-ignore-comments
  ;;;; Auto-generated analyzers.
  simple-wy--<block>-block-analyzer
  simple-wy--<keyword>-keyword-analyzer
  ;;;;
  semantic-lex-default-action)

;;; simple.wy ends here

Over the years it is apparent to me that I am not a good
   documentation writer.  If anyone out there wishes to help resolve
   these issues via the contribution of text and structure, please
   let me know.

Your email responses to my questions have generally been quite
helpful.  Maybe looking in the mailing list archives, you'll see
other answers to user's questions that could go into the
documentation.  I think that, if possible, its good to take a look
and see what questions people are actually asking and make these the
central focus for documentation.

>>> Joe Corneli <jcorneli@...> seems to think that:
>
>   Over the years it is apparent to me that I am not a good
>   documentation writer.  If anyone out there wishes to help resolve
>   these issues via the contribution of text and structure, please
>   let me know.
>
>Your email responses to my questions have generally been quite
>helpful.  Maybe looking in the mailing list archives, you'll see
>other answers to user's questions that could go into the
>documentation.  I think that, if possible, its good to take a look
>and see what questions people are actually asking and make these the
>central focus for documentation.

If you look in the .texi file, you will find some of those email
messages in comment blocks, waiting to be better described there.
This had been graciously done for us by Richard Kim.

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Here are a couple more simple questions.  I noticed that when I (1)
change the grammar file, (2) run `semantic-grammar-create-package'
(C-c C-c) and load the resulting generated parser support file, (3)
bovinate the example source file that I'm working with, the changes
to the grammar are not always reflected in the *Parser Output*
buffer.  When this happens, (4) I have to kill the source code
buffer and (5) open it again, then (6) re-bovinate.  Is this the
expected behavior?  Is there a shortcut I can use to make sure that
the current parser is being used when I bovinate?

Second question is that after repeating (1)-(3) and/or (1)-(6), my
whole system (X window manager and Emacs) begins to handle much more
sluggishly than usual.  Has anyone seen anything like this happen
before?  Where might the slow-down be coming from?

Things are beginning to come along with this code for me.  But here
is another problem that seems somewhat weird.

As part of my definition of expression (`expr') I have the following
rule:

  | expr EQ expr
    (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")"))
  | expr NEQ expr
    (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))"))

But this rule does not seem to be dealt with properly by the parser.
For example, neither of these two inputs is recognized:

  foo <> bar
  "foo" <> "bar"

If I modify the rule to say

  | STRING_LITERAL EQ STRING_LITERAL
    (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")"))
  | STRING_LITERAL NEQ STRING_LITERAL
    (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))"))

then

  "foo" <> "bar"

is recognized just fine.  Running semantic-lex-debug seems to
indicate that  foo <> bar  is lexically OK.

I'm including the revised file below.  Perhaps someone can tell
me why this particular bit of code is causing trouble?



;;; simple.wy -- LALR grammar for (simplified) Tiger

%package simple-wy

;; Not really necessary, as it is the default start symbol
%start expr
%start function_parameters

%type  <punctuation>
%token <punctuation> PLUS                     "+"
%token <punctuation> MINUS                    "-"
%token <punctuation> TIMES                    "*"
%token <punctuation> DIVIDE                   "/"

%type  <symbol>
%token <symbol> symbol "[A-Za-z][_A-Za-z0-9]*"

%type  <string>
%token <string> STRING_LITERAL

%type  <number>
%token <number> NUMBER_LITERAL

%type  <keyword>
%keyword WHILE                   "while"
%keyword FOR                     "for"
%keyword NIL                     "nil"
%keyword GREATER_THAN             ">"
%keyword LESS_THAN                "<"
 ;; these seem to need to be handled as keywords (not punctuation)
%keyword GREATER_THAN_OR_EQUAL   ">="
%keyword LESS_THAN_OR_EQUAL      "<="
%keyword EQ                      "="
%keyword NEQ                     "<>"

%left UMINUS
%left PLUS
%left MINUS
%left TIMES
%left DIVIDE

%type  <open-paren>
%token <open-paren>  LPAREN      "("
%token <open-paren>  LBRACE      "{"
%token <open-paren>  LBRACK      "["

%type  <close-paren>
%token <close-paren> RPAREN      ")"
%token <close-paren> RBRACE      "}"
%token <close-paren> RBRACK      "]"

%type  <block>
%token <block> PAREN_BLOCK "(LPAREN RPAREN)"
%token <block> BRACE_BLOCK "(LBRACE RBRACE)"
%token <block> BRACK_BLOCK "(LBRACK RBRACK)"

%%

;; For use with Semantic, must return valid semantic tags!
expr
  : ;; Empty
  | PAREN_BLOCK
    (EXPANDFULL $1 function_parameters)
  | FOR
    (TAG "expr" 'expr :value $1)
  | STRING_LITERAL
    (TAG "string" 'expr :value $1)
  | NUMBER_LITERAL
    (TAG "number" 'expr :value $1)
  | MINUS NUMBER_LITERAL %prec UMINUS
    (TAG "expr" 'expr :value (concat "-" $2))
  | NUMBER_LITERAL binop NUMBER_LITERAL 
    (TAG "binop result" 'expr :value (concat "(" $2 " " $1 " " $3 ")"))
  | string_comparison
  | expr EQ expr
    (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")"))
  | expr NEQ expr
    (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))"))
;;   | STRING_LITERAL EQ STRING_LITERAL
;;     (TAG "expr eq" 'expr :value (concat "(equal " $1 " " $3 ")"))
;;   | STRING_LITERAL NEQ STRING_LITERAL
;;     (TAG "expr neq" 'expr :value (concat "(not (equal " $1 " " $3 "))"))
  | symbol
    (TAG "expr" 'expr :value $1)
  | NIL
    (TAG "expr" 'expr :value $1)
  ;

;; parameters: '(' [varargslist] ')'
function_parameters
  : LPAREN
    ()
  | RPAREN
    ()
 ;;  | function_parameter COMMA
  | function_parameter RPAREN
  ;

function_parameter
  : WHILE
    (TAG "function_parameter" 'function_parameter :value $1)
  ;

binop
  : PLUS
  | MINUS
  | TIMES
  | DIVIDE
  | GREATER_THAN
  | LESS_THAN
  | GREATER_THAN_OR_EQUAL
  | LESS_THAN_OR_EQUAL
  ;

string_comparison
  : STRING_LITERAL GREATER_THAN STRING_LITERAL
    (TAG "sum" 'expr :value (concat "(string> " $1 " " $3 ")"))
  | STRING_LITERAL LESS_THAN STRING_LITERAL
    (TAG "sum" 'expr :value (concat "(string< " $1 " " $3 ")"))
  | STRING_LITERAL LESS_THAN_OR_EQUAL STRING_LITERAL
    (TAG "sum" 'expr :value (concat "(or (string< " $1 " " $3 ") "
                                        "(string= " $1 " " $3 "))"))
  | STRING_LITERAL GREATER_THAN_OR_EQUAL STRING_LITERAL
    (TAG "sum" 'expr :value (concat "(or (string> " $1 " " $3 ") "
                                        "(string= " $1 " " $3 "))"))
  ;

;; expr_comparison
;;   : expr EQ expr
;;     (TAG "sum" 'expr :value (concat "(equal" $1 " " $3 ")"))
;;   | expr NEQ expr
;;     (TAG "sum" 'expr :value (concat "(not (equal" $1 " " $3 "))"))
;;   ;

%%

;; this turns out not to be needed.
;; (define-lex-regex-type-analyzer simple-jac--<symbol>-regexp-analyzer
;;   "regexp analyzer for <symbol> tokens."
;;   "\\([A-Za-z][_A-Za-z0-9]\\)+"
;;   '((symbol . "[A-Za-z][_A-Za-z0-9]*"))
;;   'symbol)

(define-lex simple-lexer
  "Simple lexical analyzer."
  semantic-lex-ignore-whitespace
  semantic-lex-ignore-newline
  semantic-lex-ignore-comments
  ;;;; Auto-generated analyzers.
  simple-wy--<block>-block-analyzer
  simple-wy--<keyword>-keyword-analyzer
  simple-wy--<punctuation>-string-analyzer
  simple-wy--<number>-regexp-analyzer
  ;; From the documentation string for `semantic-lex-tokens':
  ;; "Always add this analyzer *after* `semantic-lex-number', or other
  ;; analyzers that match its regular expression." (my emphasis)
  simple-wy--<symbol>-regexp-analyzer
  simple-wy--<string>-sexp-analyzer
  ;;;;
  semantic-lex-default-action)

;;; simple.wy ends here


Just in case its relevant, here's the other file too.


;;; simple-mode.el -- major mode for a simple language

;;; Semantic parsing support

(require 'semantic-wisent)
(require 'simple-wy)

;;;###autoload
(defun semantic-default-simple-setup ()
  "Set up a buffer for semantic parsing of a SIMPLE language."
    ;; Do some useful things.
  (setq 
   semantic-ignore-comments t)
  ;; Install the parser
  (simple-wy--install-parser)
  ;; Setup the lexer
  (setq semantic-lex-analyzer 'simple-lexer
        ;; Do a full depth lexical analysis.
        semantic-lex-depth nil))

;;;###autoload
(add-hook 'simple-mode-hook 'semantic-default-simple-setup)

;;; post parser processing

;; I'm not sure why Bovine is not getting rid of comments -- I suppose
;; I haven't dealt with them properly.  I was sort of under the
;; impression that this would be handled automatically after setting
;; the document comment start and end -- but that doesn't seem to be
;; the case.  Maybe I just need to remove comments as part of my own
;; preprocessing phase.  Wouldn't be hard to do.

(defun clean-up-parser-output ()
  (interactive)
  (set-buffer (get-buffer "*Parser Output*"))
  (flush-lines "overlay")
  (flush-lines "expr")
;;  (flush-lines "\"expr\" expr")
  (goto-char (point-min))
  (replace-regexp "  (:value \"\\|\")$" "")
  (goto-char (point-min))
  (replace-regexp "\\\\\"" "\""))

(defun novinate ()
  (interactive)
  (bovinate)
  (clean-up-parser-output))

;;; simple major mode

(defvar simple-mode-syntax-table
  (let ((table (make-syntax-table (standard-syntax-table))))
    (modify-syntax-entry ?\+ "."     table) ;; Operator PLUS
    (modify-syntax-entry ?\- "."     table) ;; Operator MINUS
    ;; also deals with comments
    (modify-syntax-entry ?\* ". 23"  table) ;; Operator MULT
    (modify-syntax-entry ?\/ ". 14"  table) ;; Operator DIV
    ;; deal with brackets
    (modify-syntax-entry ?\( "()"    table)
    (modify-syntax-entry ?\{ "(}"    table)
    (modify-syntax-entry ?\[ "(]"    table)
    (modify-syntax-entry ?\) ")("    table)
    (modify-syntax-entry ?\} "){"    table)
    (modify-syntax-entry ?\] ")["    table)
    table)
  "Syntax table used in simple mode buffers.
Define operators as punctuations.")

;;;###autoload
(define-derived-mode simple-mode fundamental-mode "simple"
  (setq comment-start-skip "/\\*+ *"))

;;;###autoload
(add-to-list 'auto-mode-alist '("\\.simple\\'" . simple-mode))

;(defvar simple-mode-hook nil)

(provide 'simple-mode)

;;; simple-mode.el ends here

>>> Joe Corneli <jcorneli@...> seems to think that:
>Here are a couple more simple questions.  I noticed that when I (1)
>change the grammar file, (2) run `semantic-grammar-create-package'
>(C-c C-c) and load the resulting generated parser support file, (3)
>bovinate the example source file that I'm working with, the changes
>to the grammar are not always reflected in the *Parser Output*
>buffer.  When this happens, (4) I have to kill the source code
>buffer and (5) open it again, then (6) re-bovinate.  Is this the
>expected behavior?  Is there a shortcut I can use to make sure that
>the current parser is being used when I bovinate?

My expectation (meaning, what it did when I last was using and fixing
bugs in it) is the following:

1) I edit the grammar
2) I hit C-c C-c to compile the grammar.
3) The grammar generator looks at the %languagemode declaration,
   finds all buffers matching those major modes and reinitializes
   them with the major-mode function.
4) Wait a moment, and the auto-parse mechanism will refresh the tags
   table on those buffers.

>Second question is that after repeating (1)-(3) and/or (1)-(6), my
>whole system (X window manager and Emacs) begins to handle much more
>sluggishly than usual.  Has anyone seen anything like this happen
>before?  Where might the slow-down be coming from?
  [ ... ]

I have not seen this problem, though I almost always work on a dual
processor system.  (Either two processors, or Emacs on one machine
displaying to a second machine.)

When Emacs becomes unresponsive, I often start turning off all
the different semantic modes until I'm done debugging whatever
feature started breaking things.  I don't recall language
modification doing that to me though.

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org