CEDET

Eric,

 > Someone emailed me a question today about Common Lisp.  He had
 > identified that the Emacs Lisp parser could parse Common Lisp quite
 > well, and wanted to set this up.
 >
 > In semantic 1.4 all he needed to do was set his lisp-mode-hook to
 > call semantic-default-elisp-setup.  In semantic 2.0 this function is
 > empty!  It is empty because everything is handled through our fancy
 > new mode local mechanisms.
 >
 > So how could his situation be handled?  I put in an experiment for a
 > semantic-major-mode, or a specifier that says "This buffer belongs to
 > this parser" type relation.  The below patch appears to work though I
 > don't have any specific tests for it at this time.
 >
 > Thoughts?

This is an interesting situation!  However, I am not sure it is worth
(I don't see another case a parser can be reused like this for a
different language) introducing a new notion of `semantic-major-mode'
that could be confusing compared to the true `major-mode'.

IMO, what we could do is to move mode local stuff from
`emacs-lisp-mode' to `lisp-mode', which will be consistent with the
fact that `emacs-lisp-mode' is derived from `lisp-mode'.  Probably
that could be a general design to always define mode local stuff at
the higher consistent mode possible (another example is `jde-mode',
derived from `java-mode', which uses java mode local definitions).

David

>>> David Ponce <david.ponce@...> seems to think that:
>Eric,
>
> > Someone emailed me a question today about Common Lisp.  He had
> > identified that the Emacs Lisp parser could parse Common Lisp quite
> > well, and wanted to set this up.
> >
> > In semantic 1.4 all he needed to do was set his lisp-mode-hook to
> > call semantic-default-elisp-setup.  In semantic 2.0 this function is
> > empty!  It is empty because everything is handled through our fancy
> > new mode local mechanisms.
> >
> > So how could his situation be handled?  I put in an experiment for a
> > semantic-major-mode, or a specifier that says "This buffer belongs to
> > this parser" type relation.  The below patch appears to work though I
> > don't have any specific tests for it at this time.
> >
> > Thoughts?
>
>This is an interesting situation!  However, I am not sure it is worth
>(I don't see another case a parser can be reused like this for a
>different language) introducing a new notion of `semantic-major-mode'
>that could be confusing compared to the true `major-mode'.

This is an existing problem with C & C++.  I just never adapted C to
the new mode-local variable type ideas because there was no way to
handle it.

>IMO, what we could do is to move mode local stuff from
>`emacs-lisp-mode' to `lisp-mode', which will be consistent with the
>fact that `emacs-lisp-mode' is derived from `lisp-mode'.  Probably
>that could be a general design to always define mode local stuff at
>the higher consistent mode possible (another example is `jde-mode',
>derived from `java-mode', which uses java mode local definitions).
  [ ... ]

This is a good idea in general.  Perhaps an alternate solution would
be to make lisp-mode be the primary mode for the Lisp parser, and
then we could do this:

(put 'emacs-lisp-mode 'semantic-parent-mode 'lisp-mode)

or something equivalent.  Anything that indicates that one mode can
use mode variables and defuns from some other mode.

Perhaps the logic would look like this instead:

(if (variable-binding-for-local-mode-p)
   (get binding for major-mode)
  (if (variable-binding-for-parent-mode-p)
      (get binding for parent mode)
    (get default binding)))

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Hi Eric,

[...]
 >>IMO, what we could do is to move mode local stuff from
 >>`emacs-lisp-mode' to `lisp-mode', which will be consistent with the
 >>fact that `emacs-lisp-mode' is derived from `lisp-mode'.  Probably
 >>that could be a general design to always define mode local stuff at
 >>the higher consistent mode possible (another example is `jde-mode',
 >>derived from `java-mode', which uses java mode local definitions).
 >
 >   [ ... ]
 >
 > This is a good idea in general.  Perhaps an alternate solution would
 > be to make lisp-mode be the primary mode for the Lisp parser, and
 > then we could do this:
 >
 > (put 'emacs-lisp-mode 'semantic-parent-mode 'lisp-mode)
 >
 > or something equivalent.  Anything that indicates that one mode can
 > use mode variables and defuns from some other mode.
 >
 > Perhaps the logic would look like this instead:
 >
 > (if (variable-binding-for-local-mode-p)
 >    (get binding for major-mode)
 >   (if (variable-binding-for-parent-mode-p)
 >       (get binding for parent mode)
 >     (get default binding)))

That is exactly what does the current code in
`semantic-current-bindings', looking at parent mode using the
`derived-mode-parent' property instead of a specific
`semantic-parent-mode' one.

I wonder if finally it wouldn't be better to revert to 1.4 design and
use a mode hook to setup buffers for semantic.  It is simple,
flexible, and, after all, it is what Emacs users are used to ;-)

David

>>> David Ponce <david.ponce@...> seems to think that:
>Hi Eric,
>
>[...]
> >>IMO, what we could do is to move mode local stuff from
> >>`emacs-lisp-mode' to `lisp-mode', which will be consistent with the
> >>fact that `emacs-lisp-mode' is derived from `lisp-mode'.  Probably
> >>that could be a general design to always define mode local stuff at
> >>the higher consistent mode possible (another example is `jde-mode',
> >>derived from `java-mode', which uses java mode local definitions).
> >
> >   [ ... ]
> >
> > This is a good idea in general.  Perhaps an alternate solution would
> > be to make lisp-mode be the primary mode for the Lisp parser, and
> > then we could do this:
> >
> > (put 'emacs-lisp-mode 'semantic-parent-mode 'lisp-mode)
> >
> > or something equivalent.  Anything that indicates that one mode can
> > use mode variables and defuns from some other mode.
> >
> > Perhaps the logic would look like this instead:
> >
> > (if (variable-binding-for-local-mode-p)
> >    (get binding for major-mode)
> >   (if (variable-binding-for-parent-mode-p)
> >       (get binding for parent mode)
> >     (get default binding)))
>
>That is exactly what does the current code in
>`semantic-current-bindings', looking at parent mode using the
>`derived-mode-parent' property instead of a specific
>`semantic-parent-mode' one.

Unfortunately, Emacs Lisp is not derived from Lisp, or at least that
variable doesn't seem to be set.

>I wonder if finally it wouldn't be better to revert to 1.4 design and
>use a mode hook to setup buffers for semantic.  It is simple,
>flexible, and, after all, it is what Emacs users are used to ;-)
  [ ... ]

I always envisioned hooks as means for users to customize some
application.  We are trying to develop a system for developers to
write parser support for languages which makes maintenance easier.
With our system, we write a function once, and label it as belonging
to some mode.  In 1.4, we wrote a function, gave it a special name,
and added it to the hook in one of sections installing special
override functions.  For it to take effect, you then need to
re-evaluate the hook in all modes.

I find maintaining a parser to be quite challenging so anything we can
do to make my life easier in this respect is worth the effort. :)

Perhaps we should submit patches to Emacs and XEmacs to stack the
modes via the derived mode mechanism in a way more to our liking. :)

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Hi Eric,

[...]
 >
 > Unfortunately, Emacs Lisp is not derived from Lisp, or at least that
 > variable doesn't seem to be set.

Oops! you're right!  So it seems your proposal to explicitly build a
mode hierarchy used by semantic make sense ;-)

[...]
 >
 > I always envisioned hooks as means for users to customize some
 > application.  We are trying to develop a system for developers to
 > write parser support for languages which makes maintenance easier.
 > With our system, we write a function once, and label it as belonging
 > to some mode.  In 1.4, we wrote a function, gave it a special name,
 > and added it to the hook in one of sections installing special
 > override functions.  For it to take effect, you then need to
 > re-evaluate the hook in all modes.
 >
 > I find maintaining a parser to be quite challenging so anything we can
 > do to make my life easier in this respect is worth the effort. :)

All that make a lot of sense, and I must confess that I finally agree
with you that mode hooks are better targeted for users to customize
major mode behavior.

 > Perhaps we should submit patches to Emacs and XEmacs to stack the
 > modes via the derived mode mechanism in a way more to our liking. :)

I am not sure it is a good thing for semantic to depend on such Emacs
features.  Also, it seems that RMS is not favorable to make use of
`define-...-mode' macros mandatory (see recent thread on emacs-devel
about `define-derived-mode' and a proposed `define-major-mode' at
<http://mail.gnu.org/archive/html/emacs-devel/2003-07/msg00293.html>)!

Finally, following your initial idea, here is a possible
implementation (untested) of "derived mode like" for semantic, that
should be compatible with current implementation:

(defsubst semantic-set-parent-mode (mode parent)
   "Set parent of major mode MODE to PARENT mode.
To work properly, this function should be called after PARENT mode
local variables have been defined."
   (put mode 'semantic-mode-parent parent)
   ;; Refresh mode bindings to get mode local variables inherited from
   ;; PARENT.
   (semantic-map-mode-buffers mode 'semantic-activate-mode-bindings))

(defsubst semantic-get-parent-mode (mode)
   "Return the mode parent of the major mode MODE.
  Return nil if MODE has no parent."
   (or (get mode 'semantic-mode-parent)
       (get mode 'derived-mode-parent)))

(defmacro define-semantic-inherited-mode (mode parent &optional docstring)
   "Make major mode MODE inherits semantic stuff from PARENT mode.
DOCSTRING is optional and not used.
To work properly, this should be put after PARENT mode local variables
definition."
   `(semantic-set-parent-mode ',mode ',parent))

(defsubst semantic-current-bindings (&optional mode)
   "Return the current semantic symbol table.
That is `semantic-symbol-table' if locally set, or the symbol table of
current major mode or its parents.
If optional argument MODE is specified return the symbol table of that
mode or its parents."
   (let (table)
     (or mode (setq table semantic-symbol-table
                    mode  major-mode))
     (while (and mode (not table))
       (or (setq table (get mode 'semantic-symbol-table))
           (setq mode  (semantic-get-parent-mode mode))))
     table))

Using the above stuff, it should suffice to add the following in
semantic-el.el, after the definitions of `emacs-lisp-mode' mode local
variables, to get them defined in `lisp-mode' too ;-)

...

(define-semantic-inherited-mode lisp-mode emacs-lisp-mode
   "Make `lisp-mode' inherits semantic stuff from `emacs-lisp-mode'.")

What do you think?

David

>>> David Ponce <david.ponce@...> seems to think that:
  [ ... ]
> > I find maintaining a parser to be quite challenging so anything we can
> > do to make my life easier in this respect is worth the effort. :)
>
>All that make a lot of sense, and I must confess that I finally agree
>with you that mode hooks are better targeted for users to customize
>major mode behavior.
>
> > Perhaps we should submit patches to Emacs and XEmacs to stack the
> > modes via the derived mode mechanism in a way more to our liking. :)
>
>I am not sure it is a good thing for semantic to depend on such Emacs
>features.  Also, it seems that RMS is not favorable to make use of
>`define-...-mode' macros mandatory (see recent thread on emacs-devel
>about `define-derived-mode' and a proposed `define-major-mode' at
><http://mail.gnu.org/archive/html/emacs-devel/2003-07/msg00293.html>)!
>
>Finally, following your initial idea, here is a possible
>implementation (untested) of "derived mode like" for semantic, that
>should be compatible with current implementation:
>
>(defsubst semantic-set-parent-mode (mode parent)
>   "Set parent of major mode MODE to PARENT mode.
  [ ... ]
>(defsubst semantic-current-bindings (&optional mode)
>   "Return the current semantic symbol table.
>That is `semantic-symbol-table' if locally set, or the symbol table of
>current major mode or its parents.
>If optional argument MODE is specified return the symbol table of that
>mode or its parents."
>   (let (table)
>     (or mode (setq table semantic-symbol-table
>                    mode  major-mode))
>     (while (and mode (not table))
>       (or (setq table (get mode 'semantic-symbol-table))
>           (setq mode  (semantic-get-parent-mode mode))))
>     table))
>
>Using the above stuff, it should suffice to add the following in
>semantic-el.el, after the definitions of `emacs-lisp-mode' mode local
>variables, to get them defined in `lisp-mode' too ;-)
>
>...
>
>(define-semantic-inherited-mode lisp-mode emacs-lisp-mode
>   "Make `lisp-mode' inherits semantic stuff from `emacs-lisp-mode'.")
  [ ... ]

That's a good idea.

Currently a language file (at least, the .by file) cane set the
mode to multiple modes, such as:

%languagemode c-mode c++-mode

which implies equivalence, not inheritance.  In the language file
itself, perhaps we can do this:

%languagemode c++-mode
%derivedmodes c-mode

which better implies a parent/child relationship.  The language
author then knows to set all overrides explicitly to c++-mode, and
provides the option of specifying something with more detail to
c-mode.

If the language author does not realize there is a derived mode, a
hook plus your proposed `semantic-set-parent-mode' function will do
the trick.

Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

Eric,

[...]
 >>Using the above stuff, it should suffice to add the following in
 >>semantic-el.el, after the definitions of `emacs-lisp-mode' mode local
 >>variables, to get them defined in `lisp-mode' too ;-)
 >>
 >>...
 >>
 >>(define-semantic-inherited-mode lisp-mode emacs-lisp-mode
 >>  "Make `lisp-mode' inherits semantic stuff from `emacs-lisp-mode'.")
 >
 >   [ ... ]
 >
 > That's a good idea.
 >
 > Currently a language file (at least, the .by file) cane set the
 > mode to multiple modes, such as:
 >
 > %languagemode c-mode c++-mode
 >
 > which implies equivalence, not inheritance.  In the language file
 > itself, perhaps we can do this:
 >
 > %languagemode c++-mode
 > %derivedmodes c-mode
 >
 > which better implies a parent/child relationship.  The language
 > author then knows to set all overrides explicitly to c++-mode, and
 > provides the option of specifying something with more detail to
 > c-mode.
 >
 > If the language author does not realize there is a derived mode, a
 > hook plus your proposed `semantic-set-parent-mode' function will do
 > the trick.

I like that idea to separate the root %languagemode from its
%derivedmodes.  What do you thing of renaming %derivedmodes to
%inheritedmodes (or another different name) to avoid confusion with
the true Emacs derived modes?

Also, when we will implement compilation of grammar into ".[bw]yc"
files, it will be easy to automatically generate the corresponding
`define-semantic-inherited-mode' definitions from %languagemode and
%derivedmodes (or %inheritedmodes) statements ;-)

David

Hi Eric,

Following our discussion about mode local variables, here is a patch
to semantic-fw.el and bovine/semantic-el.el that implements
inheritance of semantic behavior between modes.

I finally adopted the name `define-semantic-child-mode' in place of
`define-semantic-inherited-mode', because I found that name simple,
clear and shorter ;-)

I tested it with some Common Lisp files, and it seems to work as
expected :-) However there is a limitation due to the use of the Elisp
reader to parse Lisp syntax.  It fails to read forms that contain Lisp
characters like #\, with an (invalid-read-syntax "#") error.  Perhaps
have you an idea on how to workaround that?

What do you think?

David

Index: semantic/semantic-fw.el
===================================================================
RCS file: /cvsroot/cedet/cedet/semantic/semantic-fw.el,v
retrieving revision 1.20
diff -c -r1.20 semantic-fw.el
*** semantic/semantic-fw.el	4 Apr 2003 14:37:18 -0000	1.20
--- semantic/semantic-fw.el	22 Jul 2003 09:19:48 -0000
***************
*** 157,162 ****
--- 157,185 ----

   ;;; Core Semantic bindings API
   ;;
+ (defsubst semantic-set-parent-mode (mode parent)
+   "Set parent of major mode MODE to PARENT mode.
+ To work properly, this function should be called after PARENT mode
+ local variables have been defined."
+   (put mode 'semantic-mode-parent parent)
+   ;; Refresh mode bindings to get mode local variables inherited from
+   ;; PARENT. To work properly, the following should be called after
+   ;; PARENT mode local variables have been defined.
+   (semantic-map-mode-buffers mode 'semantic-activate-mode-bindings))
+
+ (defsubst semantic-get-parent-mode (mode)
+   "Return the mode parent of the major mode MODE.
+  Return nil if MODE has no parent."
+   (or (get mode 'semantic-mode-parent)
+       (get mode 'derived-mode-parent)))
+
+ (defmacro define-semantic-child-mode (mode parent &optional docstring)
+   "Make major mode MODE inherits semantic behavior from PARENT mode.
+ DOCSTRING is optional and not used.
+ To work properly, this should be put after PARENT mode local variables
+ definition."
+   `(semantic-set-parent-mode ',mode ',parent))
+
   (defvar semantic-symbol-table nil
     "Buffer local semantic obarray.
   These symbols provide a hook for a `major-mode' to specify specific
***************
*** 174,180 ****
                      mode  major-mode))
       (while (and mode (not table))
         (or (setq table (get mode 'semantic-symbol-table))
!           (setq mode  (get mode 'derived-mode-parent))))
       table))

   (defun semantic-new-bindings (&optional table)
--- 197,203 ----
                      mode  major-mode))
       (while (and mode (not table))
         (or (setq table (get mode 'semantic-symbol-table))
!           (setq mode  (semantic-get-parent-mode mode))))
       table))

   (defun semantic-new-bindings (&optional table)
***************
*** 517,522 ****
--- 540,546 ----
                     "define-lex-regex-analyzer"
                     "define-lex-simple-regex-analyzer"
                     "define-mode-overload-implementation"
+                   "define-semantic-child-mode"
                     "define-overload"
                     "define-wisent-lexer"
   		  "semantic-alias-obsolete"
Index: semantic/bovine/semantic-el.el
===================================================================
RCS file: /cvsroot/cedet/cedet/semantic/bovine/semantic-el.el,v
retrieving revision 1.12
diff -c -r1.12 semantic-el.el
*** semantic/bovine/semantic-el.el	6 Apr 2003 01:06:50 -0000	1.12
--- semantic/bovine/semantic-el.el	22 Jul 2003 09:19:49 -0000
***************
*** 443,448 ****
--- 443,451 ----
   (defvar-mode-local emacs-lisp-mode imenu-create-index-function
     'semantic-create-imenu-index)

+ (define-semantic-child-mode lisp-mode emacs-lisp-mode
+   "Make `lisp-mode' inherits semantic behavior from `emacs-lisp-mode'.")
+
   ;;;###autoload
   (defun semantic-default-elisp-setup ()
     "Setup hook function for Emacs Lisp files and Semantic."
***************
*** 450,455 ****
--- 453,460 ----

   ;;;###autoload
   (add-hook 'emacs-lisp-mode-hook 'semantic-default-elisp-setup)
+ ;;;###autoload
+ (add-hook 'lisp-mode-hook 'semantic-default-elisp-setup)

   ;;;###autoload
   (eval-after-load "semanticdb"

>>> David Ponce <david.ponce@...> seems to think that:
>Hi Eric,
>
>Following our discussion about mode local variables, here is a patch
>to semantic-fw.el and bovine/semantic-el.el that implements
>inheritance of semantic behavior between modes.
>
>I finally adopted the name `define-semantic-child-mode' in place of
>`define-semantic-inherited-mode', because I found that name simple,
>clear and shorter ;-)
>
>I tested it with some Common Lisp files, and it seems to work as
>expected :-) However there is a limitation due to the use of the Elisp
>reader to parse Lisp syntax.  It fails to read forms that contain Lisp
>characters like #\, with an (invalid-read-syntax "#") error.  Perhaps
>have you an idea on how to workaround that?
  [ ... ]

This looks great.  It will let me simplify the old C parser code to be
more like that of Emacs Lisp.  Nifty.

For the invalid read syntax, we can just wrap the read call in a
condition-case, and not produce symbols for it.  At least them most
of it will work.

Thanks
Eric

-- 
          Eric Ludlam:                 zappo@..., eric@...
   Home: http://www.ludlam.net            Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net               GNU: http://www.gnu.org

[...]
 > This looks great.  It will let me simplify the old C parser code to be
 > more like that of Emacs Lisp.  Nifty.

I checked the change in, so you can play with it.

 > For the invalid read syntax, we can just wrap the read call in a
 > condition-case, and not produce symbols for it.  At least them most
 > of it will work.

The bovine parser `semantic-bovinate-stream' is already wrapped in a
condition-case, and skips forms throwing an invalid read syntax error.
So there is nothing more to do for now ;-)

David