Howdy,
It looks as though you have been busy. ;) Comments below.
>>> David Ponce <david@...> seems to think that:
>Hi Eric,
>
>I submit you the attached patch that solves the problem I had with
>partial re-parse of WY/BY semantic grammars, and improves incremental
>parsing of changes at bounds of existing tokens.
>
>In case you don't remember it, here is a snippet of my first message
>on that subject that explained the semantic grammars re-parse problem:
>
>"While I was hacking WY grammars, I got some problems with the
>incremental parser when adding new tokens between existing ones.
>
>That is when `semantic-edits-change-between-tokens' returns
>something. Here is a summary:
>
>1. The `reparse-symbol' property can't be retrieved. After
> `semantic-edits-change-between-tokens' returned a value the
> variable `tokens' is set to nil. So the following statement that
> retrieve the reparse-symbol always fails:
>
> (setq reparse-symbol (semantic-token-get
> (car tokens) 'reparse-symbol))
>
> As, in that case, "The CAR of cache-list is the token just before
> our change, but wasn't modified.", a solution could be to first try
> to get reparse-symbol from tokens, then from cache-list, like this:
>
> (setq reparse-symbol
> (semantic-token-get (car (or tokens cache-list))
> 'reparse-symbol))
>
> I tried the above change and it seems to work better.
>
>2. The inserted text is parsed using the grammar rule pointed by the
> reparse-symbol found in token just before the new text.
> Sometimes, that rule does not preserve the right semantic of the
> inserted text! Here is an example with WY grammar:
>
> 1. Initial state. The following text result in one 'nonterminal
> token: any-value, that contains five nonterminal children:
> any_symbol, STRING, NUMBER, PREFIX-EXP, PAREN_BLOCK, as 'rule
> tokens.
>
> any_value:
> any_symbol
> | STRING
> | NUMBER
> | PREFIX-EXP
> | PAREN_BLOCK
> ;
>
> 2. Now I insert a new TEST rule between STRING and NUMBER, like
> this (the change is enclosed in [...]):
>
> any_value:
> any_symbol
> | STRING[
> | TEST]
> | NUMBER
> | PREFIX-EXP
> | PAREN_BLOCK
> ;
>
> In that case `semantic-edits-change-between-tokens' returns the
> rules tokens from STRING to PAREN_BLOCK. The reparse-symbol
> `rule' is correctly retrieved from the STRING token (car
> cache-list). The parser successfully re-parse the inserted text
> "\n | TEST" using the `rule' semantic. But returns a false
> result, that is an 'empty rule token for the first `|', followed
> by a 'rule token for TEST :-(
>
> In fact without other context the parser can't determine if a rule
> like the above is actually an empty rule plus a normal rule, or
> just the latter. A short example:
>
> any_value:
> | TEST
>
> any_value:
> STRING
> | TEST
>
> In first case "| TEST" means "empty or TEST", in second one it
> means "or TEST". In other words, the meaning of the new text
> depends on its position inside the nonterminal definition (inside
> the parent 'nonterminal token)!"
>
>The patch contains the following (independent) changes to
>semantic-edit.el and semantic-grammar.el.
>
>In semantic-edit I first made the change from point 1 above. The
>other small modifications improve handling of changes at the beginning
>or end of an existing token.
>
>In semantic-grammar I solved the point 2 above using a
>`semantic-edits-new-change-hooks' local hook that, when a new change
>occurs in the scope of a nonterminal token, extends the change bounds
>to encompass the whole nonterminal token! No more need of a special
>'reparse-safepoint' property, and I can keep the nice
>nonterminal/rule token hierarchy :-)
[ ... ]
This last change you mention is interesting. It is similar to the
old `dirty-token' mechanism semantic used to use. I had switched to
tracking individual changes for two reasons:
1) Make the change-function faster. (Don't do lots of semantic token
stuff)
2) Handle changes between tokens.
A side effect of your change is that if `semantic-highlight-edits-mode'
were on, it would have an interesting visual effect. ;)
What I like about your change, however, is that it puts the knowledge
of the token bounds issue into the .wy or .by mode, AND it is a
functional knowledge, not a state setup. Perhaps when semantic-edit
requests bounds of a token (currently via semantic-token-start/end) it
could instead use an override function
`semantic-token-reparse-start/end'. Modes would then override this to
help the incremental parser make intelligent decisions. I think we
had talked about the relative merits of a reparse parent type state
added to all tokens who cared and were not convinced. I think using
override functions would be spiffy since the question asked by
semantic-edit is exactly what it wants, and the flexibility is there
for any strange state a mode may need.
Do you like that idea?
Thanks!
Eric
--
Eric Ludlam: zappo@..., eric@...
Home: http://www.ludlam.net Siege: http://www.siege-engine.com
Emacs: http://cedet.sourceforge.net GNU: http://www.gnu.org
|