> I've got all of the `bovine' directory converted over to the new
> names, and will check them in eventually.
> Anyway, starting with token constructors, I'd like a low-key movement
> to be afoot. Any NEW function that claims to act on a tag object
> created from semantic parsing should use the word `tag'. The old
> words, including `token' and `nonterminal' should be reserved for
> lex, and parsers respectively.
I checked the following changes in:
(wisent-cooked-token): Use `semantic-tag' API.
(wisent-raw-token): Renamed to...
(wisent-cook-token): Renamed to...
(wisent-cook-tag): New. Use `semantic-tag' API.
(wisent-parse-region): Use `semantic-tag' API.
(wisent-grammar-EXPANDTAG): Use `semantic-tag' API and new
(wisent-java-expand-nonterminal): Use `semantic-token-extra-specs'
in place of `semantic-token-variable-extra-specs'.
> It might also be a good time to think hard about the token reference
> functions, such as `semantic-token-extra-specs'. I think it's
> important to distinguish between language features as saved in the
> extra specifiers, and properties, stuck in the plist, but at the same
> time, they are very nearly the same. In a future token format, they
> may not be considered `extra' at all. I'll leave those -token- fcns
> till after we better understand what format tokens may take in the
I completely agree with that. In fact it appears that a tag would
have two parts:
- A public part, for "language" features.
- A private part, for internal use, where would be stored things like
buffer, overlay, persistence, etc.
IMO a clean tag API should show that. For example, we could reserve
`semantic-tag-' for public things, like `semantic-tag-name',
`semantic-tag-class' (instead of `semantic-token-token' ;-), etc.
For internal things we could use `semantic--tag-'.
We could use a plist based implementation because it keeps things very
simple (a few functions can do all what is necessary), and would
probably be efficient (we can use C built-in primitives).
It would be even possible to provide an object oriented layer on top
of that primitive API, to manipulate tags with a higher level of
Here is, a few primitive functions I think of:
The tag name.
A tag class, like type, variable, function, etc.
Add or change an additional language attribute, like parents,
throws, type, etc.
Obtain value of a language attribute.
Add or change an internal property (like bounds, overlay,
Obtain the value of an internal property.
The internal structure of a tag could reflect the public/private
(NAME CLASS ATTRIBUTES PROPERTIES)
Where ATTRIBUTES and PROPERTIES would just be property lists.