Reading bison's y.output to build the per-state token table looks like it
should be unnecessary - yypact/yycheck/yytable are sitting right there in
y.tab.c, and bison uses exactly that scan in yysyntax_error() to list expected
tokens. I tried it, and it does not work.
Those tables only carry a state's EXPLICIT actions. A token that is acceptable
through a default reduction has no entry at all. State 29 is the clearest case:
1025 module_globals_section: actual_globals_section . [$end, KW_CSTART, ...
MAIN, FORMHANDLER]
GLOBALS shift, and go to state 12
$default reduce using rule 1025
Only GLOBALS has a yypact entry, but MAIN, FUNCTION, DEFINE and the rest are
perfectly legal there - the parser reduces rule 1025 first and shifts them in
the next state. A yypact based implementation was compiled alongside the
existing one and compared over all 4732 x 2324 (state, token) pairs: they
disagree on 146338 of them, and nearly all of those are the yypact version
being too restrictive in exactly this way. Using it would silently turn
keywords into identifiers.
Answering the question from the tables alone would mean following default
reductions transitively, and performing a reduction needs the parser's state
STACK to pop and compute the goto - yyss is local to yyparse() and the lexer
only ever sees the top state. bison's LALR lookahead sets already encode the
answer, and y.output is the only place bison publishes them.
So the y.output dependency is not a shortcut, it is the only route to the
information. Comment added at the top of mk_states_c.in so the next person
does not spend the afternoon rediscovering it, including the note that the
-r itemsets,lookaheads flags are load bearing: without the lookahead sets the
generated table is wrong rather than merely incomplete.
No functional change.
