savable state support for multiple re2c blocks

With the -f flag specified, every /*!re2c */ block starts with the resume
switch statement, and a yyNext: label. However, only the last one is
correct; the other ones (as might be expected) only contain
yyFillLabels up to that point. In addition, the multiple yyNext: labels
are inappropriate.

I took a look at the code, but I couldn't see an obvious way to fix it
other than generating the yyNext: label only at the first block, and
the switch statement only at the end, perhaps triggered by
something like:

/*!re2c:emit_switch */

I think that's a bit ugly, so I'm hoping that someone has a better idea.

A couple more comments about the feature:

1) It's not clear to me why -f should suppress the declaration of
yyaccept and yych. I understand that the submitter of the patch had
a different idea of how these should be declared, but that may not
be general.

2) I'd really like to be able to hook into the yyFillLabel mechanism
myself. Not everything falls nicely into re2c tokenising (the case I'm
looking at here involves base-64 decoding.) I'd be quite content with
the following:

/*!fill:3 */ ==>

YYSETSTATE(<something>);
if (YYLIMIT - YYCURSOR < 3) YYFILL(3);
yyFillLabel<something>:

This could be achieved by adding the following snippet to
scanner.re (where the max:re2c patch went, although I think re2c:
match would have been a bit better)

"/*!fill:" [ \t]* [0-9]+ {
int n = atoi(tok + 8);
fill_index(out, n);
tok = pos = cursor;
ignore_eoc = true;
goto echo;
}

and modifying need() in code.cc:

static void need(std::ostream &o, uint n, bool & readCh)
{
fill_index(o, n);
o << "\tyych = *YYCURSOR;\n";
readCh = false;
++oline;
}

with the rest of it going into fill_index:

void fill_index(std::ostream &o, uint n)
{
uint fillIndex;
bool hasFillIndex = (0<=vFillIndexes);
if ( hasFillIndex == true )
{
fillIndex = vFillIndexes++;
o << "\tYYSETSTATE(" << fillIndex << ");\n";
++oline;
}

if (n == 1)
{
o << "\tif(YYLIMIT == YYCURSOR) YYFILL(1);\n";
++oline;
}
else
{
o << "\tif((YYLIMIT - YYCURSOR) < " << n << ") YYFILL(" <<
n << ");\n";
++oline;
}

if ( hasFillIndex == true )
{
o << "yyFillLabel" << fillIndex << ":\n";
++oline;
}
}

I haven't actually tried that yet, and I'm probably missing some
details about #line numbering.

3) I don't understand the point of YYGETSTATE() instead of just
YYGETSTATE. I guess tastes differ. But it could not be implemented
as a function unless you used a global variable, and that seems
unlikely in the context of control inversion.

4) On the other hand, while we're on the subject, it would be really
handy for me to have re2c emit:

++YYCURSOR: SAVECURSOR;
and
RESTORECURSOR;

instead of
YYMARKER = ++YYCURSOR;
and
YYCURSOR = YYMARKER;

Perhaps that's not useful to anyone else, and I know how to do it. In
the particular buffering environment I'm working inside, the saved
cursor state cannot easily be represented as a pointer. I notice that
re2c is capable of simply doing YYCURSOR -= k; for some constant
k; that wouldn't work either but I haven't run into it in the wild yet.
Maybe I've just been lucky.

rici@ricilake.net