[re2c-general] [ re2c-Feature Requests-1187127 ] savable state fails on multiple re2c blocks

[SourceForge.net <no...@so...>]

Feature Requests item #1187127, was opened at 2005-04-21 05:42
Message generated for change (Settings changed) made by helly
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=616203&aid=1187127&group_id=96864

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: None
Group: None
Status: Open
Priority: 5
Submitted By: Rici Lake (ricilake)
>Assigned to: Marcus Börger (helly)
Summary: savable state fails on multiple re2c blocks

Initial Comment:
With the -f flag specified, every /*!re2c */ block starts with the resume 
switch statement, and a yyNext: label. However, only the last one is 
correct; the other ones (as might be expected) only contain 
yyFillLabels up to that point. In addition, the multiple yyNext: labels 
are inappropriate.

I took a look at the code, but I couldn't see an obvious way to fix it 
other than generating the yyNext: label only at the first block, and 
the switch statement only at the end, perhaps triggered by 
something like:

  /*!re2c:emit_switch */

I think that's a bit ugly, so I'm hoping that someone has a better idea.

A couple more comments about the feature:

1) It's not clear to me why -f should suppress the declaration of 
yyaccept and yych. I understand that the submitter of the patch had 
a different idea of how these should be declared, but that may not 
be general.

2) I'd really like to be able to hook into the  yyFillLabel mechanism 
myself. Not everything falls nicely into re2c tokenising (the case I'm 
looking at here involves base-64 decoding.) I'd be quite content with 
the following:

/*!fill:3 */ ==>

  YYSETSTATE(<something>);
  if (YYLIMIT - YYCURSOR < 3) YYFILL(3);
yyFillLabel<something>:

This could be achieved by adding the following snippet to 
scanner.re (where the max:re2c patch went, although I think re2c:
match would have been a bit better)

        "/*!fill:" [ \t]* [0-9]+ {
                                        int n = atoi(tok + 8);
                                        fill_index(out, n);
                                        tok = pos = cursor;
                                        ignore_eoc = true;
                                        goto echo;
                                }

and modifying need() in code.cc:

static void need(std::ostream &o, uint n, bool & readCh)
{
        fill_index(o, n);
        o << "\tyych = *YYCURSOR;\n";
        readCh = false;
        ++oline;
}

with the rest of it going into fill_index:

void fill_index(std::ostream &o, uint n)
{
        uint fillIndex;
        bool hasFillIndex = (0<=vFillIndexes);
        if ( hasFillIndex == true )
        {
                fillIndex = vFillIndexes++;
                o << "\tYYSETSTATE(" << fillIndex << ");\n";
                ++oline;
        }

        if (n == 1)
        {
                o << "\tif(YYLIMIT == YYCURSOR) YYFILL(1);\n";
                ++oline;
        }
        else
        {
                o << "\tif((YYLIMIT - YYCURSOR) < " << n << ") YYFILL(" << 
n << ");\n";
                ++oline;
        }

        if ( hasFillIndex == true )
        {
                o << "yyFillLabel" << fillIndex << ":\n";
                ++oline;
        }
}

I haven't actually tried that yet, and I'm probably missing some 
details about #line numbering.

3) I don't understand the point of YYGETSTATE() instead of just 
YYGETSTATE. I guess tastes differ. But it could not be implemented 
as a function unless you used a global variable, and that seems 
unlikely in the context of control inversion.

4) On the other hand, while we're on the subject, it would be really 
handy for me to have re2c emit:

    ++YYCURSOR: SAVECURSOR;
and
    RESTORECURSOR;

instead of
    YYMARKER = ++YYCURSOR;
and
    YYCURSOR = YYMARKER;

Perhaps that's not useful to anyone else, and I know how to do it. In 
the particular buffering environment I'm working inside, the saved 
cursor state cannot easily be represented as a pointer. I notice that 
re2c is capable of simply doing YYCURSOR -= k; for some constant 
k; that wouldn't work either but I haven't run into it in the wild yet. 
Maybe I've just been lucky.

ri...@ri...

----------------------------------------------------------------------

Comment By: Marcus Börger (helly)
Date: 2005-04-23 08:49

Message:
Logged In: YES 
user_id=271023

At the moment (after 0.9.7.rc2) we issue an error in those 
cases.

----------------------------------------------------------------------

Comment By: Marcus Börger (helly)
Date: 2005-04-22 22:50

Message:
Logged In: YES 
user_id=271023

The easiest solutionis to issue an error message in case 
someone uses this with more than one re2c block.

I think we should do that for 0.9.7 and release it. A complex 
solution like described above can be done in a later version.

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=616203&aid=1187127&group_id=96864