Thread: [Flex-help] lex/yacc/gcc help
flex is a tool for generating scanners
Brought to you by:
wlestes
From: Mark H. <ma...@co...> - 2013-11-01 13:36:24
|
I am trying to port some code from an old unix box to Linux. It is a microcode compiler for an old CPU board. This code uses lex/yacc/cc. Obviously all works on this old unix box. I am completely lex/yacc ignorant but am not gcc ignorant. I am hoping someone can give me some advise as to what I need to do. I've subscribed and sent this email to both the flex-help and the help-bison mailing lists because I really have no idea what the problem really is. I hope that is appropiate. The Linux distribution OS is OpenSuSE-12.3 and the software versions in use are: gcc-4.7-7.1.1.i586 glibc-2.17-4.7.1.i586 flex-2.5.37-2.1.1.i586 bison-2.6.5-2.1.1.i586 The actual error I get from gcc when compiling this stuff on Linux is: # make ./translate5 cpu.def cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in yacc -d yacc.in cat lex1.linit lextext lexfield lex2.linit > lex.in flex -l lex.in gcc -ansi -w y.tab.c -o cpu.asm -lgcc yacc.in: In function ëyyparseí: yacc.in:2322:3: error: unknown type name ëBEGINí make: *** [cpu.asm] Error 1 This is the output from the legacy "unix" machine: # make translate5 cpu.def cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in /usr/bin/yacc -d yacc.in cat lex1.linit lextext lexfield lex2.linit > lex.in /usr/bin/lex lex.in 2277/3600 nodes(%e), 10911/17500 positions(%p), 1561/1900 (%n), 89797 transitions , 136/170 packed char classes(%k), 3614/5000 packed transitions(%a), 2433/4000 output slots(%o) /bin/cc -z -B/lib/ -t0 y.tab.c -ly -ll -v -w -o cpu.asm /lib/cpp y.tab.c /tmp/ctm030464 -DON_SEL -Dselport -DGOULD_PN -DCOFF /lib/ccom /tmp/ctm030464 /tmp/ctm030463 -Xll -w /bin/as -o y.tab.o /tmp/ctm030463 /bin/ld -X /lib/crt0.o -e start -o cpu.asm -z y.tab.o -ly -ll -lc Now from my limited mis/understanding the ëBEGINí in the compile failure above seems to be a "lex thing"? Yet it seems to appear in a yacc file? If I look at what ends up being in the y.tab.c file on both machines I see: Legacy unix box: case 207: # line 2305 "yacc.in" { if (yychar != -1) { yyclearin; while (yyleng > 0) { unput(yytext[--yyleng]); }} BEGIN ctxn;} break; } goto yystack; /* stack new state and value */ } The Linux box: case 208: /* Line 1778 of yacc.c */ #line 2318 "yacc.in" { if (yychar != -1) { yyclearin; while (yyleng > 0) { unput(yytext[--yyleng]); }} BEGIN ctxn;} break; Both the above come from the yacc3.yinit file. cnum : /*empty*/ { if (yychar != -1) { yyclearin; while (yyleng > 0) { unput(yytext[--yyleng]); }} BEGIN ctxn;} ; The only references to "ctxn" I find are from a lex1.linit and lex2.linit files. lex1.linit: %C %e 3600 %p 17500 %a 5000 %o 4000 %n 1900 %k 170 int statnum=0; int externum=0; int passc=0; int linec=1; int errc=0; int eolflag = 0; int codeflag = 0; int lque = 0; int plinec = 0; int spacec = 1; int pagesize = 55; int skippage = 0; int skipline = 0; int pagecnt = 1; int type1, type2, type3, type4, type5, type6, type7, type8; int type9, typeA, typeB, typeC, typeD, typeE, typeF; %S ex co pb lc rz bd al eq va code %S rz2 bd2 sp hd hd2 hd3 hd4 dir ctxn . . . . lex2.linit: <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { if (yytext[0] == '#') { yylval.nvaltype.nval=htoi(yytext); if (yyleng > 9) fprintf(stderr, "Literal value '%s' out of range at line %d\n", yytext, linec); } else yylval.nvaltype.nval=atoi(yytext); yylval.nvaltype.ntype=2; BEGIN code; return(NUM); } <code>[ \t] {} ^[ \t]+ {BEGIN code;} <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; return(p_symbol(yytext));} <code>\n {p_eol(); ++linec;} [ ,\t] {} \n {p_eol(); ++linec;BEGIN 0;} . {return(YYERRCODE);} %% #include "process.c" Again, I am completely ignorant of lex and yac. Very strange and cryptic looking code. I would greatly appreciate any pointers from anyone. Thanks in advance Regards Mark |
From: Arthur S. <asc...@at...> - 2013-11-01 14:29:35
|
Do a grep of your source files to locate which file eBEGINi is located in. It does not appear in any of the files included in you e-mail. ----- Original Message ----- From: Mark Hounschell <ma...@co...> To: hel...@gn...; fle...@li... Cc: Sent: Friday, November 1, 2013 6:36 AM Subject: [Flex-help] lex/yacc/gcc help I am trying to port some code from an old unix box to Linux. It is a microcode compiler for an old CPU board. This code uses lex/yacc/cc. Obviously all works on this old unix box. I am completely lex/yacc ignorant but am not gcc ignorant. I am hoping someone can give me some advise as to what I need to do. I've subscribed and sent this email to both the flex-help and the help-bison mailing lists because I really have no idea what the problem really is. I hope that is appropiate. The Linux distribution OS is OpenSuSE-12.3 and the software versions in use are: gcc-4.7-7.1.1.i586 glibc-2.17-4.7.1.i586 flex-2.5.37-2.1.1.i586 bison-2.6.5-2.1.1.i586 The actual error I get from gcc when compiling this stuff on Linux is: # make ./translate5 cpu.def cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in yacc -d yacc.in cat lex1.linit lextext lexfield lex2.linit > lex.in flex -l lex.in gcc -ansi -w y.tab.c -o cpu.asm -lgcc yacc.in: In function ëyyparseí: yacc.in:2322:3: error: unknown type name ëBEGINí make: *** [cpu.asm] Error 1 This is the output from the legacy "unix" machine: # make translate5 cpu.def cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in /usr/bin/yacc -d yacc.in cat lex1.linit lextext lexfield lex2.linit > lex.in /usr/bin/lex lex.in 2277/3600 nodes(%e), 10911/17500 positions(%p), 1561/1900 (%n), 89797 transitions , 136/170 packed char classes(%k), 3614/5000 packed transitions(%a), 2433/4000 output slots(%o) /bin/cc -z -B/lib/ -t0 y.tab.c -ly -ll -v -w -o cpu.asm /lib/cpp y.tab.c /tmp/ctm030464 -DON_SEL -Dselport -DGOULD_PN -DCOFF /lib/ccom /tmp/ctm030464 /tmp/ctm030463 -Xll -w /bin/as -o y.tab.o /tmp/ctm030463 /bin/ld -X /lib/crt0.o -e start -o cpu.asm -z y.tab.o -ly -ll -lc Now from my limited mis/understanding the ëBEGINí in the compile failure above seems to be a "lex thing"? Yet it seems to appear in a yacc file? If I look at what ends up being in the y.tab.c file on both machines I see: Legacy unix box: case 207: # line 2305 "yacc.in" { if (yychar != -1) { yyclearin; while (yyleng > 0) { unput(yytext[--yyleng]); }} BEGIN ctxn;} break; } goto yystack; /* stack new state and value */ } The Linux box: case 208: /* Line 1778 of yacc.c */ #line 2318 "yacc.in" { if (yychar != -1) { yyclearin; while (yyleng > 0) { unput(yytext[--yyleng]); }} BEGIN ctxn;} break; Both the above come from the yacc3.yinit file. cnum : /*empty*/ { if (yychar != -1) { yyclearin; while (yyleng > 0) { unput(yytext[--yyleng]); }} BEGIN ctxn;} ; The only references to "ctxn" I find are from a lex1.linit and lex2.linit files. lex1.linit: %C %e 3600 %p 17500 %a 5000 %o 4000 %n 1900 %k 170 int statnum=0; int externum=0; int passc=0; int linec=1; int errc=0; int eolflag = 0; int codeflag = 0; int lque = 0; int plinec = 0; int spacec = 1; int pagesize = 55; int skippage = 0; int skipline = 0; int pagecnt = 1; int type1, type2, type3, type4, type5, type6, type7, type8; int type9, typeA, typeB, typeC, typeD, typeE, typeF; %S ex co pb lc rz bd al eq va code %S rz2 bd2 sp hd hd2 hd3 hd4 dir ctxn . . . . lex2.linit: <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { if (yytext[0] == '#') { yylval.nvaltype.nval=htoi(yytext); if (yyleng > 9) fprintf(stderr, "Literal value '%s' out of range at line %d\n", yytext, linec); } else yylval.nvaltype.nval=atoi(yytext); yylval.nvaltype.ntype=2; BEGIN code; return(NUM); } <code>[ \t] {} ^[ \t]+ {BEGIN code;} <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; return(p_symbol(yytext));} <code>\n {p_eol(); ++linec;} [ ,\t] {} \n {p_eol(); ++linec;BEGIN 0;} . {return(YYERRCODE);} %% #include "process.c" Again, I am completely ignorant of lex and yac. Very strange and cryptic looking code. I would greatly appreciate any pointers from anyone. Thanks in advance Regards Mark ------------------------------------------------------------------------------ Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk _______________________________________________ Flex-help mailing list Fle...@li... https://lists.sourceforge.net/lists/listinfo/flex-help |
From: John P. H. <jph...@gm...> - 2013-11-01 14:36:05
|
BEGIN is a flex macro to set the scanner start state. Changing the flex state from the parser is frowned upon in some circles, but that is what your yacc does. So you need to imbed the flex generated code before the parser. (Defining the macro won't buy you anything for it sets a static variable in the flexx-generated C code.) You might look into flex' exclusive start states (%x ctxn rather than %s ctxn--lex only has the %s variety); they are likely to simplify your flexer. On 11/01/2013 02:36 PM, Mark Hounschell wrote: > I am trying to port some code from an old unix box to Linux. It is a > microcode compiler for an old CPU board. This code uses lex/yacc/cc. > Obviously all works on this old unix box. I am completely lex/yacc > ignorant but am not gcc ignorant. I am hoping someone can give me some > advise as to what I need to do. > I've subscribed and sent this email to both the flex-help and the > help-bison mailing lists because I really have no idea what the problem > really is. I hope that is appropiate. > > The Linux distribution OS is OpenSuSE-12.3 and the software versions in > use are: > > gcc-4.7-7.1.1.i586 > glibc-2.17-4.7.1.i586 > flex-2.5.37-2.1.1.i586 > bison-2.6.5-2.1.1.i586 > > > The actual error I get from gcc when compiling this stuff on Linux is: > > # make > ./translate5 cpu.def > cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in > yacc -d yacc.in > cat lex1.linit lextext lexfield lex2.linit > lex.in > flex -l lex.in > gcc -ansi -w y.tab.c -o cpu.asm -lgcc > yacc.in: In function ëyyparseí: > yacc.in:2322:3: error: unknown type name ëBEGINí > make: *** [cpu.asm] Error 1 > > This is the output from the legacy "unix" machine: > > # make > translate5 cpu.def > cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in > /usr/bin/yacc -d yacc.in > cat lex1.linit lextext lexfield lex2.linit > lex.in > /usr/bin/lex lex.in > 2277/3600 nodes(%e), 10911/17500 positions(%p), 1561/1900 (%n), 89797 > transitions > , 136/170 packed char classes(%k), 3614/5000 packed transitions(%a), > 2433/4000 output slots(%o) > /bin/cc -z -B/lib/ -t0 y.tab.c -ly -ll -v -w -o cpu.asm > /lib/cpp y.tab.c /tmp/ctm030464 -DON_SEL -Dselport -DGOULD_PN -DCOFF > /lib/ccom /tmp/ctm030464 /tmp/ctm030463 -Xll -w > /bin/as -o y.tab.o /tmp/ctm030463 > /bin/ld -X /lib/crt0.o -e start -o cpu.asm -z y.tab.o -ly -ll -lc > > Now from my limited mis/understanding the ëBEGINí in the compile failure > above seems to be a "lex thing"? Yet it seems to appear in a yacc file? > If I look at what ends up being in the y.tab.c file on both machines I see: > > Legacy unix box: > > case 207: > # line 2305 "yacc.in" > { if (yychar != -1) { > yyclearin; > while (yyleng > 0) { > unput(yytext[--yyleng]); }} > BEGIN ctxn;} break; > } > goto yystack; /* stack new state and value */ > > } > > The Linux box: > > case 208: > /* Line 1778 of yacc.c */ > #line 2318 "yacc.in" > { if (yychar != -1) { > yyclearin; > while (yyleng > 0) { > unput(yytext[--yyleng]); }} > BEGIN ctxn;} > > break; > > Both the above come from the yacc3.yinit file. > > cnum : /*empty*/ > { if (yychar != -1) { > yyclearin; > while (yyleng > 0) { > unput(yytext[--yyleng]); }} > BEGIN ctxn;} > ; > > > The only references to "ctxn" I find are from a lex1.linit and > lex2.linit files. > > lex1.linit: > > %C > > > %e 3600 > > %p 17500 > > %a 5000 > > %o 4000 > %n 1900 > > %k 170 > > > > int statnum=0; > int externum=0; > int passc=0; > int linec=1; > int errc=0; > int eolflag = 0; > int codeflag = 0; > int lque = 0; > int plinec = 0; > int spacec = 1; > int pagesize = 55; > int skippage = 0; > int skipline = 0; > int pagecnt = 1; > > int type1, type2, type3, type4, type5, type6, type7, type8; > > int type9, typeA, typeB, typeC, typeD, typeE, typeF; > > > %S ex co pb lc rz bd al eq va code > > %S rz2 bd2 sp hd hd2 hd3 hd4 dir ctxn > . > . > . > . > > > lex2.linit: > > <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { > if (yytext[0] == '#') { > yylval.nvaltype.nval=htoi(yytext); > if (yyleng > 9) > fprintf(stderr, "Literal value '%s' out of > range at line %d\n", > yytext, linec); > } else > yylval.nvaltype.nval=atoi(yytext); > > yylval.nvaltype.ntype=2; > BEGIN code; > return(NUM); > } > > <code>[ \t] {} > > ^[ \t]+ {BEGIN code;} > > <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; return(p_symbol(yytext));} > > <code>\n {p_eol(); ++linec;} > > [ ,\t] {} > \n {p_eol(); ++linec;BEGIN 0;} > . {return(YYERRCODE);} > > > %% > > #include "process.c" > > > Again, I am completely ignorant of lex and yac. Very strange and cryptic > looking code. I would greatly appreciate any pointers from anyone. > > Thanks in advance > Regards > Mark |
From: Mark H. <ma...@co...> - 2013-11-01 14:51:45
|
I notice you see in my email and responded with "eBEGINi". That must be a result of using colorgcc and the way it outputs gcc errors in color. That output was cut and pasted into the email. It is BEGIN. There is no "eBEGINi" anywhere in the code. And as I'm sure you know, BEGIN appears to be a FLEX keyword described in the FLEX manual Section 10 - Start Conditions. I have 2 Lex input files with MANY of these. And also the single yacc/bison file described below that seems to be what is not compiling on the Linux box, yet does on the legacy box.. On 11/01/2013 10:29 AM, Arthur Schwarz wrote: > Do a grep of your source files to locate which file eBEGINi is located in. It does not appear in any of the > files included in you e-mail. > > > > ----- Original Message ----- > From: Mark Hounschell <ma...@co...> > To: hel...@gn...; fle...@li... > Cc: > Sent: Friday, November 1, 2013 6:36 AM > Subject: [Flex-help] lex/yacc/gcc help > > I am trying to port some code from an old unix box to Linux. It is a > microcode compiler for an old CPU board. This code uses lex/yacc/cc. > Obviously all works on this old unix box. I am completely lex/yacc > ignorant but am not gcc ignorant. I am hoping someone can give me some > advise as to what I need to do. > I've subscribed and sent this email to both the flex-help and the > help-bison mailing lists because I really have no idea what the problem > really is. I hope that is appropiate. > > The Linux distribution OS is OpenSuSE-12.3 and the software versions in > use are: > > gcc-4.7-7.1.1.i586 > glibc-2.17-4.7.1.i586 > flex-2.5.37-2.1.1.i586 > bison-2.6.5-2.1.1.i586 > > > The actual error I get from gcc when compiling this stuff on Linux is: > > # make > ./translate5 cpu.def > cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in > yacc -d yacc.in > cat lex1.linit lextext lexfield lex2.linit > lex.in > flex -l lex.in > gcc -ansi -w y.tab.c -o cpu.asm -lgcc > yacc.in: In function ÎyyparseÌ: > yacc.in:2322:3: error: unknown type name ÎBEGINÌ > make: *** [cpu.asm] Error 1 > > This is the output from the legacy "unix" machine: > > # make > translate5 cpu.def > cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in > /usr/bin/yacc -d yacc.in > cat lex1.linit lextext lexfield lex2.linit > lex.in > /usr/bin/lex lex.in > 2277/3600 nodes(%e), 10911/17500 positions(%p), 1561/1900 (%n), 89797 > transitions > , 136/170 packed char classes(%k), 3614/5000 packed transitions(%a), > 2433/4000 output slots(%o) > /bin/cc -z -B/lib/ -t0 y.tab.c -ly -ll -v -w -o cpu.asm > /lib/cpp y.tab.c /tmp/ctm030464 -DON_SEL -Dselport -DGOULD_PN -DCOFF > /lib/ccom /tmp/ctm030464 /tmp/ctm030463 -Xll -w > /bin/as -o y.tab.o /tmp/ctm030463 > /bin/ld -X /lib/crt0.o -e start -o cpu.asm -z y.tab.o -ly -ll -lc > > Now from my limited mis/understanding the ÎBEGINÌ in the compile failure > above seems to be a "lex thing"? Yet it seems to appear in a yacc file? > If I look at what ends up being in the y.tab.c file on both machines I see: > > Legacy unix box: > > case 207: > # line 2305 "yacc.in" > { if (yychar != -1) { > yyclearin; > while (yyleng > 0) { > unput(yytext[--yyleng]); }} > BEGIN ctxn;} break; > } > goto yystack; /* stack new state and value */ > > } > > The Linux box: > > case 208: > /* Line 1778 of yacc.c */ > #line 2318 "yacc.in" > { if (yychar != -1) { > yyclearin; > while (yyleng > 0) { > unput(yytext[--yyleng]); }} > BEGIN ctxn;} > > break; > > Both the above come from the yacc3.yinit file. > > cnum : /*empty*/ > { if (yychar != -1) { > yyclearin; > while (yyleng > 0) { > unput(yytext[--yyleng]); }} > BEGIN ctxn;} > ; > > > The only references to "ctxn" I find are from a lex1.linit and > lex2.linit files. > > lex1.linit: > > %C > > > %e 3600 > > %p 17500 > > %a 5000 > > %o 4000 > %n 1900 > > %k 170 > > > > int statnum=0; > int externum=0; > int passc=0; > int linec=1; > int errc=0; > int eolflag = 0; > int codeflag = 0; > int lque = 0; > int plinec = 0; > int spacec = 1; > int pagesize = 55; > int skippage = 0; > int skipline = 0; > int pagecnt = 1; > > int type1, type2, type3, type4, type5, type6, type7, type8; > > int type9, typeA, typeB, typeC, typeD, typeE, typeF; > > > %S ex co pb lc rz bd al eq va code > > %S rz2 bd2 sp hd hd2 hd3 hd4 dir ctxn > . > . > . > . > > > lex2.linit: > > <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { > if (yytext[0] == '#') { > yylval.nvaltype.nval=htoi(yytext); > if (yyleng > 9) > fprintf(stderr, "Literal value '%s' out of > range at line %d\n", > yytext, linec); > } else > yylval.nvaltype.nval=atoi(yytext); > > yylval.nvaltype.ntype=2; > BEGIN code; > return(NUM); > } > > <code>[ \t] {} > > ^[ \t]+ {BEGIN code;} > > <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; return(p_symbol(yytext));} > > <code>\n {p_eol(); ++linec;} > > [ ,\t] {} > \n {p_eol(); ++linec;BEGIN 0;} > . {return(YYERRCODE);} > > > %% > > #include "process.c" > > > Again, I am completely ignorant of lex and yac. Very strange and cryptic > looking code. I would greatly appreciate any pointers from anyone. > > Thanks in advance > Regards > Mark > > ------------------------------------------------------------------------------ > Android is increasing in popularity, but the open development platform that > developers love is also attractive to malware creators. Download this white > paper to learn more about secure code signing practices that can help keep > Android apps secure. > http://pubads.g.doubleclick.net/gampad/clk?id=65839951&iu=/4140/ostg.clktrk > _______________________________________________ > Flex-help mailing list > Fle...@li... > https://lists.sourceforge.net/lists/listinfo/flex-help > > > > ------------------------------------------------------------- > This email was processed through Xeams to filter junk messages. > If you feel this message has been tagged incorrectly, you can > change its category by clicking the link below. > Click here http://12.186.155.30:5272/FrontController?operation=mbeu&f=00001_-930_20131101_419812.eml&chkBayesian=1&pr=1&mt=1&ma=s to mark email as junk. > ------------------------------------------------------------- > . > |
From: Arthur S. <asc...@at...> - 2013-11-01 15:30:38
|
<str>\\[0-7]{1,3} <str>\\[0-9]+ The string description in Section 10 will not work correctly. A number, e.g., 123, will be considered an octal number (first rule matches) and never a decimal number. Using the C/C++ definition for an octal number may be better. <str>\\0[0-7]{1,3} <str>\\[1-9][0-9]+ This still has the defect that the rules allow an indefinite sized decimal number. It might be better to limit the number received for decimal numbers or remove the restriction on octal numbers for consistency. <str>\\0[0-7]{1,3} <str>\\[1-9][0-9]{1,2} And you do not include hexadecimal numbers. Putting it all together and normalizing on existing definitions of numbers gets us. <str>\\0[0-7]{1,3} <str>\\[1-9][[:digit:]]{1,2} <str>\\x[[:xdigit:]]{1,2} We are left with the notion that \0 and all that follow are octal numbers. \1 and all that follow are decimal numbers. \x and all that follow are hexadecimal numbers. And \xBEEF leads to a binary \xBE followed by the characters EF. For somewhat different reasons you might want to reconsider your comment example. In this context what does '*', zero or more mean? <comment>[^*\n]* /* eat anything that's not a '*' */ <comment>"*"+[^*/\n]* /* eat up '*'s not followed by '/'s */ <comment>\n ++line_num; <comment>"*"+"/" BEGIN(comment_caller); Another way of addressing comment recognition may be: <comment>"*"+ /* eat up all '*' */ <comment>.*"*/" BEGIN(INITIAL); /* eat up everything */ <comment>[^*\n]+ /* eat up everything except newline and '*' */ <comment>\n ++line_num; /* eat up newline */ Which allows the manual to introduce the notion that the longer match wins ("*"+ loses to .*"*/") and that we really don't care what the comments are (.*"*/") and that if the comment extends to multiple lines ([^*\n]+) why that's ok. The manual example seems to address cases that address the semantics of a comment (where can a '*' appear, what can follow it, etc.) The second example focuses on what ends a comment (*/) and comments which can extend across multiple lines ([*\n]+ and \n). It's all subjective, I admit, and counter-examples don't win the day. But some editing of the examples might be a good idea. art |
From: Mark H. <ma...@co...> - 2013-11-01 19:56:33
|
On 11/01/2013 10:35 AM, John P. Hartmann wrote: > BEGIN is a flex macro to set the scanner start state. Changing the flex > state from the parser is frowned upon in some circles, but that is what > your yacc does. So you need to imbed the flex generated code before the > parser. (Defining the macro won't buy you anything for it sets a static > variable in the flexx-generated C code.) > If I understand you correctly, your saying I need to replace that "BEGIN ctxn;" in that yacc file with the actual code that would be generated if that was in one of the lex files? Somehow I think that's not what you are saying. This is the entire lex file containing reference to ctxn. I'm not sure which of the 2 are the ctxn code. The entire yacc file causing the compile failure is already shown below. <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { if (yytext[0] == '#') { yylval.nvaltype.nval=htoi(yytext); if (yyleng > 9) fprintf(stderr, "Literal value '%s' out of range at line %d\n", yytext, linec); } else yylval.nvaltype.nval=atoi(yytext); yylval.nvaltype.ntype=2; BEGIN code; return(NUM); } <code>[ \t] {} ^[ \t]+ {BEGIN code;} <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; return(p_symbol(yytext));} <code>\n {p_eol(); ++linec;} [ ,\t] {} \n {p_eol(); ++linec;BEGIN 0;} . {return(YYERRCODE);} %% #include "process.c" > You might look into flex' exclusive start states (%x ctxn rather than %s > ctxn--lex only has the %s variety); they are likely to simplify your > flexer. > Lost me there. Remember I am lex/yacc ignorant. Would sending a tarball of the code help? Mark > On 11/01/2013 02:36 PM, Mark Hounschell wrote: >> I am trying to port some code from an old unix box to Linux. It is a >> microcode compiler for an old CPU board. This code uses lex/yacc/cc. >> Obviously all works on this old unix box. I am completely lex/yacc >> ignorant but am not gcc ignorant. I am hoping someone can give me some >> advise as to what I need to do. >> I've subscribed and sent this email to both the flex-help and the >> help-bison mailing lists because I really have no idea what the problem >> really is. I hope that is appropiate. >> >> The Linux distribution OS is OpenSuSE-12.3 and the software versions in >> use are: >> >> gcc-4.7-7.1.1.i586 >> glibc-2.17-4.7.1.i586 >> flex-2.5.37-2.1.1.i586 >> bison-2.6.5-2.1.1.i586 >> >> >> The actual error I get from gcc when compiling this stuff on Linux is: >> >> # make >> ./translate5 cpu.def >> cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in >> yacc -d yacc.in >> cat lex1.linit lextext lexfield lex2.linit > lex.in >> flex -l lex.in >> gcc -ansi -w y.tab.c -o cpu.asm -lgcc >> yacc.in: In function ÎyyparseÌ: >> yacc.in:2322:3: error: unknown type name ÎBEGINÌ >> make: *** [cpu.asm] Error 1 >> >> This is the output from the legacy "unix" machine: >> >> # make >> translate5 cpu.def >> cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in >> /usr/bin/yacc -d yacc.in >> cat lex1.linit lextext lexfield lex2.linit > lex.in >> /usr/bin/lex lex.in >> 2277/3600 nodes(%e), 10911/17500 positions(%p), 1561/1900 (%n), 89797 >> transitions >> , 136/170 packed char classes(%k), 3614/5000 packed transitions(%a), >> 2433/4000 output slots(%o) >> /bin/cc -z -B/lib/ -t0 y.tab.c -ly -ll -v -w -o cpu.asm >> /lib/cpp y.tab.c /tmp/ctm030464 -DON_SEL -Dselport -DGOULD_PN -DCOFF >> /lib/ccom /tmp/ctm030464 /tmp/ctm030463 -Xll -w >> /bin/as -o y.tab.o /tmp/ctm030463 >> /bin/ld -X /lib/crt0.o -e start -o cpu.asm -z y.tab.o -ly -ll -lc >> >> Now from my limited mis/understanding the ÎBEGINÌ in the compile failure >> above seems to be a "lex thing"? Yet it seems to appear in a yacc file? >> If I look at what ends up being in the y.tab.c file on both machines I >> see: >> >> Legacy unix box: >> >> case 207: >> # line 2305 "yacc.in" >> { if (yychar != -1) { >> yyclearin; >> while (yyleng > 0) { >> unput(yytext[--yyleng]); }} >> BEGIN ctxn;} break; >> } >> goto yystack; /* stack new state and value */ >> >> } >> >> The Linux box: >> >> case 208: >> /* Line 1778 of yacc.c */ >> #line 2318 "yacc.in" >> { if (yychar != -1) { >> yyclearin; >> while (yyleng > 0) { >> unput(yytext[--yyleng]); }} >> BEGIN ctxn;} >> >> break; >> >> Both the above come from the yacc3.yinit file. >> >> cnum : /*empty*/ >> { if (yychar != -1) { >> yyclearin; >> while (yyleng > 0) { >> unput(yytext[--yyleng]); }} >> BEGIN ctxn;} >> ; >> >> >> The only references to "ctxn" I find are from a lex1.linit and >> lex2.linit files. >> >> lex1.linit: >> >> %C >> >> >> %e 3600 >> >> %p 17500 >> >> %a 5000 >> >> %o 4000 >> %n 1900 >> >> %k 170 >> >> >> >> int statnum=0; >> int externum=0; >> int passc=0; >> int linec=1; >> int errc=0; >> int eolflag = 0; >> int codeflag = 0; >> int lque = 0; >> int plinec = 0; >> int spacec = 1; >> int pagesize = 55; >> int skippage = 0; >> int skipline = 0; >> int pagecnt = 1; >> >> int type1, type2, type3, type4, type5, type6, type7, type8; >> >> int type9, typeA, typeB, typeC, typeD, typeE, typeF; >> >> >> %S ex co pb lc rz bd al eq va code >> >> %S rz2 bd2 sp hd hd2 hd3 hd4 dir ctxn >> . >> . >> . >> . >> >> >> lex2.linit: >> >> <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { >> if (yytext[0] == '#') { >> yylval.nvaltype.nval=htoi(yytext); >> if (yyleng > 9) >> fprintf(stderr, "Literal value '%s' out of >> range at line %d\n", >> yytext, linec); >> } else >> yylval.nvaltype.nval=atoi(yytext); >> >> yylval.nvaltype.ntype=2; >> BEGIN code; >> return(NUM); >> } >> >> <code>[ \t] {} >> >> ^[ \t]+ {BEGIN code;} >> >> <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; >> return(p_symbol(yytext));} >> >> <code>\n {p_eol(); ++linec;} >> >> [ ,\t] {} >> \n {p_eol(); ++linec;BEGIN 0;} >> . {return(YYERRCODE);} >> >> >> %% >> >> #include "process.c" >> >> >> Again, I am completely ignorant of lex and yac. Very strange and cryptic >> looking code. I would greatly appreciate any pointers from anyone. >> >> Thanks in advance >> Regards >> Mark > > > > ------------------------------------------------------------- > This email was processed through Xeams to filter junk messages. > If you feel this message has been tagged incorrectly, you can > change its category by clicking the link below. Click here > http://12.186.155.30:5272/FrontController?operation=mbeu&f=00001_-970_20131101_419884.eml&chkBayesian=1&pr=1&mt=1&ma=s > to mark email as junk. > ------------------------------------------------------------- > . > |
From: Arthur S. <asc...@at...> - 2013-11-01 23:12:46
|
Section 18 "The first way is to simply compile a scanner generated by flex using a C++ compiler instead of a C compiler. You should not encounter any compilation errors (see Reporting Bugs)." Compilation errors occur because flex input must be changed to flex yyinput. Change the manual, the C code must be altered to use yyinput() instead of input(). The flex statement which causes this condition is given below. #ifdef __cplusplus static int yyinput (void) #else static int input (void) #endif |
From: John P. H. <jph...@gm...> - 2013-11-02 08:22:37
|
I'm saying that the flex variable set by the BEGIN macro is static. That is, it has file scope. Thus, to reference this variable in a bison rule, you must include the flex output file in the first code block of the grammar file (x.y). You'll also have to save the token list to a file so that it can be included in the first code block of the scanner file (x.l). On 11/01/2013 08:56 PM, Mark Hounschell wrote: > On 11/01/2013 10:35 AM, John P. Hartmann wrote: >> BEGIN is a flex macro to set the scanner start state. Changing the flex >> state from the parser is frowned upon in some circles, but that is what >> your yacc does. So you need to imbed the flex generated code before the >> parser. (Defining the macro won't buy you anything for it sets a static >> variable in the flexx-generated C code.) >> > > If I understand you correctly, your saying I need to replace that > "BEGIN ctxn;" in that yacc file with the actual code that would be > generated if that was in one of the lex files? Somehow I think that's > not what you are saying. > > This is the entire lex file containing reference to ctxn. I'm not sure > which of the 2 are the ctxn code. The entire yacc file causing the > compile failure is already shown below. > > <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { > if (yytext[0] == '#') { > yylval.nvaltype.nval=htoi(yytext); > if (yyleng > 9) > fprintf(stderr, "Literal value '%s' out of > range at line %d\n", > yytext, linec); > } else > yylval.nvaltype.nval=atoi(yytext); > > yylval.nvaltype.ntype=2; > BEGIN code; > return(NUM); > } > > <code>[ \t] {} > > ^[ \t]+ {BEGIN code;} > > <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; > return(p_symbol(yytext));} > > <code>\n {p_eol(); ++linec;} > > [ ,\t] {} > \n {p_eol(); ++linec;BEGIN 0;} > . {return(YYERRCODE);} > > > %% > > #include "process.c" > > >> You might look into flex' exclusive start states (%x ctxn rather than %s >> ctxn--lex only has the %s variety); they are likely to simplify your >> flexer. >> > > Lost me there. Remember I am lex/yacc ignorant. Would sending a > tarball of the code help? > > Mark > >> On 11/01/2013 02:36 PM, Mark Hounschell wrote: >>> I am trying to port some code from an old unix box to Linux. It is a >>> microcode compiler for an old CPU board. This code uses lex/yacc/cc. >>> Obviously all works on this old unix box. I am completely lex/yacc >>> ignorant but am not gcc ignorant. I am hoping someone can give me some >>> advise as to what I need to do. >>> I've subscribed and sent this email to both the flex-help and the >>> help-bison mailing lists because I really have no idea what the problem >>> really is. I hope that is appropiate. >>> >>> The Linux distribution OS is OpenSuSE-12.3 and the software versions in >>> use are: >>> >>> gcc-4.7-7.1.1.i586 >>> glibc-2.17-4.7.1.i586 >>> flex-2.5.37-2.1.1.i586 >>> bison-2.6.5-2.1.1.i586 >>> >>> >>> The actual error I get from gcc when compiling this stuff on Linux is: >>> >>> # make >>> ./translate5 cpu.def >>> cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in >>> yacc -d yacc.in >>> cat lex1.linit lextext lexfield lex2.linit > lex.in >>> flex -l lex.in >>> gcc -ansi -w y.tab.c -o cpu.asm -lgcc >>> yacc.in: In function ÎyyparseÌ: >>> yacc.in:2322:3: error: unknown type name ÎBEGINÌ >>> make: *** [cpu.asm] Error 1 >>> >>> This is the output from the legacy "unix" machine: >>> >>> # make >>> translate5 cpu.def >>> cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in >>> /usr/bin/yacc -d yacc.in >>> cat lex1.linit lextext lexfield lex2.linit > lex.in >>> /usr/bin/lex lex.in >>> 2277/3600 nodes(%e), 10911/17500 positions(%p), 1561/1900 (%n), 89797 >>> transitions >>> , 136/170 packed char classes(%k), 3614/5000 packed transitions(%a), >>> 2433/4000 output slots(%o) >>> /bin/cc -z -B/lib/ -t0 y.tab.c -ly -ll -v -w -o cpu.asm >>> /lib/cpp y.tab.c /tmp/ctm030464 -DON_SEL -Dselport -DGOULD_PN -DCOFF >>> /lib/ccom /tmp/ctm030464 /tmp/ctm030463 -Xll -w >>> /bin/as -o y.tab.o /tmp/ctm030463 >>> /bin/ld -X /lib/crt0.o -e start -o cpu.asm -z y.tab.o -ly -ll -lc >>> >>> Now from my limited mis/understanding the ÎBEGINÌ in the compile >>> failure >>> above seems to be a "lex thing"? Yet it seems to appear in a yacc file? >>> If I look at what ends up being in the y.tab.c file on both machines I >>> see: >>> >>> Legacy unix box: >>> >>> case 207: >>> # line 2305 "yacc.in" >>> { if (yychar != -1) { >>> yyclearin; >>> while (yyleng > 0) { >>> unput(yytext[--yyleng]); }} >>> BEGIN ctxn;} break; >>> } >>> goto yystack; /* stack new state and value */ >>> >>> } >>> >>> The Linux box: >>> >>> case 208: >>> /* Line 1778 of yacc.c */ >>> #line 2318 "yacc.in" >>> { if (yychar != -1) { >>> yyclearin; >>> while (yyleng > 0) { >>> unput(yytext[--yyleng]); }} >>> BEGIN ctxn;} >>> >>> break; >>> >>> Both the above come from the yacc3.yinit file. >>> >>> cnum : /*empty*/ >>> { if (yychar != -1) { >>> yyclearin; >>> while (yyleng > 0) { >>> unput(yytext[--yyleng]); }} >>> BEGIN ctxn;} >>> ; >>> >>> >>> The only references to "ctxn" I find are from a lex1.linit and >>> lex2.linit files. >>> >>> lex1.linit: >>> >>> %C >>> >>> >>> %e 3600 >>> >>> %p 17500 >>> >>> %a 5000 >>> >>> %o 4000 >>> %n 1900 >>> >>> %k 170 >>> >>> >>> >>> int statnum=0; >>> int externum=0; >>> int passc=0; >>> int linec=1; >>> int errc=0; >>> int eolflag = 0; >>> int codeflag = 0; >>> int lque = 0; >>> int plinec = 0; >>> int spacec = 1; >>> int pagesize = 55; >>> int skippage = 0; >>> int skipline = 0; >>> int pagecnt = 1; >>> >>> int type1, type2, type3, type4, type5, type6, type7, type8; >>> >>> int type9, typeA, typeB, typeC, typeD, typeE, typeF; >>> >>> >>> %S ex co pb lc rz bd al eq va code >>> >>> %S rz2 bd2 sp hd hd2 hd3 hd4 dir ctxn >>> . >>> . >>> . >>> . >>> >>> >>> lex2.linit: >>> >>> <code,ctxn>("#"[0-9a-fA-F]+|[0-9]+) { >>> if (yytext[0] == '#') { >>> yylval.nvaltype.nval=htoi(yytext); >>> if (yyleng > 9) >>> fprintf(stderr, "Literal value '%s' out of >>> range at line %d\n", >>> yytext, linec); >>> } else >>> yylval.nvaltype.nval=atoi(yytext); >>> >>> yylval.nvaltype.ntype=2; >>> BEGIN code; >>> return(NUM); >>> } >>> >>> <code>[ \t] {} >>> >>> ^[ \t]+ {BEGIN code;} >>> >>> <ctxn,code>[A-Za-z][\$\.A-Za-z0-9]* {BEGIN code; >>> return(p_symbol(yytext));} >>> >>> <code>\n {p_eol(); ++linec;} >>> >>> [ ,\t] {} >>> \n {p_eol(); ++linec;BEGIN 0;} >>> . {return(YYERRCODE);} >>> >>> >>> %% >>> >>> #include "process.c" >>> >>> >>> Again, I am completely ignorant of lex and yac. Very strange and >>> cryptic >>> looking code. I would greatly appreciate any pointers from anyone. >>> >>> Thanks in advance >>> Regards >>> Mark >> >> >> >> ------------------------------------------------------------- >> This email was processed through Xeams to filter junk messages. >> If you feel this message has been tagged incorrectly, you can >> change its category by clicking the link below. Click here >> http://12.186.155.30:5272/FrontController?operation=mbeu&f=00001_-970_20131101_419884.eml&chkBayesian=1&pr=1&mt=1&ma=s >> >> to mark email as junk. >> ------------------------------------------------------------- >> . >> |
From: Mark H. <ma...@co...> - 2013-11-05 13:11:33
|
On 11/05/2013 04:43 AM, Akim Demaille wrote: > > Le 1 nov. 2013 ? 14:36, Mark Hounschell <ma...@co...> a Ècrit : > >> # make >> ./translate5 cpu.def >> cat yacc1.yinit yacctoks yacc2.yinit yaccrules yacc3.yinit > yacc.in >> yacc -d yacc.in >> cat lex1.linit lextext lexfield lex2.linit > lex.in >> flex -l lex.in >> gcc -ansi -w y.tab.c -o cpu.asm -lgcc >> yacc.in: In function ÎyyparseÌ: >> yacc.in:2322:3: error: unknown type name ÎBEGINÌ > > BEGIN is something expected in a lex file, not in a Yacc > file. This is fishy. > >> case 207: >> # line 2305 "yacc.in" >> { if (yychar != -1) { >> yyclearin; >> while (yyleng > 0) { >> unput(yytext[--yyleng]); }} >> BEGIN ctxn;} break; > > It looks like your Yacc parser is trying to influence your > lex scanner. In a properly designed scanner/parser couple, > you should never see things such as "BEGIN" or "unput" in > the parser: that's the scanner's job only. > > Either be brave and change all this, or maybe there is a way > out, I don't know, by trying to #include the whole scanner in > the yacc file. In either case, expect pain :( > > Thanks Akim, Like I said in my original post, I am yacc/lex ignorant. The strange thing, that I haven't mentioned is that this all built successfully 4, 5, or more years ago on Linux. I was recently able to find that executable that I had checked into CVS those years back and it works. I also found some notes of what I originally had to do to get it to build. All I had to do back then was: 1. Changed all ocurrances of yylval.ntype to yylval.nvaltype.ntype 2. Changed all occurances of yylval.nval to yylval.nvaltype.nval 3. Changed all occurances of yybgin to yy_start 4. Changed all occurances of "k =/ 10" to "k /= 10" As far as including the whole scanner in the yacc file, the y.tab.c file does include the lex file but at the very end of it. After this BEGIN shows up. End of y.tab.c: /* Line 2041 of yacc.c */ #line 2355 "yacc.in" #include "mic.c" #include "fields.c" #include "lex.yy.c" I assume lex.yy.c is the whole scanner? And I also assume you mean trying to include it before the BEGIN? Regards Mark |
From: Mark H. <ma...@co...> - 2013-11-06 12:57:48
|
On 11/06/2013 03:19 AM, Akim Demaille wrote: > > Le 5 nov. 2013 ? 14:11, Mark Hounschell <ma...@co...> a Ècrit : > >> Thanks Akim, > > Hi! > >> Like I said in my original post, I am yacc/lex ignorant. The strange thing, that I haven't mentioned is that this all built successfully 4, 5, or more years ago on Linux. > > I would not have expected that, given what you sent. > >> I was recently able to find that executable that I had checked into CVS those years back and it works. I also found some notes of what I originally had to do to get it to build. All I had to do back then was: >> >> 1. Changed all ocurrances of yylval.ntype >> to yylval.nvaltype.ntype >> 2. Changed all occurances of yylval.nval >> to yylval.nvaltype.nval >> >> 3. Changed all occurances of yybgin to yy_start >> >> 4. Changed all occurrences of "k =/ 10" to "k /= 10" > > Wow, that piece of software _is_ old! Could you tell us what > it is? > It's a micro code assembler for a legacy CPU boards micro engine. It is an early 80s generation CPU board. From back in the days when powerfull CPUs were not yet on a chip. I still repair these boards and often a micro diagnostic has to be modified to create tight scope loops. The peice of software I'm trying to build creates the assembler used to compile the micros and firmware for this board. There was a time this peice of software was "Company private". That was long ago though. I would have no problem providing the code if it would help me get it to build again? >> As far as including the whole scanner in the yacc file, the y.tab.c file does include the lex file but at the very end of it. After this BEGIN shows up. > > BEGIN is a macro, so it must be #defined before it is > used. > >> I assume lex.yy.c is the whole scanner? > > Yes, that's what I meant. > >> And I also assume you mean trying to include it before the BEGIN? > > To include it before the body of the parser. You'll have to find > the right place, as the scanner certainly also wants to "see" things > from the parser, such as YYSTYPE I guess. > Since I don't remember what version of SuSE Linux I used when it actually did built right, I might just get out all my old dist CDs and see if I can pin down the versions that did work. Regards Mark |
From: Mark H. <ma...@co...> - 2013-11-08 18:57:16
|
On 11/06/2013 07:57 AM, Mark Hounschell wrote: > On 11/06/2013 03:19 AM, Akim Demaille wrote: >> >> Le 5 nov. 2013 ? 14:11, Mark Hounschell <ma...@co...> a »crit : >> >>> Thanks Akim, >> >> Hi! >> >>> Like I said in my original post, I am yacc/lex ignorant. The strange thing, that I haven't mentioned is that this all built successfully 4, 5, or more years ago on Linux. >> >> I would not have expected that, given what you sent. >> >>> I was recently able to find that executable that I had checked into CVS those years back and it works. I also found some notes of what I originally had to do to get it to build. All I had to do back then was: >>> >>> 1. Changed all ocurrances of yylval.ntype >>> to yylval.nvaltype.ntype >>> 2. Changed all occurances of yylval.nval >>> to yylval.nvaltype.nval >>> >>> 3. Changed all occurances of yybgin to yy_start >>> >>> 4. Changed all occurrences of "k =/ 10" to "k /= 10" >> >> Wow, that piece of software _is_ old! Could you tell us what >> it is? >> > > It's a micro code assembler for a legacy CPU boards micro engine. It is > an early 80s generation CPU board. From back in the days when powerfull > CPUs were not yet on a chip. I still repair these boards and often a > micro diagnostic has to be modified to create tight scope loops. The > peice of software I'm trying to build creates the assembler used to > compile the micros and firmware for this board. There was a time this > peice of software was "Company private". That was long ago though. I > would have no problem providing the code if it would help me get it to > build again? > >>> As far as including the whole scanner in the yacc file, the y.tab.c file does include the lex file but at the very end of it. After this BEGIN shows up. >> >> BEGIN is a macro, so it must be #defined before it is >> used. >> >>> I assume lex.yy.c is the whole scanner? >> >> Yes, that's what I meant. >> >>> And I also assume you mean trying to include it before the BEGIN? >> >> To include it before the body of the parser. You'll have to find >> the right place, as the scanner certainly also wants to "see" things >> from the parser, such as YYSTYPE I guess. >> > > Since I don't remember what version of SuSE Linux I used when it > actually did built right, I might just get out all my old dist CDs and > see if I can pin down the versions that did work. > > Regards > Mark I have determined it was SuSE-8.2 that the working executable was built on. Verified by installing a SuSE-8.2 system and it builds just fine. SuSE-8.2 has the following revisions of software #rpm -q gcc glibc yacc flex bison gcc-3.3-23 glibc-2.3.2-6 yacc-91.7.30-663 flex-2.5.4a-97 bison-1.75-39 What I've found is lex/flex is NOT the problem at all. The problem is bison -y is NOT doing the same thing yacc does. On the 8.2 system the makefile specifies "YACC = yacc". If I change it to "YACC = bison -y", I get the same failure I do on the current system. But, I found a package called byacc. Using this instead of "bison -y" or "yacc" has fixed my problem. Back on SuSE-8.2 yacc was it's own package. I guess when yacc and bison were combined someone had the insight to still provide the original yacc package as byacc. I'm happy... Mark |