Re: [Flex-help] Including Newline and Carriage Return in yytext

[Allen B. <Al...@ep...>]

Here is an example parser:


%option never-interactive
%option reentrant

%{
char my_buf[1024];
%}

%%

MY[ ]STRING=.*\r?\n {
	int i;
	printf("We got the string <%s>\n",&(yytext[10]));
	printf("Total match length is <%d> long\n",yyleng);
	printf("The characters in yytext are:\n");
	for(i = 0; i < strlen(yytext); ++i)
	{
		printf("\t<%x>\n",(unsigned int)yytext[i]);
	}
	memcpy(my_buf,yytext,yyleng); 
}

. {
	// Do nothing, just consume the text
}

%%

int main(int argc, char ** argv)
{
	yyscan_t scanner;
	yylex_init( &scanner );
	yyset_in(stdin, scanner);
    yylex(scanner);
	yylex_destroy(scanner);
    return 0;
}


I then take this file and use it against the following text in a file:

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin ornare orci id 
neque imperdiet tristique. Nulla faucibus, lacus ut aliquet mollis, mi tortor 
fermentum tellus, suscipit MY STRING=THE QUICK BROWN FOX
bibendum lorem enim vitae massa. Nulla ultrices mi 
vel lorem tempor tincidunt. Sed urna tortor, commodo at maximus eget, 
pretium ac dolor. Nam auctor


## END OF FILE

It is important to note that the line endings for the above text should be (carriage return)(line feed).

What I would like to see in yytext is "MY STRING=THE QUICK BROWN FOX(\0x0D\0x0A)" but what I get is: "MY STRING=THE QUICK BROWN FOX(\0x0A)" so there is no carriage return.
It is important for my parser to be able to parse certain strings byte for byte as they appear in the input file.


Allen Blaylock

-----Original Message-----
From: Harjot kaur [mailto:har...@gm...] 
Sent: Thursday, December 11, 2014 10:29 PM
To: Allen Blaylock
Subject: Re: [Flex-help] Including Newline and Carriage Return in yytext

On Fri, Dec 12, 2014 at 2:51 AM, Allen Blaylock <Al...@ep...> wrote:
> I am struggling to get the newline and carriage return characters to appear in the yytext "buffer."
>
I think it shows your token is not declared in yytext or yylval.
> For instance let's say I was trying to match with the rule:
>
> MY[ ]STRING=.*\r?\n {
>               memcpy(my_buf,yytext,yyleng); }
Are u using this function in Flex file or Bison file?
>
> Which you can see copies the buffer exactly to my_buf. Unfortunatly I only get the portion of the string matched by MY[ ]STRING=.* but not the optional carriage return or newline character.

Is it solved out or not? Please show your code.




-- 
Harjot Kaur
Blog: harjotpandher93.wordpress.com
github: https://github.com/jotpandher
" ਕੰਮ ਨਾਲ ਪਿਆਰ ਕਰੀਏ ਫਿਰ ਕੰਮ ਹੀ ਸ਼ੋਂਕ ਬਣ ਜਾਏਗਾ "