Re: [Flex-help] How to debug flex matching?

[Georgios P. <pet...@ya...>]

Hi all,

I think that flex, when considering the match length, it also counts the 
look-ahead.
Adding the same look-ahead as in rule 2 to rule 1, seems to fix the problem.

Regards,

George

On 18/7/2016 22:09, Georgios Petasis wrote:
> Hi all,
>
> I am having trouble understanding why gnu flex does not match a
> specific string the way I think it should be matched.
> And I don't know how to debug/trace how the match is done.
>
> My rule set tries to implement a natural language tokenizer.
>
> I have a rule that says:
>
> syz\./[:space:] -> tag token as an abbreviation (lets name this "rule 1")
>
> Further in the rule set, there is a rule that says:
> {consonant}+/\.[:space:]+{capital} -> tag as word & split sentence
>
> where {consonant} & {capital} are my definitions. (lets name this
> "rule 2")
>
> I cannot understand why when matching string "syz. test" rule 1 is
> used (what I want), but when matching "syz. Test" rule 2 is matched
> (what I do not want). Rule 2 is not only after rule 1, but it also
> matches a shorter text (as the period is in the trailing context). Why
> is this happening?
>
> I have tried the option "%option debug", but nothing is written in
> stderr (perhaps because I load the scanner as a dynamic library).
>
> Regards,
>
> George
>