Menu

#11 a c++ quex lexer code

closed
nobody
None
5
2020-11-13
2010-09-24
ollydbg
No

Hi, I just create a qx file to handle Preprocessor code, you can see the attachment file.

The forum topic can be find here:
https://sourceforge.net/projects/quex/forums/forum/574343/topic/3861677

The command line to run this is:

quex -i cpp.qx --engine tiny_lexer --token-policy single --token-memory-management-by-user --foreign-token-id-file tokenizer_id.h --token-prefix QUEX_TKN_ >> result.txt

Discussion

  • Frank-Rene Schäfer

    Hello.

    Now, I would like to apply your patch. Could you tell me real quick
    what files and where you want them to be put?

    Regards

    Frank

     
  • ollydbg

    ollydbg - 2010-10-13

    I just remove the attached file, because they are outdated. more recent files were all in google code links in my previous post.

     
  • Frank-Rene Schäfer

    Thanks, for the cpp.qx file. I downloaded it and started to edit it.
    I did not find the time to actually try it out, but did some modifications
    that might make things a little clearer:

    -- use of the 'GOTO(mode, token(...))' command.
    -- separate out elements that are common in PROGRAM and PREPROCESSOR into
    inheritable only modes (~= java interface).
    -- operators and keywords do not have to copy the lexeme that matches. They always
    look the same. Deleted 'Lexeme' from operator and keyword tokens.

    Some things still seem odd, such as allowing '##' in the PROGRAM mode. You might
    want to have a look at it.

    demo/Cpp/004/cpp.qx

    in the repository revision 2336.

    Best Regards

    Frank

     
  • ollydbg

    ollydbg - 2010-10-15

    thanks frank,

    I have just find a dead bug in the cpp,qx, that you mistakenly add a comma in the definition of a derived mode list...

    This error, the quex will generate some strange error log says that the parent mode was not defined.....

    Oh, my God,

    it takes me One hour to beat this bug. I just strip the cpp.qx file to a very small set, and look at the grammar on the pdf manual.....

    The good news is that finally I found the reason.

     
  • ollydbg

    ollydbg - 2010-10-15

    patch to fix the build error of cqq.qx

     
  • ollydbg

    ollydbg - 2010-10-15

    I just add a patch to fix:
    1, remove comma
    2, fix precedence conflict on P_IDENTIFIER

     
  • Nobody/Anonymous

    Thanks, as I said. I never tried the cpp.qx file--sorry.

    If the error message was not descriptive, please, let me know what it was exactly.
    May be, the error reporting needs to be improved.

     
  • ollydbg

    ollydbg - 2010-10-15

    the error message is like:
    cpp.qx:91:error: Mode 'PROGRAM' inherits mode 'OPERATORS,' which does not exist.
    cpp.qx:91:error: Did you mean 'OPERATORS'?

    I seems quex has detect a comma after "OPERATORS" which I have not noticed.
    Should it be:

    There is a comma after the OPEARATOR, so it is a grammar error.

     
  • Frank-Rene Schäfer

    I like the ',' separators better, actually. In 0.53.1, they will be standard.

    Can we close the issue?

     
  • ollydbg

    ollydbg - 2010-10-16

    Ok, I have tested the latest svn, and works fine.

    So, this issue can be closed. thanks.

     

Anonymous
Anonymous

Add attachments
Cancel