#1532 Can we develop syntax highlighting tools using Flex or Yacc tools?

[Notepad-- / subtwo]

I am currently using Scintilla as the core editor and need to develop syntax highlighting for specific niche languages. However, the built-in syntax writing mechanism of Scintilla is too cumbersome and requires manual work, which can easily lead to errors and low writing efficiency. Can you provide automated tools based on Flex or Yacc to do this. For example, if you create a framework that can translate syntax highlighting and folding functions into corresponding C code, how can it improve efficiency?

我目前使用scintilla做核心编辑器，需要开发一些特定小众语言的语法高亮，但是目前scintilla自带的语法编写机制太麻烦，需要手动去做这些事情，很容易出错和编写效率低下。能否提供基于flex或yacc等自动化的工具，来做这个事情。比如你做个框架，可以翻译语法高亮和折叠的功能，转换为对应的c代码，如此提高效率？

[Zufu Liu]

This probably what you are looking for https://scintilla.sourceforge.io/LPegLexer.html

[Zufu Liu]

I think this is not implementable or worth to implement:

1. C++ lexer does not support regex.
2. Scintilla lex code piece by piece instead in one go (e.g. when you edit a line, only the line and text after the line will be lexed), so lexer needs to backtrack or save/restore states when across lines.
3. Code folding code can't be inferred from Flex or Yacc file.

[Neil Hodgson]

Lexilla defines a protocol to deliver lexing to Scintilla from a shared library as well as an implementation of many lexers. Anyone can develop a lexing solution that uses Flex or Yacc or ANTLR or a PEG library that implements the Lexilla protocol so it can be used within Scintilla just as LPegLexer does.

Projects have to define their scope and lexing is outside the scope of Scintilla now that Lexilla was separated into its own project.