https://sourceforge.net/p/re-flex/wiki/Home/Recent changes to HomeenWed, 28 Feb 2024 13:46:08 -0000https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v58 +++ v59 @@ -4,9 +4,7 @@ [[members limit=20]] -A high-performance C++ regex library and lexical analyzer generator with Unicode support. - -RE/flex is compatible with Flex lexer specifications and Bison parsers. RE/flex is faster than Flex and other regex matchers. Compatible with Flex specifications. Generates reusable source code that is easy to understand. Easily integrates with Bison and other parsers. Includes a fast stand-alone regex engine and library. +A high-performance C++ regex library and lexical analyzer generator with Unicode support. Extends Flex++ with Unicode, indent/dedent anchors, lazy quantifiers, functions for lex and syntax error reporting and more. The RE/flex lexical analyzer generator accepts Flex lexer specifications and seamlessly integrates with Bison parsers. See [Constructing Lexical Analyzers with RE/flex - a Modern Alternative to Flex for C++][reflex-url] and the [RE/flex User Guide][manual-url] for more details. </pre> </div>Robert van EngelenWed, 28 Feb 2024 13:46:08 -0000https://sourceforge.net4f3ad6985a378557b5998634c8b78a0260c862a4https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v57 +++ v58 @@ -4,34 +4,11 @@ [[members limit=20]] -The regex-centric, fast lexical analyzer generator for C++ with full Unicode support. Faster than Flex and other regex matchers. Compatible with Flex specifications. Generates reusable source code that is easy to understand. Easily integrates with Bison and other parsers. Includes a fast stand-alone regex engine and library. +A high-performance C++ regex library and lexical analyzer generator with Unicode support. + +RE/flex is compatible with Flex lexer specifications and Bison parsers. RE/flex is faster than Flex and other regex matchers. Compatible with Flex specifications. Generates reusable source code that is easy to understand. Easily integrates with Bison and other parsers. Includes a fast stand-alone regex engine and library. See [Constructing Lexical Analyzers with RE/flex - a Modern Alternative to Flex for C++][reflex-url] and the [RE/flex User Guide][manual-url] for more details. - -RE/flex is faster than Flex and much faster than regex libraries such as Boost.Regex, C++11 std::regex, PCRE2 and RE2. For example, tokenizing a representative C source code file into 244 tokens takes: - -| Command / Function | Software | Time (μs) -| ------------------------------------------------------- | ----------------------------------- | --------- -| reflex −−fast −−noindent | RE/flex 2.0.0 | 8 -| reflex −−fast | RE/flex 2.0.0 | 9 -| flex −+ −−full | Flex 2.5.35 | 17 -| reflex −−full | RE/flex 2.0.0 | 18 -| boost::spirit::lex::lexertl::actor_lexer::iterator_type | Boost.Spirit.Lex 1.66.0 | 40 -| pcre2_jit_match() | PCRE2 (jit) 10.32 | 60 -| hs_compile_multi(), hs_scan() | Hyperscan 5.1.0 | 209 -| reflex −m=boost-perl | Boost.Regex 1.66.0 | 230 -| pcre2_match() | PCRE2 10.32 | 318 -| RE2::Consume() | RE2 (pre-compiled) 2018-04-01 | 417 -| reflex −m=boost | Boost.Regex POSIX 1.66.0 | 450 -| RE2::Consume() | RE2 POSIX (pre-compiled) 2018-04-01 | 1226 -| flex −+ | Flex 2.5.35 | 3968 -| pcre2_dfa_match() | PCRE2 POSIX (dfa) 10.32 | 4094 -| regcomp(), regexec() | GNU C POSIX.2 regex | 5800 -| std::regex::cregex_iterator() | C++11 std::regex | 5979 - -Note: *Best times of 30 tests with average time in microseconds over 100 runs (using Mac OS X 10.12.6 with clang 9.0.0 -O2, 2.9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3). Hyperscan disqualifies as a potential scanner due to its "All matches reported" semantics resulting in 1915 matches for his test, and due to its event handler requirements.* [Download the tests](https://www.genivia.com/files/perfcomp.zip) *Timings on other platforms may differ, though in the worst cases tested, reflex ran equally fast or slightly faster than the best times of Flex.* - -The RE/flex matcher tracks line numbers, column numbers, and indentations, whereas Flex does not (option noyylineno) and neither do the other regex matchers in the table, except Boost.Regex with reflex. Tracking this information incurs some overhead. Features -------- </pre> </div>Robert van EngelenSun, 05 Nov 2023 19:13:33 -0000https://sourceforge.net968ae5dc3e487f4e8bee9b6bfce22caf2abf7631https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v56 +++ v57 @@ -1,6 +1,6 @@ [![logo][logo-url]][reflex-url] -[![build status][travis-image]][travis-url][![Language grade: C/C++][lgtm-image]][lgtm-url][![license][bsd-3-image]][bsd-3-url] +[![build status][travis-image]][travis-url][![license][bsd-3-image]][bsd-3-url] [[members limit=20]] @@ -273,7 +273,5 @@ [travis-url]: https://travis-ci.com/Genivia/RE-flex [sourceforge-image]: https://img.shields.io/sourceforge/dw/re-flex.svg [sourceforge-url]: https://sourceforge.net/projects/re-flex/files/latest/download -[lgtm-image]: https://img.shields.io/lgtm/grade/cpp/g/Genivia/RE-flex.svg?logo=lgtm&amp;logoWidth=18 -[lgtm-url]: https://lgtm.com/projects/g/Genivia/RE-flex/context:cpp [bsd-3-image]: https://img.shields.io/badge/License-BSD%203--Clause-blue.svg [bsd-3-url]: https://opensource.org/licenses/BSD-3-Clause </pre> </div>Robert van EngelenSat, 07 Jan 2023 01:49:12 -0000https://sourceforge.net3e9d55e8c037e176f1635d484477a0572303f726https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v55 +++ v56 @@ -269,8 +269,8 @@ [dot-url]: http://www.graphviz.org [FSM-url]: https://www.genivia.com/images/reflex-FSM.png [boost-url]: http://www.boost.org -[travis-image]: https://travis-ci.org/Genivia/RE-flex.svg?branch=master -[travis-url]: https://travis-ci.org/Genivia/RE-flex +[travis-image]: https://travis-ci.com/Genivia/RE-flex.svg?branch=master +[travis-url]: https://travis-ci.com/Genivia/RE-flex [sourceforge-image]: https://img.shields.io/sourceforge/dw/re-flex.svg [sourceforge-url]: https://sourceforge.net/projects/re-flex/files/latest/download [lgtm-image]: https://img.shields.io/lgtm/grade/cpp/g/Genivia/RE-flex.svg?logo=lgtm&amp;logoWidth=18 </pre> </div>Robert van EngelenThu, 29 Apr 2021 12:55:04 -0000https://sourceforge.net38b8baf62ee70b2c31bea80863af5070ab842bc5https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v54 +++ v55 @@ -43,7 +43,7 @@ - Auto-detects UTF-8/16/32 input to match Unicode patterns. - Supports file encodings ISO-8859-1 through ISO-8859-15, CP 1250 through 1258, CP 437, CP 850, CP 858, KOI8, MACROMAN, EBCDIC, and custom code pages. - Generates scanners for lexical analysis on files, C++ streams, (wide) strings, and memory such as mmap files. -- Includes many examples, such as a tokenizer for C/C++ source code, a tokenizer for Python source code, a tokenizer for Java source code, and more. +- IIncludes many examples, such as a mini C compiler to Java bytecode, a tokenizer for C/C++ source code, a tokenizer for Python source code, a tokenizer for Java source code, and more. - Extensive documentation in the online [User Guide][manual-url]. - Indent/nodent/dedent anchors to match text with indentation, including custom `\t` (tab) widths. - Lazy quantifiers, no hacks are needed to work around greedy repetitions. </pre> </div>Robert van EngelenMon, 29 Jun 2020 19:28:37 -0000https://sourceforge.nete72e38a35696908e96eeb0f91fb98f032eae2f8ahttps://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v53 +++ v54 @@ -4,7 +4,7 @@ [[members limit=20]] -The regex-centric, fast lexical analyzer generator for C++ with full Unicode support. Faster than Flex and other regex matchers. Compatible with Flex specifications. Generates reusable source code that is easy to understand. Can be used with Bison and other parsers. Includes a fast stand-alone regex engine and library. Adds Boost 💪 +The regex-centric, fast lexical analyzer generator for C++ with full Unicode support. Faster than Flex and other regex matchers. Compatible with Flex specifications. Generates reusable source code that is easy to understand. Easily integrates with Bison and other parsers. Includes a fast stand-alone regex engine and library. See [Constructing Lexical Analyzers with RE/flex - a Modern Alternative to Flex for C++][reflex-url] and the [RE/flex User Guide][manual-url] for more details. @@ -12,10 +12,10 @@ | Command / Function | Software | Time (μs) | ------------------------------------------------------- | ----------------------------------- | --------- -| reflex −−fast −−noindent | RE/flex 1.5.6 | 8 -| reflex −−fast | RE/flex 1.5.6 | 9 +| reflex −−fast −−noindent | RE/flex 2.0.0 | 8 +| reflex −−fast | RE/flex 2.0.0 | 9 | flex −+ −−full | Flex 2.5.35 | 17 -| reflex −−full | RE/flex 1.5.6 | 18 +| reflex −−full | RE/flex 2.0.0 | 18 | boost::spirit::lex::lexertl::actor_lexer::iterator_type | Boost.Spirit.Lex 1.66.0 | 40 | pcre2_jit_match() | PCRE2 (jit) 10.32 | 60 | hs_compile_multi(), hs_scan() | Hyperscan 5.1.0 | 209 @@ -36,30 +36,31 @@ Features -------- -- Fully compatible with Flex to eliminate a learning curve, making a transition to RE/flex frustration-free. -- Extensive documentation in the online [manual][manual-url]. -- Generates reusable source code that is readable by humans. -- Works with Bison/Yacc and supports reentrant, bison-bridge and bison-locations. -- Generates scanners for lexical analysis on ASCII, UTF-8/16/32, EBCDIC, ISO-8859-1 files, C++ streams, and (wide) strings. -- Adds Unicode support, integrated Unicode pattern matching on UTF-8/16/32 files and wide strings (no need to write a `YY_INPUT` routine for decoding). -- Adds Unicode property matching `\p{C}` and C++11, Java, C#, and Python Unicode properties for identifier name matching. -- Adds indent `\i` and dedent `\j` patterns to match rules on text with indentation, including `\t` (tab) adjustments. -- Adds lazy quantifiers to the POSIX regular expression syntax, so not more hacks to work around the greedy repetitions in Flex. -- Adds word boundary anchors to the POSIX regular expression syntax. -- Adds `%class` and `%init` to customize the generated Lexer classes. -- Adds `%include` to modularize lex specifications.- Adds an extensible hierarchy of pattern matcher engines, with a choice of regex engines, including the RE/flex regex engine and Boost.Regex. -- Generates MT-safe (reentrant) code by default. -- Generates clean source code that defines a C++ Lexer class derived from an abstract lexer class. -- RE/flex generates lex.yy.cpp files while Flex generates lex.yy.cc files (in C++ mode with flex option -+), to distinguish the generated files. +- Compatible with Flex to eliminate a learning curve, making a transition from Flex++ to RE/flex frustration-free. +- Generates reusable source code that is easy to understand. +- Integrates seamlessly with Bison and supports reentrant, bison-bridge, bison-locations, Bison 3.0 C++ interface `%skeleton "lalr1.cc"` and Bison complete symbols. +- Fully supports Unicode and Unicode properties `\p{C}`, including Unicode identifier matching for C++11, Java, C#, and Python source code. +- Auto-detects UTF-8/16/32 input to match Unicode patterns. +- Supports file encodings ISO-8859-1 through ISO-8859-15, CP 1250 through 1258, CP 437, CP 850, CP 858, KOI8, MACROMAN, EBCDIC, and custom code pages. +- Generates scanners for lexical analysis on files, C++ streams, (wide) strings, and memory such as mmap files. +- Includes many examples, such as a tokenizer for C/C++ source code, a tokenizer for Python source code, a tokenizer for Java source code, and more. +- Extensive documentation in the online [User Guide][manual-url]. +- Indent/nodent/dedent anchors to match text with indentation, including custom `\t` (tab) widths. +- Lazy quantifiers, no hacks are needed to work around greedy repetitions. +- Word boundary anchors. +- Freespace mode option to improve readability of lexer specifications. +- `%class` and `%init` to customize the generated Lexer classes. +- `%include` to modularize lexer specifications. +- Multiple lexer classes can be combined and used in one application, e.g. by multiple threads in a thread-safe manner. +- Configurable Lexer class generation to customize the interface for various parsers, including Yacc and Bison. - Generates Graphviz files to visualize FSMs with the Graphviz dot tool. -- Includes many examples, such as a tokenizer for C/C++ code, a tokenizer for Python code, a tokenizer for Java code, and more. -- Converts the official Unicode scripts Scripts.txt and UnicodeData.txt to UTF-8 patterns by applying a RE/flex scanner to convert these scripts to C++ code. Future Unicode standards can be automatically converted using these scanners that are written in RE/flex itself. -- Conversion of regex expressions, for regex engines that lack regex features. -- The RE/flex regex library makes C++11 std::regex and Boost.Regex much easier to use in plain C++ code for pattern matching on (wide) strings, files, and streams. - -The RE/flex software is self-contained. No other libraries are required. Boost.Regex is optional to use as a regex engine. - -The RE/flex repo includes tokenizers for Java, Python, and C/C++. +- Includes an extensible hierarchy of pattern matcher engines, with a choice of regex engines, including the RE/flex regex engine, PCRE2, and Boost.Regex. +- The RE/flex regex library makes C++11 std::regex, PCRE2, and Boost.Regex much easier to use for pattern matching on (wide) strings, files, and streams. +- Lots of other improvements over Flex++, such as `yypush_buffer_state` saves the scanner state (line, column, and indentation positions), not just the input buffer; no input buffer length limit (Flex has a 16KB limit); `line()` returns the current line (e.g. for error reporting) + +The RE/flex software is self-contained. No other libraries are required. PCRE2 and Boost.Regex are optional to use as regex engines. + +The RE/flex repo includes tokenizers for Java, Python, C/C++, JSON, XML, YAML. Installation </pre> </div>Robert van EngelenThu, 14 May 2020 19:24:30 -0000https://sourceforge.net4bdfd291de1045bb8f50946e4bf466915fe899fbhttps://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v52 +++ v53 @@ -20,11 +20,12 @@ | pcre2_jit_match() | PCRE2 (jit) 10.32 | 60 | hs_compile_multi(), hs_scan() | Hyperscan 5.1.0 | 209 | reflex −m=boost-perl | Boost.Regex 1.66.0 | 230 -| pcre2_match() | PCRE2 (pre-compiled) 10.32 | 318 +| pcre2_match() | PCRE2 10.32 | 318 | RE2::Consume() | RE2 (pre-compiled) 2018-04-01 | 417 | reflex −m=boost | Boost.Regex POSIX 1.66.0 | 450 | RE2::Consume() | RE2 POSIX (pre-compiled) 2018-04-01 | 1226 | flex −+ | Flex 2.5.35 | 3968 +| pcre2_dfa_match() | PCRE2 POSIX (dfa) 10.32 | 4094 | regcomp(), regexec() | GNU C POSIX.2 regex | 5800 | std::regex::cregex_iterator() | C++11 std::regex | 5979 </pre> </div>Robert van EngelenSun, 01 Mar 2020 13:51:03 -0000https://sourceforge.netb24cdb9c627284505e682a120f9656da99a5f1a6https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v51 +++ v52 @@ -17,9 +17,10 @@ | flex −+ −−full | Flex 2.5.35 | 17 | reflex −−full | RE/flex 1.5.6 | 18 | boost::spirit::lex::lexertl::actor_lexer::iterator_type | Boost.Spirit.Lex 1.66.0 | 40 +| pcre2_jit_match() | PCRE2 (jit) 10.32 | 60 | hs_compile_multi(), hs_scan() | Hyperscan 5.1.0 | 209 | reflex −m=boost-perl | Boost.Regex 1.66.0 | 230 -| pcre2_match() | PCRE2 (pre-compiled) 10.30 | 318 +| pcre2_match() | PCRE2 (pre-compiled) 10.32 | 318 | RE2::Consume() | RE2 (pre-compiled) 2018-04-01 | 417 | reflex −m=boost | Boost.Regex POSIX 1.66.0 | 450 | RE2::Consume() | RE2 POSIX (pre-compiled) 2018-04-01 | 1226 </pre> </div>Robert van EngelenWed, 26 Feb 2020 19:17:18 -0000https://sourceforge.netfc91af0969e442141e28f4fa56236fb06c1419f5https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v50 +++ v51 @@ -27,8 +27,9 @@ | regcomp(), regexec() | GNU C POSIX.2 regex | 5800 | std::regex::cregex_iterator() | C++11 std::regex | 5979 -Note: *Best times of 10 tests with average time in microseconds over 100 runs (using Mac OS X 10.12.6 with clang 9.0.0 -O2, 2.9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3). Hyperscan disqualifies as a potential scanner due to its "All matches reported" semantics resulting in 1915 matches for his test and its event handler requirements.* [Download the tests](https://www.genivia.com/files/perfcomp.zip) - +Note: *Best times of 30 tests with average time in microseconds over 100 runs (using Mac OS X 10.12.6 with clang 9.0.0 -O2, 2.9 GHz Intel Core i7, 16 GB 2133 MHz LPDDR3). Hyperscan disqualifies as a potential scanner due to its "All matches reported" semantics resulting in 1915 matches for his test, and due to its event handler requirements.* [Download the tests](https://www.genivia.com/files/perfcomp.zip) *Timings on other platforms may differ, though in the worst cases tested, reflex ran equally fast or slightly faster than the best times of Flex.* + +The RE/flex matcher tracks line numbers, column numbers, and indentations, whereas Flex does not (option noyylineno) and neither do the other regex matchers in the table, except Boost.Regex with reflex. Tracking this information incurs some overhead. Features -------- </pre> </div>Robert van EngelenTue, 31 Dec 2019 15:03:02 -0000https://sourceforge.neta2e13ca9eb4b465a07b47fe593a14af86ace90d7https://sourceforge.net/p/re-flex/wiki/Home/<div class="markdown_content"><pre>--- v49 +++ v50 @@ -12,7 +12,7 @@ | Command / Function | Software | Time (μs) | ------------------------------------------------------- | ----------------------------------- | --------- -| reflex −−fast −−noindent | RE/flex 1.5.6 | 9 +| reflex −−fast −−noindent | RE/flex 1.5.6 | 8 | reflex −−fast | RE/flex 1.5.6 | 9 | flex −+ −−full | Flex 2.5.35 | 17 | reflex −−full | RE/flex 1.5.6 | 18 </pre> </div>Robert van EngelenSat, 28 Dec 2019 22:01:05 -0000https://sourceforge.net0193716b7280d4992f3e0e519864b5801c560254