Did you have a look a 'demo/Cpp/11-ManualBufferFilling', there it is explained how a feeder or gavager can help out. The example 'lexer-point.cpp' shows how simply to point to a piece of allocated memory. For plain C look at 'demo/C/11-ManualBufferFilling'.
Hi! I really want to use Quex in a project, but I'm having a problem with lexing large input programs. The buffer always fills up to the buffer limit specified in the configuration, but I want to be able to support arbitrary file sizes. This happens even if I create a ByteLoader from a file. Quex just loads it up to the buffer limit. I'm writing this in C, and using a UTF-8 converter. For context, I am given the entire program as a char* string in memory (not a file!). The lexing works perfectly...
Still does not work here on python3.8. It does on python 3.9. Maybe is my environment? However I am happy with it.
OK I opened a homebrew PR to update to 0.71.2: https://github.com/Homebrew/homebrew-core/pull/76539
I just uploaded version 0.71.2 in which I am definitely not able to see the issue anymore. Please, verify
Changes to UTs
Fixing UTs; 3 fails / 1919;
step 12; UTs 100% ok / 1913;
Getting back; fixing unit tests.
step 13; UTs fails 1/1913;
Save
Fixing UTs; UTs 3 fails/1919;
OK, I am on it. However, on my linux machine I am not able to reproduce the error. I will try to submit a tar file this evening with a current stable version of Quex. If I get a chance, I am also investigating on MacOs.
If I do python3.8 quex-exe.py I get Exception occurred -- please, report a bug under Getting a little inside I do python3.8 from quex.blackboard import setup Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/var/tmp/portage/dev-python/quex-0.71.0-r1/work/quex-0.71.0/quex/blackboard.py", line 22, in <module> setup = QuexSetup(SETUP_INFO, bc_factory) File "/var/tmp/portage/dev-python/quex-0.71.0-r1/work/quex-0.71.0/quex/input/setup.py", line 72, in init self.init(SetupInfo)...
I am not an end-user of quex. I was only looking into it as part of doing some maintenance works on Homebrew. Homebrew has had a quex build formula for a long time but until recently it was using a python-2 version (which has been deprecated in homebrew) and am just trying to get a reliable python 3 version integrated. See: https://github.com/Homebrew/homebrew-core/blob/master/Formula/quex.rb As mentioned above we are now using 0.71.0 but are using it with python 3.7 (the earliest homebrew supports)...
This weekend, sorry.
I will look into it this weekend. Python 3.9 should be supported, then. Can you please, send me your use-case?
Is there documentation about what python versions are supported now? I took a look at updating the homebrew formula for quex. Homebrew has python 3.7, 3.8, and 3.9 available currently. I hit some immediate issues with 3.8 but that seems to be fixed if I backported one little fix from the git head, but I'm not sure if that is really a complete fix for 3.7 use: https://github.com/Homebrew/homebrew-core/pull/70789 Of course best would be an upstream release with an explicit indication of what python...
Hi, have you found what could be wrong?
Sorry, for the delay. I might have a look into it the next couple of days.
UTF8 lexer Assertion failed. with latin script letters
step 11; UTs 100% ok / 1913;
step 9; UTs 100% ok / 1913;
step 10; UTs 100% ok / 1913;
step 8; UTs 100% ok / 1913;
step 7; UTs 100% ok / 1913;
step 6; UTs 100% 0 fails/1913;
step 3; UTs 100% 0 fails/1913;
step 4; UTs 100% 0 fails/1913;
step 2; UTs 100% 0 fails/1913;
step 5; UTs 100% 0 fails/1913;
1st steps; UTs 100% 0 fails/1913;
Make master clean again.
Fix git issue with packing
Refactor event handling infrastructure
Refactor event handling infrastructure
Refactor event handling infrastructure
Refactor event handling infrastructure
Documentation
Porting to python 3.8
Porting to python 3.7
Documentation;
Documentation;
The MacOS installer for version 0.71 does not work
Documentation: unary operations
UTs improved;
Adding UTs for epsilon transition--required also for documentation;
Operations on singular dfas; UTs 7 fails/1905;
Documentation work: Unary Operations;
Documentation and UTs on singular DFAs;
Operations on singular dfas; UTs 49 fails/1904;
Quex 0.70.0 generates invalid include search paths
Adding \lexemes command;
The lexeme determination command in define section
The lexeme determination command in define section; UTs 1 fail/1894;
Documentation; Elementary RE character sets;
Documentation; RE elemetary regular expressions;
Unicode General Category 'Currency Symbol' bad UTF8 State Machine Plot
Not an error: Replace \plot (fh) Ahex.dot A with \plot (fh) Ahex.dot {A} and everything is fine
Unicode General Category 'Currency Symbol' bad UTF8 State Machine Plot
Changes to state machine transformation; UTs 100% ok/1893;
Omitting pesky __init__.py files
Merge conflicts
Documentation: syntax re review
Documentation work;
RE: Backslash + Space causes assertion error
Solved in Revision: 71403dc33 Unit Test: TEST/bug-306.sh
Terminated Bugfix + minor issue: UTs 100% Ok / 1893;
Terminate bugfix
Terminate bugfix 306;
Bug fix done; UTs 100% ok / 1893;
RE: Backslash + Space causes assertion error
Documentation; syntax/top-level done
Documentation; syntax on RE
Documentation; token identifier precision
Merged branch master; UTs 1 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 1 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 2 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 2 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 2 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 2 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 16 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 6 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 34 fails/1892;
Migration Development to Ubuntu 20.4 LTS; UTs 53 fails/1892;
Progress with UTs
Migration Development to Ubuntu 20.4 LTS; UTs 63 fails/1892;
Migration with development to Ubuntu 20.4 LTS; UTs 113 fails / 1892
Changes in command line interpretation; UTs 288 fails / 1892;
begin