Re: [Flex-devel] [Flex-help] 8 bit input and flex?

[Will E. <wl...@us...>]

Aaron, you're right that we don't have any 8-bit input tests in the test
suite. It's also worth noting that flex doesn't do unicode as such at
the moment (but that's on the todo list to address); flex just has some
support for 8-bit input.


On Tuesday, 27 November 2007,  9:39 am -0800, Aaron Stone <aa...@se...> wrote:

> Hi Steve,
> 
> What version of flex are you using?
> 
> To be honest, I've never worked with 8-bit patterns, and actually I
> wonder if we have any regression tests with Unicode input in current
> flex CVS.
> 
> Aaron
> 
> On Tue, 2007-11-27 at 11:13 -0600, Steve Murphy wrote:
> > Hello!
> > 
> > This is perplexing me! How do you do this? I've tried all sorts of
> > tactics, and nothing seems to work... a tip or two would be greatly
> > appreciated.
> > 
> > Just to be sure, I set the %option 8bit (even tho flex builds 8-bit
> > scanners by default, or so it says).
> > 
> > First of all, I noted that chars with the high-order bit set were
> > getting shovelled out to stdout. This was natural, as the rule that
> > was meant to capture them doesn't include any 8-bit chars, and looks
> > like this:
> > 
> > [a-zA-Z0-9_][a-zA-Z0-9_$#@%&]*
> > 
> > Well, using range notation didn't work:
> > 
> > [a-zA-Z0-9_\x80-\xff][a-zA-Z0-9_$#@%&\x80-\xff]*
> > 
> > And octal ranges didn't work, either:
> > 
> > [a-zA-Z0-9_\200-\377][a-zA-Z0-9_$#@%&\200-\377]*
> > 
> > So, I tried the {name} substitution approach:
> > 
> > HIBIT  (\x80|\x81|\x82....)
> > %%
> > ([a-zA-Z0-9_]|{HIBIT})([a-zA-Z0-9_$#@%&]|{HIBIT})*
> > 
> > and that's not working either.  The documentation says flex is an
> > 8-bit scanner
> > by default, but I'm not seeing evidence of this... what am I doing
> > wrong?
> > 
> > 
> > -------------------------------------------------------------------------
> > This SF.net email is sponsored by: Microsoft
> > Defy all challenges. Microsoft(R) Visual Studio 2005.
> > http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> > _______________________________________________ Flex-help mailing list Fle...@li... https://lists.sourceforge.net/lists/listinfo/flex-help
> 
> 
> -------------------------------------------------------------------------
> This SF.net email is sponsored by: Microsoft
> Defy all challenges. Microsoft(R) Visual Studio 2005.
> http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/
> _______________________________________________
> Flex-help mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-help

-- 
Will Estes
Flex Project Maintainer
http://flex.sourceforge.net/