flex-devel — development of flex itself

[Flex-devel] UTF-8

[Paul <pa...@pr...>]

Looking at Peter Martini's work, I think that when set as 8 bit, it will 
accept both ascii and utf-8.
I would prefer to only accept utf-8 with a flex flag like:  %option utf8.
I believe a generated scanner can be more efficient if it does not have 
to deal with utf-8.
This seems to be a philosophical issue, so I would like opinions.
I may be off the rails here.

Paul Neelands

Re: [Flex-devel] Git branch for utf16

[Will E. <wes...@gm...>]

Paul,

Yes, it's going to be a bit before I can get to this. I've got a couple
cleanup chores yet to do in the git repo and then we can look at how to
include the unicode work.

Thanks for working on this so much.

--Will

On Saturday, 21 July 2012,  2:36 pm -0400, Paul <pa...@pr...> wrote:

> Have created here a git branch utf16v1 for the UTF16 changes.
> Passes the tests.
> I assume now, I wait for the changes to the source and test trees.
> Thanks for doing the git.
> 
> Paul Neelands
> 
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel

-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

Re: [Flex-devel] flex git repo at sourceforge

[Will E. <wes...@gm...>]

Paul,


The git repo includes all the latest stuff from the cvs tree, up through
the 2.5.36 release that I did yesterday. (I see my post to flex-announce
is stuck in the moderation queue...)

Any differences between 2.5.35 and current git are due to that.

On Saturday, 21 July 2012,  1:18 pm -0400, Paul <pa...@pr...> wrote:

> Works nicely. You've added some tests.
> 
> Paul
> 
> On 12-07-20 09:37 PM, Will Estes wrote:
> > All,
> >
> > See the flex repo, which is now in git, at:
> >
> > git://flex.git.sourceforge.net/gitroot/flex/flex
> >
> > for read only access.
> >
> > At the moment, it's just an import of the (now legacy) cvs tree, with
> > only the initial git cvsimport done. I see that the .cvsignore files
> > need converted to .gitignore files and we could do with a proper tagging
> > as well. I'll also be putting git2cl in place so that we can build the
> > GNU-style ChangeLog again, but there's git log for the mean time.
> >
> > Happy hacking,
> >
> > --Will
> >
> >
> 
> 
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel

-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

[Flex-devel] utf16 diff line count

[Paul <pa...@pr...>]

Here is a git diff list of the # of changes utf16 introduces.
The numbers are the line count changed for each file.

Paul

git diff --stat --color master..utf16v1
  FlexLexer.h                           |   30 ++-
  ccl.c                                      |    2 +-
  ecs.c                                     |    2 +-
  flex.skl                                  |  352 
++++++++++++++++++++++-------
  flexdef.h                                |   50 +++-
  gen.c                                     |   99 ++++++--
  main.c                                    |  135 +++++++++--
  misc.c                                    |   33 ++-
  options.c                                |   10 +-
  options.h                                |    3 +
  parse.y                                   |    2 +-
   scan.l                                     |    8 +-
  scanopt.c                               |    8 +-
  tblcmp.c                                 |    2 +-

[Flex-devel] Git branch for utf16

[Paul <pa...@pr...>]

Have created here a git branch utf16v1 for the UTF16 changes.
Passes the tests.
I assume now, I wait for the changes to the source and test trees.
Thanks for doing the git.

Paul Neelands

Re: [Flex-devel] flex git repo at sourceforge

[Paul <pa...@pr...>]

Works nicely. You've added some tests.

Paul

On 12-07-20 09:37 PM, Will Estes wrote:
> All,
>
> See the flex repo, which is now in git, at:
>
> git://flex.git.sourceforge.net/gitroot/flex/flex
>
> for read only access.
>
> At the moment, it's just an import of the (now legacy) cvs tree, with
> only the initial git cvsimport done. I see that the .cvsignore files
> need converted to .gitignore files and we could do with a proper tagging
> as well. I'll also be putting git2cl in place so that we can build the
> GNU-style ChangeLog again, but there's git log for the mean time.
>
> Happy hacking,
>
> --Will
>
>

[Flex-devel] flex git repo at sourceforge

[Will E. <wes...@gm...>]

All,

See the flex repo, which is now in git, at:

git://flex.git.sourceforge.net/gitroot/flex/flex

for read only access.

At the moment, it's just an import of the (now legacy) cvs tree, with
only the initial git cvsimport done. I see that the .cvsignore files
need converted to .gitignore files and we could do with a proper tagging
as well. I'll also be putting git2cl in place so that we can build the
GNU-style ChangeLog again, but there's git log for the mean time.

Happy hacking,

--Will


-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

[Flex-devel] Unicode food for thought

[Peter M. <pet...@gm...>]

Unicode has just released its newest version of UTS-18, dealing with
regular expression in Unicode.

http://www.unicode.org/reports/tr18/tr18-15.html

I think as far as our implementation of Unicode handling is concerned, we
should see which sections of Level 1 we're most concerned about and aim for
that. Some of the Level 2 features would be nice down the road, but I don't
think we'd even want to go for full support for that or anything further.

The elements of Level 1 that stick out the most to me are:

"Some caseless matches may match one character against two: for example,
U+00DF "ß" matches the two characters "SS". And case matching may vary by
locale. However, because many implementations are not set up to handle
this, at Level 1 only simple case matches are necessary. To correctly
implement a caseless match, see* Chapter 3, Conformance* of
[Unicode<http://www.unicode.org/reports/tr18/tr18-15.html#Unicode>].
The data file supporting caseless matching is
[CaseData<http://www.unicode.org/reports/tr18/tr18-15.html#CaseData>
]."

Definition of a newline character is :
"\u{A} | \u{B} | \u{C} | \u{D} | \u{85} | \u{2028} | \u{2029} | \u{D A}"

"It is strongly recommended that there be a regular expression
meta-character, such as "\R", for matching all line ending characters and
sequences listed above (for example, in #1). This would correspond to
something equivalent to the following expression. That expression is
slightly complicated by the need to avoid backup.

(?:\u{D A}|(?!\u{D A})[\u{A}-\u{D}\u{85}\u{2028}\u{2029})"


"
RL1.7 <http://www.unicode.org/reports/tr18/tr18-15.html#RL1.7>Supplementary
Code PointsTo meet this requirement, an implementation shall handle the
full range of Unicode code points, including values from U+FFFF to
U+10FFFF. In particular, where UTF-16 is used, a sequence consisting of a
leading surrogate followed by a trailing surrogate shall be handled as a
single code point in matching.
*Note: *It is permissible, but not required, to match an isolated surrogate
code point (such as \u{D800}), which may occur in Unicode Strings. SeeUnicode
String <http://www.unicode.org/glossary/#unicode_string> in the Unicode
glossary."

Re: [Flex-devel] UTF-8 integration

[Will E. <wes...@gm...>]

Paul,

This is great.

I'm not opposed in principle to the change you suggest, but I want to
think about some other issues first.

It looks like importing the cvs tree into git is going to be a breeze,
and it looks like it'll come out quite pretty.

I've got time on my books tomorrow to put out a new release of flex to
catch up the released code to what's in cvs. Then we can get flex into
git and make all of the UTF work that much easier.

Fair warning that once flex is in git, the source code will get moved
into an src/ directory and I want to look at redoing the way the test
suite is handled, especially since you've added so many tests. You're
right that flex does need a lot more test coverage.

--Will

On Thursday, 19 July 2012,  6:55 pm -0400, Paul <pa...@pr...> wrote:

> Have the barest bones of an integration of flex-cvs, flex utf-16, &
> flex utf-8.
> Passes 106 of 107 tests. The last is a problem with ccl lists in utf16-mode.
> Besides this fix it needs many more tests.
> 
> I'm wondering if the change from the ccl bit map to lists is ok in
> general in all modes.
> I'm using it for the tests and there is no perceptible speed difference.
> Having a bit map for non utf-8 and a list for it would be a pain.
> 
> Paul
> 
> On 12-07-19 05:25 AM, Peter Martini wrote:
> >I'd be thrilled, actually.  I started trying to integrate the
> >changes that flex had gone through and  the patches you were
> >sending into my branch directly, and ended up deferring the work
> >each time when I got intimidated by the scale of the diffs :-)
> >
> >Hopefully the commits in my branch are discrete enough to be useful.
> >
> >On Thu, Jul 19, 2012 at 5:21 AM, Paul <pa...@pr...
> ><mailto:pa...@pr...>> wrote:
> >
> >    Would you mind if I took your changes, made them conditional on
> >    the utf8 flag and integrated them into the unicode16 version.
> >    I would also like to add more tests.
> >
> >
> >    Paul
> >
> >    On 12-07-18 11:29 AM, Peter Martini wrote:
> >>    Not nearly as thoroughly as you did.  I confirmed that with or
> >>    without UTF-8 parsing enabled, if its only ASCII text, the same
> >>    exact tables are generated, and I added tests to make sure that
> >>    the unicode escapes and the '.' work as designed.  As designed is
> >>    a very broad definition though.
> >>
> >>    Take a look at
> >>    https://github.com/PeterMartini/flex/commits/master - I made a
> >>    lot of small commits, so you can see the changes I was
> >>making.     The biggest change structurally was I changed the
> >>CCL bitmap to
> >>    be a linked list of ranges, to speed up processing the 0x10 FFFF
> >>    allowed values, and then added a secondary step to convert those
> >>    unicode characters to an equivalent 8-bit pattern.
> >>
> >>    But my commits also track some housekeeping I did to get rid of
> >>    warnings and get those C++ tests to pass, and I think some of
> >>    that has been merged into the main branch now.
> >>
> >>    On Mon, Jul 16, 2012 at 5:03 PM, Paul <pa...@pr...
> >>    <mailto:pa...@pr...>> wrote:
> >>
> >>        How completely have you tested your utf8 version of flex?
> >>
> >>        Paul
> >>
> >>
> >
> >
> >
> 
> 

> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/

> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel


-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

[Flex-devel] UTF-8 integration

[Paul <pa...@pr...>]

Have the barest bones of an integration of flex-cvs, flex utf-16, & flex 
utf-8.
Passes 106 of 107 tests. The last is a problem with ccl lists in utf16-mode.
Besides this fix it needs many more tests.

I'm wondering if the change from the ccl bit map to lists is ok in 
general in all modes.
I'm using it for the tests and there is no perceptible speed difference.
Having a bit map for non utf-8 and a list for it would be a pain.

Paul

On 12-07-19 05:25 AM, Peter Martini wrote:
> I'd be thrilled, actually.  I started trying to integrate the changes 
> that flex had gone through and  the patches you were sending into my 
> branch directly, and ended up deferring the work each time when I got 
> intimidated by the scale of the diffs :-)
>
> Hopefully the commits in my branch are discrete enough to be useful.
>
> On Thu, Jul 19, 2012 at 5:21 AM, Paul <pa...@pr... 
> <mailto:pa...@pr...>> wrote:
>
>     Would you mind if I took your changes, made them conditional on
>     the utf8 flag and integrated them into the unicode16 version.
>     I would also like to add more tests.
>
>
>     Paul
>
>     On 12-07-18 11:29 AM, Peter Martini wrote:
>>     Not nearly as thoroughly as you did.  I confirmed that with or
>>     without UTF-8 parsing enabled, if its only ASCII text, the same
>>     exact tables are generated, and I added tests to make sure that
>>     the unicode escapes and the '.' work as designed.  As designed is
>>     a very broad definition though.
>>
>>     Take a look at
>>     https://github.com/PeterMartini/flex/commits/master - I made a
>>     lot of small commits, so you can see the changes I was making. 
>>     The biggest change structurally was I changed the CCL bitmap to
>>     be a linked list of ranges, to speed up processing the 0x10 FFFF
>>     allowed values, and then added a secondary step to convert those
>>     unicode characters to an equivalent 8-bit pattern.
>>
>>     But my commits also track some housekeeping I did to get rid of
>>     warnings and get those C++ tests to pass, and I think some of
>>     that has been merged into the main branch now.
>>
>>     On Mon, Jul 16, 2012 at 5:03 PM, Paul <pa...@pr...
>>     <mailto:pa...@pr...>> wrote:
>>
>>         How completely have you tested your utf8 version of flex?
>>
>>         Paul
>>
>>
>
>
>

Re: [Flex-devel] UTF8

[Peter M. <pet...@gm...>]

I'd be thrilled, actually.  I started trying to integrate the changes that
flex had gone through and  the patches you were sending into my branch
directly, and ended up deferring the work each time when I got intimidated
by the scale of the diffs :-)

Hopefully the commits in my branch are discrete enough to be useful.

On Thu, Jul 19, 2012 at 5:21 AM, Paul <pa...@pr...> wrote:

>  Would you mind if I took your changes, made them conditional on the utf8
> flag and integrated them into the unicode16 version.
> I would also like to add more tests.
>
>
> Paul
>
> On 12-07-18 11:29 AM, Peter Martini wrote:
>
> Not nearly as thoroughly as you did.  I confirmed that with or without
> UTF-8 parsing enabled, if its only ASCII text, the same exact tables are
> generated, and I added tests to make sure that the unicode escapes and the
> '.' work as designed.  As designed is a very broad definition though.
>
> Take a look at https://github.com/PeterMartini/flex/commits/master - I
> made a lot of small commits, so you can see the changes I was making.  The
> biggest change structurally was I changed the CCL bitmap to be a linked
> list of ranges, to speed up processing the 0x10 FFFF allowed values, and
> then added a secondary step to convert those unicode characters to an
> equivalent 8-bit pattern.
>
> But my commits also track some housekeeping I did to get rid of warnings
> and get those C++ tests to pass, and I think some of that has been merged
> into the main branch now.
>
> On Mon, Jul 16, 2012 at 5:03 PM, Paul <pa...@pr...> wrote:
>
>> How completely have you tested your utf8 version of flex?
>>
>> Paul
>>
>
>
>
>

Re: [Flex-devel] UTF8

[Paul <pa...@pr...>]

Would you mind if I took your changes, made them conditional on the utf8 
flag and integrated them into the unicode16 version.
I would also like to add more tests.

Paul

On 12-07-18 11:29 AM, Peter Martini wrote:
> Not nearly as thoroughly as you did.  I confirmed that with or without 
> UTF-8 parsing enabled, if its only ASCII text, the same exact tables 
> are generated, and I added tests to make sure that the unicode escapes 
> and the '.' work as designed.  As designed is a very broad definition 
> though.
>
> Take a look at https://github.com/PeterMartini/flex/commits/master - I 
> made a lot of small commits, so you can see the changes I was making.  
> The biggest change structurally was I changed the CCL bitmap to be a 
> linked list of ranges, to speed up processing the 0x10 FFFF allowed 
> values, and then added a secondary step to convert those unicode 
> characters to an equivalent 8-bit pattern.
>
> But my commits also track some housekeeping I did to get rid of 
> warnings and get those C++ tests to pass, and I think some of that has 
> been merged into the main branch now.
>
> On Mon, Jul 16, 2012 at 5:03 PM, Paul <pa...@pr... 
> <mailto:pa...@pr...>> wrote:
>
>     How completely have you tested your utf8 version of flex?
>
>     Paul
>
>

Re: [Flex-devel] UTF8

[Paul <pa...@pr...>]

Thanks Peter, I'll give it a try.

Paul

On 12-07-18 11:29 AM, Peter Martini wrote:
> Not nearly as thoroughly as you did.  I confirmed that with or without 
> UTF-8 parsing enabled, if its only ASCII text, the same exact tables 
> are generated, and I added tests to make sure that the unicode escapes 
> and the '.' work as designed.  As designed is a very broad definition 
> though.
>
> Take a look at https://github.com/PeterMartini/flex/commits/master - I 
> made a lot of small commits, so you can see the changes I was making.  
> The biggest change structurally was I changed the CCL bitmap to be a 
> linked list of ranges, to speed up processing the 0x10 FFFF allowed 
> values, and then added a secondary step to convert those unicode 
> characters to an equivalent 8-bit pattern.
>
> But my commits also track some housekeeping I did to get rid of 
> warnings and get those C++ tests to pass, and I think some of that has 
> been merged into the main branch now.
>
> On Mon, Jul 16, 2012 at 5:03 PM, Paul <pa...@pr... 
> <mailto:pa...@pr...>> wrote:
>
>     How completely have you tested your utf8 version of flex?
>
>     Paul
>
>

Re: [Flex-devel] UTF8

[Peter M. <pet...@gm...>]

Not nearly as thoroughly as you did.  I confirmed that with or without
UTF-8 parsing enabled, if its only ASCII text, the same exact tables are
generated, and I added tests to make sure that the unicode escapes and the
'.' work as designed.  As designed is a very broad definition though.

Take a look at https://github.com/PeterMartini/flex/commits/master - I made
a lot of small commits, so you can see the changes I was making.  The
biggest change structurally was I changed the CCL bitmap to be a linked
list of ranges, to speed up processing the 0x10 FFFF allowed values, and
then added a secondary step to convert those unicode characters to an
equivalent 8-bit pattern.

But my commits also track some housekeeping I did to get rid of warnings
and get those C++ tests to pass, and I think some of that has been merged
into the main branch now.

On Mon, Jul 16, 2012 at 5:03 PM, Paul <pa...@pr...> wrote:

> How completely have you tested your utf8 version of flex?
>
> Paul
>

[Flex-devel] UTF8

[Paul <pa...@pr...>]

How completely have you tested your utf8 version of flex?

Paul

Re: [Flex-devel] Conversion to Git?

[Will E. <wes...@gm...>]

On Sunday, 15 July 2012,  7:17 pm -0700, Isaac Dunham <id...@la...> wrote:

> Does gitcl [http://josefsson.org/git2cl/] suffice?

I'll check it out.

Thanks,

--Will

-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

Re: [Flex-devel] Conversion to Git?

[Isaac D. <id...@la...>]

On Sun, 15 Jul 2012 20:10:05 -0400
Will Estes <wes...@gm...> wrote:

> The basic plan is this:
> 
> * Get a release of flex out so that all the recent bug fixes are in
> the generally available version of flex.

> * Get flex imported into git and cleaned up and buildable with the git
> * workflow/toolchain. (In particular, I want to move the *.[ch] files
>   into an src directory and I want to look at making the test suite
> more sane. But I'll also need the git equivalent of cvs2cl, which the
> current cvs tree relies on to build new ChangeLog files.)

Does gitcl [http://josefsson.org/git2cl/] suffice?
<snip>
> --Will

HTH,
Isaac Dunham

Re: [Flex-devel] Conversion to Git?

[Aaron S. <aa...@se...>]

Absolutely fantastic! Thank you, Will.

On Sun, Jul 15, 2012 at 5:10 PM, Will Estes <wes...@gm...> wrote:
> All,
>
> I'm making arrangements to get flex into git so that things like this
> can happen way more easily.
>
> The basic plan is this:
>
> * Get a release of flex out so that all the recent bug fixes are in the
>   generally available version of flex.
>
> * Get flex imported into git and cleaned up and buildable with the git
> * workflow/toolchain. (In particular, I want to move the *.[ch] files
>   into an src directory and I want to look at making the test suite more
> sane. But I'll also need the git equivalent of cvs2cl, which the
> current cvs tree relies on to build new ChangeLog files.)
>
> If this were the sort of change that didn't touch flex in all kinds of
> places, it'd be more-or-less easy to do in cvs, but what with all the
> changes that Paul has been working on, once I patched the cvs tree, and
> made additional changes to continue work on flex, it'd be a mess again,
> and branching and remerging in cvs are as Aaron notes, dirty words. Git
> rebase, not so much.
>
> --Will
>
> On Tuesday, 10 July 2012,  2:12 pm -0700, Aaron Stone <aa...@se...> wrote:
>
>> In CVS, forking is a dirty word, because merging is extremely hard. In
>> git, a fork is simply your workspace, ready o be pulled back to
>> mainline. It's a fantastically easier workflow.
>>
>> However your point is valid: if mainline flex is in CVS, patches are
>> its workflow :/
>>
>> On Jul 10, 2012, at 1:53 PM, Paul <pa...@pr...> wrote:
>>
>> > I really would rather not effectively fork flex. I would prefer to do
>> > whatever is necessary to integrate with the mainline.
>> >
>> > Paul
>> >
>> > On 07/10/2012 04:43 PM, ruertar wrote:
>> >> On 07/10/2012 04:33 PM, Peter Martini wrote:
>> >>> Any thoughts on converting to Git?  Sourceforge already has a
>> >>> (empty) git repository for flex, and it's not that difficult to
>> >>> convert the CVS repo to Git while preserving all of the history,
>> >>> branches, and tags.  (I've done it locally already).
>> >>>
>> >> I would like this as well.
>> >>
>> >> ------------------------------------------------------------------------------
>> >> Live Security Virtual Conference
>> >> Exclusive live event will cover all the ways today's security and
>> >> threat landscape has changed and how IT managers can respond. Discussions
>> >> will include endpoint security, mobile security and the latest in malware
>> >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> >> _______________________________________________
>> >> Flex-devel mailing list
>> >> Fle...@li...
>> >> https://lists.sourceforge.net/lists/listinfo/flex-devel
>> >>
>> >>
>> >
>> >
>> >
>> >
>> > ------------------------------------------------------------------------------
>> > Live Security Virtual Conference
>> > Exclusive live event will cover all the ways today's security and
>> > threat landscape has changed and how IT managers can respond. Discussions
>> > will include endpoint security, mobile security and the latest in malware
>> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> > _______________________________________________
>> > Flex-devel mailing list
>> > Fle...@li...
>> > https://lists.sourceforge.net/lists/listinfo/flex-devel
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Flex-devel mailing list
>> Fle...@li...
>> https://lists.sourceforge.net/lists/listinfo/flex-devel
>
> --
> Will Estes (wl...@us...)
> Flex Project Maintainer
> http://flex.sourceforge.net/

Re: [Flex-devel] Conversion to Git?

[Will E. <wes...@gm...>]

All,

I'm making arrangements to get flex into git so that things like this
can happen way more easily.

The basic plan is this:

* Get a release of flex out so that all the recent bug fixes are in the
  generally available version of flex.

* Get flex imported into git and cleaned up and buildable with the git
* workflow/toolchain. (In particular, I want to move the *.[ch] files
  into an src directory and I want to look at making the test suite more
sane. But I'll also need the git equivalent of cvs2cl, which the
current cvs tree relies on to build new ChangeLog files.)

If this were the sort of change that didn't touch flex in all kinds of
places, it'd be more-or-less easy to do in cvs, but what with all the
changes that Paul has been working on, once I patched the cvs tree, and
made additional changes to continue work on flex, it'd be a mess again,
and branching and remerging in cvs are as Aaron notes, dirty words. Git
rebase, not so much.

--Will

On Tuesday, 10 July 2012,  2:12 pm -0700, Aaron Stone <aa...@se...> wrote:

> In CVS, forking is a dirty word, because merging is extremely hard. In
> git, a fork is simply your workspace, ready o be pulled back to
> mainline. It's a fantastically easier workflow.
> 
> However your point is valid: if mainline flex is in CVS, patches are
> its workflow :/
> 
> On Jul 10, 2012, at 1:53 PM, Paul <pa...@pr...> wrote:
> 
> > I really would rather not effectively fork flex. I would prefer to do
> > whatever is necessary to integrate with the mainline.
> >
> > Paul
> >
> > On 07/10/2012 04:43 PM, ruertar wrote:
> >> On 07/10/2012 04:33 PM, Peter Martini wrote:
> >>> Any thoughts on converting to Git?  Sourceforge already has a
> >>> (empty) git repository for flex, and it's not that difficult to
> >>> convert the CVS repo to Git while preserving all of the history,
> >>> branches, and tags.  (I've done it locally already).
> >>>
> >> I would like this as well.
> >>
> >> ------------------------------------------------------------------------------
> >> Live Security Virtual Conference
> >> Exclusive live event will cover all the ways today's security and
> >> threat landscape has changed and how IT managers can respond. Discussions
> >> will include endpoint security, mobile security and the latest in malware
> >> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> >> _______________________________________________
> >> Flex-devel mailing list
> >> Fle...@li...
> >> https://lists.sourceforge.net/lists/listinfo/flex-devel
> >>
> >>
> >
> >
> >
> >
> > ------------------------------------------------------------------------------
> > Live Security Virtual Conference
> > Exclusive live event will cover all the ways today's security and
> > threat landscape has changed and how IT managers can respond. Discussions
> > will include endpoint security, mobile security and the latest in malware
> > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> > _______________________________________________
> > Flex-devel mailing list
> > Fle...@li...
> > https://lists.sourceforge.net/lists/listinfo/flex-devel
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel

-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

Re: [Flex-devel] Flex unicode additional files

[Will E. <wes...@gm...>]

Paul,

I'm making arrangements to get this hosted in flex provided resources.
I'll be in touch off list to communicate the details so we can set this
up and make sure it's working before we announce it as generally
available.

Thanks much for your work on this.

--Will

On Sunday, 15 July 2012,  9:24 am -0400, Paul <pa...@pr...> wrote:

> I have two additional files to send.
> One is the diff including the config & makefile changes for the tests.
> About 161kb including all flex changes.
> The second is the file containing the 60+ unicode tests
> About 519kb.
> These two files together are the complete set of changes.
> The previously sent file is strictly flex itself.
> Please advise.
> 
> Paul Neelands
> 
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and 
> threat landscape has changed and how IT managers can respond. Discussions 
> will include endpoint security, mobile security and the latest in malware 
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel

-- 
Will Estes (wl...@us...)
Flex Project Maintainer
http://flex.sourceforge.net/

[Flex-devel] Unicode diff all

[Paul <pa...@pr...>]

Attached is the compressed unicode tar.gz of all flex files except the 
actual additional unicode tests. The tests are named in the configure & 
make files.

Paul Neelands

[Flex-devel] Flex unicode additional files

[Paul <pa...@pr...>]

I have two additional files to send.
One is the diff including the config & makefile changes for the tests.
About 161kb including all flex changes.
The second is the file containing the 60+ unicode tests
About 519kb.
These two files together are the complete set of changes.
The previously sent file is strictly flex itself.
Please advise.

Paul Neelands

[Flex-devel] flex unicode diff

[Paul <pa...@pr...>]

Attached is the flex to flex-unicode diff of the 1 July 2012 flex cvs.
This diff does not include the added tests, and only includes modified 
files.
Some debugging code is included.
The added options are %option utf16, %option utf16le and %option utf16be.
%option utf16 accepts the machine native utf16 without a BOM.
%option utf16le accepts UTF16-LE regardless of the machine endedness.
%option utf16be accepts UTF16-BE regardless of the machine endedness.
The macro YY_ACCEPT_UTF16LE sets flex to accept UTF-16LE.
The macro YY_ACCEPT_UTF16BE sets flex to accept UTF-16BE.
These can be used with any of the %option utf... options.
The diff is ~33kb.

Paul Neelands

Re: [Flex-devel] %option utf16 utf16le & utf16be

[Paul <pa...@pr...>]

Yes

Paul

On 07/10/2012 04:59 PM, Peter Martini wrote:
> I'm not familiar with ARM, although it seems like it would be a good 
> idea to start looking into it now. Is that bit setting something that 
> can change at runtime?
>
> On Tue, Jul 10, 2012 at 4:50 PM, Paul <pa...@pr... 
> <mailto:pa...@pr...>> wrote:
>
>     I'm not aware of a fully general compile time check. I use flex on
>     pc's with Linux & Windows, and cross compiled to small micro's
>     (ARM) that can be either little or big endian depending on a bit
>     setting.
>
>
>     Paul
>
>
>     On 07/10/2012 03:59 PM, Peter Martini wrote:
>
>         if we do the check at compile time
>
>
>
>
>

Re: [Flex-devel] Conversion to Git?

[Aaron S. <aa...@se...>]

In CVS, forking is a dirty word, because merging is extremely hard. In
git, a fork is simply your workspace, ready o be pulled back to
mainline. It's a fantastically easier workflow.

However your point is valid: if mainline flex is in CVS, patches are
its workflow :/

On Jul 10, 2012, at 1:53 PM, Paul <pa...@pr...> wrote:

> I really would rather not effectively fork flex. I would prefer to do
> whatever is necessary to integrate with the mainline.
>
> Paul
>
> On 07/10/2012 04:43 PM, ruertar wrote:
>> On 07/10/2012 04:33 PM, Peter Martini wrote:
>>> Any thoughts on converting to Git?  Sourceforge already has a
>>> (empty) git repository for flex, and it's not that difficult to
>>> convert the CVS repo to Git while preserving all of the history,
>>> branches, and tags.  (I've done it locally already).
>>>
>> I would like this as well.
>>
>> ------------------------------------------------------------------------------
>> Live Security Virtual Conference
>> Exclusive live event will cover all the ways today's security and
>> threat landscape has changed and how IT managers can respond. Discussions
>> will include endpoint security, mobile security and the latest in malware
>> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
>> _______________________________________________
>> Flex-devel mailing list
>> Fle...@li...
>> https://lists.sourceforge.net/lists/listinfo/flex-devel
>>
>>
>
>
>
>
> ------------------------------------------------------------------------------
> Live Security Virtual Conference
> Exclusive live event will cover all the ways today's security and
> threat landscape has changed and how IT managers can respond. Discussions
> will include endpoint security, mobile security and the latest in malware
> threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
> _______________________________________________
> Flex-devel mailing list
> Fle...@li...
> https://lists.sourceforge.net/lists/listinfo/flex-devel