re2c-general — For discussion of re2c

[re2c-general] Switch to a widely-recognized OSI and FSF approved license.

[Ulya <skv...@gm...>]

Hi re2c users and contributors!


This is a proposal to change re2c license to one of the widely
recognized, OSI and FSF approved licenses (most likely MIT). To
clarify, re2c will remain free and open source software, and this
change should not affect the users.

What is the current license?
----------------------------

Historically re2c uses a custom "public domain" license, worded as
follows:

    Re2c is in the public domain. The data structures and algorithms
    used in re2c are all either taken from documents available to the
    general public or are inventions of the author. Programs generated
    by re2c may be distributed freely. Re2c itself may be distributed
    freely, in source or binary, unchanged or modified. Distributors
    may charge whatever fees they can obtain for re2c. If you do make
    use of re2c, or incorporate it into a larger project an
    acknowledgement somewhere (documentation, research report, etc.)
    would be appreciated. Re2c is distributed with no warranty
    whatsoever. The code is certain to contain errors. Neither the
    author nor any contributor takes responsibility for any
    consequences of its use.

Why change?
-----------

It is getting increasingly difficult to use current license as a proof
that re2c is free software. There is no SPDX. Github and journals like
Software Impacts do recognize it. Personally I see nothing wrong with
public domain, but at the same time I feel that not changing the
license to an FSF and OSI approved one is the equivalent of Asimov's
"...through inaction, allow a human being to come to harm".

Do I have the right?
--------------------

I don't know. This is not my own project --- many people have
contributed to re2c over the years, and some have spent considerable
time on the project. I am the only active author: none of the other
authors get in touch (sadly, not all of them are alive by now). I have
made by far the most contributions to the project in the number of
commits, and I am the only remaining developer with commit access to
the source code repositories. I am not worried that someone will sue me
(it seems extremely unlikely), but rather that it may alert or create
problems for some re2c users.

Change to what?
---------------

There are two aspects: 1) what the license says and 2) how it is
interpreted. Personally I prefer "public domain" licenses that don't
have any requirements, like Unlicense and 0-BSD. However, they are not
as widely accepted as MIT, and not always approved by various projects
and organizations. Since the only reason to change license is to make
re2c easier to use, it makes sense to choose a less appealing, but more
default option.

- MIT. Pros: it is the most widely recognized OSI and FSF approved
    license, its text is short and readable, it is technically very
    close to the current license (unlikely to cause license troubles
    for any re2c users). Cons: it adds one requirement not present in
    the current license (copy-pasting the license), it doesn't have the
    words "public domain" in it, and it requires the year and name.

- Unlicense. Pros: it is FSF approved, it is recognized as a "public
    domain" license, it does not require copy-pasting the license text,
    and I like the wording, the name and the project goals. Cons: I
    don't thinks it's OSI approved, and some companies like Google
    exclude it from the allowed list. I'm afraid it may create more
    technical problems for re2c uses than the seemingly more
    restrictive MIT license.

- Zero-Clause BSD / Free Public License 1.0.0 (0BSD). Pros: OSI
    approved, short, recognized as public domain, no requirement to
    copy license notice. Cons: not FSF approved, less widely used.

Other alternatives were also considered, but seem to be worse than the
above: ISC is like MIT but less widely used, CC0 seems more for data
than code (and the webcite says not to use it for public domain).

I'm leaning more towards MIT, although I personally like the Unlicense
better. Please let me know if you have any concers/advice.


--
Ulya

Re: [re2c-general] re2c-2.0 released

[Stephen L. <ste...@st...>]

Thanks for your work on this!

Ulya <skv...@gm...> writes:

> Hi,
>
>
> Re2c-2.0 has been released!
>
> - Release notes: https://re2c.org/releases/release_notes.html
> - Tarballs on Github: https://github.com/skvadrik/re2c/releases/tag/2.0
>
>
> Ulya
>
>
> _______________________________________________
> Re2c-general mailing list
> Re2...@li...
> https://lists.sourceforge.net/lists/listinfo/re2c-general
>
>

-- 
-- Stephe

[re2c-general] re2c-2.0 released

[Ulya <skv...@gm...>]

Hi,


Re2c-2.0 has been released!

- Release notes: https://re2c.org/releases/release_notes.html
- Tarballs on Github: https://github.com/skvadrik/re2c/releases/tag/2.0


Ulya

Re: [re2c-general] supporting Gnu-format error messages

[Stephen L. <ste...@st...>]

Ulya Trofimovich <skv...@gm...> writes:

>> I've run into a problem.
>> 
>> In most places where options are already passed, they are passed as
>> 'const opt_t *opts'. However, that type does not contain 'source_file';
>> 'source_file' is only in the higher level Opt type.
>> 
>> So either I need to add an Opt arg to all the places that already get
>> opt_t (that seems awkward), or move source_file into conopt_t.
>> 
>> Any advice?
>
> The right way would be to move 'source_file' into 'mutopt_t': the name
> of the source file might be changed by one of the '#line' directives
> (which are handled by re2c).

Hmm. Opt.source_file is only set in parse_opts.

As far as I can tell, the #line directives are inserted into the output
file by re2c; they are not in the input .re file.

>> I'm tempted to give up; I won't be editing that many .re files, so the
>> error message format is not too annoying. 
>
> Sure, you can file an issue on github -- then I'll fix it myself.

I'll do that; you are in a better position to decide how to proceed.

-- 
-- Stephe

Re: [re2c-general] supporting Gnu-format error messages

[Ulya T. <skv...@gm...>]

> I've run into a problem.
> 
> In most places where options are already passed, they are passed as
> 'const opt_t *opts'. However, that type does not contain 'source_file';
> 'source_file' is only in the higher level Opt type.
> 
> So either I need to add an Opt arg to all the places that already get
> opt_t (that seems awkward), or move source_file into conopt_t.
> 
> Any advice?

The right way would be to move 'source_file' into 'mutopt_t': the name
of the source file might be changed by one of the '#line' directives
(which are handled by re2c).

> I'm tempted to give up; I won't be editing that many .re files, so the
> error message format is not too annoying. 

Sure, you can file an issue on github -- then I'll fix it myself.

Re: [re2c-general] supporting Gnu-format error messages

[Stephen L. <ste...@st...>]

Stephen Leake <ste...@st...> writes:

>> As for the code, your guess is correct: error reporting is scattered all
>> around the codebase and the patch would probably end up quite big. This
>> is not a problem, as long as all the changes are related to the message
>> format. If you feel that it could be broken into a series of smaller
>> patches, you are welcome.
>
> Even without a new option, we still have to pass 'opts' to the error
> message functions, since that holds the current file name.

I've run into a problem.

In most places where options are already passed, they are passed as
'const opt_t *opts'. However, that type does not contain 'source_file';
'source_file' is only in the higher level Opt type.

So either I need to add an Opt arg to all the places that already get
opt_t (that seems awkward), or move source_file into conopt_t.

Any advice?

I'm tempted to give up; I won't be editing that many .re files, so the
error message format is not too annoying.

-- 
-- Stephe

Re: [re2c-general] supporting Gnu-format error messages

[Ulya T. <skv...@gm...>]

>> - what about the errors/warnings that have no column info?
> 
> Then the column is left out. This is from the Gnu coding standards:
> https://www.gnu.org/prep/standards/standards.html#Errors

Ah, thanks. Probably worth referencing somewhere in the comments.

> Even without a new option, we still have to pass 'opts' to the error
> message functions, since that holds the current file name.

I think it is better to pass the filename only rather than the entire
'opts' (unless the message depends on other options).

> One approach would be to simply change the error message format, then
> see whether it would be better to introduce an option or change the test
> expected results.

The changes should be trivial (easily verifiable with a simple shell
script, except maybe for a couple of tests that include binary blobs).

> I've cloned from the SourceForge repository.

The primary repository is on github:

    https://github.com/skvadrik/re2c.git

Sourceforge is a mirror and may be slightly outdated (as of now it is
one commit behind github).


Ulya

Re: [re2c-general] supporting Gnu-format error messages

[Stephen L. <ste...@st...>]

Ulya Trofimovich <skv...@gm...> writes:

> Hi Stephen,
>
>
> I'm glad you find re2c useful.
>
> What you suggest sounds good to me. Only I think the option is not
> necessary: until now message format was not specified -- or at least I
> know nothing about it. 

I assumed there are tests that rely on it.

> The general rule is to make the message as clear and precise as
> possible and somehow relate it to re2c, therefore we use something
> like:
>
>     re2c: line X, column Y: error:
>
> But if you add the filename instead, then one could deduce from the
> '.re' extension (or just the filename) that the message is related to
> re2c. As I understand, you suggest that the new format will be:
>
>     filename.re: line: column: <error>:
>
> The two questions I have:
>
> - why the '<error>' in angle brackets?

That was just an oversite; they are not necessary in the actual message.

> - what about the errors/warnings that have no column info?

Then the column is left out. This is from the Gnu coding standards:
https://www.gnu.org/prep/standards/standards.html#Errors

> As for the code, your guess is correct: error reporting is scattered all
> around the codebase and the patch would probably end up quite big. This
> is not a problem, as long as all the changes are related to the message
> format. If you feel that it could be broken into a series of smaller
> patches, you are welcome.

Even without a new option, we still have to pass 'opts' to the error
message functions, since that holds the current file name.

> Don't forget about the warnings (src/conf/warn.{h,cc}).

Ok, thanks.

> Also, don't forget about the test suite. There is a couple of tests that
> check various error messages, and a lot of tests contain warnings (so
> don't be scared by the number of broken tests).

Right.

One approach would be to simply change the error message format, then
see whether it would be better to introduce an option or change the test
expected results.

> The patch should be based on top of 'master'. 

I've cloned from the SourceForge repository.

I'll make a branch for this, so it will be easy to trash and start over
if necessary.

> If you need any advice or pre-review, I would be glad to help.

Ok.

-- 
-- Stephe

Re: [re2c-general] supporting Gnu-format error messages

[Ulya T. <skv...@gm...>]

Hi Stephen,


I'm glad you find re2c useful.

What you suggest sounds good to me. Only I think the option is not
necessary: until now message format was not specified -- or at least I
know nothing about it. The general rule is to make the message as clear
and precise as possible and somehow relate it to re2c, therefore we use
something like:

    re2c: line X, column Y: error:

But if you add the filename instead, then one could deduce from the
'.re' extension (or just the filename) that the message is related to
re2c. As I understand, you suggest that the new format will be:

    filename.re: line: column: <error>:

The two questions I have:

- why the '<error>' in angle brackets?
- what about the errors/warnings that have no column info?

As for the code, your guess is correct: error reporting is scattered all
around the codebase and the patch would probably end up quite big. This
is not a problem, as long as all the changes are related to the message
format. If you feel that it could be broken into a series of smaller
patches, you are welcome.

Don't forget about the warnings (src/conf/warn.{h,cc}).

Also, don't forget about the test suite. There is a couple of tests that
check various error messages, and a lot of tests contain warnings (so
don't be scared by the number of broken tests).

The patch should be based on top of 'master'. If you need any advice or
pre-review, I would be glad to help.


Ulya

[re2c-general] supporting Gnu-format error messages

[Stephen L. <ste...@st...>]

I've just started using re2c, and I like it; the generated lexer is easy
to debug and understand, and I can even read the generator source code
:). 

I'd like to add an option to output Gnu format error messages:

file:line:column: <error>

That make it easy to have an IDE (Emacs, in my case) jump to the error
location in the source file.

Currently, the error messages don't contain the source file name; at a
minimum, that would need to be added.

To support the Gnu format, I need to do two things:

- add an option to specify the error message format

- modify the error-reporting functions to respect that option.

It seems options are defined in src/conf/opt.h, and the
error-reporting functions in src/conf/msg.h.

I haven't quite figured out how to declare a new option yet, but I'll
work on it some more.

The functions in msg.h would all need a new argument 'opts'. That same
argument might need to be added up the call chain where each of them is
used. That's a fairly invasive change, so I'm asking if it's ok to start
on this.

-- 
-- Stephe

Re: [re2c-general] [re2c-devel] Release 1.0!

[Ulya T. <skv...@gm...>]

> I think the web page http://re2c.org/install/install.html still need some adjustments.
> For example, it currently still shows "Install — re2c 0.16 documentation" as the web page title.
> In the left navigation panel, it still shows "re2c-0.16", all those can be updated to "1.0".
True! Thanks for reminding. I will update it shortly.

[re2c-general] Release 1.0!

[Ulya T. <skv...@gm...>]

After a year and a half of development, finally re2c-1.0 is ready!
As usually, I mixed up a couple of things and 1.0 was almost immediately
followed by 1.0.1. See the install page for details:

http://re2c.org/install/install.html

The main news, which explains the change of major version from 0.x to 1.x,
as well as the long gap between releases, is that re2c now supports submatch
extraction. The implementation is based on a slightly modified version of
Laurikari algorithm described in this paper:

http://re2c.org/2017_trofimovich_tagged_deterministic_finite_automata_with_lookahead.pdf 


Release notes: http://re2c.org/news/release_notes/1_0.html

As always, bug reports and comments are appreciated, and thanks to all the
people who contributed to this release!

Ulya

Re: [re2c-general] Wikipedia's article on re2c

[Ulya T. <skv...@gm...>]

Good work, thank you! I'll contribute my part as soon as I get to fixing 
documentation.

Re: [re2c-general] Wikipedia's article on re2c

[Rui M. <rui...@gm...>]

On 01/08/2017 11:09 AM, Ulya Trofimovich wrote:
> Good idea, thank you!
>
> One minor issue: I believe the title should be either "re2c" or "RE2C",
> not "Re2c" :)

Thanks for pointing that out.  It seems wikipedia imposes some technical 
restrictions on page names, which includes the forced capitalization of 
the page title's leading character.[¹]  After a bit of digging I believe 
I've found a fix.


Rui Maciel

[¹] 
https://en.wikipedia.org/wiki/Wikipedia:Naming_conventions_%28technical_restrictions%29#Lowercase_first_letter

Re: [re2c-general] Wikipedia's article on re2c

[Ulya T. <skv...@gm...>]

Good idea, thank you!

One minor issue: I believe the title should be either "re2c" or "RE2C", 
not "Re2c" :)

[re2c-general] Wikipedia's article on re2c

[Rui M. <rui...@gm...>]

I've noticed that wikipedia didn't hosted an article on re2c.
Consequenty, I've started one.

Here's a link to wikipedia's article on re2c:

https://en.wikipedia.org/wiki/Re2c


If anyone is willing to contribute anything to the article, go for it.


Rui Maciel

[re2c-general] Released re2c-0.15.2

[Ulya F. <skv...@gm...>]

Next bugfix release in 0.15x series: re2c-0.15.2 includes a fix in build 
system (release notes: http://re2c.org/news/release_notes/0_15_2.html ).

[re2c-general] Relesed re2c-0.15 and re2c-0.15.1

[Ulya F. <skv...@gm...>]

Hello everybody!

I'm glad to announce re2c-0.15 release. It's been quite a while; I tried 
hard not to break anything. You can find release notes here: 
http://re2c.org/news/release_notes/0_15.html .

Major release was immediately followed by a minor re2c-0.15.1 release, 
which includes a small fix in the testing script (locale-sensitive 
things always beat me).Release notes here: 
http://re2c.org/news/release_notes/0_15_1.html.

New site home page: http://re2c.org (note the Atom feed).

Awaiting bug reports. ;)

Ulya

Re: [re2c-general] re2c: flex syntax and parsing ambiguity

[Jakub Z. <bu...@ph...>]

On Thu, Jul 30, 2015 at 3:56 PM, Ulya Fokanova <skv...@gm...> wrote:

>  Found an easy hack to disambiguate grammar by forbidding newlines in
> flex-style named definitions. All the dirty work is done by lexer. ;)
>
> view commit
> <https://github.com/skvadrik/re2c/commit/e02b761a748833293b39fd69766f8073c0e6fe29#diff-9351147abfd7dc33744eaa3f4f30f371R199>
>
> So no need to break backwards compatibility. Thanks everyone. :)
>

Nice! ;)

Re: [re2c-general] re2c: flex syntax and parsing ambiguity

[Ulya F. <skv...@gm...>]

Found an easy hack to disambiguate grammar by forbidding newlines in 
flex-style named definitions. All the dirty work is done by lexer. ;)

view commit 
<https://github.com/skvadrik/re2c/commit/e02b761a748833293b39fd69766f8073c0e6fe29#diff-9351147abfd7dc33744eaa3f4f30f371R199>

So no need to break backwards compatibility. Thanks everyone. :)

[re2c-general] flex syntax and parsing ambiguity

[Ulya F. <skv...@gm...>]

Hello everyone,

in short:

  * Partial flex syntax support (enabled with '-F' option) causes
    ambiguity in re2c grammar and introduces parsing conflicts.
  * These conflicts are mitigated in '-c' mode, so the one major user of
    '-F' that I know of (PHP team) didn't notice them until now.
    However, they lead to severe parsing failures in normal mode.
  * There's a couple of ways to fix the problem, but all of them have
    major downsides (mostly backwards compatibility breaks).

Please read the full analyses 
<https://github.com/skvadrik/re2c/issues/115>and tell what you think.

Ulya

Re: [re2c-general] re2c.org domain name

[Ulya F. <skv...@gm...>]

I didn't realize that github allows to add binaries to an existing 
release (and customize it in general). Now that I do, a separate branch 
doesn't make sense. :)

Re: [re2c-general] re2c.org domain name

[Ulya F. <skv...@gm...>]

I tried to give your github user (dnuffer) write access to the repo by 
adding you to collaborators 
<https://help.github.com/articles/adding-collaborators-to-a-personal-repository/>. 
Could you check that you have the right permissions now?

Re: [re2c-general] re2c.org domain name

[Ulya F. <skv...@gm...>]

> Is expanding the tarball really necessary? It looks like one can just 
> attach a binary so it probably isn't necessary to mess with putting 
> all the contents of it in to the branch.
It don't think it's that messy if we make a script that does the whole 
thing. The resulting tarball looks better without an extra layer of 
packaging. And more important, some users/maintainers might have an 
installation script which relies on the structure of tarball.

I experimented a bit and here is the result 
<https://github.com/skvadrik/re2c/releases>. Those starting with 
"release-" are not important (they are automatically fabricated from 
tags in master). The single real release is '0.13.6' in (from tag 
'0.13.6' in branch release). Clicking it will lead to downloading 
're2c-0.13.6.tar.gz', which is identical to that one on sourceforge.

Not bad?
> I tried visiting re2c.org <http://re2c.org>, and it looks like the dns 
> is resolving correctly but there's something incorrect about the 
> github setup. It returns a 404 not found page. I think we need to 
> create a CNAME file as described on 
> https://help.github.com/articles/adding-a-cname-file-to-your-repository/
Yes! I'm sorry, just completely forgot about it. Fixed already. :)

[re2c-general] re2c.org domain name

[Ulya F. <skv...@gm...>]

Hello Dan,

since sourceforge has been down for a while, I think re2c should move 
somewhere.
Things to move:

 1. repository
 2. bugtracker
 3. mailing lists
 4. site
 5. tarballs

Github is ok for 1, maybe 2 and 4.
Unfortunately 're2c' username has already been taken.
I used 'skvadrik/re2c' repo as a mirror for a while, I think it will do.

However, default site URL will look like 'skvarik.github.io/re2c' -- too 
bad compared to 're2c.org'. Github allows to customize domain name 
<https://help.github.com/articles/setting-up-a-custom-domain-with-github-pages/>, 
but the new domain name must be registered on DNS servers. I know that 
you registered 're2c.org' somehow, could you possibly re-register it to 
point to the new site (provided I will create the site)?

Ulya