gawkextlib-users — Discussing user level aspects of extended gawk

Re: [Gawkextlib-users] What's the GAWK way to report errors on individual records?

[Vinícius d. S. O. <vin...@gm...>]

Em dom., 6 de jun. de 2021 às 23:15, Andrew J. Schorr <
as...@te...> escreveu:

> Are you asking how an input parser get_record function should behave?
>

Not really. Taking the extensions side aside, what would be the pure AWK
code way to handle parsing errors on individual records? I only started to
code in AWK last year, so I'm not much familiarized with AWK idioms and the
like.

[...]
>
> Do you have something else in mind besides the above behavior?
>

Just an idea. Maybe the user should fill some field in PROCINFO informing
whether he wants the input parser to silently discard broken records, to
print a warning before discarding the broken record, or to allow the user
to consume and handle the error explicitly.

In general, if an input parser cannot read
> a record, then I think it has to barf. What else could make sense?
>

For some streams it's possible to ignore just the broken record and proceed
to parsing the rest of the stream.

It might not make sense to gawkextlib's current plugins, but it does make
sense to the plugin I'm writing right now.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] What's the GAWK way to report errors on individual records?

[Manuel C. <mco...@gm...>]

Resent. The previous attempt to respond failed.

El 04/06/2021 a las 12:25, Vinícius dos Santos Oliveira escribió:
> Hi,
>
> I've been playing with GAWK extension API again and I'm wondering what's
> the proper way to report errors on individual records.
>
> If there was a problem to extract a single record on the stream, how
> should I report this to the user?

The relevant sections of the gawk manual are:

17.4.6 Printing Messages
17.4.7 Updating ERRNO

>
> I only know two GAWK extensions that play with input modes. XML where
> failing the whole stream is fine, and CSV where I just don't see errors
> on single records. Therefore I don't know where to look for current
> practice.

The gawk-csv extension reports syntax errors on a per-record basis. It 
can even give several error messages for the same input record.

$ echo 'a"b" ,"c,"x' | gawk -lcsv -v CSVMODE=1 '{1}'
gawk: error: csvinput: Unexpected quote
   a"
    ^
gawk: error: csvinput: Unexpected quote
   a"b"
      ^
gawk: error: csvinput: Unexpected character
   a"b" ,"c,"x
             ^
Regards.

-- 
Manuel Collado - http://mcollado.z15.es

Re: [Gawkextlib-users] What's the GAWK way to report errors on individual records?

[Manuel C. <mco...@gm...>]

El 04/06/2021 a las 12:25, Vinícius dos Santos Oliveira escribió:
> Hi,
>
> I've been playing with GAWK extension API again and I'm wondering what's
> the proper way to report errors on individual records.
>
> If there was a problem to extract a single record on the stream, how
> should I report this to the user?

The relevant sections of the gawk manual are:

17.4.6 Printing Messages
17.4.7 Updating ERRNO

>
> I only know two GAWK extensions that play with input modes. XML where
> failing the whole stream is fine, and CSV where I just don't see errors
> on single records. Therefore I don't know where to look for current
> practice.

The gawk-csv extension reports syntax errors on a per-record basis. It 
can even give several error messages for the same input record.

$ echo 'a"b" ,"c,"x' | gawk -lcsv -v CSVMODE=1 '{1}'
gawk: error: csvinput: Unexpected quote
   a"
    ^
gawk: error: csvinput: Unexpected quote
   a"b"
      ^
gawk: error: csvinput: Unexpected character
   a"b" ,"c,"x
             ^
Regards.

-- 
Manuel Collado - http://mcollado.z15.es

Re: [Gawkextlib-users] What's the GAWK way to report errors on individual records?

[Andrew J. S. <as...@te...>]

Hi,

On Fri, Jun 04, 2021 at 07:25:35AM -0300, Vinícius dos Santos Oliveira wrote:
> I've been playing with GAWK extension API again and I'm wondering what's the
> proper way to report errors on individual records.

Are you asking how an input parser get_record function should behave?
>From gawkapi.h:

         * If an error does occur, the function should return EOF and set
         * *errcode to a positive value.  In that case, if *errcode is greater
         * than zero, gawk will automatically update the ERRNO variable based
         * on the value of *errcode (e.g., setting *errcode = errno should do
         * the right thing).

> If there was a problem to extract a single record on the stream, how should I
> report this to the user?

Do you have something else in mind besides the above behavior?

> I only know two GAWK extensions that play with input modes. XML where failing
> the whole stream is fine, and CSV where I just don't see errors on single
> records. Therefore I don't know where to look for current practice.

The CSV extension is not really in production, and I think we are leaning
towards a pure gawk solution. In general, if an input parser cannot read
a record, then I think it has to barf. What else could make sense?

Regards,
Andy

[Gawkextlib-users] What's the GAWK way to report errors on individual records?

[Vinícius d. S. O. <vin...@gm...>]

Hi,

I've been playing with GAWK extension API again and I'm wondering what's
the proper way to report errors on individual records.

If there was a problem to extract a single record on the stream, how should
I report this to the user?

I only know two GAWK extensions that play with input modes. XML where
failing the whole stream is fine, and CSV where I just don't see errors on
single records. Therefore I don't know where to look for current practice.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

[Gawkextlib-users] unsubscribe

[Didier C. <did...@un...>]

unsubscribe

-- 
Didier COURTAUD
Professeur
UFR Sciences Fondamentales et Appliquées
+33 1 64 85 34 44
did...@un... <mailto:did...@un...>
Institut de Biologie Génétique et Bio-Informatique (IBGBI)
23, Boulevard de France 91037 Evry Cedex - France
www.univ-evry.fr <http://www.univ-evry.fr>
Rejoignez-nous sur twitter @UnivEvry <https://twitter.com/UnivEvry> et 
Facebook ! 
<https://www.facebook.com/pages/Universit%C3%A9-dEvry-Val-dEssonne/309222508983>Toutes 
nos vidéos sur notre webTV <http://www.webtv.univ-evry.fr/>

[Gawkextlib-users] Fwd: Re: CSVQUOTE in gawk extension CSV

[Manuel C. <mco...@gm...>]

-------- Mensaje reenviado --------
Asunto: Re: CSVQUOTE in gawk extension CSV
Fecha: Tue, 6 Oct 2020 12:51:04 +0200
De: Manuel Collado <mco...@gm...>
Para: Koch, Wilfried <wk...@dx...>, m-c...@us... 
<m-c...@us...>

El 05/10/2020 a las 13:19, Koch, Wilfried escribió:
> Oh, I see,
>
> the CSVQUOTE is only applied, if a separator is part of the field, or
> part of an array element in case of csvcompose.
>
> That makes sense.
>
> BUT SQLs needs the quotes in any case,
>
> Some or many csv files/filereaders contain/need unconditional quotes.
>
> Do I have to quote myself in gawk?

The feature you need is in my TODO list, but never implemented. I'll try
to add it to the CSV extension, but it may take some time. I've not
touched the code for several years.

As a quick fix, you can try to remove the conditional in the csvquote()
function, in the "csv.awk" source code. No need to recompile, just patch
it in the installed library.

-----------
# Quote a CSV field value, if necessary
function csvquote(str, comma, quote) {
     comma = comma ? comma : (CSVCOMMA ? CSVCOMMA : ",")
     quote = quote ? quote : (CSVQUOTE ? CSVQUOTE : "\"")    #"
##    if (index(str, comma) || index(str, quote) || index(str, "\n")) {
         gsub(quote, quote quote, str)
         str = quote str quote
##    }
     return str
}
-----------

I've not tested it yet. Please let me know if it works.

Regards.
-- 
Manuel Collado - http://mcollado.z15.es

[Gawkextlib-users] Fwd: RE: CSVQUOTE in gawk extension CSV

[Manuel C. <mco...@gm...>]

-------- Mensaje reenviado --------
Asunto: 	RE: CSVQUOTE in gawk extension CSV
Fecha: 	Mon, 5 Oct 2020 11:19:01 +0000
De: 	Koch, Wilfried <wk...@dx...>
Para: 	m-c...@us... <m-c...@us...>



Oh, I see,

the CSVQUOTE is only applied, if a separator is part of the field, or
part of an array element in case of csvcompose.

That makes sense.

BUT SQLs needs the quotes in any case,

Some or many csv files/filereaders contain/need unconditional quotes.

Do I have to quote myself in gawk?

*From:*Koch, Wilfried
*Sent:* Monday, October 5, 2020 12:58 PM
*To:* 'm-c...@us...' <m-c...@us...>
*Subject:* CSVQUOTE in gawk extension CSV

Hi Manuel ,

I found out, that your extension is/would be extremely helpful

for re/generating SQL lists, not just csv files.

But for neither for the one nor for the other

I can get this CSVQUOTE working.

$ echo "" | gawk -i CSV  -v CSVQUOTE="\047" '{CSVQUOTE="!"; print "where
(name,forename) in (" csvformat("hugo smith","" , "",  "@") ")" ;print
CSVQUOTE}'

where (name,forename) in (hugo,smith)

!

I apply form outside, define it in the loop and sip it as parameter for
the csvformat() call.

At the end gawk has understood the  assignment in the loop,

But csvformat does not apply any of them.

I expect

where (name,forename) in (@hugo@,@smith@).

---------------------

$ echo "" | gawk -i CSV  -v CSVQUOTE="\047" '{CSVQUOTE="!"; print "where
(name,forename) in (" csvformat("hugo smith") ")" ;print CSVQUOTE}'

where (name,forename) in (hugo,smith)

!

I expect

where (name,forename) in (!hugo!,!smith!).

--------------------- now the one wich I would use for SQL eventually

$ echo "" | gawk -i CSV  -v CSVQUOTE="'" '{print "where (name,forename)
in (" csvformat("hugo smith") ")" ;print CSVQUOTE}'

where (name,forename) in (hugo,smith)

'

I expect

where (name,forename) in ('hugo','smith').

---------------------- and even the internal one (")  is not applied

$ echo "" | gawk -i CSV  '{print "where (name,forename) in ("
csvformat("hugo smith") ")" ;print CSVQUOTE}'

where (name,forename) in (hugo,smith)

"

I expect

where (name,forename) in ("hugo","smith")


DXC Technology Deutschland GmbH: Alfred-Herrhausen-Allee 3-5, 65760
Eschborn, Germany - Board of Directors: Joachim Löffler, Karl Anzboeck,
Claus Schünemann - Registered in Frankfurt: HRB 116802.
DXC Technology Company -- This message is transmitted to you by or on
behalf of DXC Technology Company or one of its affiliates. It is
intended exclusively for the addressee. The substance of this message,
along with any attachments, may contain proprietary, confidential or
privileged information or information that is otherwise legally exempt
from disclosure. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient of
this message, you are not authorized to read, print, retain, copy or
disseminate any part of this message. If you have received this message
in error, please destroy and delete all copies and notify the sender by
return e-mail. Regardless of content, this e-mail shall not operate to
bind DXC Technology Company or any of its affiliates to any order or
other contract unless pursuant to explicit written agreement or
government initiative expressly permitting the use of e-mail for such
purpose.

[Gawkextlib-users] Fwd: CSVQUOTE in gawk extension CSV

[Manuel C. <mco...@gm...>]

-------- Mensaje reenviado --------
Asunto: 	CSVQUOTE in gawk extension CSV
Fecha: 	Mon, 5 Oct 2020 10:58:00 +0000
De: 	Koch, Wilfried <wk...@dx...>
Para: 	m-c...@us... <m-c...@us...>



Hi Manuel ,

I found out, that your extension is/would be extremely helpful

for re/generating SQL lists, not just csv files.

But for neither for the one nor for the other

I can get this CSVQUOTE working.

$ echo "" | gawk -i CSV  -v CSVQUOTE="\047" '{CSVQUOTE="!"; print "where
(name,forename) in (" csvformat("hugo smith","" , "",  "@") ")" ;print
CSVQUOTE}'

where (name,forename) in (hugo,smith)

!

I apply form outside, define it in the loop and sip it as parameter for
the csvformat() call.

At the end gawk has understood the  assignment in the loop,

But csvformat does not apply any of them.

I expect

where (name,forename) in (@hugo@,@smith@).

---------------------

$ echo "" | gawk -i CSV  -v CSVQUOTE="\047" '{CSVQUOTE="!"; print "where
(name,forename) in (" csvformat("hugo smith") ")" ;print CSVQUOTE}'

where (name,forename) in (hugo,smith)

!

I expect

where (name,forename) in (!hugo!,!smith!).

--------------------- now the one wich I would use for SQL eventually

$ echo "" | gawk -i CSV  -v CSVQUOTE="'" '{print "where (name,forename)
in (" csvformat("hugo smith") ")" ;print CSVQUOTE}'

where (name,forename) in (hugo,smith)

'

I expect

where (name,forename) in ('hugo','smith').

---------------------- and even the internal one (")  is not applied

$ echo "" | gawk -i CSV  '{print "where (name,forename) in ("
csvformat("hugo smith") ")" ;print CSVQUOTE}'

where (name,forename) in (hugo,smith)

"

I expect

where (name,forename) in ("hugo","smith")


DXC Technology Deutschland GmbH: Alfred-Herrhausen-Allee 3-5, 65760
Eschborn, Germany - Board of Directors: Joachim Löffler, Karl Anzboeck,
Claus Schünemann - Registered in Frankfurt: HRB 116802.
DXC Technology Company -- This message is transmitted to you by or on
behalf of DXC Technology Company or one of its affiliates. It is
intended exclusively for the addressee. The substance of this message,
along with any attachments, may contain proprietary, confidential or
privileged information or information that is otherwise legally exempt
from disclosure. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient of
this message, you are not authorized to read, print, retain, copy or
disseminate any part of this message. If you have received this message
in error, please destroy and delete all copies and notify the sender by
return e-mail. Regardless of content, this e-mail shall not operate to
bind DXC Technology Company or any of its affiliates to any order or
other contract unless pursuant to explicit written agreement or
government initiative expressly permitting the use of e-mail for such
purpose.

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Em qui., 10 de set. de 2020 às 10:39, Andrew J. Schorr <
as...@te...> escreveu:

> FYI, Fedora will update to the new version 2.0.3 in Fedora 33. It had
> gotten stale because the project moved, so the automatic upstream
> monitoring
> system failed to give an alert about the new version. They have now updated
> the URL based on my bug report.
>

Thanks for the heads up. I also use re2c in other projects and it'd also be
a problem for them.

Now that I think about it, re2c is similar to AWK in its modus operandi.
You configure pattern -> action pairs. The example I gave earlier already
illustrates this idea, so I won't bother copy'past'ing a new example.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

On Tue, Sep 08, 2020 at 07:00:16PM -0300, Vinícius dos Santos Oliveira wrote:
> Em ter., 8 de set. de 2020 às 18:32, Andrew J. Schorr <
> as...@te...> escreveu:
> 
>     I tried that, but now I get a different error instead:
> 
>     [...]
> 
>     Fedora usually has the latest versions. I'm surprised about re2c; 1.1.1
>     is pretty old at this point. I opened a Fedora bug:
>        https://bugzilla.redhat.com/show_bug.cgi?id=1877125
> 
> 
> I have pushed a workaround (a pregenerated re2c output) to a different branch
> (branch tmp) if you want to try before Fedora upgrades the re2c package.

FYI, Fedora will update to the new version 2.0.3 in Fedora 33. It had
gotten stale because the project moved, so the automatic upstream monitoring
system failed to give an alert about the new version. They have now updated
the URL based on my bug report.

Regards,
Andy

Re: [Gawkextlib-users] Announcing a new JSON extension

[Galen T. <gta...@ya...>]

> I think that comp.lang.awk has a much bigger audience where you're more likely
> to get feedback. There are very few people on this list -- I'd guess 5 or so.
> Certainly not more than 10. For a real discussion, comp.lang.awk is a better
> place to go. Of course, I can't guarantee a high quality discussion on
> comp.lang.awk. :-)
> 
Seconded. Even there, the signal-to-noise ratio isn’t terrible, and the “noise” (if it’s fair to call it that) mostly comes from inexperienced users.
> Regards,
> Andy
> 
> 
> _______________________________________________
> Gawkextlib-users mailing list
> Gaw...@li...
> https://lists.sourceforge.net/lists/listinfo/gawkextlib-users

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

On Wed, Sep 09, 2020 at 09:53:29AM -0300, Vinícius dos Santos Oliveira wrote:
> It seemed to me this would be the best place to discuss with other plug-in
> writers for gawk. Maybe it was a misjudgement from my part. I'm actually new to
> AWK. I learnt it earlier this year to write a tester script for another project
> of mine.

It's not a big problem, of course, but this list sees very little activity
and has only a few members. In my view, it's for discussion of gawkextlib
projects, not gawk extension libraries in general, but I'm open to other
points of view. 

> Is it okay to discuss plug-in ideas here or should I move to another mailing
> list?

I think that comp.lang.awk has a much bigger audience where you're more likely
to get feedback. There are very few people on this list -- I'd guess 5 or so.
Certainly not more than 10. For a real discussion, comp.lang.awk is a better
place to go. Of course, I can't guarantee a high quality discussion on
comp.lang.awk. :-)

Regards,
Andy

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Em qua., 9 de set. de 2020 às 09:35, Andrew J. Schorr <
as...@te...> escreveu:

> Which license are you using? There doesn't seem to be any mention of a
> license
> in your project.
>

I don't have much time available until the end of the year, so I don't want
to get distracted while I can dedicate some time to it -- coding, I mean --
for a few more days. As far as it concerns me, putting a license file is a
distraction, but a distraction that doesn't demand a focused state of my
mind, so I can do it between tasks later this year.

But the license will be Boost Software License which is similar to MIT. As
stated earlier, it's the same license from the JSON library that I use in
the project.

That's all fine, but if you would not like to contribute this project
> to gawkextlib, why did you post your message to the gawkextlib mailing
> list instead of to the broader community in comp.lang.awk? This mailing
> list is for gawkextlib projects, whereas comp.lang.awk is a better place
> to reach more gawk users who may be interested in various extensions.
>

It seemed to me this would be the best place to discuss with other plug-in
writers for gawk. Maybe it was a misjudgement from my part. I'm actually
new to AWK. I learnt it earlier this year to write a tester script for
another project of mine.

Is it okay to discuss plug-in ideas here or should I move to another
mailing list?


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

On Wed, Sep 09, 2020 at 04:52:48AM -0300, Vinícius dos Santos Oliveira wrote:
> Em ter., 8 de set. de 2020 às 22:33, Andrew J. Schorr <
> as...@te...> escreveu:
> 
>     No worries. Overall, I'd say that there are some packaging challenges here.
>     Never before have I felt such love for autoconf, automake, etc. Would you
>     consider converting it to the standard toolchain and contributing it to
>     the gawkextlib project?
> 
> 
> I thought about contributing to the gawkextlib, but after a few days I realized
> I'd like to keep it under a more permissive license and be sure to adopt the
> least risk approach to leave it in a separate repo.

Which license are you using? There doesn't seem to be any mention of a license
in your project.

> The reason I'd like to keep under a permissive license is that I want to make
> it easier to port algorithms back and forth between the JSON library I use and
> the plug-in.

That's all fine, but if you would not like to contribute this project
to gawkextlib, why did you post your message to the gawkextlib mailing
list instead of to the broader community in comp.lang.awk? This mailing
list is for gawkextlib projects, whereas comp.lang.awk is a better place
to reach more gawk users who may be interested in various extensions.

> Anyway, I'm really impressed on how quick I was able to create a plug-in for
> gawk knowing nothing beforehand. I wasn't expecting an API that was this
> documented and easy-to-use. It was a pleasant surprise and I'm satisfied. I
> intend to create other plug-ins in the future after this.

You're welcome. I'm not sure it's perfect yet, but we did spend quite
a bit of time on it.

Regards,
Andy

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Em ter., 8 de set. de 2020 às 22:33, Andrew J. Schorr <
as...@te...> escreveu:

> No worries. Overall, I'd say that there are some packaging challenges here.
> Never before have I felt such love for autoconf, automake, etc. Would you
> consider converting it to the standard toolchain and contributing it to
> the gawkextlib project?
>

I thought about contributing to the gawkextlib, but after a few days I
realized I'd like to keep it under a more permissive license and be sure to
adopt the least risk approach to leave it in a separate repo.

The reason I'd like to keep under a permissive license is that I want to
make it easier to port algorithms back and forth between the JSON library I
use and the plug-in.

re2c is not that strange of a project. It is used in projects such as PHP,
ninja-build (which in turn is used in chrome), yasm, and others. I can
spend some time reviewing the flags I use when I spare some time to write
the README file, but this effort would require me to at least know the
flags accepted in re2c's old versions. Maybe a better approach would be to
create release tarballs for which re2c has already run, but I don't think
the project is ready for tagged releases before I properly add support for
gawk-compatible multibyte locale-dependent encoding.

Using re2c, I can write rules such as:

* {
    if (*begin != '\0')
        return bad_path();
    jpat[i].components.emplace_back(std::move(current_component));
    goto end;
}
{prefix} {
    jpat[i].components.emplace_back(std::move(current_component));
    current_component = std::string{};
    goto loop;
}
{unescaped} {
    current_component.append(begin, YYCURSOR - begin);
    goto loop;
}
{escaped_tilde} {
    current_component.push_back('~');
    goto loop;
}
{escaped_slash} {
    current_component.push_back('/');
    goto loop;
}

And re2c will generate an efficient state machine to parse that. I could
try to use Boost.Xpressive to create the same state machine, but as far as
I'm aware, re2c is still faster:


   - https://lists.boost.org/boost-users/2006/04/18794.php
   -
   https://www.boost.org/doc/libs/1_73_0/doc/html/xpressive/appendices.html#boost_xpressive.appendices.appendix_2__not_yet_implemented


Anyway, I'm really impressed on how quick I was able to create a plug-in
for gawk knowing nothing beforehand. I wasn't expecting an API that was
this documented and easy-to-use. It was a pleasant surprise and I'm
satisfied. I intend to create other plug-ins in the future after this.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

No worries. Overall, I'd say that there are some packaging challenges here.
Never before have I felt such love for autoconf, automake, etc. Would you
consider converting it to the standard toolchain and contributing it to
the gawkextlib project?

Regards,
Andy

On Tue, Sep 08, 2020 at 07:17:01PM -0300, Vinícius dos Santos Oliveira wrote:
> Sorry, I forgot to include what to do with the git commands.
> 
> If you already cloned the repo, you'll need to sync again to get the new
> branch:
> 
> $ git fetch --all
> 
> Once you have a clone with the new branch, you just run this:
> 
> $ git checkout tmp
> 
> Em ter., 8 de set. de 2020 às 19:00, Vinícius dos Santos Oliveira <
> vin...@gm...> escreveu:
> 
>     Em ter., 8 de set. de 2020 às 18:32, Andrew J. Schorr <
>     as...@te...> escreveu:
> 
>         I tried that, but now I get a different error instead:
> 
>         [...]
> 
>         Fedora usually has the latest versions. I'm surprised about re2c; 1.1.1
>         is pretty old at this point. I opened a Fedora bug:
>            https://bugzilla.redhat.com/show_bug.cgi?id=1877125
> 
> 
>     I have pushed a workaround (a pregenerated re2c output) to a different
>     branch (branch tmp) if you want to try before Fedora upgrades the re2c
>     package.
> 
> 
>     --
>     Vinícius dos Santos Oliveira
>     https://vinipsmaker.github.io/
> 
> 
> 
> --
> Vinícius dos Santos Oliveira
> https://vinipsmaker.github.io/


> _______________________________________________
> Gawkextlib-users mailing list
> Gaw...@li...
> https://lists.sourceforge.net/lists/listinfo/gawkextlib-users


-- 
Andrew Schorr                      e-mail: as...@te...
Telemetry Investments, L.L.C.      phone:  917-305-1748
152 W 36th St, #402                fax:    212-425-5550
New York, NY 10018-8765

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Sorry, I forgot to include what to do with the git commands.

If you already cloned the repo, you'll need to sync again to get the new
branch:

$ git fetch --all

Once you have a clone with the new branch, you just run this:

$ git checkout tmp

Em ter., 8 de set. de 2020 às 19:00, Vinícius dos Santos Oliveira <
vin...@gm...> escreveu:

> Em ter., 8 de set. de 2020 às 18:32, Andrew J. Schorr <
> as...@te...> escreveu:
>
>> I tried that, but now I get a different error instead:
>>
>> [...]
>>
>> Fedora usually has the latest versions. I'm surprised about re2c; 1.1.1
>> is pretty old at this point. I opened a Fedora bug:
>>    https://bugzilla.redhat.com/show_bug.cgi?id=1877125
>>
>
> I have pushed a workaround (a pregenerated re2c output) to a different
> branch (branch tmp) if you want to try before Fedora upgrades the re2c
> package.
>
>
> --
> Vinícius dos Santos Oliveira
> https://vinipsmaker.github.io/
>


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Em ter., 8 de set. de 2020 às 18:32, Andrew J. Schorr <
as...@te...> escreveu:

> I tried that, but now I get a different error instead:
>
> [...]
>
> Fedora usually has the latest versions. I'm surprised about re2c; 1.1.1
> is pretty old at this point. I opened a Fedora bug:
>    https://bugzilla.redhat.com/show_bug.cgi?id=1877125
>

I have pushed a workaround (a pregenerated re2c output) to a different
branch (branch tmp) if you want to try before Fedora upgrades the re2c
package.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

On Tue, Sep 08, 2020 at 04:54:23PM -0300, Vinícius dos Santos Oliveira wrote:
> Em ter., 8 de set. de 2020 às 16:35, Andrew J. Schorr <
> as...@te...> escreveu:
> 
>     [1/5] Generating 'jsonstream.so.0.1.0.p/jpat.cpp'.
>     FAILED: jsonstream.so.0.1.0.p/jpat.cpp
>     /usr/bin/re2c -W --flex-syntax --case-ranges --empty-class match-none
>     --input-encoding ascii ../src/jpat.ypp -o jsonstream.so.0.1.0.p/jpat.cpp
>     re2c: error: bad long option: --case-ranges
> 
> 
> I have re2c 2.0.3 on my system. I didn't know --case-ranges were an option
> exclusive to the newer version.
> 
> But this flag is not important. I have submitted a new commit that just removes
> this line from meson.build file. Could you try again?

I tried that, but now I get a different error instead:

bash$ ninja
[1/5] Generating 'jsonstream.so.0.1.0.p/jpat.cpp'.
FAILED: jsonstream.so.0.1.0.p/jpat.cpp 
/usr/bin/re2c -W --flex-syntax --empty-class match-none --input-encoding ascii ../src/jpat.ypp -o jsonstream.so.0.1.0.p/jpat.cpp
re2c: error: bad long option: --input-encoding
[2/5] Precompiling header ../src/pchheader.hpp
ninja: build stopped: subcommand failed.

Fedora usually has the latest versions. I'm surprised about re2c; 1.1.1
is pretty old at this point. I opened a Fedora bug:
   https://bugzilla.redhat.com/show_bug.cgi?id=1877125

Regards,
Andy

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Em ter., 8 de set. de 2020 às 17:30, Jürgen Kahrs via Gawkextlib-users <
gaw...@li...> escreveu:

> I just tried again with your latest patches.
> Looks like the "charconv" header that you use is only
> available with C++17 and later
>

Yup, that's true. Could you try to build with clang?

$ mkdir build
$ cd build
$ CXX=clang++ CC=clang meson ..
$ ninja

The project does build with GCC, but the GCC I have installed here is
10.2.0. clang is faster to catch up with recent standards (so you don't
need the latest clang to build the project) and distros also usually ship
newer clang versions faster.

There are other C++17 features that I use, so getting rid of <charconv>
will only result in another problem somewhere else.


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[<jue...@go...>]

I just tried again with your latest patches.
Looks like the "charconv" header that you use is only
available with C++17 and later:

    c++ -Ijsonstream@sha -I. -I.. -I../include -I../3rd/trial.protocol/include -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -std=c++17 -O0 -g -fPIC  -MD -MQ 'jsonstream@sha/pchheader.hpp.gch' -MF 'jsonstream@sha/pchheader.hpp.gch.d' -o 'jsonstream@sha/pchheader.hpp.gch' -c ../src/pchheader.hpp
    ../src/pchheader.hpp:6:10: fatal error: charconv: No such file or directory
     #include <charconv>
              ^~~~~~~~~~
    compilation terminated.
    ninja: build stopped: subcommand failed.


  https://en.cppreference.com/w/cpp/header/charconv

My gcc 7.5 seems to not know charconv.


> Em ter., 8 de set. de 2020 às 16:35, Andrew J. Schorr <as...@te... <mailto:as...@te...>> escreveu:
>
>     [1/5] Generating 'jsonstream.so.0.1.0.p/jpat.cpp'.
>     FAILED: jsonstream.so.0.1.0.p/jpat.cpp
>     /usr/bin/re2c -W --flex-syntax --case-ranges --empty-class match-none --input-encoding ascii ../src/jpat.ypp -o jsonstream.so.0.1.0.p/jpat.cpp
>     re2c: error: bad long option: --case-ranges
>
>
> I have re2c 2.0.3 on my system. I didn't know --case-ranges were an option exclusive to the newer version.
>
> But this flag is not important. I have submitted a new commit that just removes this line from meson.build file. Could you try again?
>
>
> -- 
> Vinícius dos Santos Oliveira
> https://vinipsmaker.github.io/
>
>
> _______________________________________________
> Gawkextlib-users mailing list
> Gaw...@li...
> https://lists.sourceforge.net/lists/listinfo/gawkextlib-users

Re: [Gawkextlib-users] Announcing a new JSON extension

[Vinícius d. S. O. <vin...@gm...>]

Em ter., 8 de set. de 2020 às 16:35, Andrew J. Schorr <
as...@te...> escreveu:

> [1/5] Generating 'jsonstream.so.0.1.0.p/jpat.cpp'.
> FAILED: jsonstream.so.0.1.0.p/jpat.cpp
> /usr/bin/re2c -W --flex-syntax --case-ranges --empty-class match-none
> --input-encoding ascii ../src/jpat.ypp -o jsonstream.so.0.1.0.p/jpat.cpp
> re2c: error: bad long option: --case-ranges
>

I have re2c 2.0.3 on my system. I didn't know --case-ranges were an option
exclusive to the newer version.

But this flag is not important. I have submitted a new commit that just
removes this line from meson.build file. Could you try again?


-- 
Vinícius dos Santos Oliveira
https://vinipsmaker.github.io/

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

On Tue, Sep 08, 2020 at 04:31:04PM -0300, Vinícius dos Santos Oliveira wrote:
> This is true. Meson is similar to cmake/autotools. After it runs, it generates
> a build.ninja that can be used to run ninja and compile the project. So
> typically these are the steps you have to run:
> 
> $ mkdir build
> $ cd build
> $ meson ..
> $ ninja
> 
> This will generate a jsonstream.so file in the build dir that can be loaded in
> gawk through -l jsonstream (if you also set the AWKLIBPATH env var). I'll be
> renaming the plug-in to tabjson later on.

Thanks. This is helpful, but I have the same problem:

bash$ mkdir build
bash$ cd build/
bash$ meson ..
The Meson build system
Version: 0.55.1
Source dir: /home/users/schorr/gawk-tabjson
Build dir: /home/users/schorr/gawk-tabjson/build
Build type: native build
Project name: gawk-jsonstream
Project version: undefined
C++ compiler for the host machine: c++ (gcc 10.2.1 "c++ (GCC) 10.2.1 20200723 (Red Hat 10.2.1-1)")
C++ linker for the host machine: c++ ld.bfd 2.34-4
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (1.6.3)
Run-time dependency Boost found: YES 1.69.0 (/usr)
Program re2c found: YES
Build targets in project: 1

Found ninja-1.10.1 at /usr/bin/ninja
bash$ ninja
[1/5] Generating 'jsonstream.so.0.1.0.p/jpat.cpp'.
FAILED: jsonstream.so.0.1.0.p/jpat.cpp 
/usr/bin/re2c -W --flex-syntax --case-ranges --empty-class match-none --input-encoding ascii ../src/jpat.ypp -o jsonstream.so.0.1.0.p/jpat.cpp
re2c: error: bad long option: --case-ranges
[2/5] Precompiling header ../src/pchheader.hpp
ninja: build stopped: subcommand failed.

Regards,
Andy

Re: [Gawkextlib-users] Announcing a new JSON extension

[Andrew J. S. <as...@te...>]

On Tue, Sep 08, 2020 at 04:25:15PM -0300, Vinícius dos Santos Oliveira wrote:
> Please run:
> 
> $ git submodule update --init
> 
> It'll download submodule dependencies (just one; the JSON library that I use in
> the project).

I did that. Now "meson builddir" succeeds. But there's
trouble after that:

bash$ git submodule update --init
Submodule '3rd/trial.protocol' (https://github.com/breese/trial.protocol.git) registered for path '3rd/trial.protocol'
Cloning into '/home/users/schorr/gawk-tabjson/3rd/trial.protocol'...
Submodule path '3rd/trial.protocol': checked out '17e606b0a193cd2db9f44b3cc35d43cfb91cdb89'
bash$ meson builddir
The Meson build system
Version: 0.55.1
Source dir: /home/users/schorr/gawk-tabjson
Build dir: /home/users/schorr/gawk-tabjson/builddir
Build type: native build
Project name: gawk-jsonstream
Project version: undefined
C++ compiler for the host machine: c++ (gcc 10.2.1 "c++ (GCC) 10.2.1 20200723 (Red Hat 10.2.1-1)")
C++ linker for the host machine: c++ ld.bfd 2.34-4
Host machine cpu family: x86_64
Host machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (1.6.3)
Run-time dependency Boost found: YES 1.69.0 (/usr)
Program re2c found: YES
Build targets in project: 1

Found ninja-1.10.1 at /usr/bin/ninja
bash$ cd builddir/
bash$ meson compile
Found runner: /usr/bin/ninja
ninja: Entering directory `.'
[1/5] Generating 'jsonstream.so.0.1.0.p/jpat.cpp'.
FAILED: jsonstream.so.0.1.0.p/jpat.cpp 
/usr/bin/re2c -W --flex-syntax --case-ranges --empty-class match-none --input-encoding ascii ../src/jpat.ypp -o jsonstream.so.0.1.0.p/jpat.cpp
re2c: error: bad long option: --case-ranges
[2/5] Precompiling header ../src/pchheader.hpp
ninja: build stopped: subcommand failed.

I have re2c version 1.1.1 on my system:

bash$ rpm -q re2c
re2c-1.1.1-4.fc32.x86_64

Regards,
Andy