xmlstar-devel — XMLStarlet Toolkit Developers list

Re: [Xmlstar-devel] I can try to help

[Noam P. <npo...@us...>]

On Fri, May 25, 2012 at 1:59 AM, Алексей Штыков <if...@ma...> wrote:
> I've seen this. I will try to get some more details from author.

So it seems unlikely that we'll hear back from them. My take is this:
currently the ed subcommand can add an element node, a text node, or
an attribute node; it would make sense to allow adding a comment too.
The syntax would be just the same as the other node types:

  xml ed -i -t comment -n '' -v 'comment contents here'

Re: [Xmlstar-devel] I can try to help

[Noam P. <npo...@us...>]

On Fri, May 25, 2012 at 1:44 AM, Алексей Штыков <if...@ma...> wrote:
> Hello, Noam
> I will try to do it.
> Where I can find detailed information about this request?

Heh, well there's not too much detail, the request was this forum
post: http://sourceforge.net/projects/xmlstar/forums/forum/226076/topic/5286058

PS if it's not too much trouble, could you send XMLStarlet related to
the list (xml...@li...), it's useful to have a
public archive.

Re: [Xmlstar-devel] I can try to help

[Noam P. <npo...@us...>]

> Am 23.05.2012 um 14:30 schrieb Aleksey Shtykov:
>> I developed software similar to Yours. It was based on
>> libhml2 and allowed to validate / format XML files from
>> Windows/Solaris/BSD/Linux command line. I\'m going to
>> register my project in sourceforge, but I found Yours. I can
>> try to help You in Your project, if You wish

Sure, that would be cool. Someone just asked about adding XML
comments; I think that would a pretty simple feature to add if you're
looking for a place to start.

[Xmlstar-devel] XMLStarlet 1.3.1 Released

[Noam P. <npo...@us...>]

1.3.1: Jan 14, 2012

    - handle multiple values for --value-of properly (Bug #2563866)
    - substitute external entities (Bug #3467320)
    - pyx output needs space between attribute name and value (Bug #3440797)

Re: [Xmlstar-devel] Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]

[Ming C. <cim...@ya...>]

Yeah! That's it.
 
Thank you!
Ming
 

________________________________
 From: Noam Postavsky <npo...@us...>
To: chen ming <cim...@ya...> 
Cc: "xml...@li..." <xml...@li...> 
Sent: Thursday, January 12, 2012 11:48 AM
Subject: Re: Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]
  
"chen ming" <cim...@ya...> writes:
> I used "xml ed -d /descendant::node()/rec/child::node()[1] test.xml

Okay, I see what's happening. The behaviour depends on whether libxml's
keepBlanks option is set. xml ed has it off by default, but xml sel and
xmllint have it on. You can get the other behaviour by passing the
appropriate options:

xml ed --pf (or -P) ...
xml sel --noblanks (or -B) ...
xmllint --noblanks ...

Noam

Re: [Xmlstar-devel] Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]

[Noam P. <npo...@us...>]

"chen ming" <cim...@ya...> writes:
> I used "xml ed -d /descendant::node()/rec/child::node()[1] test.xml

Okay, I see what's happening. The behaviour depends on whether libxml's
keepBlanks option is set. xml ed has it off by default, but xml sel and
xmllint have it on. You can get the other behaviour by passing the
appropriate options:

xml ed --pf (or -P) ...
xml sel --noblanks (or -B) ...
xmllint --noblanks ...

Noam

Re: [Xmlstar-devel] Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]

[chen m. <cim...@ya...>]

I used "xml ed -d /descendant::node()/rec/child::node()[1] test.xml > result.xml
 
Thanks，
Ming
已通过MOTOBLUR™连接

-----原始信息-----
From: Noam Postavsky <npo...@us...>
To: Ming Chen <cim...@ya...>
抄送: "xml...@li..." <xml...@li...>
已发送: 2012 1月, 周二, 10 16:26:59 格林尼治标准时间+0000
主题: Re: Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]

On Tue, Jan 10, 2012 at 12:05 AM, Ming Chen <cim...@ya...> wrote:

> While my original expected outputs (which really is XmlStar and
> XPathBuilder's output) are: "<para type="error" position="1"/><para
> type="warning" position="1"/><para type="info" position="1"/>", the
> XmlLint and XPathTester give some empty lines actually.

I get blank lines from XMLStarlet as well, which is not surprising
since it's based on the same code as xmllint. What command line
arguments did you use for XMLStarlet? I used

  xml sel -t -c "/descendant::node()/rec/child::node()[1]" test.xml

Noam

Re: [Xmlstar-devel] Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]

[Noam P. <npo...@us...>]

On Tue, Jan 10, 2012 at 12:05 AM, Ming Chen <cim...@ya...> wrote:

> While my original expected outputs (which really is XmlStar and
> XPathBuilder's output) are: "<para type="error" position="1"/><para
> type="warning" position="1"/><para type="info" position="1"/>", the
> XmlLint and XPathTester give some empty lines actually.

I get blank lines from XMLStarlet as well, which is not surprising
since it's based on the same code as xmllint. What command line
arguments did you use for XMLStarlet? I used

  xml sel -t -c "/descendant::node()/rec/child::node()[1]" test.xml

Noam

[Xmlstar-devel] Should the first node be text node for xpath: /descendant::node()/rec/child::node()[1]

[Ming C. <cim...@ya...>]

Hi Noam,
 
Recently I get different outputs with those tools: XmlStar, XPathBuilder, XmlLint (win32 version, with libxml2 underneath) and XPathTester (online tool), for below XML file:
 
<?xml version="1.0" encoding="UTF-8"?>
<xml>
   <table>
      <rec id="1">
         <para type="error" position="1"/>
         <para type="error" position="2"/>
         <para type="error" position="3"/>
      </rec>
      <rec id="2">
         <para type="warning" position="1"/>
         <para type="warning" position="2"/>
         <para type="warning" position="3"/>
      </rec>
      <rec id="3">
         <para type="info" position="1"/>
         <para type="info" position="2"/>
         <para type="info" position="3"/>
      </rec>
   </table>
</xml>
 
While my original expected outputs (which really is XmlStar and XPathBuilder's output) are: "<para type="error" position="1"/><para type="warning" position="1"/><para type="info" position="1"/>", the XmlLint and XPathTester give some empty lines actually.
 
I asked this question to the libxml2 mailing list, and got below answers:
 
From: Liam R E Quin <li...@ho...>
To: Ming Chen <cim...@ya...> 
Cc: "xm...@gn..." <xm...@gn...> 
Sent: Tuesday, January 10, 2012 11:22 AM
Subject: Re: [xml] Does not support the expression : /descendant::node()/rec/child:­:node()[1]?
 
On Mon, 2012-01-09 at 18:54 -0800, Ming Chen wrote:

> According to the XPath spec (V2.0 section 3.2.3 Unabbreviated 
> Syntax) : child::node() selects all the children of the context node. Note 
> that no attribute nodes are returned, because attributes are not 
> children.

Note, libxml2 actually only supports XPath 1, not XPath 2.

However,
/descendant::node()/rec/child::node()[1]
will match text nodes, and you're getting the blank (whitespace-only)
text node that's the first child of elements, since your input is
"indented".

> Shouldn’t it have the same output as 
> /descendant::node()/rec/child::*[1] and /descendant::node()/rec/para[1]?
No.

The first child node in
<rec id="1">
<para type="error" position="1"/>
is the newline and spaces between id="1"> and <para.


Liam

-- 
Liam Quin - XML Activity Lead, W3C, http://www.w3.org/People/Quin/
Pictures from old books: http://fromoldbooks.org/


I think that explanation is legal. What's your opinion on this? 
 
 
Thanks,
Ming

Re: [Xmlstar-devel] Line ending issue for unicode XML files

[Ming C. <cim...@ya...>]

Hi Noam,
 
Thanks for the new release, 1.3.0. It unified the line endings to 0A 00 (or 00 0A) for all UTF16 format files. It meets my current requirements although it would be better to have the 00 0D 00 0A (or 0D 00 0A 00) line ending on Windows (I understand that is controlled by the C runtime).

Best regards,
Ming


________________________________
From: Noam Postavsky <npo...@us...>
To: Ming Chen <cim...@ya...>; xml...@li...
Sent: Sunday, October 2, 2011 12:08 AM
Subject: Re: [Xmlstar-devel] Line ending issue for unicode XML files

Ming Chen <cim...@ya...> writes:

> I basically understood what has happened. So is there a good solution for this?
> I didn't get the picture of what you said about mingw and binary
> mode...

There is a semi-good solution: switch stdout to binary (as opposed to
text) mode so that the ascii carriage returns won't be added. Ideally I
would want to switch stdout to UTF16-text mode but this is only possible
with a newer version of the C runtime. 

I don't want to use this newer version because it is not guaranteed to
be installed on Windows so it would have to be packaged with XMLStarlet,
and also it's not supported by mingw (gcc
 for Windows).

Noam

Re: [Xmlstar-devel] XMLStarlet 1.3.0 Released

[Mònica R. A. <mo...@pr...>]

El dg 02 de 10 de 2011 a les 17:12 -0400, en/na Noam Postavsky va
escriure:
> 1.3.0: Oct 7, 2011
> 
>     - avoid ASCII CRs in UTF-16/32 text (reported by Ming Chen)
>     - --value-of outputs concat values of all nodes (Req #2563866)
>     - encode special chars for ed -u -x
>     - allow use of exslt functions in ed -u -x
>     - add --var to select (allow --var <name>=<value> as well as --var
>       <name> <value> --break)
>     - work around libxml bug that passes bogus data to error handler
>       (Bug #3362217)

I packaged xmlstarlet 1.3.0 for Debian. It's already in unstable
distribution. Hopefully, it'll be in testing distribution in 10 days.

http://packages.qa.debian.org/x/xmlstarlet.html

Cheers,
Mònica

[Xmlstar-devel] XMLStarlet 1.3.0 Released

[Noam P. <npo...@us...>]

1.3.0: Oct 7, 2011

    - avoid ASCII CRs in UTF-16/32 text (reported by Ming Chen)
    - --value-of outputs concat values of all nodes (Req #2563866)
    - encode special chars for ed -u -x
    - allow use of exslt functions in ed -u -x
    - add --var to select (allow --var <name>=<value> as well as --var
      <name> <value> --break)
    - work around libxml bug that passes bogus data to error handler
      (Bug #3362217)

Re: [Xmlstar-devel] Line ending issue for unicode XML files

[Noam P. <npo...@us...>]

Ming Chen <cim...@ya...> writes:

> I basically understood what has happened. So is there a good solution for this?
> I didn't get the picture of what you said about mingw and binary
> mode...

There is a semi-good solution: switch stdout to binary (as opposed to
text) mode so that the ascii carriage returns won't be added. Ideally I
would want to switch stdout to UTF16-text mode but this is only possible
with a newer version of the C runtime. 

I don't want to use this newer version because it is not guaranteed to
be installed on Windows so it would have to be packaged with XMLStarlet,
and also it's not supported by mingw (gcc for Windows).

Noam

Re: [Xmlstar-devel] Line ending issue for unicode XML files

[Noam P. <npo...@us...>]

On Mon, Sep 26, 2011 at 6:22 AM, Ming Chen <cim...@ya...> wrote:
> In all output files, the line endings have strange format. I think they should be 00 0D 00 0A or 0D 00 0A 00.
>
> I am using the latest windows version (1.2.1). My OS is Win7 64bit.
>
> Could someone help to check this?

Yes, I think I see what is happening: the libxml output routine is
writing just a newline character and expecting the c-runtime to do the
conversion to the system specific line ending. Problem is, nobody told
the c-runtime that the output is in UTF16, so it's just waiting until
it sees an ascii newline (0A) and then it adds a carriage return (0D).

Unfortunately, it seems there is no way to tell Windows to use switch
stdout to UTF16 until the VS2005 c-runtime. XMLStarlet uses mingw, so
the best I can do is change to binary mode, meaning the line endings
will be 00 0A.

[Xmlstar-devel] Line ending issue for unicode XML files

[Ming C. <cim...@ya...>]

Hi,


Recently I run the XmlStarlet tool on my Win7 OS with different Unicode XML files, and found some strange things:

1. XmlStarlet supports UTF-16BE (no BOM, encoding is UTF-16BE), with command line like "xml ed -d //d UTF-16BE.xml > d-UTF-16BE.xml", the output file has UTF-16BE format, the line endings becomes 00 0D 0A whatever the original line endings are (00 0D 00 0A, 00 0A or 00 0D).
2. XmlStarlet supports UTF-16LE (no BOM, encoding is UTF-16LE), with command line like "xml ed -d //d UTF-16LE.xml > d-UTF-16LE.xml", the output file has UTF-16LE format, the line endings becomes 0D 0A 00 whatever the original line endings are (0D 00 0A 00, 0A 00 or 0D 00).
3. XmlStarlet supports UTF-16LE-BOM (BOM FF FE, encoding is UTF-16), with command line like "xml ed -d //d UTF-16LE-BOM.xml > d-UTF-16LE-BOM.xml", the output file has UTF-16LE-BOM format, the line endings becomes 0D 0A 00 whatever the original line endings are (0D 00 0A 00, 0A 00 or 0D 00).
4. XmlStarlet supports UTF-16BE-BOM (BOM FE FF, encoding is UTF-16), with command line like "xml ed -d //d UTF-16BE-BOM.xml > d-UTF-16BE-BOM.xml", the output file has UTF-16LE-BOM format, the line endings becomes 0D 0A 00 whatever the original line endings are (00 0D 00 0A, 00 0A or 00 0D).

Please note that the case #4, in which the output file has an reversed byte order.

In all output files, the line endings have strange format. I think they should be 00 0D 00 0A or 0D 00 0A 00.

I am using the latest windows version (1.2.1). My OS is Win7 64bit.

Could someone help to check this?

Thanks,
Ming

[Xmlstar-devel] XMLStarlet 1.2.1 Released

[Noam P. <npo...@us...>]

1.2.1: July 07, 2011

    - check for NULL nodeset result (Bugs #3323189, #3323196)
    - "-" was being confused with --elif
    - generated XSLT should also have automatic namespaces
    - allow -N after other option (Bug #3325166)
    - namespace values were being registered as prefixes
    - avoid segfault when asked to move namespace nodes
    - missing newline in ed --help message
    - test scripts portability
      - no bashisms allowed in NetBSD sh
      - make BRE portable: '+' is not allowed
      - deal with msys path conversion properly (Bug #3178657)
    - don't use XML_SAVE_WSNONSIG #if libxml < 2.7.8 (Bug #3310475)

Re: [Xmlstar-devel] xmlstarlet portability problems

[Noam P. <npo...@us...>]

Matthias Drochner <M.D...@fz...> writes:

> Seems so... I don't have a good idea how to do this which
> portable shell commands or sed -- At the point "$@" is evaluated,
> the original quoting is lost.

Actually "$@" preserves tokenization, "$*" doesn't. Fixed in commit
014ed317a2e49618aeaa841dfcf4d6f3cc9478ee.

http://xmlstar.git.sourceforge.net/git/gitweb.cgi?p=xmlstar/xmlstar;a=patch;h=014ed31

>
>> P.S. please send xmlstarlet related mail to the mailing list (I've added
>> it to Cc), or use the Sourceforge website.
>
> I've tried the latter but it wanted a registration. Maybe I even
> have one, but generally I'm sick of leaving passwords everywhere
> and have them sent around in cleartext every first of the month.

Sourceforge does support OpenID, although that only helps if you're
using at least one other site that supports it...

Re: [Xmlstar-devel] xmlstarlet portability problems

[Noam P. <npo...@us...>]

Matthias Drochner <M.D...@fz...> writes:
> Hi -
> thanks for caring about xmlstarlet - I've successfully used
> it with osmarender.
> Just found two subtle dependencies on GNU sed and bash
> which make the selftests fail at least on NetBSD.
> See the patches -- the second one is somewhat brute-force;
> this should be done by someone who can test on Windows.

I've applied the first patch. The second won't quite work as is. Does
the NetBSD shell really check the syntax on statements that aren't
evaluated? If so I guess I'll have to split that part into a different
file.

thanks, Noam

P.S. please send xmlstarlet related mail to the mailing list (I've added
it to Cc), or use the Sourceforge website.

[Xmlstar-devel] XMLStarlet 1.2.0 Released

[Noam P. <npo...@us...>]

I just released version 1.2.0:

    - implement ed --update --expr
    - use top-level namespace definitions from first input file, this
      should remove the need to define namespaces on the command line
      with -N in most cases.
    - select exits with 0 only if result is non-empty (Req #3155702)
    - add -Q to select, like grep's -q
    - add column number to error messages
    - restore input context (lost in version 1.0.3) to error messages
      (Bug #3305659)
    - print extra string information in error messages
    - use entity definitions from dtd (Bug #3305659)
    - add --net option to c14n, ed, fo, and val (Req #1071398)
    - remove --catalog from tr --help message since it isn't actually supported
    - add --elif and --else to sel --help message

Noam

Re: [Xmlstar-devel] Offer to help with XMLStarlet

[Noam P. <npo...@us...>]

Rowan Thorpe <ro...@ro...> writes:

> It is not as straightforward as I first thought, for a mix of reasons
> (primarily that there aren't many up-to-date 64-bit GCC-or-ANSI-C
> compiled versions of the dependencies - zlib, iconv, libxml, libxslt,
> and some of their Win32 Makefiles are quirky, out-of-date, and 32-bit
> centric).

You can leave out zlib and iconv for now if that makes things easier.
Actually now that I think of it, I may have left zlib out of the win32
package!

> I presume that a statically compiled version is all you are looking
> for? I meandered into dll-and-autoconf-hell a few times so far
> (confusedly, being a *nix user), and might just leave dynamic linking
> out of the equation if not important.

Yeah, since we can't assume any of the dlls would be present on a
Windows system might as well statically link everything.

Re: [Xmlstar-devel] Bug#619445: Should XPath arguments interpret XML entities? (Bug 3240736)

[Mònica R. A. <mo...@pr...>]

El dt 12 de 04 de 2011 a les 01:31 +0200, en/na Vincent Lefevre va
escriure:
> On 2011-04-11 19:13:14 -0400, Noam Postavsky wrote:
> > Ah, that is a wrinkle I overlooked. Fortunately, the first command does
> > give a tab in version 1.1.0 since in that version the stylesheet xml
> > tree is built directly in memory instead of parsing it from a string.
> 
> I confirm that the problem with the tab character is solved in 1.1.0.
> 
> Perhaps the Debian bug should be retitled as:
> 
>   xmlstarlet doesn't interpret XPath arguments in a consistent way
> 
> and be closed as fixed in 1.1.0-1.

So, we can happily close the bug in both BTSs :-)

Re: [Xmlstar-devel] Should XPath arguments interpret XML entities? (Bug 3240736)

[Noam P. <npo...@us...>]

I've cc'd to the Debian BTS.

Mònica Ramírez Arceda <mo...@pr...> writes:

> The original submitter of this bug has sent an argument to Debian BTS. I
> forward our dialog: 
[...]
> XMLStarlet should be fixed in one way or the other, so that at least
> one of the following works, i.e. gives a line with "a" followed by a
> tab character followed by "b":
>
> $ echo '<root/>' | xml sel -T -t -v "'a$(printf \\t)b'" -n
> $ echo '<root/>' | xml sel -T -t -v "'a&#9;b'" -n
>
> Currently the former command gives: "a b" (with a space).

Ah, that is a wrinkle I overlooked. Fortunately, the first command does
give a tab in version 1.1.0 since in that version the stylesheet xml
tree is built directly in memory instead of parsing it from a string.

Noam

Re: [Xmlstar-devel] XMLStarlet 1.1.0 Released

[Aleksandar I. <ale...@iv...>]

On 04/05/2011 02:58 AM, Noam Postavsky wrote:
> Aleksandar Ivanisevic<ale...@iv...>  writes:
>
>> I don't see a point in me uploading EL5 rpms, except that rpmforge has a
>> patch renaming the binary from 'xml' to 'xmlstarlet'. Or is there a
>> political issue I've missed?
>
> A political issue about the binary name or about using rpmforge? Either

Both, don't know ;)

> way I don't think there is. If you, as a Redhat user, think that using
> the rpmforge packages make sense, then I guess we should do that.

Probably a note in the README or something along those lines should 
suffice, but I could continue creating packages, its really only < 10 
mins work to rebuild. I think Dag Wieers, the master of rpmforge 
personally uses xmlstarlet so it gets updated there really fast.

Re: [Xmlstar-devel] Offer to help with XMLStarlet

[Noam P. <npo...@us...>]

Dagobert Michelsen <da...@op...> writes:

> Hi Rowan,
>
> Am 04.04.2011 um 03:14 schrieb row...@us...:
>> I realised it makes more sense to just use XMLStarlet (or maybe a
>> trimmed-down version for speed reasons).

There isn't much to trim from XMLStarlet itself, the bulk of the code
and run time is in libxml.

> Sure, I'll forward your mail to xmlstar-devel@ where I would recommend
> you also subscribe to. Maybe the other developers (and notable Noam here)
> have a suggestion on how you could help?
>
>> Like yourself I have very limited and sporadic free-time, and also I am not 
>> very experienced with C (yet) as Perl/shell-scripting/PHP/etc have been 
>> my forté more, but so far I've not found C too foreign, but I'm certainly not 
>> a C guru...

The feature requests tracker has some ideas that are worth pursuing, or
since you mentioned you were working on a framework maybe you have some
cool ideas of your own. If your lack of C experience gets in the way I
am willing to help with that.

Another thing that really needs to get done is updating the
documentation. Right now we don't even have the makefile rules to
generate it.

Noam

Re: [Xmlstar-devel] XMLStarlet 1.1.0 Released

[Noam P. <npo...@us...>]

Aleksandar Ivanisevic <ale...@iv...> writes:

> I don't see a point in me uploading EL5 rpms, except that rpmforge has a 
> patch renaming the binary from 'xml' to 'xmlstarlet'. Or is there a 
> political issue I've missed?

A political issue about the binary name or about using rpmforge? Either
way I don't think there is. If you, as a Redhat user, think that using
the rpmforge packages make sense, then I guess we should do that.

Noam