libxmlplusplus-general — Open discusions about libXML++

Re: [libxml++] ISO-8859-2 enocdings

[Jonathan W. <co...@co...>]

On Tue, Dec 02, 2003 at 04:55:07PM +0100, Rainer Stransky wrote:

> I use libxml++ in a application dealing with iso-8859-2.
> 
> When creating a xmlpp::Document including special characaters (valid within 
> iso8859-2) the method-call document.write_to_string("iso-8859-2") (see code 
> below) puts a error message to stdout:
>   output conversion failed due to conv error
>   Bytes: 0xB5 0x72 0x61 0x69
> 
> and the resulting xml-string is corrupted (truncated at the postion of the 
> special character (Dec 174)
> 
> What is the problem ? Do i use libxml++ in a wrong manner ?

I can't tell from your code snippet, but I suspect the problem is that
you added ISO-8859-2 characters to the xmlpp::Document (using
set_child_content() or similar).

libxml2 (and therefore libxml++) uses UTF-8 as its internal encoding, so
any data you read from or write to an xmlDoc (or xmlpp::Document) must
be in UTF-8.

If you do this:

xmlNode->set_child_content("£££");
// ...
theRequest = document.write_to_string("iso-8859-2");

Then libxml will try to convert the xmlDoc structure from UTF-8 to
ISO-88590-2, but the "£££" data is not valid in UTF-8, so the conversion
fails.

When you add data to the document the data *must* be encoded in UTF-8.

When libxml outputs the doc it will be in the requested encoding.

Hope that helps,

jon


> code fragment:
> ------------------------------------------------------------
> xmlpp::Document document;
> xmlpp::Element* xmlPNode = 
> document.create_root_node(theCommand+XML_CM_REQUEST);
> 
> xmlPNode->add_attribute(XML_MA_XMLNS, XML_NAMESPACE_VALUE);
> xmlNode = xmlPNode->add_child(XML_NN_TASK);
> xmlNode->set_child_content(getHeaderAttr(OH_TASK_ID));
> xmlNode->add_attribute(XML_MA_NAMESPACE, ourNamespace);
> xmlNode->add_attribute(XML_MA_VERSION, ourVersion);
>  
> theRequest = document.write_to_string("iso-8859-2");
> 
> 
> 
> -- 
> --------------------------------------------------------------------------
> Software Factory GmbH                      mailto:Rai...@so...
> Panoramastr. 47, 73084 Salach              phone  : +49 7162 460592
>  o technical software development          fax    : +49 7162 460593
>  o system engineering                      mobile : +49 172 6714084
>  o consulting                              www    : http://www.so-fa.de
> --------------------------------------------------------------------------
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: SF.net Giveback Program.
> Does SourceForge.net help you be more productive?  Does it
> help you create better code?  SHARE THE LOVE, and help us help
> YOU!  Click Here: http://sourceforge.net/donate/
> _______________________________________________
> Libxmlplusplus-general mailing list
> Lib...@li...
> https://lists.sourceforge.net/lists/listinfo/libxmlplusplus-general

-- 
"A scientific theory should be as simple as possible, but no simpler."
	- Albert Einstein

Re: [libxml++] ISO-8859-2 enocdings

[Christophe de V. <cde...@al...>]

Rainer Stransky wrote:

>I use libxml++ in a application dealing with iso-8859-2.
>
>When creating a xmlpp::Document including special characaters (valid within 
>iso8859-2) the method-call document.write_to_string("iso-8859-2") (see code 
>below) puts a error message to stdout:
>  output conversion failed due to conv error
>  Bytes: 0xB5 0x72 0x61 0x69
>
>  
>
iconv cannot convert your datas from UTF-8 to ISO-8859-2.

>and the resulting xml-string is corrupted (truncated at the postion of the 
>special character (Dec 174)
>
>What is the problem ? Do i use libxml++ in a wrong manner ?
>
>  
>

Don't forget that whatever you final document encoding is, the in-memory 
encoding that libxml2 uses is UTF-8. As a consequence all strings you 
give to libxml++ should be UTF-8 encoded.

So I suggest you check that you give UTF-8 strings to the API.

Cheers,

Christophe

[libxml++] ISO-8859-2 enocdings

[Rainer S. <Rai...@so...>]

I use libxml++ in a application dealing with iso-8859-2.

When creating a xmlpp::Document including special characaters (valid within 
iso8859-2) the method-call document.write_to_string("iso-8859-2") (see code 
below) puts a error message to stdout:
  output conversion failed due to conv error
  Bytes: 0xB5 0x72 0x61 0x69

and the resulting xml-string is corrupted (truncated at the postion of the 
special character (Dec 174)

What is the problem ? Do i use libxml++ in a wrong manner ?



Rainer


code fragment:
------------------------------------------------------------
xmlpp::Document document;
xmlpp::Element* xmlPNode = 
document.create_root_node(theCommand+XML_CM_REQUEST);

xmlPNode->add_attribute(XML_MA_XMLNS, XML_NAMESPACE_VALUE);
xmlNode = xmlPNode->add_child(XML_NN_TASK);
xmlNode->set_child_content(getHeaderAttr(OH_TASK_ID));
xmlNode->add_attribute(XML_MA_NAMESPACE, ourNamespace);
xmlNode->add_attribute(XML_MA_VERSION, ourVersion);
 
theRequest = document.write_to_string("iso-8859-2");



-- 
--------------------------------------------------------------------------
Software Factory GmbH                      mailto:Rai...@so...
Panoramastr. 47, 73084 Salach              phone  : +49 7162 460592
 o technical software development          fax    : +49 7162 460593
 o system engineering                      mobile : +49 172 6714084
 o consulting                              www    : http://www.so-fa.de
--------------------------------------------------------------------------

Re: [libxml++] Dependencies...

[Christophe de V. <cde...@al...>]

Christophe de Vienne wrote:

> Paul Breslin a =E9crit :
>
>> It would useful if the libxml++ home page specified which Version of=20
>> libxml2
>> it is dependent on.
>> =20
>>
>
> Yes. Thanks for suggestion. I'll do it soon.


Done.

Re: [libxml++] Re: [xml] xmlCreatePushParserCtxt() docs wrong (was RE: [libxml++] sax_pars er example: entity not defined)

[Christophe de V. <cde...@al...>]

Sorry about the noise, I clicked too fast :-/

RE: [libxml++] sax_parser example: entity not defined

[Dan D. <da...@de...>]

On Tue, 2003-11-25 at 04:12, Mur...@Co... wrote:
> Dan Dennedy wrote:
> > I just posted a revised 
> > patch that embeds this handling inside SaxParser proper. 
> > Note, it uses an empty dummy Document to perform its magic. 
> 
> I am very close to being able to check this in, but I would like some
> response to these questions:
> http://sourceforge.net/tracker/index.php?func=detail&aid=829721&group_id=129
> 99&atid=312999

done

> In particular, I am concerned that we must always use true in the
> constructor, or normal entity substitution is broken.

I do not like the boolean in the constructor and recommend removing it.

> I would be surprised
> if we always had to use this dummy document just to get normal entity
> substitution.

It can be made to work without using a Document object, just don't ask
me how because this is my preferred solution ;-). I think another
solution would end up duplicating much logic in libxml2.

RE: [libxml++] sax_parser example: entity not defined

[<Mur...@Co...>]

Dan Dennedy wrote:
> I just posted a revised 
> patch that embeds this handling inside SaxParser proper. 
> Note, it uses an empty dummy Document to perform its magic. 

I am very close to being able to check this in, but I would like some
response to these questions:
http://sourceforge.net/tracker/index.php?func=detail&aid=829721&group_id=129
99&atid=312999

In particular, I am concerned that we must always use true in the
constructor, or normal entity substitution is broken. I would be surprised
if we always had to use this dummy document just to get normal entity
substitution.

This is really cool, and I really want to get it checked in.

Murray Cumming
www.murrayc.com
mu...@us...

[libxml++] [ libxmlplusplus-Patches-846883 ] derived DOM example

[SourceForge.net <no...@so...>]

Patches item #846883, was opened at 2003-11-21 20:53
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=312999&aid=846883&group_id=12999

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Dan Dennedy (ddennedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: derived DOM example

Initial Comment:
Attached is perhaps a better example of a derived DOM
than my first example submitted some time ago. This one
is an example of using SVG and requires that the SAX
parser is capable of resolving entities (see my other
patch). 

The motivation: handle SVG animation by manipulating
attributes as required per frame. Then, for each frame
iteration, traverse the tree synthesizing SAX events to
feed (a suitably modified) librsvg (SAX-based SVG
renderer). This may likely result in more libxmlpp
enhancements.

For now, consider this a test case for the sax entity
handling as well as an exploration of deriving an
application-specific DOM, rather than using a generic
DOM in your application.


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=312999&aid=846883&group_id=12999

Re: [libxml++] sax_parser example: entity not defined

[Dan D. <da...@de...>]

On Fri, 2003-11-21 at 12:20, Murray Cumming wrote:
> There's just one bug that is bothering me before the API freeze:
> 
> examples/sax_parser/example.xml defines an entity and uses it, but the
> example reports an "entity not defined" error. This also happens with
> libxml++ 0.24, so I don't think it has anything to do with our recent
> entity-parsing enhancements.
> 
> Does anybody have any ideas? If not, I guess I need to try it in C.

As you know, I had been working on sax and entities some, and in October
submitted a patch to the tracker that partially addressed my ultimate
goal. I was never able to achieve the ultimate goal, but I figured out
quite some time ago one way to handle entity substitutions in SaxParser.
The patch on the tracker did not really make sense without the derived
SaxParser in my unposted example. I just posted a revised patch that
embeds this handling inside SaxParser proper. Note, it uses an empty
dummy Document to perform its magic. In any case, it does work, and it
is worth considering. The nice thing about using the dummy document is
that we leverage the facilities with libxml2 for entity resolution.
Check out the updated patch on the tracker and let me know what you
think.

Re: [libxml++] Dependencies...

[Christophe de V. <cde...@al...>]

Paul Breslin a =E9crit :

>It would useful if the libxml++ home page specified which Version of lib=
xml2
>it is dependent on.
> =20
>

Yes. Thanks for suggestion. I'll do it soon.

Cheers,

Christophe

[libxml++] Dependencies...

[Paul B. <pbr...@al...>]

It would useful if the libxml++ home page specified which Version of =
libxml2
it is dependent on.

Cheers.

Paul H. Breslin

[libxml++] [ libxmlplusplus-Patches-842730 ] TextReader

[SourceForge.net <no...@so...>]

Patches item #842730, was opened at 2003-11-15 15:28
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=312999&aid=842730&group_id=12999

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Christophe de Vienne (cdevienne)
Assigned to: Nobody/Anonymous (nobody)
Summary: TextReader

Initial Comment:
This patch provides an implementation of textreader. 
A few points needs to be discussed I think : 
- member functions names: I used the names present in 
the XmlTextReader of C#, since it is were it comes 
from. However it does not respect the convention we 
used in the rest of the API, with lower case letters 
and underscores. 
- do we include it in version 1.0 of libxml++ ? This 
would force users to use libxml2 2.6.1 at least. 
- the implementation is not complete: I still need to 
implement the input buffer stuffs. 
 

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=312999&aid=842730&group_id=12999

Re: [libxml++] DomParser Thread segmentation fault

[Christophe de V. <cde...@al...>]

Christophe de VIENNE wrote:

>
> I will commit the patch in the CVS in a few minutes...
>

done.

Re: [libxml++] DomParser Thread segmentation fault

[Christophe de V. <cde...@al...>]

Christophe de VIENNE wrote:

> gr...@te... wrote:
>
>>     Hi:
>>  
>>     I'm using
>>         gcc 3.3.2
>>         libxml++ 0.26/0.27
>>         commonc++2 / ptypes (pthread)
>>         libxml2 v2.5.11.
>
>
>
> I've just upgraded to this version of libxml2 and I got the same 
> problem :-(
> I currently investigating this.
>

I found... When --with-threads is on in libxml2, which is the default 
since 2.5.7, the callbacks we set in Init are set only for the main thread.
To enable them in all threads we have to use 
xmlThrDefRegisterNodeDefault and xmlThrDefDeregisterNodeDefault in 
addition to xmlRegisterNodeDefault and xmlDeregisterNodeDefault.

I will commit the patch in the CVS in a few minutes...

Cheers

Christophe

Re: [libxml++] DomParser Thread segmentation fault

[Fredrik A. <e97_far@e.kth.se>]

On 2003.11.10 22:27, gr...@te... wrote:
>     I have also tried to use the xmlInitParser in the first
> instruction of my main, but nothing changes.

xmlInitParser is called in Document::Init::Init(), so that should not 
be necessary.

/Fredrik Arnerup

Re: [libxml++] DomParser Thread segmentation fault

[Christophe de V. <cde...@al...>]

gr...@te... wrote:

>     Hi:
>  
>     I'm using
>         gcc 3.3.2
>         libxml++ 0.26/0.27
>         commonc++2 / ptypes (pthread)
>         libxml2 v2.5.11.


I've just upgraded to this version of libxml2 and I got the same problem :-(
I currently investigating this.

Cheers

Christophe

[libxml++] DomParser Thread segmentation fault

[<gr...@te...>]

    Hi:

    I'm using=20
        gcc 3.3.2
        libxml++ 0.26/0.27
        commonc++2 / ptypes (pthread)
        libxml2 v2.5.11.=20

    I having a segmentation fault when using any kind of DomParser in a =
Thread code. The get_document returns a valid pointer (no-NULL), but =
get_root_node always returns NULL. (The xml document is valid) And i =
don't have more than one thread running.

    If i execute the same code outside any thread it works.=20

    I have also tried to use the xmlInitParser in the first instruction =
of my main, but nothing changes.

    Any clue about this extrange behaviour?

    Thanks in advance.

   =20

[libxml++] how do i get just one result from the domparser

[Tim <ti...@st...>]

This is a little hard to explain but i will try, after the responses
from my 
previous email i switched to using the DomParser .

ok just until i get this all right i am just using the domparser 
example(libxml++/examples/dom_parser).

i am using this very tiny cut down xml file (see below)

ok in the int main() part in main.cc i would like to be able get the
data out 
of 1 certain node-element for eg the value of "DOWNLOADS" out of "normal
billable traffic". I need to be able to minus that amount from the 
"MEAGABYTES" amount to get the usage remaing for the month and been able
to 
pull the data out one by one instead of just printing it all in order
like 
print_node(pNode) does, i can see how handy in some cases it is to just
print 
it all out as it lies in the XML file but for my case i need to
re-arrange 
things and also store things in "stings" "int" etc etc for later use,
such as 
what i mentioned above. Another example...the last part of the xml file
the 
PLAN DESCRIPTION, that would print last using the print_node call,
really 
something like that i would prefer to print first along with START_DATE,
END_DATE. Does any of this make sense??. I don't know why my ISP does
the XML
feed in that order but there is nothing i can do about that, actually 
sometimes they change the order of things too but never do they change
the 
node names.

Thanks again guys

Tim




***************** xml file ************************

<USAGE START_DATE="21-10-2003" GRANULARITY="MONTH" VERSION="3" 
END_DATE="20-11-2003">
  <TRAFFIC DESCRIPTION="Normal Billable Traffic">
    <DATA UPLOADS="0" DOWNLOADS="316" TYPE="Standard"
TIMESTAMP="2003-10-21 
00:00:00" />
  </TRAFFIC>
  <TRAFFIC DESCRIPTION="Free Traffic">
    <DATA UPLOADS="0" DOWNLOADS="3" TYPE="Standard"
TIMESTAMP="2003-10-21 
00:00:00" />
  </TRAFFIC>
  <PLAN DESCRIPTION="Home 512 Flat Rate">
    <LIMIT NAME="Standard" MEGABYTES="6000" />
  </PLAN>
</USAGE>
**********************************************************************

[libxml++] Re: [xml] xmlReader and _private

[Christophe de V. <cde...@al...>]

Daniel Veillard wrote:

>  Anyway, fixed in CVS.
>
>  
>

That's great ! Thanks very much...


Regards,

Christophe

[libxml++] Re: [xml] xmlReader and _private

[Daniel V. <vei...@re...>]

On Sun, Oct 26, 2003 at 12:30:57PM +0100, Christophe de Vienne wrote:
> I see 2 potential solutions to the problem :
> 1) Change libxml xmlreader implementation so it does not use 
> _xmlNode._private.

  This wasn't possible before 2.6.0 , I commited the change to CVS

Daniel

-- 
Daniel Veillard      | Red Hat Network https://rhn.redhat.com/
vei...@re...  | libxml GNOME XML XSLT toolkit  http://xmlsoft.org/
http://veillard.com/ | Rpmfind RPM search engine http://rpmfind.net/

[libxml++] xmlReader and _private

[Christophe de V. <cde...@al...>]

Hi,

While implementing xmlReader wrapper for libxml++ I faced a little 
problem with the _private field of xmlNode. xmlReader implementation in 
libxml2 do use this field, and since we use it in libxml++ to store a 
pointer to a xmlpp::Node which is automatically instanciated in the 
xmlRegisterNodeDefault callback, it leads to a segfault when it is 
deleted in the xmlDeregisterNodeDefault callback.

I see 2 potential solutions to the problem :
1) Change libxml xmlreader implementation so it does not use 
_xmlNode._private.
2) Detect in the callbacks wether or not the node is created by 
xmlReader, and then don't do anything if it is the case. I would then 
implement Expand by copying the subtree to avoid some non wrapped nodes 
to be given to libxml++ user.

Is one of this options possible ? And if not is there any other way to 
solve this issue ?


Thanks,

Christophe de Vienne

Re: [libxml++] sax parser

[Dan D. <da...@de...>]

On Thu, 2003-10-23 at 23:03, Tim wrote:
> Hi everyone
> 
> I am using the sax parser example together with this xml file (see below). The 
> xml file is from a ISP and just reports your usage and any meassges, your 
> plan etc,the format for this file doesn't change only the data obviouslly. My 
> problem is the order in which things get printed to the screen, for eg... in 
> the <MESSAGE the "TITLE" comes after "TEXT", when using that sax_example how 

I don't like this either. It is due to the use of a map in the sax
parser to collect the xml attributes, which by nature is key-ordered.
Since there are not typically a huge number of attributes, would it be
better to just use a vector and iterate on lookup?

[libxml++] [ libxmlplusplus-Patches-829721 ] entity handling for sax parser

[SourceForge.net <no...@so...>]

Patches item #829721, was opened at 2003-10-24 17:11
Message generated for change (Tracker Item Submitted) made by Item Submitter
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=312999&aid=829721&group_id=12999

Category: None
Group: None
Status: Open
Resolution: None
Priority: 5
Submitted By: Dan Dennedy (ddennedy)
Assigned to: Nobody/Anonymous (nobody)
Summary: entity handling for sax parser

Initial Comment:
This patch allows entity declarations and entity
references to be properly resolved within the SAX parser.

Please note, in the SAX parser, I set the
substitute_entities(true) in the base Parser's context
since this is the most sensible default for SAX.
Otherwise, entity references will be encoded as
&entityReference;.

This patch probably does not make much sense outside
the context of my use case: using SAX to build a
derived DOM. Shall I submit it again as a separate
patch? Nevertheless, it is clear that the new Document
methods do afford some new features not yet introduced.

+-DRD-+


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=312999&aid=829721&group_id=12999

Re: [libxml++] sax parser

[Dan D. <da...@de...>]

On Fri, 2003-10-24 at 12:20, Christophe de VIENNE wrote:

> Nothing in the SaxParser itself will do that for you : informations are 
> given to the callbacks as soon as they are parsed.
> If you need to reorder the informations, keep them in memory and do what 
> you need when you got everything you need.

On a re-read of the original post, I see the order request he stated is
different than what I was talking about. Since the SAX parser uses a map
internally, it is trivial for him to get them in whatever order he
desires. In the on_start_element() callback, just get an attribute value
using attributes["TITLE"] and then attributes["TEXT"]. I am sorry I made
this more confusing by raising a different issue--original order
preservation.

Re: [libxml++] sax parser

[Christophe de V. <cde...@al...>]

Tim wrote:

>Hi everyone
>
>I am using the sax parser example together with this xml file (see below). The 
>xml file is from a ISP and just reports your usage and any meassges, your 
>plan etc,the format for this file doesn't change only the data obviouslly. My 
>problem is the order in which things get printed to the screen, for eg... in 
>the <MESSAGE the "TITLE" comes after "TEXT", when using that sax_example how 
>can i change the order in which it is printed because i would like to be able 
>to print the "title" and than "test" in that order. Also i need to be able to 
>add up the "DOWNLOADS" in <DATA to give a total. Any help would be great
>
>  
>

Nothing in the SaxParser itself will do that for you : informations are 
given to the callbacks as soon as they are parsed.
If you need to reorder the informations, keep them in memory and do what 
you need when you got everything you need.
However for this task I suggest you use DomParser : your file is small, 
and you will not have to code data manipulations. Unless of course you 
want to practice.

If you just want to transform your file into formatted text or html, you 
can have a look to XSLT.

>Thanks
>
>  
>

You're welcome


Christophe