vtd-xml-users — This mailing list is for users or people interested in VTD-XML

Re: [Vtd-xml-users] Performance in comparison to libxml2

[Jimmy Z. <cra...@co...>]

Tatu, we have done some internal testing, Java and C version are very close 
in
performance for most cases, when XML is small, C out performs Java version.
The other thing: by java what I mean is using server JVM (hotspot) which 
does a
phenominal job doing JIT compiler... An other observation: C# is the 
slowest,
20% slower than Java and C...
Anyway, we are doing a version 1.8 release, which should come out pretty 
soon,
the big feature is XMLModifier, which is a easy to use API that allows one 
to take
advantage of VTD's incremental update capability, by then, we will try to 
post
updated benchmark results includes side-by-side comparison of C and Java...
Cheers,
Jimmy

----- Original Message ----- 
From: "Tatu Saloranta" <cow...@ya...>
To: "Jimmy Zhang" <cra...@co...>; "John Kraal" <jk...@in...>
Cc: <vtd...@li...>
Sent: Tuesday, November 07, 2006 11:29 AM
Subject: Re: [Vtd-xml-users] Performance in comparison to libxml2


> --- Jimmy Zhang <cra...@co...> wrote:
>
>> The performnace test doesn't look right to me...
> ...
>> I parsed a 170MB today on windows (1.7 ghz pentium
>> M)
>> running XML parsing  it took about 3 seconds to
>> parse that
>
> Hi Jimmy! Besides comparing libxml2 and vtd-xml/c, it
> would be quite interesting to see how vtd-xml c and
> java implementations differ regarding performance. I
> know how to measure Java version, but since it's been
> a while I did anything in c/c++, I haven't had time to
> figure out if performance differs significantly
> (probably not, but who knows?)
>
> So... would it be possible for you to maybe run some
> side-by-side comparisons between Java and C
> implementations of vtd-xml? These would be quite
> enlightening. Also, for c version, it'd be good to do
> similar warmup as is done for Java version, to try to
> minimize startup costs (which are much less of a
> concern for native code... but there's still some
> overhead for the first run).
>
> I suspect original poster should also try running the
> same/similar parsing multiple times per test -- just
> running executable once may give skewed results (maybe
> there are lib deps to load or such). Especially when
> runtime is very short, couple of seconds.
>
> -+ Tatu +-
>
>
>
>
>
>
> ____________________________________________________________________________________
> Want to start your own business?
> Learn how on Yahoo! Small Business.
> http://smallbusiness.yahoo.com/r-index
> 

Re: [Vtd-xml-users] Performance in comparison to libxml2

[Tatu S. <cow...@ya...>]

--- Jimmy Zhang <cra...@co...> wrote:

> The performnace test doesn't look right to me...
...
> I parsed a 170MB today on windows (1.7 ghz pentium
> M)
> running XML parsing  it took about 3 seconds to
> parse that

Hi Jimmy! Besides comparing libxml2 and vtd-xml/c, it
would be quite interesting to see how vtd-xml c and
java implementations differ regarding performance. I
know how to measure Java version, but since it's been
a while I did anything in c/c++, I haven't had time to
figure out if performance differs significantly
(probably not, but who knows?)

So... would it be possible for you to maybe run some
side-by-side comparisons between Java and C
implementations of vtd-xml? These would be quite
enlightening. Also, for c version, it'd be good to do
similar warmup as is done for Java version, to try to
minimize startup costs (which are much less of a
concern for native code... but there's still some
overhead for the first run).

I suspect original poster should also try running the
same/similar parsing multiple times per test -- just
running executable once may give skewed results (maybe
there are lib deps to load or such). Especially when
runtime is very short, couple of seconds.

-+ Tatu +-





 
____________________________________________________________________________________
Want to start your own business?
Learn how on Yahoo! Small Business.
http://smallbusiness.yahoo.com/r-index

Re: [Vtd-xml-users] Performance in comparison to libxml2

[Jimmy Z. <cra...@co...>]

My experience is that, when comparing VTD-XML with expat (no content 
handler)
VTD-XML is typically twice as fast...
----- Original Message ----- 
From: "John Kraal" <jk...@in...>
To: "Jimmy Zhang" <cra...@co...>
Sent: Tuesday, November 07, 2006 12:57 AM
Subject: Re: [Vtd-xml-users] Performance in comparison to libxml2


> Inline reply:
>
> Jimmy Zhang wrote:
>> The performnace test doesn't look right to me...
>> did you take into accoutn the file IO? Did you benchmark
>> parsing, or navigation?
>
> I benchmarked the total equation; which is of our importance, we receive
> a file, parse it once, and delete it. Most of the time the file is in
> memory, this was on disk though, for your pleasure, disk-benchmarks:
>
> /dev/sda:
> Timing cached reads:   1030 MB in  2.00 seconds = 514.56 MB/sec
> Timing buffered disk reads:  138 MB in  3.04 seconds =  45.39 MB/sec
>
> On the Xeon machine I used ramdisks; I'm pretty sure that cannot be
> obstructive
>
>>
>> What platform did you test? What OS? Did you tune the
>> compile flags for that platform?? Did you inline the getChar()?
>
> Tested on debian-sarge running on VMware, Pentium-M 1.7Ghz with 1GB of
> memory (not full), repeated the test to make sure the file was in linux'
> buffer. I intentionally did _nothing_ to both libraries to keep it a
> "fair" test. I can optimize the libraries madly, but that's not how it's
> distributed. libxml2 was used in the way it's distributed through apt
> (no platform optimalisations), vtdxml was built with "-march=pentium-m
> -mtune=pentium-m -O2".
>
> On the other system it ran on a native System on a Dual Xeon 3.6 with
> 4GB of memory on Fedora Core 3-smp, with comparable results.
>
>>
>> I parsed a 170MB today on windows (1.7 ghz pentium M)
>> running XML parsing  it took about 3 seconds to parse that
>> file... so the performance measuremetn on my end is quite
>> consistent with published performance benchmark...
>
> Was that Java or C? And Windows handles I/O and buffering differently
> than *nix.
>
>> ----- Original Message ----- From: "John Kraal" <jk...@in...>
>> To: "Jimmy Zhang" <cra...@co...>
>> Cc: <vtd...@li...>
>> Sent: Tuesday, November 07, 2006 12:12 AM
>> Subject: Re: [Vtd-xml-users] Performance in comparison to libxml2
>>
>>
>>> No, I'm sorry - the files I used to test contains data from my employer;
>>> I will not be the one to distribute that :-). But it's quite easy to
>>> create a fairly simple xml file like mine; and test it.
>>>
>>> Remember that I'm not testing a Java application, but merely C.
>>>
>>> The sources are in the previous email.
>>>
>>> The structure is quite simple, and as I'm not able to create a XSD
>>> schema from a XML File easily (not really sure with what), here the
>>> structure at it's simplest, just add some elements:
>>>
>>> <shipments>
>>> <shipment dosvlg="123" afd="abc123">
>>> <from>NLAMS</from>
>>> <to>NLRTM</to>
>>> <goods>
>>> <goodsline>
>>> <merknr>abc</merknr>
>>> <col code="pl">12345</col>
>>> </goodsline>
>>> </goods>
>>> <relations>
>>> <relation srtnaw="123" tsroln="123" relnr="123" zoek="abc">
>>> <tsnam1>abc</tsnam1>
>>> <coordinates>
>>> <tscoox>-12.3456</tscoox>
>>> <tscooy>-12.3456</tscoox>
>>> </coordinates>
>>> </relation>
>>> </relations>
>>> </shipment>
>>> </shipments>
>>>
>>>
>>>
>>>
>>> Jimmy Zhang wrote:
>>>> Can you provide a few sample XML files that you used for the testing?
>>>>
>>>> I am sure there can be additioal performance tuning for performance
>>>> evaluation...
>>>>
>>>> So far the response we got compare VTD-XML favorably with Xerces...
>>>> for the java version..
>>>> ----- Original Message ----- From: "John Kraal - Kewill Interchain NL"
>>>> <jk...@in...>
>>>> To: <vtd...@li...>
>>>> Sent: Monday, November 06, 2006 4:23 AM
>>>> Subject: [Vtd-xml-users] Performance in comparison to libxml2
>>>>
>>>>
>>>>> Hello,
>>>>>
>>>>> I've been playing with vtdxml for a while now, and my latest 
>>>>> experience
>>>>> is not very encouraging to go on :(...
>>>>>
>>>>> I have created a file with _lots_ of recurring structures and
>>>>> amounts of
>>>>> recurring data (about 137 MB), eventually I created a xpath to select
>>>>> the name of every relation in the 4th role (doesn't really matter).
>>>>>
>>>>> So, I executed it, with the code included for vtdxml (see vtdxml.c),
>>>>> and
>>>>> it performed like this:
>>>>>
>>>>> unims@gxvm1:~/src/xmltest$ time ./test ./jkl.files.xml
>>>>> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>>>>>
>>>>> real    0m4.940s
>>>>> user    0m4.625s
>>>>> sys     0m0.256s
>>>>>
>>>>> With 162450 bytes of terminal output.
>>>>>
>>>>> Then, the horrible thing happened, I used lixml2 with an adjusted
>>>>> reference xpath-program. (xpath1.c from their website, only with
>>>>> content
>>>>> retrieval):
>>>>>
>>>>> unims@gxvm1:~/src/libxmltest$ time ./test ./jkl.files.xml
>>>>> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>>>>>
>>>>> real    0m2.545s
>>>>> user    0m2.201s
>>>>> sys     0m0.321s
>>>>>
>>>>> The size of the output was 150808 bytes.
>>>>>
>>>>> This is an incredible difference, am I doing something wrong? Or did I
>>>>> have too many expectations of vtd?
>>>>>
>>>>> Regards,
>>>>> John Kraal
>>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>> /**
>>>>> * section: XPath
>>>>> * synopsis: Evaluate XPath expression and prints result node set.
>>>>> * purpose: Shows how to evaluate XPath expression and register
>>>>> *          known namespaces in XPath context.
>>>>> * usage: xpath1 <xml-file> <xpath-expr> [<known-ns-list>]
>>>>> * test: ./xpath1 test3.xml '//child2' > xpath1.tmp ; diff xpath1.tmp
>>>>> xpath1.res ; rm xpath1.tmp
>>>>> * author: Aleksey Sanin
>>>>> * copy: see Copyright for the status of this software.
>>>>> */
>>>>> #include <stdlib.h>
>>>>> #include <stdio.h>
>>>>> #include <string.h>
>>>>> #include <assert.h>
>>>>>
>>>>> #include <libxml/tree.h>
>>>>> #include <libxml/parser.h>
>>>>> #include <libxml/xpath.h>
>>>>> #include <libxml/xpathInternals.h>
>>>>>
>>>>> #if defined(LIBXML_XPATH_ENABLED) && defined(LIBXML_SAX1_ENABLED)
>>>>>
>>>>>
>>>>> static void usage(const char *name);
>>>>> int  execute_xpath_expression(const char* filename, const xmlChar*
>>>>> xpathExpr, const xmlChar* nsList);
>>>>> int  register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar*
>>>>> nsList);
>>>>> void print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output);
>>>>>
>>>>> int
>>>>> main(int argc, char **argv) {
>>>>>    /* Parse command line and process file */
>>>>>    if((argc < 3) || (argc > 4)) {
>>>>> fprintf(stderr, "Error: wrong number of arguments.\n");
>>>>> usage(argv[0]);
>>>>> return(-1);
>>>>>    }
>>>>>
>>>>>    /* Init libxml */
>>>>>    xmlInitParser();
>>>>>    LIBXML_TEST_VERSION
>>>>>
>>>>>    /* Do the main job */
>>>>>    if(execute_xpath_expression(argv[1], BAD_CAST argv[2], (argc > 3) ?
>>>>> BAD_CAST argv[3] : NULL) < 0) {
>>>>> usage(argv[0]);
>>>>> return(-1);
>>>>>    }
>>>>>
>>>>>    /* Shutdown libxml */
>>>>>    xmlCleanupParser();
>>>>>
>>>>>    /*
>>>>>     * this is to debug memory for regression tests
>>>>>     */
>>>>>    xmlMemoryDump();
>>>>>    return 0;
>>>>> }
>>>>>
>>>>> /**
>>>>> * usage:
>>>>> * @name: the program name.
>>>>> *
>>>>> * Prints usage information.
>>>>> */
>>>>> static void
>>>>> usage(const char *name) {
>>>>>    assert(name);
>>>>>
>>>>>    fprintf(stderr, "Usage: %s <xml-file> <xpath-expr>
>>>>> [<known-ns-list>]\n", name);
>>>>>    fprintf(stderr, "where <known-ns-list> is a list of known
>>>>> namespaces\n");
>>>>>    fprintf(stderr, "in \"<prefix1>=<href1> <prefix2>=href2> ...\"
>>>>> format\n");
>>>>> }
>>>>>
>>>>> /**
>>>>> * execute_xpath_expression:
>>>>> * @filename: the input XML filename.
>>>>> * @xpathExpr: the xpath expression for evaluation.
>>>>> * @nsList: the optional list of known namespaces in
>>>>> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
>>>>> *
>>>>> * Parses input XML file, evaluates XPath expression and prints 
>>>>> results.
>>>>> *
>>>>> * Returns 0 on success and a negative value otherwise.
>>>>> */
>>>>> int
>>>>> execute_xpath_expression(const char* filename, const xmlChar*
>>>>> xpathExpr, const xmlChar* nsList) {
>>>>>    xmlDocPtr doc;
>>>>>    xmlXPathContextPtr xpathCtx;
>>>>>    xmlXPathObjectPtr xpathObj;
>>>>>
>>>>>    assert(filename);
>>>>>    assert(xpathExpr);
>>>>>
>>>>>    /* Load XML document */
>>>>>    doc = xmlParseFile(filename);
>>>>>    if (doc == NULL) {
>>>>> fprintf(stderr, "Error: unable to parse file \"%s\"\n", filename);
>>>>> return(-1);
>>>>>    }
>>>>>
>>>>>    /* Create xpath evaluation context */
>>>>>    xpathCtx = xmlXPathNewContext(doc);
>>>>>    if(xpathCtx == NULL) {
>>>>>        fprintf(stderr,"Error: unable to create new XPath context\n");
>>>>>        xmlFreeDoc(doc);
>>>>>        return(-1);
>>>>>    }
>>>>>
>>>>>    /* Register namespaces from list (if any) */
>>>>>    if((nsList != NULL) && (register_namespaces(xpathCtx, nsList) <
>>>>> 0)) {
>>>>>        fprintf(stderr,"Error: failed to register namespaces list
>>>>> \"%s\"\n", nsList);
>>>>>        xmlXPathFreeContext(xpathCtx);
>>>>>        xmlFreeDoc(doc);
>>>>>        return(-1);
>>>>>    }
>>>>>
>>>>>    /* Evaluate xpath expression */
>>>>>    xpathObj = xmlXPathEvalExpression(xpathExpr, xpathCtx);
>>>>>    if(xpathObj == NULL) {
>>>>>        fprintf(stderr,"Error: unable to evaluate xpath expression
>>>>> \"%s\"\n", xpathExpr);
>>>>>        xmlXPathFreeContext(xpathCtx);
>>>>>        xmlFreeDoc(doc);
>>>>>        return(-1);
>>>>>    }
>>>>>
>>>>>    /* Print results */
>>>>>    print_xpath_nodes(xpathObj->nodesetval, stdout);
>>>>>
>>>>>    /* Cleanup */
>>>>>    xmlXPathFreeObject(xpathObj);
>>>>>    xmlXPathFreeContext(xpathCtx);
>>>>>    xmlFreeDoc(doc);
>>>>>
>>>>>    return(0);
>>>>> }
>>>>>
>>>>> /**
>>>>> * register_namespaces:
>>>>> * @xpathCtx: the pointer to an XPath context.
>>>>> * @nsList: the list of known namespaces in
>>>>> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
>>>>> *
>>>>> * Registers namespaces from @nsList in @xpathCtx.
>>>>> *
>>>>> * Returns 0 on success and a negative value otherwise.
>>>>> */
>>>>> int
>>>>> register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar*
>>>>> nsList) {
>>>>>    xmlChar* nsListDup;
>>>>>    xmlChar* prefix;
>>>>>    xmlChar* href;
>>>>>    xmlChar* next;
>>>>>
>>>>>    assert(xpathCtx);
>>>>>    assert(nsList);
>>>>>
>>>>>    nsListDup = xmlStrdup(nsList);
>>>>>    if(nsListDup == NULL) {
>>>>> fprintf(stderr, "Error: unable to strdup namespaces list\n");
>>>>> return(-1);
>>>>>    }
>>>>>
>>>>>    next = nsListDup;
>>>>>    while(next != NULL) {
>>>>> /* skip spaces */
>>>>> while((*next) == ' ') next++;
>>>>> if((*next) == '\0') break;
>>>>>
>>>>> /* find prefix */
>>>>> prefix = next;
>>>>> next = (xmlChar*)xmlStrchr(next, '=');
>>>>> if(next == NULL) {
>>>>>     fprintf(stderr,"Error: invalid namespaces list format\n");
>>>>>     xmlFree(nsListDup);
>>>>>     return(-1);
>>>>> }
>>>>> *(next++) = '\0';
>>>>>
>>>>> /* find href */
>>>>> href = next;
>>>>> next = (xmlChar*)xmlStrchr(next, ' ');
>>>>> if(next != NULL) {
>>>>>     *(next++) = '\0';
>>>>> }
>>>>>
>>>>> /* do register namespace */
>>>>> if(xmlXPathRegisterNs(xpathCtx, prefix, href) != 0) {
>>>>>     fprintf(stderr,"Error: unable to register NS with prefix=\"%s\"
>>>>> and href=\"%s\"\n", prefix, href);
>>>>>     xmlFree(nsListDup);
>>>>>     return(-1);
>>>>> }
>>>>>    }
>>>>>
>>>>>    xmlFree(nsListDup);
>>>>>    return(0);
>>>>> }
>>>>>
>>>>> /**
>>>>> * print_xpath_nodes:
>>>>> * @nodes: the nodes set.
>>>>> * @output: the output file handle.
>>>>> *
>>>>> * Prints the @nodes content to @output.
>>>>> */
>>>>> void
>>>>> print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output) {
>>>>>    xmlNodePtr cur;
>>>>> xmlChar* content;
>>>>>    int size;
>>>>>    int i;
>>>>>
>>>>>    assert(output);
>>>>>    size = (nodes) ? nodes->nodeNr : 0;
>>>>>
>>>>>    fprintf(output, "Result (%d nodes):\n", size);
>>>>>    for(i = 0; i < size; ++i) {
>>>>> assert(nodes->nodeTab[i]);
>>>>>
>>>>> if(nodes->nodeTab[i]->type == XML_NAMESPACE_DECL) {
>>>>>     xmlNsPtr ns;
>>>>>
>>>>>     ns = (xmlNsPtr)nodes->nodeTab[i];
>>>>>     cur = (xmlNodePtr)ns->next;
>>>>>     if(cur->ns) {
>>>>>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s:%s\n",
>>>>>     ns->prefix, ns->href, cur->ns->href, cur->name);
>>>>>     } else {
>>>>>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s\n",
>>>>>     ns->prefix, ns->href, cur->name);
>>>>>     }
>>>>> } else if(nodes->nodeTab[i]->type == XML_ELEMENT_NODE) {
>>>>>     cur = nodes->nodeTab[i];
>>>>> content = xmlNodeGetContent(cur);
>>>>>     if(cur->ns) {
>>>>>            fprintf(output, "= element node \"%s:%s\": \"%s\"\n",
>>>>>     cur->ns->href, cur->name, content);
>>>>>     } else {
>>>>>            fprintf(output, "= element node \"%s\": \"%s\"\n",
>>>>>     cur->name, content);
>>>>>     }
>>>>> xmlFree(content);
>>>>> } else {
>>>>>     cur = nodes->nodeTab[i];
>>>>> content = xmlNodeGetContent(cur);
>>>>>     fprintf(output, "= node \"%s\": type %d: value \"%s\"\n",
>>>>> cur->name, cur->type, content);
>>>>> xmlFree(content);
>>>>> }
>>>>>    }
>>>>> }
>>>>>
>>>>> #else
>>>>> int main(void) {
>>>>>    fprintf(stderr, "XPath support not compiled in\n");
>>>>>    exit(1);
>>>>> }
>>>>> #endif
>>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>> #include <string.h>
>>>>> #include <stdio.h>
>>>>> #include <wchar.h>
>>>>> #include <stdlib.h>
>>>>> #include <fcntl.h>
>>>>> #include <sys/types.h>
>>>>> #include <sys/stat.h>
>>>>> #include <vtdxml/xpath1.h>
>>>>> #include <vtdxml/helper.h>
>>>>> #include <vtdxml/vtdGen.h>
>>>>>
>>>>> struct exception_context the_exception_context[1];
>>>>>
>>>>> int main(int argc, char **argv, char **envv)
>>>>> {
>>>>> exception e;
>>>>> FILE *f = NULL;
>>>>> int i=0,t,result,count=0;
>>>>> wchar_t *tmpString, *tmpString2;
>>>>> char *filename;
>>>>> wchar_t *xpath;
>>>>> struct stat s;
>>>>> UByte *xml = NULL;
>>>>> VTDGen *vg = NULL;
>>>>> VTDNav *vn = NULL;
>>>>> AutoPilot *ap = NULL;
>>>>>
>>>>> Try
>>>>> {
>>>>> if (argc > 2)
>>>>> {
>>>>> filename = (char *)malloc(strlen(argv[1]) + 1);
>>>>> xpath = (wchar_t *)malloc((strlen(argv[2]) + 1) * sizeof(wchar_t));
>>>>> strcpy(filename, argv[1]);
>>>>> mbstowcs(xpath , argv[2], strlen(argv[2])+1);
>>>>> wprintf(L"Using xpath: %ls\n", xpath);
>>>>> }
>>>>> else
>>>>> {
>>>>> filename = "test.xml";
>>>>> xpath    = L"//*";
>>>>> }
>>>>> f = fopen(filename, "r");
>>>>> stat(filename, &s);
>>>>> i = (int)s.st_size;
>>>>> wprintf(L"size of the file is %d bytes\n", i);
>>>>> xml = (UByte *)malloc(sizeof(UByte) * i);
>>>>> i = fread(xml,sizeof(UByte),i,f);
>>>>>
>>>>> vg = createVTDGen();
>>>>> setDoc(vg,xml,i);
>>>>> parse(vg,TRUE);
>>>>>
>>>>> vn = getNav(vg);
>>>>> ap = createAutoPilot2();
>>>>> declareXPathNameSpace(ap,L"ns1",L"");
>>>>>
>>>>> if (selectXPath(ap, xpath))
>>>>> {
>>>>> bind(ap, vn);
>>>>> while ((result = evalXPath(ap)) != -1)
>>>>> {
>>>>> tmpString = toString(vn, result);
>>>>> t = getText(vn);
>>>>> if (t != -1)
>>>>> {
>>>>> tmpString2 = toNormalizedString(vn,t);
>>>>> wprintf(L"r: %d e: %ls (%d): %ls \n",
>>>>> result,
>>>>> tmpString,
>>>>> t,
>>>>> tmpString2);
>>>>> free(tmpString);
>>>>> free(tmpString2);
>>>>> }
>>>>> count++;
>>>>> }
>>>>> }
>>>>> wprintf(L"\nTotal number of elements %d \n", count);
>>>>> fclose(f);
>>>>>
>>>>> freeVTDNav(vn);
>>>>> freeVTDGen(vg);
>>>>> freeAutoPilot(ap);
>>>>> } Catch (e) {
>>>>> wprintf (L"exception occurred\nerr: %s\nmoreerr: %s\n", e.msg,
>>>>> e.sub_msg);
>>>>> return 1;
>>>>> }
>>>>>
>>>>> return 0;
>>>>> }
>>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>> -------------------------------------------------------------------------
>>>>>
>>>>> Using Tomcat but need to do more? Need to support web services,
>>>>> security?
>>>>> Get stuff done quickly with pre-integrated technology to make your job
>>>>> easier
>>>>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>>>>> Geronimo
>>>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------------------
>>>>
>>>>
>>>>
>>>>
>>>>> _______________________________________________
>>>>> Vtd-xml-users mailing list
>>>>> Vtd...@li...
>>>>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>>>>
>>>>
>>>>
>>> =
>>>
>>
>>
>
> -- 
> Met vriendelijke groeten,
> With kind regards,
>
> John Kraal
> Programmer
>
> Kewill Interchain
> A member of the Kewill Group of companies
>
> (T) +31 (0)78 6 123 310
> (E)  jk...@in...
> (W) www.interchain.nl
>
> IMPORTANT NOTICE: This e-mail, and each of any attachments transmitted
> with it, is confidential, is the property of Kewill Interchain, and is
> intended solely for the information of the individual/entity to whom
> this e-mail is addressed. It may also be privileged or protected by
> other legal rules. If you have received it by mistake, please notify the
>
> author immediately by replying to this e-mail and then destroy/delete
> this e-mail and all attachments (if any). If you are not the intended
> recipient you must not use, disclose, distribute, copy, print, or rely
> on this e-mail. In accordance with Kewill Interchain's internal policy
> e-mails sent and/or received may be monitored. Whilst Kewill Interchain
> takes reasonable precautions (including use of anti-virus software) it
> accepts no responsibility for any loss or damage should this e-mail
> contain any virus, or similar destructive or mischievous code. (c)
> Kewill Interchain 2006. 

Re: [Vtd-xml-users] Performance in comparison to libxml2

[Jimmy Z. <cra...@co...>]

The performnace test doesn't look right to me...
did you take into accoutn the file IO? Did you benchmark
parsing, or navigation?

What platform did you test? What OS? Did you tune the
compile flags for that platform?? Did you inline the getChar()?

I parsed a 170MB today on windows (1.7 ghz pentium M)
running XML parsing  it took about 3 seconds to parse that
file... so the performance measuremetn on my end is quite
consistent with published performance benchmark...
----- Original Message ----- 
From: "John Kraal" <jk...@in...>
To: "Jimmy Zhang" <cra...@co...>
Cc: <vtd...@li...>
Sent: Tuesday, November 07, 2006 12:12 AM
Subject: Re: [Vtd-xml-users] Performance in comparison to libxml2


> No, I'm sorry - the files I used to test contains data from my employer;
> I will not be the one to distribute that :-). But it's quite easy to
> create a fairly simple xml file like mine; and test it.
>
> Remember that I'm not testing a Java application, but merely C.
>
> The sources are in the previous email.
>
> The structure is quite simple, and as I'm not able to create a XSD
> schema from a XML File easily (not really sure with what), here the
> structure at it's simplest, just add some elements:
>
> <shipments>
> <shipment dosvlg="123" afd="abc123">
> <from>NLAMS</from>
> <to>NLRTM</to>
> <goods>
> <goodsline>
> <merknr>abc</merknr>
> <col code="pl">12345</col>
> </goodsline>
> </goods>
> <relations>
> <relation srtnaw="123" tsroln="123" relnr="123" zoek="abc">
> <tsnam1>abc</tsnam1>
> <coordinates>
> <tscoox>-12.3456</tscoox>
> <tscooy>-12.3456</tscoox>
> </coordinates>
> </relation>
> </relations>
> </shipment>
> </shipments>
>
>
>
>
> Jimmy Zhang wrote:
>> Can you provide a few sample XML files that you used for the testing?
>>
>> I am sure there can be additioal performance tuning for performance
>> evaluation...
>>
>> So far the response we got compare VTD-XML favorably with Xerces...
>> for the java version..
>> ----- Original Message ----- From: "John Kraal - Kewill Interchain NL"
>> <jk...@in...>
>> To: <vtd...@li...>
>> Sent: Monday, November 06, 2006 4:23 AM
>> Subject: [Vtd-xml-users] Performance in comparison to libxml2
>>
>>
>>> Hello,
>>>
>>> I've been playing with vtdxml for a while now, and my latest experience
>>> is not very encouraging to go on :(...
>>>
>>> I have created a file with _lots_ of recurring structures and amounts of
>>> recurring data (about 137 MB), eventually I created a xpath to select
>>> the name of every relation in the 4th role (doesn't really matter).
>>>
>>> So, I executed it, with the code included for vtdxml (see vtdxml.c), and
>>> it performed like this:
>>>
>>> unims@gxvm1:~/src/xmltest$ time ./test ./jkl.files.xml
>>> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>>>
>>> real    0m4.940s
>>> user    0m4.625s
>>> sys     0m0.256s
>>>
>>> With 162450 bytes of terminal output.
>>>
>>> Then, the horrible thing happened, I used lixml2 with an adjusted
>>> reference xpath-program. (xpath1.c from their website, only with content
>>> retrieval):
>>>
>>> unims@gxvm1:~/src/libxmltest$ time ./test ./jkl.files.xml
>>> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>>>
>>> real    0m2.545s
>>> user    0m2.201s
>>> sys     0m0.321s
>>>
>>> The size of the output was 150808 bytes.
>>>
>>> This is an incredible difference, am I doing something wrong? Or did I
>>> have too many expectations of vtd?
>>>
>>> Regards,
>>> John Kraal
>>>
>>
>>
>> --------------------------------------------------------------------------------
>>
>>
>>
>>> /**
>>> * section: XPath
>>> * synopsis: Evaluate XPath expression and prints result node set.
>>> * purpose: Shows how to evaluate XPath expression and register
>>> *          known namespaces in XPath context.
>>> * usage: xpath1 <xml-file> <xpath-expr> [<known-ns-list>]
>>> * test: ./xpath1 test3.xml '//child2' > xpath1.tmp ; diff xpath1.tmp
>>> xpath1.res ; rm xpath1.tmp
>>> * author: Aleksey Sanin
>>> * copy: see Copyright for the status of this software.
>>> */
>>> #include <stdlib.h>
>>> #include <stdio.h>
>>> #include <string.h>
>>> #include <assert.h>
>>>
>>> #include <libxml/tree.h>
>>> #include <libxml/parser.h>
>>> #include <libxml/xpath.h>
>>> #include <libxml/xpathInternals.h>
>>>
>>> #if defined(LIBXML_XPATH_ENABLED) && defined(LIBXML_SAX1_ENABLED)
>>>
>>>
>>> static void usage(const char *name);
>>> int  execute_xpath_expression(const char* filename, const xmlChar*
>>> xpathExpr, const xmlChar* nsList);
>>> int  register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar*
>>> nsList);
>>> void print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output);
>>>
>>> int
>>> main(int argc, char **argv) {
>>>    /* Parse command line and process file */
>>>    if((argc < 3) || (argc > 4)) {
>>> fprintf(stderr, "Error: wrong number of arguments.\n");
>>> usage(argv[0]);
>>> return(-1);
>>>    }
>>>
>>>    /* Init libxml */
>>>    xmlInitParser();
>>>    LIBXML_TEST_VERSION
>>>
>>>    /* Do the main job */
>>>    if(execute_xpath_expression(argv[1], BAD_CAST argv[2], (argc > 3) ?
>>> BAD_CAST argv[3] : NULL) < 0) {
>>> usage(argv[0]);
>>> return(-1);
>>>    }
>>>
>>>    /* Shutdown libxml */
>>>    xmlCleanupParser();
>>>
>>>    /*
>>>     * this is to debug memory for regression tests
>>>     */
>>>    xmlMemoryDump();
>>>    return 0;
>>> }
>>>
>>> /**
>>> * usage:
>>> * @name: the program name.
>>> *
>>> * Prints usage information.
>>> */
>>> static void
>>> usage(const char *name) {
>>>    assert(name);
>>>
>>>    fprintf(stderr, "Usage: %s <xml-file> <xpath-expr>
>>> [<known-ns-list>]\n", name);
>>>    fprintf(stderr, "where <known-ns-list> is a list of known
>>> namespaces\n");
>>>    fprintf(stderr, "in \"<prefix1>=<href1> <prefix2>=href2> ...\"
>>> format\n");
>>> }
>>>
>>> /**
>>> * execute_xpath_expression:
>>> * @filename: the input XML filename.
>>> * @xpathExpr: the xpath expression for evaluation.
>>> * @nsList: the optional list of known namespaces in
>>> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
>>> *
>>> * Parses input XML file, evaluates XPath expression and prints results.
>>> *
>>> * Returns 0 on success and a negative value otherwise.
>>> */
>>> int
>>> execute_xpath_expression(const char* filename, const xmlChar*
>>> xpathExpr, const xmlChar* nsList) {
>>>    xmlDocPtr doc;
>>>    xmlXPathContextPtr xpathCtx;
>>>    xmlXPathObjectPtr xpathObj;
>>>
>>>    assert(filename);
>>>    assert(xpathExpr);
>>>
>>>    /* Load XML document */
>>>    doc = xmlParseFile(filename);
>>>    if (doc == NULL) {
>>> fprintf(stderr, "Error: unable to parse file \"%s\"\n", filename);
>>> return(-1);
>>>    }
>>>
>>>    /* Create xpath evaluation context */
>>>    xpathCtx = xmlXPathNewContext(doc);
>>>    if(xpathCtx == NULL) {
>>>        fprintf(stderr,"Error: unable to create new XPath context\n");
>>>        xmlFreeDoc(doc);
>>>        return(-1);
>>>    }
>>>
>>>    /* Register namespaces from list (if any) */
>>>    if((nsList != NULL) && (register_namespaces(xpathCtx, nsList) < 0)) {
>>>        fprintf(stderr,"Error: failed to register namespaces list
>>> \"%s\"\n", nsList);
>>>        xmlXPathFreeContext(xpathCtx);
>>>        xmlFreeDoc(doc);
>>>        return(-1);
>>>    }
>>>
>>>    /* Evaluate xpath expression */
>>>    xpathObj = xmlXPathEvalExpression(xpathExpr, xpathCtx);
>>>    if(xpathObj == NULL) {
>>>        fprintf(stderr,"Error: unable to evaluate xpath expression
>>> \"%s\"\n", xpathExpr);
>>>        xmlXPathFreeContext(xpathCtx);
>>>        xmlFreeDoc(doc);
>>>        return(-1);
>>>    }
>>>
>>>    /* Print results */
>>>    print_xpath_nodes(xpathObj->nodesetval, stdout);
>>>
>>>    /* Cleanup */
>>>    xmlXPathFreeObject(xpathObj);
>>>    xmlXPathFreeContext(xpathCtx);
>>>    xmlFreeDoc(doc);
>>>
>>>    return(0);
>>> }
>>>
>>> /**
>>> * register_namespaces:
>>> * @xpathCtx: the pointer to an XPath context.
>>> * @nsList: the list of known namespaces in
>>> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
>>> *
>>> * Registers namespaces from @nsList in @xpathCtx.
>>> *
>>> * Returns 0 on success and a negative value otherwise.
>>> */
>>> int
>>> register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar* nsList) 
>>> {
>>>    xmlChar* nsListDup;
>>>    xmlChar* prefix;
>>>    xmlChar* href;
>>>    xmlChar* next;
>>>
>>>    assert(xpathCtx);
>>>    assert(nsList);
>>>
>>>    nsListDup = xmlStrdup(nsList);
>>>    if(nsListDup == NULL) {
>>> fprintf(stderr, "Error: unable to strdup namespaces list\n");
>>> return(-1);
>>>    }
>>>
>>>    next = nsListDup;
>>>    while(next != NULL) {
>>> /* skip spaces */
>>> while((*next) == ' ') next++;
>>> if((*next) == '\0') break;
>>>
>>> /* find prefix */
>>> prefix = next;
>>> next = (xmlChar*)xmlStrchr(next, '=');
>>> if(next == NULL) {
>>>     fprintf(stderr,"Error: invalid namespaces list format\n");
>>>     xmlFree(nsListDup);
>>>     return(-1);
>>> }
>>> *(next++) = '\0';
>>>
>>> /* find href */
>>> href = next;
>>> next = (xmlChar*)xmlStrchr(next, ' ');
>>> if(next != NULL) {
>>>     *(next++) = '\0';
>>> }
>>>
>>> /* do register namespace */
>>> if(xmlXPathRegisterNs(xpathCtx, prefix, href) != 0) {
>>>     fprintf(stderr,"Error: unable to register NS with prefix=\"%s\"
>>> and href=\"%s\"\n", prefix, href);
>>>     xmlFree(nsListDup);
>>>     return(-1);
>>> }
>>>    }
>>>
>>>    xmlFree(nsListDup);
>>>    return(0);
>>> }
>>>
>>> /**
>>> * print_xpath_nodes:
>>> * @nodes: the nodes set.
>>> * @output: the output file handle.
>>> *
>>> * Prints the @nodes content to @output.
>>> */
>>> void
>>> print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output) {
>>>    xmlNodePtr cur;
>>> xmlChar* content;
>>>    int size;
>>>    int i;
>>>
>>>    assert(output);
>>>    size = (nodes) ? nodes->nodeNr : 0;
>>>
>>>    fprintf(output, "Result (%d nodes):\n", size);
>>>    for(i = 0; i < size; ++i) {
>>> assert(nodes->nodeTab[i]);
>>>
>>> if(nodes->nodeTab[i]->type == XML_NAMESPACE_DECL) {
>>>     xmlNsPtr ns;
>>>
>>>     ns = (xmlNsPtr)nodes->nodeTab[i];
>>>     cur = (xmlNodePtr)ns->next;
>>>     if(cur->ns) {
>>>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s:%s\n",
>>>     ns->prefix, ns->href, cur->ns->href, cur->name);
>>>     } else {
>>>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s\n",
>>>     ns->prefix, ns->href, cur->name);
>>>     }
>>> } else if(nodes->nodeTab[i]->type == XML_ELEMENT_NODE) {
>>>     cur = nodes->nodeTab[i];
>>> content = xmlNodeGetContent(cur);
>>>     if(cur->ns) {
>>>            fprintf(output, "= element node \"%s:%s\": \"%s\"\n",
>>>     cur->ns->href, cur->name, content);
>>>     } else {
>>>            fprintf(output, "= element node \"%s\": \"%s\"\n",
>>>     cur->name, content);
>>>     }
>>> xmlFree(content);
>>> } else {
>>>     cur = nodes->nodeTab[i];
>>> content = xmlNodeGetContent(cur);
>>>     fprintf(output, "= node \"%s\": type %d: value \"%s\"\n",
>>> cur->name, cur->type, content);
>>> xmlFree(content);
>>> }
>>>    }
>>> }
>>>
>>> #else
>>> int main(void) {
>>>    fprintf(stderr, "XPath support not compiled in\n");
>>>    exit(1);
>>> }
>>> #endif
>>>
>>
>>
>> --------------------------------------------------------------------------------
>>
>>
>>
>>> #include <string.h>
>>> #include <stdio.h>
>>> #include <wchar.h>
>>> #include <stdlib.h>
>>> #include <fcntl.h>
>>> #include <sys/types.h>
>>> #include <sys/stat.h>
>>> #include <vtdxml/xpath1.h>
>>> #include <vtdxml/helper.h>
>>> #include <vtdxml/vtdGen.h>
>>>
>>> struct exception_context the_exception_context[1];
>>>
>>> int main(int argc, char **argv, char **envv)
>>> {
>>> exception e;
>>> FILE *f = NULL;
>>> int i=0,t,result,count=0;
>>> wchar_t *tmpString, *tmpString2;
>>> char *filename;
>>> wchar_t *xpath;
>>> struct stat s;
>>> UByte *xml = NULL;
>>> VTDGen *vg = NULL;
>>> VTDNav *vn = NULL;
>>> AutoPilot *ap = NULL;
>>>
>>> Try
>>> {
>>> if (argc > 2)
>>> {
>>> filename = (char *)malloc(strlen(argv[1]) + 1);
>>> xpath = (wchar_t *)malloc((strlen(argv[2]) + 1) * sizeof(wchar_t));
>>> strcpy(filename, argv[1]);
>>> mbstowcs(xpath , argv[2], strlen(argv[2])+1);
>>> wprintf(L"Using xpath: %ls\n", xpath);
>>> }
>>> else
>>> {
>>> filename = "test.xml";
>>> xpath    = L"//*";
>>> }
>>> f = fopen(filename, "r");
>>> stat(filename, &s);
>>> i = (int)s.st_size;
>>> wprintf(L"size of the file is %d bytes\n", i);
>>> xml = (UByte *)malloc(sizeof(UByte) * i);
>>> i = fread(xml,sizeof(UByte),i,f);
>>>
>>> vg = createVTDGen();
>>> setDoc(vg,xml,i);
>>> parse(vg,TRUE);
>>>
>>> vn = getNav(vg);
>>> ap = createAutoPilot2();
>>> declareXPathNameSpace(ap,L"ns1",L"");
>>>
>>> if (selectXPath(ap, xpath))
>>> {
>>> bind(ap, vn);
>>> while ((result = evalXPath(ap)) != -1)
>>> {
>>> tmpString = toString(vn, result);
>>> t = getText(vn);
>>> if (t != -1)
>>> {
>>> tmpString2 = toNormalizedString(vn,t);
>>> wprintf(L"r: %d e: %ls (%d): %ls \n",
>>> result,
>>> tmpString,
>>> t,
>>> tmpString2);
>>> free(tmpString);
>>> free(tmpString2);
>>> }
>>> count++;
>>> }
>>> }
>>> wprintf(L"\nTotal number of elements %d \n", count);
>>> fclose(f);
>>>
>>> freeVTDNav(vn);
>>> freeVTDGen(vg);
>>> freeAutoPilot(ap);
>>> } Catch (e) {
>>> wprintf (L"exception occurred\nerr: %s\nmoreerr: %s\n", e.msg,
>>> e.sub_msg);
>>> return 1;
>>> }
>>>
>>> return 0;
>>> }
>>>
>>
>>
>> --------------------------------------------------------------------------------
>>
>>
>>
>>> -------------------------------------------------------------------------
>>> Using Tomcat but need to do more? Need to support web services, 
>>> security?
>>> Get stuff done quickly with pre-integrated technology to make your job
>>> easier
>>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>>> Geronimo
>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>
>>
>> --------------------------------------------------------------------------------
>>
>>
>>
>>> _______________________________________________
>>> Vtd-xml-users mailing list
>>> Vtd...@li...
>>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>>
>>
>>
> =
> 

Re: [Vtd-xml-users] Performance in comparison to libxml2

[John K. <jk...@in...>]

No, I'm sorry - the files I used to test contains data from my employer;
I will not be the one to distribute that :-). But it's quite easy to
create a fairly simple xml file like mine; and test it.

Remember that I'm not testing a Java application, but merely C.

The sources are in the previous email.

The structure is quite simple, and as I'm not able to create a XSD
schema from a XML File easily (not really sure with what), here the
structure at it's simplest, just add some elements:

<shipments>
	<shipment dosvlg="123" afd="abc123">
		<from>NLAMS</from>
		<to>NLRTM</to>
		<goods>
			<goodsline>
				<merknr>abc</merknr>
				<col code="pl">12345</col>
			</goodsline>
		</goods>
		<relations>
			<relation srtnaw="123" tsroln="123" relnr="123" zoek="abc">
				<tsnam1>abc</tsnam1>
				<coordinates>
					<tscoox>-12.3456</tscoox>
					<tscooy>-12.3456</tscoox>
				</coordinates>
			</relation>
		</relations>
	</shipment>
</shipments>




Jimmy Zhang wrote:
> Can you provide a few sample XML files that you used for the testing?
> 
> I am sure there can be additioal performance tuning for performance
> evaluation...
> 
> So far the response we got compare VTD-XML favorably with Xerces...
> for the java version..
> ----- Original Message ----- From: "John Kraal - Kewill Interchain NL"
> <jk...@in...>
> To: <vtd...@li...>
> Sent: Monday, November 06, 2006 4:23 AM
> Subject: [Vtd-xml-users] Performance in comparison to libxml2
> 
> 
>> Hello,
>>
>> I've been playing with vtdxml for a while now, and my latest experience
>> is not very encouraging to go on :(...
>>
>> I have created a file with _lots_ of recurring structures and amounts of
>> recurring data (about 137 MB), eventually I created a xpath to select
>> the name of every relation in the 4th role (doesn't really matter).
>>
>> So, I executed it, with the code included for vtdxml (see vtdxml.c), and
>> it performed like this:
>>
>> unims@gxvm1:~/src/xmltest$ time ./test ./jkl.files.xml
>> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>>
>> real    0m4.940s
>> user    0m4.625s
>> sys     0m0.256s
>>
>> With 162450 bytes of terminal output.
>>
>> Then, the horrible thing happened, I used lixml2 with an adjusted
>> reference xpath-program. (xpath1.c from their website, only with content
>> retrieval):
>>
>> unims@gxvm1:~/src/libxmltest$ time ./test ./jkl.files.xml
>> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>>
>> real    0m2.545s
>> user    0m2.201s
>> sys     0m0.321s
>>
>> The size of the output was 150808 bytes.
>>
>> This is an incredible difference, am I doing something wrong? Or did I
>> have too many expectations of vtd?
>>
>> Regards,
>> John Kraal
>>
> 
> 
> --------------------------------------------------------------------------------
> 
> 
> 
>> /**
>> * section: XPath
>> * synopsis: Evaluate XPath expression and prints result node set.
>> * purpose: Shows how to evaluate XPath expression and register
>> *          known namespaces in XPath context.
>> * usage: xpath1 <xml-file> <xpath-expr> [<known-ns-list>]
>> * test: ./xpath1 test3.xml '//child2' > xpath1.tmp ; diff xpath1.tmp
>> xpath1.res ; rm xpath1.tmp
>> * author: Aleksey Sanin
>> * copy: see Copyright for the status of this software.
>> */
>> #include <stdlib.h>
>> #include <stdio.h>
>> #include <string.h>
>> #include <assert.h>
>>
>> #include <libxml/tree.h>
>> #include <libxml/parser.h>
>> #include <libxml/xpath.h>
>> #include <libxml/xpathInternals.h>
>>
>> #if defined(LIBXML_XPATH_ENABLED) && defined(LIBXML_SAX1_ENABLED)
>>
>>
>> static void usage(const char *name);
>> int  execute_xpath_expression(const char* filename, const xmlChar*
>> xpathExpr, const xmlChar* nsList);
>> int  register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar*
>> nsList);
>> void print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output);
>>
>> int
>> main(int argc, char **argv) {
>>    /* Parse command line and process file */
>>    if((argc < 3) || (argc > 4)) {
>> fprintf(stderr, "Error: wrong number of arguments.\n");
>> usage(argv[0]);
>> return(-1);
>>    }
>>
>>    /* Init libxml */
>>    xmlInitParser();
>>    LIBXML_TEST_VERSION
>>
>>    /* Do the main job */
>>    if(execute_xpath_expression(argv[1], BAD_CAST argv[2], (argc > 3) ?
>> BAD_CAST argv[3] : NULL) < 0) {
>> usage(argv[0]);
>> return(-1);
>>    }
>>
>>    /* Shutdown libxml */
>>    xmlCleanupParser();
>>
>>    /*
>>     * this is to debug memory for regression tests
>>     */
>>    xmlMemoryDump();
>>    return 0;
>> }
>>
>> /**
>> * usage:
>> * @name: the program name.
>> *
>> * Prints usage information.
>> */
>> static void
>> usage(const char *name) {
>>    assert(name);
>>
>>    fprintf(stderr, "Usage: %s <xml-file> <xpath-expr>
>> [<known-ns-list>]\n", name);
>>    fprintf(stderr, "where <known-ns-list> is a list of known
>> namespaces\n");
>>    fprintf(stderr, "in \"<prefix1>=<href1> <prefix2>=href2> ...\"
>> format\n");
>> }
>>
>> /**
>> * execute_xpath_expression:
>> * @filename: the input XML filename.
>> * @xpathExpr: the xpath expression for evaluation.
>> * @nsList: the optional list of known namespaces in
>> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
>> *
>> * Parses input XML file, evaluates XPath expression and prints results.
>> *
>> * Returns 0 on success and a negative value otherwise.
>> */
>> int
>> execute_xpath_expression(const char* filename, const xmlChar*
>> xpathExpr, const xmlChar* nsList) {
>>    xmlDocPtr doc;
>>    xmlXPathContextPtr xpathCtx;
>>    xmlXPathObjectPtr xpathObj;
>>
>>    assert(filename);
>>    assert(xpathExpr);
>>
>>    /* Load XML document */
>>    doc = xmlParseFile(filename);
>>    if (doc == NULL) {
>> fprintf(stderr, "Error: unable to parse file \"%s\"\n", filename);
>> return(-1);
>>    }
>>
>>    /* Create xpath evaluation context */
>>    xpathCtx = xmlXPathNewContext(doc);
>>    if(xpathCtx == NULL) {
>>        fprintf(stderr,"Error: unable to create new XPath context\n");
>>        xmlFreeDoc(doc);
>>        return(-1);
>>    }
>>
>>    /* Register namespaces from list (if any) */
>>    if((nsList != NULL) && (register_namespaces(xpathCtx, nsList) < 0)) {
>>        fprintf(stderr,"Error: failed to register namespaces list
>> \"%s\"\n", nsList);
>>        xmlXPathFreeContext(xpathCtx);
>>        xmlFreeDoc(doc);
>>        return(-1);
>>    }
>>
>>    /* Evaluate xpath expression */
>>    xpathObj = xmlXPathEvalExpression(xpathExpr, xpathCtx);
>>    if(xpathObj == NULL) {
>>        fprintf(stderr,"Error: unable to evaluate xpath expression
>> \"%s\"\n", xpathExpr);
>>        xmlXPathFreeContext(xpathCtx);
>>        xmlFreeDoc(doc);
>>        return(-1);
>>    }
>>
>>    /* Print results */
>>    print_xpath_nodes(xpathObj->nodesetval, stdout);
>>
>>    /* Cleanup */
>>    xmlXPathFreeObject(xpathObj);
>>    xmlXPathFreeContext(xpathCtx);
>>    xmlFreeDoc(doc);
>>
>>    return(0);
>> }
>>
>> /**
>> * register_namespaces:
>> * @xpathCtx: the pointer to an XPath context.
>> * @nsList: the list of known namespaces in
>> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
>> *
>> * Registers namespaces from @nsList in @xpathCtx.
>> *
>> * Returns 0 on success and a negative value otherwise.
>> */
>> int
>> register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar* nsList) {
>>    xmlChar* nsListDup;
>>    xmlChar* prefix;
>>    xmlChar* href;
>>    xmlChar* next;
>>
>>    assert(xpathCtx);
>>    assert(nsList);
>>
>>    nsListDup = xmlStrdup(nsList);
>>    if(nsListDup == NULL) {
>> fprintf(stderr, "Error: unable to strdup namespaces list\n");
>> return(-1);
>>    }
>>
>>    next = nsListDup;
>>    while(next != NULL) {
>> /* skip spaces */
>> while((*next) == ' ') next++;
>> if((*next) == '\0') break;
>>
>> /* find prefix */
>> prefix = next;
>> next = (xmlChar*)xmlStrchr(next, '=');
>> if(next == NULL) {
>>     fprintf(stderr,"Error: invalid namespaces list format\n");
>>     xmlFree(nsListDup);
>>     return(-1);
>> }
>> *(next++) = '\0';
>>
>> /* find href */
>> href = next;
>> next = (xmlChar*)xmlStrchr(next, ' ');
>> if(next != NULL) {
>>     *(next++) = '\0';
>> }
>>
>> /* do register namespace */
>> if(xmlXPathRegisterNs(xpathCtx, prefix, href) != 0) {
>>     fprintf(stderr,"Error: unable to register NS with prefix=\"%s\"
>> and href=\"%s\"\n", prefix, href);
>>     xmlFree(nsListDup);
>>     return(-1);
>> }
>>    }
>>
>>    xmlFree(nsListDup);
>>    return(0);
>> }
>>
>> /**
>> * print_xpath_nodes:
>> * @nodes: the nodes set.
>> * @output: the output file handle.
>> *
>> * Prints the @nodes content to @output.
>> */
>> void
>> print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output) {
>>    xmlNodePtr cur;
>> xmlChar* content;
>>    int size;
>>    int i;
>>
>>    assert(output);
>>    size = (nodes) ? nodes->nodeNr : 0;
>>
>>    fprintf(output, "Result (%d nodes):\n", size);
>>    for(i = 0; i < size; ++i) {
>> assert(nodes->nodeTab[i]);
>>
>> if(nodes->nodeTab[i]->type == XML_NAMESPACE_DECL) {
>>     xmlNsPtr ns;
>>
>>     ns = (xmlNsPtr)nodes->nodeTab[i];
>>     cur = (xmlNodePtr)ns->next;
>>     if(cur->ns) {
>>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s:%s\n",
>>     ns->prefix, ns->href, cur->ns->href, cur->name);
>>     } else {
>>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s\n",
>>     ns->prefix, ns->href, cur->name);
>>     }
>> } else if(nodes->nodeTab[i]->type == XML_ELEMENT_NODE) {
>>     cur = nodes->nodeTab[i];
>> content = xmlNodeGetContent(cur);
>>     if(cur->ns) {
>>            fprintf(output, "= element node \"%s:%s\": \"%s\"\n",
>>     cur->ns->href, cur->name, content);
>>     } else {
>>            fprintf(output, "= element node \"%s\": \"%s\"\n",
>>     cur->name, content);
>>     }
>> xmlFree(content);
>> } else {
>>     cur = nodes->nodeTab[i];
>> content = xmlNodeGetContent(cur);
>>     fprintf(output, "= node \"%s\": type %d: value \"%s\"\n",
>> cur->name, cur->type, content);
>> xmlFree(content);
>> }
>>    }
>> }
>>
>> #else
>> int main(void) {
>>    fprintf(stderr, "XPath support not compiled in\n");
>>    exit(1);
>> }
>> #endif
>>
> 
> 
> --------------------------------------------------------------------------------
> 
> 
> 
>> #include <string.h>
>> #include <stdio.h>
>> #include <wchar.h>
>> #include <stdlib.h>
>> #include <fcntl.h>
>> #include <sys/types.h>
>> #include <sys/stat.h>
>> #include <vtdxml/xpath1.h>
>> #include <vtdxml/helper.h>
>> #include <vtdxml/vtdGen.h>
>>
>> struct exception_context the_exception_context[1];
>>
>> int main(int argc, char **argv, char **envv)
>> {
>> exception e;
>> FILE *f = NULL;
>> int i=0,t,result,count=0;
>> wchar_t *tmpString, *tmpString2;
>> char *filename;
>> wchar_t *xpath;
>> struct stat s;
>> UByte *xml = NULL;
>> VTDGen *vg = NULL;
>> VTDNav *vn = NULL;
>> AutoPilot *ap = NULL;
>>
>> Try
>> {
>> if (argc > 2)
>> {
>> filename = (char *)malloc(strlen(argv[1]) + 1);
>> xpath = (wchar_t *)malloc((strlen(argv[2]) + 1) * sizeof(wchar_t));
>> strcpy(filename, argv[1]);
>> mbstowcs(xpath , argv[2], strlen(argv[2])+1);
>> wprintf(L"Using xpath: %ls\n", xpath);
>> }
>> else
>> {
>> filename = "test.xml";
>> xpath    = L"//*";
>> }
>> f = fopen(filename, "r");
>> stat(filename, &s);
>> i = (int)s.st_size;
>> wprintf(L"size of the file is %d bytes\n", i);
>> xml = (UByte *)malloc(sizeof(UByte) * i);
>> i = fread(xml,sizeof(UByte),i,f);
>>
>> vg = createVTDGen();
>> setDoc(vg,xml,i);
>> parse(vg,TRUE);
>>
>> vn = getNav(vg);
>> ap = createAutoPilot2();
>> declareXPathNameSpace(ap,L"ns1",L"");
>>
>> if (selectXPath(ap, xpath))
>> {
>> bind(ap, vn);
>> while ((result = evalXPath(ap)) != -1)
>> {
>> tmpString = toString(vn, result);
>> t = getText(vn);
>> if (t != -1)
>> {
>> tmpString2 = toNormalizedString(vn,t);
>> wprintf(L"r: %d e: %ls (%d): %ls \n",
>> result,
>> tmpString,
>> t,
>> tmpString2);
>> free(tmpString);
>> free(tmpString2);
>> }
>> count++;
>> }
>> }
>> wprintf(L"\nTotal number of elements %d \n", count);
>> fclose(f);
>>
>> freeVTDNav(vn);
>> freeVTDGen(vg);
>> freeAutoPilot(ap);
>> } Catch (e) {
>> wprintf (L"exception occurred\nerr: %s\nmoreerr: %s\n", e.msg,
>> e.sub_msg);
>> return 1;
>> }
>>
>> return 0;
>> }
>>
> 
> 
> --------------------------------------------------------------------------------
> 
> 
> 
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> 
> 
> --------------------------------------------------------------------------------
> 
> 
> 
>> _______________________________________________
>> Vtd-xml-users mailing list
>> Vtd...@li...
>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>
> 
> 
=

Re: [Vtd-xml-users] built-in entity references

[John K. - K. I. NL <jk...@in...>]

And I've posted the solution; just increment offset before the next
getChar() (or GetCharUnit()).

Jimmy Zhang wrote:
> They should have the same issue since the porting is a literal translat=
ion
> ----- Original Message ----- From: "John Kraal - Kewill Interchain NL"
> <jk...@in...>
> To: "John Kraal - Kewill Interchain NL" <jk...@in...>
> Cc: "Harrie Hoogeveen" <Har...@KB...>;
> <vtd...@li...>
> Sent: Monday, November 06, 2006 2:51 AM
> Subject: Re: [Vtd-xml-users] built-in entity references
>=20
>=20
>> Oops,
>>
>> I'm using c_1.7, not the java-one. Though it seems the errors have the
>> same origin.
>>
>> Kind regards,
>> John Kraal
>>
>> John Kraal - Kewill Interchain NL wrote:
>>> In the new code this actually has nothing to do with "&amps;" - there=
 is
>>> no such check, my guess it's a typo of the website. It does have to d=
o
>>> something with offsets:
>>>
>>> For the package I built, add a ++ after offset in getChar() (in the
>>> function getCharResolved())
>>>
>>> Example, it was this:
>>>
>>>                      static Long getCharResolved(VTDNav *vn,int offse=
t){
>>>                          exception e;
>>>                          int ch =3D 0;
>>>                          int val =3D 0;
>>>                          Long inc =3D2;
>>>                          Long l =3D getChar(vn,offset);
>>>
>>> Change it to:
>>>
>>>                      static Long getCharResolved(VTDNav *vn,int offse=
t){
>>>                          exception e;
>>>                          int ch =3D 0;
>>>                          int val =3D 0;
>>>                          Long inc =3D2;
>>>                          Long l =3D getChar(vn,offset++);
>>>
>>> I'm not sure if this'll work on older versions.
>>>
>>> In a new version I threw in a bunch of typecasts too, to get rid of
>>> those pesky compiler warnings (on HP-UX a killer), but I'll release t=
he
>>> package somewhat later when I'm convinced it works.
>>>
>>> Regards,
>>> John Kraal
>>>
>>>
>>> Harrie Hoogeveen wrote:
>>>> Hello,
>>>>
>>>> I had a question about the built-in entity references. You support, =
as
>>>> told on the website only a few:
>>>>
>>>> &quot; &amps; &apos; &gt; &lt;
>>>>
>>>> Q1: I was wondering why is it not &amp; and &apo; as in the standard=
?
>>>>
>>>> For example a lot of parsers do not understand &amps; but do underst=
and
>>>> &amp;
>>>>
>>>>
>>>>
>>>> Now, I have to adapt my xml, or the VTD code. I am looking at this
>>>> function getCharResolved() from VTDNav.
>>>>
>>>> If you look at the swich, after the & sign, it detects this:
>>>>
>>>>
>>>>
>>>> *case* 'a' :
>>>>
>>>>                         ch =3D getCharUnit(offset);
>>>>
>>>>                         *if* (ch =3D=3D 'm') {
>>>>
>>>>                               *if* (getCharUnit(offset + 1) =3D=3D '=
p'
>>>>
>>>>                                     && getCharUnit(offset + 2) =3D=3D
>>>> ';') {
>>>>
>>>>                                     inc =3D 5;
>>>>
>>>>                                     val =3D '&';
>>>>
>>>>                               } *else*
>>>>
>>>>                                     *throw* *new* NavException("ille=
gal
>>>> builtin reference");
>>>>
>>>>                         } *else* *if* (ch =3D=3D 'p') {
>>>>
>>>>                               *if* (getCharUnit(offset + 1) =3D=3D '=
o'
>>>>
>>>>                                     && getCharUnit(offset + 2) =3D=3D=
 's'
>>>>
>>>>                                     && getCharUnit(offset + 3) =3D=3D
>>>> ';') {
>>>>
>>>>                                     inc =3D 6;
>>>>
>>>>                                     val =3D '\'';
>>>>
>>>>                               } *else*
>>>>
>>>>                                     *throw* *new* NavException("ille=
gal
>>>> builtin reference");
>>>>
>>>>                         } *else*
>>>>
>>>>                               *throw* *new* NavException("illegal
>>>> builtin reference");
>>>>
>>>>                         *break*;
>>>>
>>>>
>>>>
>>>> Which in translation is amp; and apos;
>>>>
>>>>
>>>>
>>>> Now it realy is getting confusing.
>>>>
>>>> If a put &amps; in my xml, it throws an exception any way:
>>>> =E2=80=9Cjavax.ejb.EJBException: com.ximpleware.EntityException: Err=
ors in
>>>> Entity: Illegal builtin reference=E2=80=9D on line 499 in VTDGen.
>>>>
>>>> If I put &amp; in my xml I get a: =E2=80=9CNavException: Invalid ent=
ity
>>>> char=E2=80=9D on
>>>> line 830 in VTDNav.
>>>>
>>>> Of which you can see the code above, which should work for &amp;.
>>>>
>>>> Also the &#38; is supported in VTDNav, if I see correct on line 740,
>>>> but
>>>> also gets a exception
>>>>
>>>> What is going on?
>>>>
>>>> Mabey it=E2=80=99s in this piece of code on line 729 in VTDNav:
>>>>
>>>>
>>>>
>>>>             ch =3D (*int*)l;
>>>>
>>>>
>>>>
>>>>             *if* (ch !=3D '&')
>>>>
>>>>                   *return* l;
>>>>
>>>>
>>>>
>>>> I have some time to correct this. What is the problem? Why did you
>>>> block
>>>> it out like that?
>>>>
>>>>
>>>>
>>>> Harrie Hoogeveen
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------=
----
>>>>
>>>>
>>>> --------------------------------------------------------------------=
-----
>>>>
>>>> Using Tomcat but need to do more? Need to support web services,
>>>> security?
>>>> Get stuff done quickly with pre-integrated technology to make your
>>>> job easier
>>>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>>>> Geronimo
>>>> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&=
dat=3D121642
>>>>
>>>>
>>>>
>>>> --------------------------------------------------------------------=
----
>>>>
>>>>
>>>> _______________________________________________
>>>> Vtd-xml-users mailing list
>>>> Vtd...@li...
>>>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>>
>>
>> ----------------------------------------------------------------------=
---
>> Using Tomcat but need to do more? Need to support web services, securi=
ty?
>> Get stuff done quickly with pre-integrated technology to make your job
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&da=
t=3D121642
>> _______________________________________________
>> Vtd-xml-users mailing list
>> Vtd...@li...
>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>
>=20
>=20

--=20
Met vriendelijke groeten,
With kind regards,

John Kraal
Programmer

Kewill Interchain
A member of the Kewill Group of companies

(T) +31 (0)78 6 123 310
(E)  jk...@in...
(W) www.interchain.nl

IMPORTANT NOTICE: This e-mail, and each of any attachments transmitted
with it, is confidential, is the property of Kewill Interchain, and is
intended solely for the information of the individual/entity to whom
this e-mail is addressed. It may also be privileged or protected by
other legal rules. If you have received it by mistake, please notify the

author immediately by replying to this e-mail and then destroy/delete
this e-mail and all attachments (if any). If you are not the intended
recipient you must not use, disclose, distribute, copy, print, or rely
on this e-mail. In accordance with Kewill Interchain's internal policy
e-mails sent and/or received may be monitored. Whilst Kewill Interchain
takes reasonable precautions (including use of anti-virus software) it
accepts no responsibility for any loss or damage should this e-mail
contain any virus, or similar destructive or mischievous code. (c)
Kewill Interchain 2006.

Re: [Vtd-xml-users] built-in entity references

[Jimmy Z. <cra...@co...>]

It is actually a typo on the vtd-xml project page, teh right entity is & 
not &amps...
----- Original Message ----- 
From: "John Kraal - Kewill Interchain NL" <jk...@in...>
To: "Harrie Hoogeveen" <Har...@KB...>
Cc: <vtd...@li...>
Sent: Monday, November 06, 2006 2:35 AM
Subject: Re: [Vtd-xml-users] built-in entity references


> In the new code this actually has nothing to do with "&amps;" - there is
> no such check, my guess it's a typo of the website. It does have to do
> something with offsets:
>
> For the package I built, add a ++ after offset in getChar() (in the
> function getCharResolved())
>
> Example, it was this:
>
>                     static Long getCharResolved(VTDNav *vn,int offset){
>                         exception e;
>                         int ch = 0;
>                         int val = 0;
>                         Long inc =2;
>                         Long l = getChar(vn,offset);
>
> Change it to:
>
>                     static Long getCharResolved(VTDNav *vn,int offset){
>                         exception e;
>                         int ch = 0;
>                         int val = 0;
>                         Long inc =2;
>                         Long l = getChar(vn,offset++);
>
> I'm not sure if this'll work on older versions.
>
> In a new version I threw in a bunch of typecasts too, to get rid of
> those pesky compiler warnings (on HP-UX a killer), but I'll release the
> package somewhat later when I'm convinced it works.
>
> Regards,
> John Kraal
>
>
> Harrie Hoogeveen wrote:
>> Hello,
>>
>> I had a question about the built-in entity references. You support, as
>> told on the website only a few:
>>
>> &quot; &amps; &apos; &gt; &lt;
>>
>> Q1: I was wondering why is it not &amp; and &apo; as in the standard?
>>
>> For example a lot of parsers do not understand &amps; but do understand
>> &amp;
>>
>>
>>
>> Now, I have to adapt my xml, or the VTD code. I am looking at this
>> function getCharResolved() from VTDNav.
>>
>> If you look at the swich, after the & sign, it detects this:
>>
>>
>>
>> *case* 'a' :
>>
>>                         ch = getCharUnit(offset);
>>
>>                         *if* (ch == 'm') {
>>
>>                               *if* (getCharUnit(offset + 1) == 'p'
>>
>>                                     && getCharUnit(offset + 2) == ';') {
>>
>>                                     inc = 5;
>>
>>                                     val = '&';
>>
>>                               } *else*
>>
>>                                     *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         } *else* *if* (ch == 'p') {
>>
>>                               *if* (getCharUnit(offset + 1) == 'o'
>>
>>                                     && getCharUnit(offset + 2) == 's'
>>
>>                                     && getCharUnit(offset + 3) == ';') {
>>
>>                                     inc = 6;
>>
>>                                     val = '\'';
>>
>>                               } *else*
>>
>>                                     *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         } *else*
>>
>>                               *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         *break*;
>>
>>
>>
>> Which in translation is amp; and apos;
>>
>>
>>
>> Now it realy is getting confusing.
>>
>> If a put &amps; in my xml, it throws an exception any way:
>> “javax.ejb.EJBException: com.ximpleware.EntityException: Errors in
>> Entity: Illegal builtin reference” on line 499 in VTDGen.
>>
>> If I put &amp; in my xml I get a: “NavException: Invalid entity char” on
>> line 830 in VTDNav.
>>
>> Of which you can see the code above, which should work for &amp;.
>>
>> Also the &#38; is supported in VTDNav, if I see correct on line 740, but
>> also gets a exception
>>
>> What is going on?
>>
>> Mabey it’s in this piece of code on line 729 in VTDNav:
>>
>>
>>
>>             ch = (*int*)l;
>>
>>
>>
>>             *if* (ch != '&')
>>
>>                   *return* l;
>>
>>
>>
>> I have some time to correct this. What is the problem? Why did you block
>> it out like that?
>>
>>
>>
>> Harrie Hoogeveen
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job 
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache 
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Vtd-xml-users mailing list
>> Vtd...@li...
>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>
> -- 
> Met vriendelijke groeten,
> With kind regards,
>
> John Kraal
> Programmer
>
> Kewill Interchain
> A member of the Kewill Group of companies
>
> (T) +31 (0)78 6 123 310
> (E)  jk...@in...
> (W) www.interchain.nl
>
> IMPORTANT NOTICE: This e-mail, and each of any attachments transmitted
> with it, is confidential, is the property of Kewill Interchain, and is
> intended solely for the information of the individual/entity to whom
> this e-mail is addressed. It may also be privileged or protected by
> other legal rules. If you have received it by mistake, please notify the
>
> author immediately by replying to this e-mail and then destroy/delete
> this e-mail and all attachments (if any). If you are not the intended
> recipient you must not use, disclose, distribute, copy, print, or rely
> on this e-mail. In accordance with Kewill Interchain's internal policy
> e-mails sent and/or received may be monitored. Whilst Kewill Interchain
> takes reasonable precautions (including use of anti-virus software) it
> accepts no responsibility for any loss or damage should this e-mail
> contain any virus, or similar destructive or mischievous code. (c)
> Kewill Interchain 2006.
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

Re: [Vtd-xml-users] Performance in comparison to libxml2

[Jimmy Z. <cra...@co...>]

Can you provide a few sample XML files that you used for the testing?

I am sure there can be additioal performance tuning for performance
evaluation...

So far the response we got compare VTD-XML favorably with Xerces...
for the java version..
----- Original Message ----- 
From: "John Kraal - Kewill Interchain NL" <jk...@in...>
To: <vtd...@li...>
Sent: Monday, November 06, 2006 4:23 AM
Subject: [Vtd-xml-users] Performance in comparison to libxml2


> Hello,
>
> I've been playing with vtdxml for a while now, and my latest experience
> is not very encouraging to go on :(...
>
> I have created a file with _lots_ of recurring structures and amounts of
> recurring data (about 137 MB), eventually I created a xpath to select
> the name of every relation in the 4th role (doesn't really matter).
>
> So, I executed it, with the code included for vtdxml (see vtdxml.c), and
> it performed like this:
>
> unims@gxvm1:~/src/xmltest$ time ./test ./jkl.files.xml
> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>
> real    0m4.940s
> user    0m4.625s
> sys     0m0.256s
>
> With 162450 bytes of terminal output.
>
> Then, the horrible thing happened, I used lixml2 with an adjusted
> reference xpath-program. (xpath1.c from their website, only with content
> retrieval):
>
> unims@gxvm1:~/src/libxmltest$ time ./test ./jkl.files.xml
> '//relation[@tsroln=04]/tsnam1' >> /dev/null
>
> real    0m2.545s
> user    0m2.201s
> sys     0m0.321s
>
> The size of the output was 150808 bytes.
>
> This is an incredible difference, am I doing something wrong? Or did I
> have too many expectations of vtd?
>
> Regards,
> John Kraal
>


--------------------------------------------------------------------------------


> /**
> * section: XPath
> * synopsis: Evaluate XPath expression and prints result node set.
> * purpose: Shows how to evaluate XPath expression and register
> *          known namespaces in XPath context.
> * usage: xpath1 <xml-file> <xpath-expr> [<known-ns-list>]
> * test: ./xpath1 test3.xml '//child2' > xpath1.tmp ; diff xpath1.tmp 
> xpath1.res ; rm xpath1.tmp
> * author: Aleksey Sanin
> * copy: see Copyright for the status of this software.
> */
> #include <stdlib.h>
> #include <stdio.h>
> #include <string.h>
> #include <assert.h>
>
> #include <libxml/tree.h>
> #include <libxml/parser.h>
> #include <libxml/xpath.h>
> #include <libxml/xpathInternals.h>
>
> #if defined(LIBXML_XPATH_ENABLED) && defined(LIBXML_SAX1_ENABLED)
>
>
> static void usage(const char *name);
> int  execute_xpath_expression(const char* filename, const xmlChar* 
> xpathExpr, const xmlChar* nsList);
> int  register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar* 
> nsList);
> void print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output);
>
> int
> main(int argc, char **argv) {
>    /* Parse command line and process file */
>    if((argc < 3) || (argc > 4)) {
> fprintf(stderr, "Error: wrong number of arguments.\n");
> usage(argv[0]);
> return(-1);
>    }
>
>    /* Init libxml */
>    xmlInitParser();
>    LIBXML_TEST_VERSION
>
>    /* Do the main job */
>    if(execute_xpath_expression(argv[1], BAD_CAST argv[2], (argc > 3) ? 
> BAD_CAST argv[3] : NULL) < 0) {
> usage(argv[0]);
> return(-1);
>    }
>
>    /* Shutdown libxml */
>    xmlCleanupParser();
>
>    /*
>     * this is to debug memory for regression tests
>     */
>    xmlMemoryDump();
>    return 0;
> }
>
> /**
> * usage:
> * @name: the program name.
> *
> * Prints usage information.
> */
> static void
> usage(const char *name) {
>    assert(name);
>
>    fprintf(stderr, "Usage: %s <xml-file> <xpath-expr> 
> [<known-ns-list>]\n", name);
>    fprintf(stderr, "where <known-ns-list> is a list of known 
> namespaces\n");
>    fprintf(stderr, "in \"<prefix1>=<href1> <prefix2>=href2> ...\" 
> format\n");
> }
>
> /**
> * execute_xpath_expression:
> * @filename: the input XML filename.
> * @xpathExpr: the xpath expression for evaluation.
> * @nsList: the optional list of known namespaces in
> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
> *
> * Parses input XML file, evaluates XPath expression and prints results.
> *
> * Returns 0 on success and a negative value otherwise.
> */
> int
> execute_xpath_expression(const char* filename, const xmlChar* xpathExpr, 
> const xmlChar* nsList) {
>    xmlDocPtr doc;
>    xmlXPathContextPtr xpathCtx;
>    xmlXPathObjectPtr xpathObj;
>
>    assert(filename);
>    assert(xpathExpr);
>
>    /* Load XML document */
>    doc = xmlParseFile(filename);
>    if (doc == NULL) {
> fprintf(stderr, "Error: unable to parse file \"%s\"\n", filename);
> return(-1);
>    }
>
>    /* Create xpath evaluation context */
>    xpathCtx = xmlXPathNewContext(doc);
>    if(xpathCtx == NULL) {
>        fprintf(stderr,"Error: unable to create new XPath context\n");
>        xmlFreeDoc(doc);
>        return(-1);
>    }
>
>    /* Register namespaces from list (if any) */
>    if((nsList != NULL) && (register_namespaces(xpathCtx, nsList) < 0)) {
>        fprintf(stderr,"Error: failed to register namespaces list 
> \"%s\"\n", nsList);
>        xmlXPathFreeContext(xpathCtx);
>        xmlFreeDoc(doc);
>        return(-1);
>    }
>
>    /* Evaluate xpath expression */
>    xpathObj = xmlXPathEvalExpression(xpathExpr, xpathCtx);
>    if(xpathObj == NULL) {
>        fprintf(stderr,"Error: unable to evaluate xpath expression 
> \"%s\"\n", xpathExpr);
>        xmlXPathFreeContext(xpathCtx);
>        xmlFreeDoc(doc);
>        return(-1);
>    }
>
>    /* Print results */
>    print_xpath_nodes(xpathObj->nodesetval, stdout);
>
>    /* Cleanup */
>    xmlXPathFreeObject(xpathObj);
>    xmlXPathFreeContext(xpathCtx);
>    xmlFreeDoc(doc);
>
>    return(0);
> }
>
> /**
> * register_namespaces:
> * @xpathCtx: the pointer to an XPath context.
> * @nsList: the list of known namespaces in
> * "<prefix1>=<href1> <prefix2>=href2> ..." format.
> *
> * Registers namespaces from @nsList in @xpathCtx.
> *
> * Returns 0 on success and a negative value otherwise.
> */
> int
> register_namespaces(xmlXPathContextPtr xpathCtx, const xmlChar* nsList) {
>    xmlChar* nsListDup;
>    xmlChar* prefix;
>    xmlChar* href;
>    xmlChar* next;
>
>    assert(xpathCtx);
>    assert(nsList);
>
>    nsListDup = xmlStrdup(nsList);
>    if(nsListDup == NULL) {
> fprintf(stderr, "Error: unable to strdup namespaces list\n");
> return(-1);
>    }
>
>    next = nsListDup;
>    while(next != NULL) {
> /* skip spaces */
> while((*next) == ' ') next++;
> if((*next) == '\0') break;
>
> /* find prefix */
> prefix = next;
> next = (xmlChar*)xmlStrchr(next, '=');
> if(next == NULL) {
>     fprintf(stderr,"Error: invalid namespaces list format\n");
>     xmlFree(nsListDup);
>     return(-1);
> }
> *(next++) = '\0';
>
> /* find href */
> href = next;
> next = (xmlChar*)xmlStrchr(next, ' ');
> if(next != NULL) {
>     *(next++) = '\0';
> }
>
> /* do register namespace */
> if(xmlXPathRegisterNs(xpathCtx, prefix, href) != 0) {
>     fprintf(stderr,"Error: unable to register NS with prefix=\"%s\" and 
> href=\"%s\"\n", prefix, href);
>     xmlFree(nsListDup);
>     return(-1);
> }
>    }
>
>    xmlFree(nsListDup);
>    return(0);
> }
>
> /**
> * print_xpath_nodes:
> * @nodes: the nodes set.
> * @output: the output file handle.
> *
> * Prints the @nodes content to @output.
> */
> void
> print_xpath_nodes(xmlNodeSetPtr nodes, FILE* output) {
>    xmlNodePtr cur;
> xmlChar* content;
>    int size;
>    int i;
>
>    assert(output);
>    size = (nodes) ? nodes->nodeNr : 0;
>
>    fprintf(output, "Result (%d nodes):\n", size);
>    for(i = 0; i < size; ++i) {
> assert(nodes->nodeTab[i]);
>
> if(nodes->nodeTab[i]->type == XML_NAMESPACE_DECL) {
>     xmlNsPtr ns;
>
>     ns = (xmlNsPtr)nodes->nodeTab[i];
>     cur = (xmlNodePtr)ns->next;
>     if(cur->ns) {
>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s:%s\n",
>     ns->prefix, ns->href, cur->ns->href, cur->name);
>     } else {
>         fprintf(output, "= namespace \"%s\"=\"%s\" for node %s\n",
>     ns->prefix, ns->href, cur->name);
>     }
> } else if(nodes->nodeTab[i]->type == XML_ELEMENT_NODE) {
>     cur = nodes->nodeTab[i];
> content = xmlNodeGetContent(cur);
>     if(cur->ns) {
>            fprintf(output, "= element node \"%s:%s\": \"%s\"\n",
>     cur->ns->href, cur->name, content);
>     } else {
>            fprintf(output, "= element node \"%s\": \"%s\"\n",
>     cur->name, content);
>     }
> xmlFree(content);
> } else {
>     cur = nodes->nodeTab[i];
> content = xmlNodeGetContent(cur);
>     fprintf(output, "= node \"%s\": type %d: value \"%s\"\n", cur->name, 
> cur->type, content);
> xmlFree(content);
> }
>    }
> }
>
> #else
> int main(void) {
>    fprintf(stderr, "XPath support not compiled in\n");
>    exit(1);
> }
> #endif
>


--------------------------------------------------------------------------------


> #include <string.h>
> #include <stdio.h>
> #include <wchar.h>
> #include <stdlib.h>
> #include <fcntl.h>
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <vtdxml/xpath1.h>
> #include <vtdxml/helper.h>
> #include <vtdxml/vtdGen.h>
>
> struct exception_context the_exception_context[1];
>
> int main(int argc, char **argv, char **envv)
> {
> exception e;
> FILE *f = NULL;
> int i=0,t,result,count=0;
> wchar_t *tmpString, *tmpString2;
> char *filename;
> wchar_t *xpath;
> struct stat s;
> UByte *xml = NULL;
> VTDGen *vg = NULL;
> VTDNav *vn = NULL;
> AutoPilot *ap = NULL;
>
> Try
> {
> if (argc > 2)
> {
> filename = (char *)malloc(strlen(argv[1]) + 1);
> xpath = (wchar_t *)malloc((strlen(argv[2]) + 1) * sizeof(wchar_t));
> strcpy(filename, argv[1]);
> mbstowcs(xpath , argv[2], strlen(argv[2])+1);
> wprintf(L"Using xpath: %ls\n", xpath);
> }
> else
> {
> filename = "test.xml";
> xpath    = L"//*";
> }
> f = fopen(filename, "r");
> stat(filename, &s);
> i = (int)s.st_size;
> wprintf(L"size of the file is %d bytes\n", i);
> xml = (UByte *)malloc(sizeof(UByte) * i);
> i = fread(xml,sizeof(UByte),i,f);
>
> vg = createVTDGen();
> setDoc(vg,xml,i);
> parse(vg,TRUE);
>
> vn = getNav(vg);
> ap = createAutoPilot2();
> declareXPathNameSpace(ap,L"ns1",L"");
>
> if (selectXPath(ap, xpath))
> {
> bind(ap, vn);
> while ((result = evalXPath(ap)) != -1)
> {
> tmpString = toString(vn, result);
> t = getText(vn);
> if (t != -1)
> {
> tmpString2 = toNormalizedString(vn,t);
> wprintf(L"r: %d e: %ls (%d): %ls \n",
> result,
> tmpString,
> t,
> tmpString2);
> free(tmpString);
> free(tmpString2);
> }
> count++;
> }
> }
> wprintf(L"\nTotal number of elements %d \n", count);
> fclose(f);
>
> freeVTDNav(vn);
> freeVTDGen(vg);
> freeAutoPilot(ap);
> } Catch (e) {
> wprintf (L"exception occurred\nerr: %s\nmoreerr: %s\n", e.msg, e.sub_msg);
> return 1;
> }
>
> return 0;
> }
>


--------------------------------------------------------------------------------


> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642


--------------------------------------------------------------------------------


> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

Re: [Vtd-xml-users] built-in entity references

[Jimmy Z. <cra...@co...>]

They should have the same issue since the porting is a literal translation
----- Original Message ----- 
From: "John Kraal - Kewill Interchain NL" <jk...@in...>
To: "John Kraal - Kewill Interchain NL" <jk...@in...>
Cc: "Harrie Hoogeveen" <Har...@KB...>; 
<vtd...@li...>
Sent: Monday, November 06, 2006 2:51 AM
Subject: Re: [Vtd-xml-users] built-in entity references


> Oops,
>
> I'm using c_1.7, not the java-one. Though it seems the errors have the
> same origin.
>
> Kind regards,
> John Kraal
>
> John Kraal - Kewill Interchain NL wrote:
>> In the new code this actually has nothing to do with "&amps;" - there is
>> no such check, my guess it's a typo of the website. It does have to do
>> something with offsets:
>>
>> For the package I built, add a ++ after offset in getChar() (in the
>> function getCharResolved())
>>
>> Example, it was this:
>>
>>                      static Long getCharResolved(VTDNav *vn,int offset){
>>                          exception e;
>>                          int ch = 0;
>>                          int val = 0;
>>                          Long inc =2;
>>                          Long l = getChar(vn,offset);
>>
>> Change it to:
>>
>>                      static Long getCharResolved(VTDNav *vn,int offset){
>>                          exception e;
>>                          int ch = 0;
>>                          int val = 0;
>>                          Long inc =2;
>>                          Long l = getChar(vn,offset++);
>>
>> I'm not sure if this'll work on older versions.
>>
>> In a new version I threw in a bunch of typecasts too, to get rid of
>> those pesky compiler warnings (on HP-UX a killer), but I'll release the
>> package somewhat later when I'm convinced it works.
>>
>> Regards,
>> John Kraal
>>
>>
>> Harrie Hoogeveen wrote:
>>> Hello,
>>>
>>> I had a question about the built-in entity references. You support, as
>>> told on the website only a few:
>>>
>>> &quot; &amps; &apos; &gt; &lt;
>>>
>>> Q1: I was wondering why is it not &amp; and &apo; as in the standard?
>>>
>>> For example a lot of parsers do not understand &amps; but do understand
>>> &amp;
>>>
>>>
>>>
>>> Now, I have to adapt my xml, or the VTD code. I am looking at this
>>> function getCharResolved() from VTDNav.
>>>
>>> If you look at the swich, after the & sign, it detects this:
>>>
>>>
>>>
>>> *case* 'a' :
>>>
>>>                         ch = getCharUnit(offset);
>>>
>>>                         *if* (ch == 'm') {
>>>
>>>                               *if* (getCharUnit(offset + 1) == 'p'
>>>
>>>                                     && getCharUnit(offset + 2) == ';') {
>>>
>>>                                     inc = 5;
>>>
>>>                                     val = '&';
>>>
>>>                               } *else*
>>>
>>>                                     *throw* *new* NavException("illegal
>>> builtin reference");
>>>
>>>                         } *else* *if* (ch == 'p') {
>>>
>>>                               *if* (getCharUnit(offset + 1) == 'o'
>>>
>>>                                     && getCharUnit(offset + 2) == 's'
>>>
>>>                                     && getCharUnit(offset + 3) == ';') {
>>>
>>>                                     inc = 6;
>>>
>>>                                     val = '\'';
>>>
>>>                               } *else*
>>>
>>>                                     *throw* *new* NavException("illegal
>>> builtin reference");
>>>
>>>                         } *else*
>>>
>>>                               *throw* *new* NavException("illegal
>>> builtin reference");
>>>
>>>                         *break*;
>>>
>>>
>>>
>>> Which in translation is amp; and apos;
>>>
>>>
>>>
>>> Now it realy is getting confusing.
>>>
>>> If a put &amps; in my xml, it throws an exception any way:
>>> “javax.ejb.EJBException: com.ximpleware.EntityException: Errors in
>>> Entity: Illegal builtin reference” on line 499 in VTDGen.
>>>
>>> If I put &amp; in my xml I get a: “NavException: Invalid entity char” on
>>> line 830 in VTDNav.
>>>
>>> Of which you can see the code above, which should work for &amp;.
>>>
>>> Also the &#38; is supported in VTDNav, if I see correct on line 740, but
>>> also gets a exception
>>>
>>> What is going on?
>>>
>>> Mabey it’s in this piece of code on line 729 in VTDNav:
>>>
>>>
>>>
>>>             ch = (*int*)l;
>>>
>>>
>>>
>>>             *if* (ch != '&')
>>>
>>>                   *return* l;
>>>
>>>
>>>
>>> I have some time to correct this. What is the problem? Why did you block
>>> it out like that?
>>>
>>>
>>>
>>> Harrie Hoogeveen
>>>
>>>
>>>
>>>
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> -------------------------------------------------------------------------
>>> Using Tomcat but need to do more? Need to support web services, 
>>> security?
>>> Get stuff done quickly with pre-integrated technology to make your job 
>>> easier
>>> Download IBM WebSphere Application Server v.1.0.1 based on Apache 
>>> Geronimo
>>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>>
>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> Vtd-xml-users mailing list
>>> Vtd...@li...
>>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

Re: [Vtd-xml-users] built-in entity references

[Jimmy Z. <cra...@co...>]

getCharResolved returns a long, upper 32 bit is the offset value of the 
detected
char or entity, lower 32 bit is the actually char detected, this is a 
rewrite in v1.7
it eliminates a instance variable which was shared among many different 
things...

the other thing, I have been doing a lot of comment conversion from // to /* 
*/ :)
----- Original Message ----- 
From: "John Kraal - Kewill Interchain NL" <jk...@in...>
To: "Harrie Hoogeveen" <Har...@KB...>
Cc: <vtd...@li...>
Sent: Monday, November 06, 2006 2:35 AM
Subject: Re: [Vtd-xml-users] built-in entity references


> In the new code this actually has nothing to do with "&amps;" - there is
> no such check, my guess it's a typo of the website. It does have to do
> something with offsets:
>
> For the package I built, add a ++ after offset in getChar() (in the
> function getCharResolved())
>
> Example, it was this:
>
>                     static Long getCharResolved(VTDNav *vn,int offset){
>                         exception e;
>                         int ch = 0;
>                         int val = 0;
>                         Long inc =2;
>                         Long l = getChar(vn,offset);
>
> Change it to:
>
>                     static Long getCharResolved(VTDNav *vn,int offset){
>                         exception e;
>                         int ch = 0;
>                         int val = 0;
>                         Long inc =2;
>                         Long l = getChar(vn,offset++);
>
> I'm not sure if this'll work on older versions.
>
> In a new version I threw in a bunch of typecasts too, to get rid of
> those pesky compiler warnings (on HP-UX a killer), but I'll release the
> package somewhat later when I'm convinced it works.
>
> Regards,
> John Kraal
>
>
> Harrie Hoogeveen wrote:
>> Hello,
>>
>> I had a question about the built-in entity references. You support, as
>> told on the website only a few:
>>
>> &quot; &amps; &apos; &gt; &lt;
>>
>> Q1: I was wondering why is it not &amp; and &apo; as in the standard?
>>
>> For example a lot of parsers do not understand &amps; but do understand
>> &amp;
>>
>>
>>
>> Now, I have to adapt my xml, or the VTD code. I am looking at this
>> function getCharResolved() from VTDNav.
>>
>> If you look at the swich, after the & sign, it detects this:
>>
>>
>>
>> *case* 'a' :
>>
>>                         ch = getCharUnit(offset);
>>
>>                         *if* (ch == 'm') {
>>
>>                               *if* (getCharUnit(offset + 1) == 'p'
>>
>>                                     && getCharUnit(offset + 2) == ';') {
>>
>>                                     inc = 5;
>>
>>                                     val = '&';
>>
>>                               } *else*
>>
>>                                     *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         } *else* *if* (ch == 'p') {
>>
>>                               *if* (getCharUnit(offset + 1) == 'o'
>>
>>                                     && getCharUnit(offset + 2) == 's'
>>
>>                                     && getCharUnit(offset + 3) == ';') {
>>
>>                                     inc = 6;
>>
>>                                     val = '\'';
>>
>>                               } *else*
>>
>>                                     *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         } *else*
>>
>>                               *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         *break*;
>>
>>
>>
>> Which in translation is amp; and apos;
>>
>>
>>
>> Now it realy is getting confusing.
>>
>> If a put &amps; in my xml, it throws an exception any way:
>> “javax.ejb.EJBException: com.ximpleware.EntityException: Errors in
>> Entity: Illegal builtin reference” on line 499 in VTDGen.
>>
>> If I put &amp; in my xml I get a: “NavException: Invalid entity char” on
>> line 830 in VTDNav.
>>
>> Of which you can see the code above, which should work for &amp;.
>>
>> Also the &#38; is supported in VTDNav, if I see correct on line 740, but
>> also gets a exception
>>
>> What is going on?
>>
>> Mabey it’s in this piece of code on line 729 in VTDNav:
>>
>>
>>
>>             ch = (*int*)l;
>>
>>
>>
>>             *if* (ch != '&')
>>
>>                   *return* l;
>>
>>
>>
>> I have some time to correct this. What is the problem? Why did you block
>> it out like that?
>>
>>
>>
>> Harrie Hoogeveen
>>
>>
>>
>>
>>
>>
>> ------------------------------------------------------------------------
>>
>> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support web services, security?
>> Get stuff done quickly with pre-integrated technology to make your job 
>> easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache 
>> Geronimo
>> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>
>>
>> ------------------------------------------------------------------------
>>
>> _______________________________________________
>> Vtd-xml-users mailing list
>> Vtd...@li...
>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>
> -- 
> Met vriendelijke groeten,
> With kind regards,
>
> John Kraal
> Programmer
>
> Kewill Interchain
> A member of the Kewill Group of companies
>
> (T) +31 (0)78 6 123 310
> (E)  jk...@in...
> (W) www.interchain.nl
>
> IMPORTANT NOTICE: This e-mail, and each of any attachments transmitted
> with it, is confidential, is the property of Kewill Interchain, and is
> intended solely for the information of the individual/entity to whom
> this e-mail is addressed. It may also be privileged or protected by
> other legal rules. If you have received it by mistake, please notify the
>
> author immediately by replying to this e-mail and then destroy/delete
> this e-mail and all attachments (if any). If you are not the intended
> recipient you must not use, disclose, distribute, copy, print, or rely
> on this e-mail. In accordance with Kewill Interchain's internal policy
> e-mails sent and/or received may be monitored. Whilst Kewill Interchain
> takes reasonable precautions (including use of anti-virus software) it
> accepts no responsibility for any loss or damage should this e-mail
> contain any virus, or similar destructive or mischievous code. (c)
> Kewill Interchain 2006.
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

Re: [Vtd-xml-users] built-in entity references

[Jimmy Z. <cra...@co...>]

Thanks for pointing that out, I have to look at the code again, it may =
be a bug that
you discovered...
  ----- Original Message -----=20
  From: Harrie Hoogeveen=20
  To: vtd...@li...=20
  Sent: Monday, November 06, 2006 1:46 AM
  Subject: [Vtd-xml-users] built-in entity references


  Hello,

  I had a question about the built-in entity references. You support, as =
told on the website only a few:

  " &amps; ' > <

  Q1: I was wondering why is it not & and &apo; as in the standard?=20

  For example a lot of parsers do not understand &amps; but do =
understand &=20

  =20

  Now, I have to adapt my xml, or the VTD code. I am looking at this =
function getCharResolved() from VTDNav.

  If you look at the swich, after the & sign, it detects this:

  =20

  case 'a' :

                          ch =3D getCharUnit(offset);

                          if (ch =3D=3D 'm') {

                                if (getCharUnit(offset + 1) =3D=3D 'p'

                                      && getCharUnit(offset + 2) =3D=3D =
';') {

                                      inc =3D 5;

                                      val =3D '&';

                                } else

                                      throw new NavException("illegal =
builtin reference");

                          } else if (ch =3D=3D 'p') {

                                if (getCharUnit(offset + 1) =3D=3D 'o'

                                      && getCharUnit(offset + 2) =3D=3D =
's'

                                      && getCharUnit(offset + 3) =3D=3D =
';') {

                                      inc =3D 6;

                                      val =3D '\'';

                                } else

                                      throw new NavException("illegal =
builtin reference");

                          } else

                                throw new NavException("illegal builtin =
reference");

                          break;

  =20

  Which in translation is amp; and apos;

  =20

  Now it realy is getting confusing.

  If a put &amps; in my xml, it throws an exception any way: =
"javax.ejb.EJBException: com.ximpleware.EntityException: Errors in =
Entity: Illegal builtin reference" on line 499 in VTDGen.

  If I put & in my xml I get a: "NavException: Invalid entity char" =
on line 830 in VTDNav.

  Of which you can see the code above, which should work for &.

  Also the & is supported in VTDNav, if I see correct on line 740, =
but also gets a exception

  What is going on?

  Mabey it's in this piece of code on line 729 in VTDNav:

  =20

              ch =3D (int)l;

             =20

              if (ch !=3D '&')

                    return l;

  =20

  I have some time to correct this. What is the problem? Why did you =
block it out like that?

  =20

  Harrie Hoogeveen

  =20

  =20



-------------------------------------------------------------------------=
-----


  =
-------------------------------------------------------------------------=

  Using Tomcat but need to do more? Need to support web services, =
security?
  Get stuff done quickly with pre-integrated technology to make your job =
easier
  Download IBM WebSphere Application Server v.1.0.1 based on Apache =
Geronimo
  =
http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D=
121642


-------------------------------------------------------------------------=
-----


  _______________________________________________
  Vtd-xml-users mailing list
  Vtd...@li...
  https://lists.sourceforge.net/lists/listinfo/vtd-xml-users

[Vtd-xml-users] Performance in comparison to libxml2

[John K. - K. I. NL <jk...@in...>]

Hello,

I've been playing with vtdxml for a while now, and my latest experience
is not very encouraging to go on :(...

I have created a file with _lots_ of recurring structures and amounts of
recurring data (about 137 MB), eventually I created a xpath to select
the name of every relation in the 4th role (doesn't really matter).

So, I executed it, with the code included for vtdxml (see vtdxml.c), and
it performed like this:

unims@gxvm1:~/src/xmltest$ time ./test ./jkl.files.xml
'//relation[@tsroln=04]/tsnam1' >> /dev/null

real    0m4.940s
user    0m4.625s
sys     0m0.256s

With 162450 bytes of terminal output.

Then, the horrible thing happened, I used lixml2 with an adjusted
reference xpath-program. (xpath1.c from their website, only with content
retrieval):

unims@gxvm1:~/src/libxmltest$ time ./test ./jkl.files.xml
'//relation[@tsroln=04]/tsnam1' >> /dev/null

real    0m2.545s
user    0m2.201s
sys     0m0.321s

The size of the output was 150808 bytes.

This is an incredible difference, am I doing something wrong? Or did I
have too many expectations of vtd?

Regards,
John Kraal

Re: [Vtd-xml-users] built-in entity references

[Harrie H. <Har...@KB...>]

I found it. The line is there. It is just commented out.
If line 736 of VTDNav is uncommented, it works fine.
Maybe a buggy feature, not for general use jet?


> -----Oorspronkelijk bericht-----
> Van: John Kraal - Kewill Interchain NL [mailto:jk...@in...]
> Verzonden: maandag 6 november 2006 11:52
> Aan: John Kraal - Kewill Interchain NL
> CC: Harrie Hoogeveen; vtd...@li...
> Onderwerp: Re: [Vtd-xml-users] built-in entity references
>=20
> Oops,
>=20
> I'm using c_1.7, not the java-one. Though it seems the errors have the
> same origin.
>=20
> Kind regards,
> John Kraal
>=20
> John Kraal - Kewill Interchain NL wrote:
> > In the new code this actually has nothing to do with "&amps;" -
there is
> > no such check, my guess it's a typo of the website. It does have to
do
> > something with offsets:
> >
> > For the package I built, add a ++ after offset in getChar() (in the
> > function getCharResolved())
> >
> > Example, it was this:
> >
> >                      static Long getCharResolved(VTDNav *vn,int
offset){
> >                          exception e;
> >                          int ch =3D 0;
> >                          int val =3D 0;
> >                          Long inc =3D2;
> >                          Long l =3D getChar(vn,offset);
> >
> > Change it to:
> >
> >                      static Long getCharResolved(VTDNav *vn,int
offset){
> >                          exception e;
> >                          int ch =3D 0;
> >                          int val =3D 0;
> >                          Long inc =3D2;
> >                          Long l =3D getChar(vn,offset++);
> >
> > I'm not sure if this'll work on older versions.
> >
> > In a new version I threw in a bunch of typecasts too, to get rid of
> > those pesky compiler warnings (on HP-UX a killer), but I'll release
the
> > package somewhat later when I'm convinced it works.
> >
> > Regards,
> > John Kraal
> >
> >
> > Harrie Hoogeveen wrote:
> >> Hello,
> >>
> >> I had a question about the built-in entity references. You support,
as
> >> told on the website only a few:
> >>
> >> " &amps; ' > <
> >>
> >> Q1: I was wondering why is it not & and &apo; as in the
standard?
> >>
> >> For example a lot of parsers do not understand &amps; but do
understand
> >> &
> >>
> >>
> >>
> >> Now, I have to adapt my xml, or the VTD code. I am looking at this
> >> function getCharResolved() from VTDNav.
> >>
> >> If you look at the swich, after the & sign, it detects this:
> >>
> >>
> >>
> >> *case* 'a' :
> >>
> >>                         ch =3D getCharUnit(offset);
> >>
> >>                         *if* (ch =3D=3D 'm') {
> >>
> >>                               *if* (getCharUnit(offset + 1) =3D=3D =
'p'
> >>
> >>                                     && getCharUnit(offset + 2) =
=3D=3D
';')
> {
> >>
> >>                                     inc =3D 5;
> >>
> >>                                     val =3D '&';
> >>
> >>                               } *else*
> >>
> >>                                     *throw* *new*
NavException("illegal
> >> builtin reference");
> >>
> >>                         } *else* *if* (ch =3D=3D 'p') {
> >>
> >>                               *if* (getCharUnit(offset + 1) =3D=3D =
'o'
> >>
> >>                                     && getCharUnit(offset + 2) =
=3D=3D
's'
> >>
> >>                                     && getCharUnit(offset + 3) =
=3D=3D
';')
> {
> >>
> >>                                     inc =3D 6;
> >>
> >>                                     val =3D '\'';
> >>
> >>                               } *else*
> >>
> >>                                     *throw* *new*
NavException("illegal
> >> builtin reference");
> >>
> >>                         } *else*
> >>
> >>                               *throw* *new* NavException("illegal
> >> builtin reference");
> >>
> >>                         *break*;
> >>
> >>
> >>
> >> Which in translation is amp; and apos;
> >>
> >>
> >>
> >> Now it realy is getting confusing.
> >>
> >> If a put &amps; in my xml, it throws an exception any way:
> >> "javax.ejb.EJBException: com.ximpleware.EntityException: Errors in
> >> Entity: Illegal builtin reference" on line 499 in VTDGen.
> >>
> >> If I put & in my xml I get a: "NavException: Invalid entity
char"
> on
> >> line 830 in VTDNav.
> >>
> >> Of which you can see the code above, which should work for &.
> >>
> >> Also the & is supported in VTDNav, if I see correct on line
740,
> but
> >> also gets a exception
> >>
> >> What is going on?
> >>
> >> Mabey it's in this piece of code on line 729 in VTDNav:
> >>
> >>
> >>
> >>             ch =3D (*int*)l;
> >>
> >>
> >>
> >>             *if* (ch !=3D '&')
> >>
> >>                   *return* l;
> >>
> >>
> >>
> >> I have some time to correct this. What is the problem? Why did you
> block
> >> it out like that?
> >>
> >>
> >>
> >> Harrie Hoogeveen
> >>
> >>
> >>
> >>
> >>
> >>
> >>
-----------------------------------------------------------------------
> -
> >>
> >>
-----------------------------------------------------------------------
> --
> >> Using Tomcat but need to do more? Need to support web services,
> security?
> >> Get stuff done quickly with pre-integrated technology to make your
job
> easier
> >> Download IBM WebSphere Application Server v.1.0.1 based on Apache
> Geronimo
> >> http://sel.as-
> us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=3D121642
> >>
> >>
> >>
-----------------------------------------------------------------------
> -
> >>
> >> _______________________________________________
> >> Vtd-xml-users mailing list
> >> Vtd...@li...
> >> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> >

Re: [Vtd-xml-users] built-in entity references

[John K. - K. I. NL <jk...@in...>]

Oops,

I'm using c_1.7, not the java-one. Though it seems the errors have the
same origin.

Kind regards,
John Kraal

John Kraal - Kewill Interchain NL wrote:
> In the new code this actually has nothing to do with "&amps;" - there i=
s
> no such check, my guess it's a typo of the website. It does have to do
> something with offsets:
>=20
> For the package I built, add a ++ after offset in getChar() (in the
> function getCharResolved())
>=20
> Example, it was this:
>=20
>                      static Long getCharResolved(VTDNav *vn,int offset)=
{
>                          exception e;
>                          int ch =3D 0;
>                          int val =3D 0;
>                          Long inc =3D2;
>                          Long l =3D getChar(vn,offset);
>=20
> Change it to:
>=20
>                      static Long getCharResolved(VTDNav *vn,int offset)=
{
>                          exception e;
>                          int ch =3D 0;
>                          int val =3D 0;
>                          Long inc =3D2;
>                          Long l =3D getChar(vn,offset++);
>=20
> I'm not sure if this'll work on older versions.
>=20
> In a new version I threw in a bunch of typecasts too, to get rid of
> those pesky compiler warnings (on HP-UX a killer), but I'll release the
> package somewhat later when I'm convinced it works.
>=20
> Regards,
> John Kraal
>=20
>=20
> Harrie Hoogeveen wrote:
>> Hello,
>>
>> I had a question about the built-in entity references. You support, as
>> told on the website only a few:
>>
>> " &amps; ' > <
>>
>> Q1: I was wondering why is it not & and &apo; as in the standard?
>>
>> For example a lot of parsers do not understand &amps; but do understan=
d
>> &
>>
>> =20
>>
>> Now, I have to adapt my xml, or the VTD code. I am looking at this
>> function getCharResolved() from VTDNav.
>>
>> If you look at the swich, after the & sign, it detects this:
>>
>> =20
>>
>> *case* 'a' :
>>
>>                         ch =3D getCharUnit(offset);
>>
>>                         *if* (ch =3D=3D 'm') {
>>
>>                               *if* (getCharUnit(offset + 1) =3D=3D 'p'
>>
>>                                     && getCharUnit(offset + 2) =3D=3D =
';') {
>>
>>                                     inc =3D 5;
>>
>>                                     val =3D '&';
>>
>>                               } *else*
>>
>>                                     *throw* *new* NavException("illega=
l
>> builtin reference");
>>
>>                         } *else* *if* (ch =3D=3D 'p') {
>>
>>                               *if* (getCharUnit(offset + 1) =3D=3D 'o'
>>
>>                                     && getCharUnit(offset + 2) =3D=3D =
's'
>>
>>                                     && getCharUnit(offset + 3) =3D=3D =
';') {
>>
>>                                     inc =3D 6;
>>
>>                                     val =3D '\'';
>>
>>                               } *else*
>>
>>                                     *throw* *new* NavException("illega=
l
>> builtin reference");
>>
>>                         } *else*
>>
>>                               *throw* *new* NavException("illegal
>> builtin reference");
>>
>>                         *break*;
>>
>> =20
>>
>> Which in translation is amp; and apos;
>>
>> =20
>>
>> Now it realy is getting confusing.
>>
>> If a put &amps; in my xml, it throws an exception any way:
>> =E2=80=9Cjavax.ejb.EJBException: com.ximpleware.EntityException: Error=
s in
>> Entity: Illegal builtin reference=E2=80=9D on line 499 in VTDGen.
>>
>> If I put & in my xml I get a: =E2=80=9CNavException: Invalid entit=
y char=E2=80=9D on
>> line 830 in VTDNav.
>>
>> Of which you can see the code above, which should work for &.
>>
>> Also the & is supported in VTDNav, if I see correct on line 740, b=
ut
>> also gets a exception
>>
>> What is going on?
>>
>> Mabey it=E2=80=99s in this piece of code on line 729 in VTDNav:
>>
>> =20
>>
>>             ch =3D (*int*)l;
>>
>>           =20
>>
>>             *if* (ch !=3D '&')
>>
>>                   *return* l;
>>
>> =20
>>
>> I have some time to correct this. What is the problem? Why did you blo=
ck
>> it out like that?
>>
>> =20
>>
>> Harrie Hoogeveen
>>
>> =20
>>
>> =20
>>
>>
>> ----------------------------------------------------------------------=
--
>>
>> ----------------------------------------------------------------------=
---
>> Using Tomcat but need to do more? Need to support web services, securi=
ty?
>> Get stuff done quickly with pre-integrated technology to make your job=
 easier
>> Download IBM WebSphere Application Server v.1.0.1 based on Apache Gero=
nimo
>> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&da=
t=3D121642
>>
>>
>> ----------------------------------------------------------------------=
--
>>
>> _______________________________________________
>> Vtd-xml-users mailing list
>> Vtd...@li...
>> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>=20

Re: [Vtd-xml-users] built-in entity references

[John K. - K. I. NL <jk...@in...>]

In the new code this actually has nothing to do with "&amps;" - there is
no such check, my guess it's a typo of the website. It does have to do
something with offsets:

For the package I built, add a ++ after offset in getChar() (in the
function getCharResolved())

Example, it was this:

                     static Long getCharResolved(VTDNav *vn,int offset){
                         exception e;
                         int ch =3D 0;
                         int val =3D 0;
                         Long inc =3D2;
                         Long l =3D getChar(vn,offset);

Change it to:

                     static Long getCharResolved(VTDNav *vn,int offset){
                         exception e;
                         int ch =3D 0;
                         int val =3D 0;
                         Long inc =3D2;
                         Long l =3D getChar(vn,offset++);

I'm not sure if this'll work on older versions.

In a new version I threw in a bunch of typecasts too, to get rid of
those pesky compiler warnings (on HP-UX a killer), but I'll release the
package somewhat later when I'm convinced it works.

Regards,
John Kraal


Harrie Hoogeveen wrote:
> Hello,
>=20
> I had a question about the built-in entity references. You support, as
> told on the website only a few:
>=20
> " &amps; ' > <
>=20
> Q1: I was wondering why is it not & and &apo; as in the standard?
>=20
> For example a lot of parsers do not understand &amps; but do understand
> &
>=20
> =20
>=20
> Now, I have to adapt my xml, or the VTD code. I am looking at this
> function getCharResolved() from VTDNav.
>=20
> If you look at the swich, after the & sign, it detects this:
>=20
> =20
>=20
> *case* 'a' :
>=20
>                         ch =3D getCharUnit(offset);
>=20
>                         *if* (ch =3D=3D 'm') {
>=20
>                               *if* (getCharUnit(offset + 1) =3D=3D 'p'
>=20
>                                     && getCharUnit(offset + 2) =3D=3D '=
;') {
>=20
>                                     inc =3D 5;
>=20
>                                     val =3D '&';
>=20
>                               } *else*
>=20
>                                     *throw* *new* NavException("illegal
> builtin reference");
>=20
>                         } *else* *if* (ch =3D=3D 'p') {
>=20
>                               *if* (getCharUnit(offset + 1) =3D=3D 'o'
>=20
>                                     && getCharUnit(offset + 2) =3D=3D '=
s'
>=20
>                                     && getCharUnit(offset + 3) =3D=3D '=
;') {
>=20
>                                     inc =3D 6;
>=20
>                                     val =3D '\'';
>=20
>                               } *else*
>=20
>                                     *throw* *new* NavException("illegal
> builtin reference");
>=20
>                         } *else*
>=20
>                               *throw* *new* NavException("illegal
> builtin reference");
>=20
>                         *break*;
>=20
> =20
>=20
> Which in translation is amp; and apos;
>=20
> =20
>=20
> Now it realy is getting confusing.
>=20
> If a put &amps; in my xml, it throws an exception any way:
> =E2=80=9Cjavax.ejb.EJBException: com.ximpleware.EntityException: Errors=
 in
> Entity: Illegal builtin reference=E2=80=9D on line 499 in VTDGen.
>=20
> If I put & in my xml I get a: =E2=80=9CNavException: Invalid entity=
 char=E2=80=9D on
> line 830 in VTDNav.
>=20
> Of which you can see the code above, which should work for &.
>=20
> Also the & is supported in VTDNav, if I see correct on line 740, bu=
t
> also gets a exception
>=20
> What is going on?
>=20
> Mabey it=E2=80=99s in this piece of code on line 729 in VTDNav:
>=20
> =20
>=20
>             ch =3D (*int*)l;
>=20
>           =20
>=20
>             *if* (ch !=3D '&')
>=20
>                   *return* l;
>=20
> =20
>=20
> I have some time to correct this. What is the problem? Why did you bloc=
k
> it out like that?
>=20
> =20
>=20
> Harrie Hoogeveen
>=20
> =20
>=20
> =20
>=20
>=20
> -----------------------------------------------------------------------=
-
>=20
> -----------------------------------------------------------------------=
--
> Using Tomcat but need to do more? Need to support web services, securit=
y?
> Get stuff done quickly with pre-integrated technology to make your job =
easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geron=
imo
> http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057&dat=
=3D121642
>=20
>=20
> -----------------------------------------------------------------------=
-
>=20
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users

--=20
Met vriendelijke groeten,
With kind regards,

John Kraal
Programmer

Kewill Interchain
A member of the Kewill Group of companies

(T) +31 (0)78 6 123 310
(E)  jk...@in...
(W) www.interchain.nl

IMPORTANT NOTICE: This e-mail, and each of any attachments transmitted
with it, is confidential, is the property of Kewill Interchain, and is
intended solely for the information of the individual/entity to whom
this e-mail is addressed. It may also be privileged or protected by
other legal rules. If you have received it by mistake, please notify the

author immediately by replying to this e-mail and then destroy/delete
this e-mail and all attachments (if any). If you are not the intended
recipient you must not use, disclose, distribute, copy, print, or rely
on this e-mail. In accordance with Kewill Interchain's internal policy
e-mails sent and/or received may be monitored. Whilst Kewill Interchain
takes reasonable precautions (including use of anti-virus software) it
accepts no responsibility for any loss or damage should this e-mail
contain any virus, or similar destructive or mischievous code. (c)
Kewill Interchain 2006.

[Vtd-xml-users] built-in entity references

[Harrie H. <Har...@KB...>]

Hello,

I had a question about the built-in entity references. You support, as
told on the website only a few:

" &amps; ' > <

Q1: I was wondering why is it not & and &apo; as in the standard?=20

For example a lot of parsers do not understand &amps; but do understand
&=20

=20

Now, I have to adapt my xml, or the VTD code. I am looking at this
function getCharResolved() from VTDNav.

If you look at the swich, after the & sign, it detects this:

=20

case 'a' :

                        ch =3D getCharUnit(offset);

                        if (ch =3D=3D 'm') {

                              if (getCharUnit(offset + 1) =3D=3D 'p'

                                    && getCharUnit(offset + 2) =3D=3D =
';') {

                                    inc =3D 5;

                                    val =3D '&';

                              } else

                                    throw new NavException("illegal
builtin reference");

                        } else if (ch =3D=3D 'p') {

                              if (getCharUnit(offset + 1) =3D=3D 'o'

                                    && getCharUnit(offset + 2) =3D=3D =
's'

                                    && getCharUnit(offset + 3) =3D=3D =
';') {

                                    inc =3D 6;

                                    val =3D '\'';

                              } else

                                    throw new NavException("illegal
builtin reference");

                        } else

                              throw new NavException("illegal builtin
reference");

                        break;

=20

Which in translation is amp; and apos;

=20

Now it realy is getting confusing.

If a put &amps; in my xml, it throws an exception any way:
"javax.ejb.EJBException: com.ximpleware.EntityException: Errors in
Entity: Illegal builtin reference" on line 499 in VTDGen.

If I put & in my xml I get a: "NavException: Invalid entity char" on
line 830 in VTDNav.

Of which you can see the code above, which should work for &.

Also the & is supported in VTDNav, if I see correct on line 740, but
also gets a exception

What is going on?

Mabey it's in this piece of code on line 729 in VTDNav:

=20

            ch =3D (int)l;

           =20

            if (ch !=3D '&')

                  return l;

=20

I have some time to correct this. What is the problem? Why did you block
it out like that?

=20

Harrie Hoogeveen

=20

=20

Re: [Vtd-xml-users] vtd-xml 1.7 compiling

[Jimmy Z. <cra...@co...>]

One more thing: can you change vtd-xml.cpp to vtd-xml.c just to be =
safe...
Jimmy
  ----- Original Message -----=20
  From: Chinh Ho=20
  To: cra...@co...=20
  Sent: Friday, November 03, 2006 8:08 AM
  Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling


  Jimmy,
  If I remove the y.tab.c, there are the other errors.  Please take a =
look at=20
  the log file.

  Thanks


  >From: "Jimmy Zhang" <cra...@co...>
  >To: "Chinh Ho" <ho_...@ho...>
  >CC: <vtd...@li...>
  >Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
  >Date: Thu, 2 Nov 2006 10:05:52 -0800
  >
  >Chinh, there is a file called y.tab.c that must be removed, then =
everything
  >will compile...
  >JImmy
  >----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
  >To: <cra...@co...>
  >Cc: <vtd...@li...>
  >Sent: Wednesday, November 01, 2006 12:42 PM
  >Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
  >
  >
  >>For the second error, I included every file from the 1.7_c folder =
that I=20
  >>extracted from the downloaded ximpleware.  That means the l8.y and =
l8.l=20
  >>were included.
  >>
  >>
  >>>From: "Jimmy Zhang" <cra...@co...>
  >>>To: "Chinh Ho" <ho_...@ho...>
  >>>CC: <vtd...@li...>
  >>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
  >>>Date: Tue, 31 Oct 2006 14:10:09 -0800
  >>>
  >>>For the first error, I think the file size has exceeded 2GB, which =
is why=20
  >>>a negative
  >>>number is shown...
  >>>
  >>>for the second error, did you include "l8.y" and "l8.l" as part of =
the=20
  >>>project?
  >>>----- Original Message ----- From: "Chinh Ho" =
<ho_...@ho...>
  >>>To: <cra...@co...>
  >>>Cc: <vtd...@li...>
  >>>Sent: Monday, October 30, 2006 3:55 PM
  >>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
  >>>
  >>>
  >>>>Yes.  If you have the instruction, please let me know the link.
  >>>>
  >>>>Thanks
  >>>>
  >>>>
  >>>>>From: "Jimmy Zhang" <cra...@co...>
  >>>>>To: "Chinh Ho" <ho_...@ho...>
  >>>>>CC: <vtd...@li...>
  >>>>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
  >>>>>Date: Thu, 26 Oct 2006 19:46:14 -0700
  >>>>>
  >>>>>do you still have the same issue compiling v 1.7??
  >>>>>----- Original Message ----- From: "Jimmy Zhang" =
<cra...@co...>
  >>>>>To: "Chinh Ho" <ho_...@ho...>
  >>>>>Cc: <vtd...@li...>
  >>>>>Sent: Wednesday, October 25, 2006 11:14 AM
  >>>>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
  >>>>>
  >>>>>
  >>>>>>The file size must have been greater than 2GB, as the size of =
file=20
  >>>>>>becomes
  >>>>>>negative (> 0x7fffffff), have to look into the other problem
  >>>>>>----- Original Message ----- From: "Chinh Ho" =
<ho_...@ho...>
  >>>>>>To: <cra...@co...>
  >>>>>>Cc: <vtd...@li...>
  >>>>>>Sent: Wednesday, October 25, 2006 10:16 AM
  >>>>>>Subject: Re: vtd-xml 1.7 compiling
  >>>>>>
  >>>>>>
  >>>>>>>For 1.7_c_light, there is no errors.  However, when I run the=20
  >>>>>>>benchmark to
  >>>>>>>open a 2 Gig xml file, it displayed "size of the file is=20
  >>>>>>>-1841250597".
  >>>>>>>That's an error.
  >>>>>>>
  >>>>>>>For 1.7_c, the errors are on the l8.* files.  For example, one =
of the
  >>>>>>>errors is:
  >>>>>>>
  >>>>>>>l8.y(583) : error C2065: 'xpathInputPtr' : undeclared =
identifier
  >>>>>>>
  >>>>>>>
  >>>>>>>>From: "Jimmy Zhang" <cra...@co...>
  >>>>>>>>To: "Chinh Ho" <ho_...@ho...>
  >>>>>>>>CC: <vtd...@li...>
  >>>>>>>>Subject: Re: vtd-xml 1.7 compiling
  >>>>>>>>Date: Wed, 25 Oct 2006 09:46:16 -0700
  >>>>>>>>
  >>>>>>>>What is the error message?
  >>>>>>>>----- Original Message ----- From: "Chinh Ho" =
<ho_...@ho...>
  >>>>>>>>To: <cra...@co...>; =
<vtd...@li...>
  >>>>>>>>Sent: Wednesday, October 25, 2006 8:04 AM
  >>>>>>>>Subject: vtd-xml 1.7 compiling
  >>>>>>>>
  >>>>>>>>
  >>>>>>>>>Hi Jimmy,
  >>>>>>>>>I downloaded 2 versions of the vtd-xml 1.7, ximpleware_1.7_c =
and
  >>>>>>>>>ximpleware_1.7_c_light.  The 1.7_c is for Unix / Linux OS, =
isn't=20
  >>>>>>>>>it? The
  >>>>>>>>>1.7_c_light is for Windows OS?
  >>>>>>>>>
  >>>>>>>>>I could not compile the 1.7_c under Windows.  I am using =
WinXP and
  >>>>>>>>>MSVC2005.
  >>>>>>>>>
  >>>>>>>>>If both of them are for windows, how do I compile them?
  >>>>>>>>>
  >>>>>>>>>Do you have any instructions?  Readme?
  >>>>>>>>>
  >>>>>>>>>Thanks
  >>>>>>>>>
  =
>>>>>>>>>________________________________________________________________=
_
  >>>>>>>>>Try Search Survival Kits: Fix up your home and better handle =
your=20
  >>>>>>>>>cash
  >>>>>>>>>with Live Search!
  =
>>>>>>>>>http://imagine-windowslive.com/search/kits/default.aspx?kit=3Dim=
prove&locale=3Den-US&source=3Dhmtagline
  >>>>>>>>>
  >>>>>>>>>
  >>>>>>>>
  >>>>>>>>
  >>>>>>>
  =
>>>>>>>_________________________________________________________________
  >>>>>>>Try the next generation of search with Windows Live Search =
today!
  =
>>>>>>>http://imagine-windowslive.com/minisites/searchlaunch/?locale=3Den=
-us&source=3Dhmtagline
  >>>>>>>
  >>>>>>>
  >>>>>>
  >>>>>>
  >>>>>>
  =
>>>>>>-------------------------------------------------------------------=
------
  >>>>>>Using Tomcat but need to do more? Need to support web services,=20
  >>>>>>security?
  >>>>>>Get stuff done quickly with pre-integrated technology to make =
your job=20
  >>>>>>easier
  >>>>>>Download IBM WebSphere Application Server v.1.0.1 based on =
Apache=20
  >>>>>>Geronimo
  =
>>>>>>http://sel.as-us.falkag.net/sel?cmd=3Dlnk&kid=3D120709&bid=3D263057=
&dat=3D121642
  >>>>>>_______________________________________________
  >>>>>>Vtd-xml-users mailing list
  >>>>>>Vtd...@li...
  >>>>>>https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
  >>>>>>
  >>>>>
  >>>>>
  >>>>
  >>>>_________________________________________________________________
  >>>>Stay in touch with old friends and meet new ones with Windows Live =

  >>>>Spaces=20
  =
>>>>http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=3Dh=
ttp://spaces.live.com/spacesapi.aspx?wx_action=3Dcreate&wx_url=3D/friends=
.aspx&mkt=3Den-us
  >>>>
  >>>>
  >>>
  >>>
  >>
  >>_________________________________________________________________
  >>Try Search Survival Kits: Fix up your home and better handle your =
cash=20
  >>with Live Search!=20
  =
>>http://imagine-windowslive.com/search/kits/default.aspx?kit=3Dimprove&l=
ocale=3Den-US&source=3Dhmtagline
  >>
  >>
  >
  >

  _________________________________________________________________
  Stay in touch with old friends and meet new ones with Windows Live =
Spaces=20
  =
http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=3Dhttp:=
//spaces.live.com/spacesapi.aspx?wx_action=3Dcreate&wx_url=3D/friends.asp=
x&mkt=3Den-us

Re: [Vtd-xml-users] design question/best practice

[Tatu S. <cow...@ya...>]

--- Jimmy Zhang <cra...@co...> wrote:

> Tatu, woodstox still has to recognize various
> encoding delcarations right,,
> the question is really what kind of encoding
> declaration a parser has to
> recognize.... for ascii,

Well, not necessarily, since I can just try to
construct JDK-based decoder (as InputStreamReader)
with the given encoding. However:

> one can have encoding="ascii" or
> encoding="us-ascii".. does woodstox
> support both?

Yes. I do check for some of aliases (from the
linked-to IANA page).

> what about for windows encoding? cp-1250 or cp1250

Those I defer to JDK. ;-)

But any encodings Woodstox natively supports, I do use
loose comparison, which essentially strips out
non-alphanum characters (spaces, dashes, underscores),
and does case-insensitive comparison. So cp-1250 and
cp1250 are considered equivalent.

-+ Tatu +-



 
____________________________________________________________________________________
Everyone is raving about the all-new Yahoo! Mail 
(http://advision.webevents.yahoo.com/mailbeta/)

Re: [Vtd-xml-users] vtd-xml 1.7 compiling

[Jimmy Z. <cra...@co...>]

Chinh, there is a file called y.tab.c that must be removed, then everything
will compile...
JImmy
----- Original Message ----- 
From: "Chinh Ho" <ho_...@ho...>
To: <cra...@co...>
Cc: <vtd...@li...>
Sent: Wednesday, November 01, 2006 12:42 PM
Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling


> For the second error, I included every file from the 1.7_c folder that I 
> extracted from the downloaded ximpleware.  That means the l8.y and l8.l 
> were included.
>
>
>>From: "Jimmy Zhang" <cra...@co...>
>>To: "Chinh Ho" <ho_...@ho...>
>>CC: <vtd...@li...>
>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>Date: Tue, 31 Oct 2006 14:10:09 -0800
>>
>>For the first error, I think the file size has exceeded 2GB, which is why 
>>a negative
>>number is shown...
>>
>>for the second error, did you include "l8.y" and "l8.l" as part of the 
>>project?
>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>To: <cra...@co...>
>>Cc: <vtd...@li...>
>>Sent: Monday, October 30, 2006 3:55 PM
>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>
>>
>>>Yes.  If you have the instruction, please let me know the link.
>>>
>>>Thanks
>>>
>>>
>>>>From: "Jimmy Zhang" <cra...@co...>
>>>>To: "Chinh Ho" <ho_...@ho...>
>>>>CC: <vtd...@li...>
>>>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>>>Date: Thu, 26 Oct 2006 19:46:14 -0700
>>>>
>>>>do you still have the same issue compiling v 1.7??
>>>>----- Original Message ----- From: "Jimmy Zhang" <cra...@co...>
>>>>To: "Chinh Ho" <ho_...@ho...>
>>>>Cc: <vtd...@li...>
>>>>Sent: Wednesday, October 25, 2006 11:14 AM
>>>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>>>
>>>>
>>>>>The file size must have been greater than 2GB, as the size of file 
>>>>>becomes
>>>>>negative (> 0x7fffffff), have to look into the other problem
>>>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>>>To: <cra...@co...>
>>>>>Cc: <vtd...@li...>
>>>>>Sent: Wednesday, October 25, 2006 10:16 AM
>>>>>Subject: Re: vtd-xml 1.7 compiling
>>>>>
>>>>>
>>>>>>For 1.7_c_light, there is no errors.  However, when I run the 
>>>>>>benchmark to
>>>>>>open a 2 Gig xml file, it displayed "size of the file is -1841250597".
>>>>>>That's an error.
>>>>>>
>>>>>>For 1.7_c, the errors are on the l8.* files.  For example, one of the
>>>>>>errors is:
>>>>>>
>>>>>>l8.y(583) : error C2065: 'xpathInputPtr' : undeclared identifier
>>>>>>
>>>>>>
>>>>>>>From: "Jimmy Zhang" <cra...@co...>
>>>>>>>To: "Chinh Ho" <ho_...@ho...>
>>>>>>>CC: <vtd...@li...>
>>>>>>>Subject: Re: vtd-xml 1.7 compiling
>>>>>>>Date: Wed, 25 Oct 2006 09:46:16 -0700
>>>>>>>
>>>>>>>What is the error message?
>>>>>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>>>>>To: <cra...@co...>; <vtd...@li...>
>>>>>>>Sent: Wednesday, October 25, 2006 8:04 AM
>>>>>>>Subject: vtd-xml 1.7 compiling
>>>>>>>
>>>>>>>
>>>>>>>>Hi Jimmy,
>>>>>>>>I downloaded 2 versions of the vtd-xml 1.7, ximpleware_1.7_c and
>>>>>>>>ximpleware_1.7_c_light.  The 1.7_c is for Unix / Linux OS, isn't it? 
>>>>>>>>The
>>>>>>>>1.7_c_light is for Windows OS?
>>>>>>>>
>>>>>>>>I could not compile the 1.7_c under Windows.  I am using WinXP and
>>>>>>>>MSVC2005.
>>>>>>>>
>>>>>>>>If both of them are for windows, how do I compile them?
>>>>>>>>
>>>>>>>>Do you have any instructions?  Readme?
>>>>>>>>
>>>>>>>>Thanks
>>>>>>>>
>>>>>>>>_________________________________________________________________
>>>>>>>>Try Search Survival Kits: Fix up your home and better handle your 
>>>>>>>>cash
>>>>>>>>with Live Search!
>>>>>>>>http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>_________________________________________________________________
>>>>>>Try the next generation of search with Windows Live Search today!
>>>>>>http://imagine-windowslive.com/minisites/searchlaunch/?locale=en-us&source=hmtagline
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>-------------------------------------------------------------------------
>>>>>Using Tomcat but need to do more? Need to support web services, 
>>>>>security?
>>>>>Get stuff done quickly with pre-integrated technology to make your job 
>>>>>easier
>>>>>Download IBM WebSphere Application Server v.1.0.1 based on Apache 
>>>>>Geronimo
>>>>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>>>>_______________________________________________
>>>>>Vtd-xml-users mailing list
>>>>>Vtd...@li...
>>>>>https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>>>>
>>>>
>>>>
>>>
>>>_________________________________________________________________
>>>Stay in touch with old friends and meet new ones with Windows Live Spaces 
>>>http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
>>>
>>>
>>
>>
>
> _________________________________________________________________
> Try Search Survival Kits: Fix up your home and better handle your cash 
> with Live Search! 
> http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline
>
> 

Re: [Vtd-xml-users] design question/best practice

[Jimmy Z. <cra...@co...>]

Tatu, woodstox still has to recognize various encoding delcarations right,,
the question is really what kind of encoding declaration a parser has to
recognize.... for ascii,
one can have encoding="ascii" or encoding="us-ascii".. does woodstox
support both?
what about for windows encoding? cp-1250 or cp1250

Jimmy

----- Original Message ----- 
From: "Tatu Saloranta" <cow...@ya...>
To: "Jimmy Zhang" <cra...@co...>
Cc: <vtd...@li...>
Sent: Wednesday, November 01, 2006 12:41 PM
Subject: Re: [Vtd-xml-users] design question/best practice


> --- Jimmy Zhang <cra...@co...> wrote:
>
>> Tatu, when detecting XML encoding types, which are
>> currrently supported
>> by Woodstox?? Examples below:
>>
>> encoding="cp-1250"
>> encoding="cp1250"
>> encoding="windows-1250"
>> encoding="win-1250"
>>
>> It seems there can be many different types of
>> encoding values...
>
> Yes. Woodstox offers 2 levels of encoding support:
> first one is "native" support, in which Woodstox
> actually has its own decoder implementations, and then
> fallback handling that uses JDK-provided decoders.
> Set of natively supported decoders is quite small:
> just ISO-Latin1 (8859-1), Ascii (7-bit), UTF-8, UTF-32
> and UTF-16 (last one with partial native support).
>
> But I do think JDK actually supports ones you listed;
> about the only encoding I know that will not work is
> IBM's EBCDIC, and I haven't received requests for
> supporting any additional ones. So I assume either
> people don't use other encodings, or JDK has support.
>
> One page that's been useful when figuring out how
> encodings are to work has been this one:
>
> http://www.iana.org/assignments/character-sets
>
> which is linked to from xml specification itself.
>
> Hope this helps!
>
> -+ Tatu +-
>
>> Jimmy
>> ----- Original Message ----- 
>> From: "Tatu Saloranta" <cow...@ya...>
>> To: <vtd...@li...>
>> Sent: Monday, October 23, 2006 10:14 AM
>> Subject: Re: [Vtd-xml-users] design question/best
>> practice
>>
>>
>> > --- Paul Tomsic <pt...@ya...> wrote:
>> >
>> >> well, by caching i mean in-memory.
>> >> guess that's my question.  is it better to parse
>> >> once,
>> >> then store the xml as a stringBuffer or
>> something?
>> >
>> > No, it's probably never good idea to use
>> StringBuffer
>> > (or builder) -- it will not have a parsed
>> > representation, just a kind of serialization.
>> There
>> > are generally 3 steps in parsing:
>> >
>> > (a) Reading the data (from file, network)
>> > (b) Decoding (byte->char)
>> > (c) Tokenization (char[] -> events, or in case of
>> > vtd-xml, int offsets + type)
>> >
>> > If you want to cache things in-memory, the most
>> > compact presentation would be a raw byte array.
>> That'd
>> > get rid of (a). If you use StringBuffer (or char[]
>> > etc), you get rid of (b), but most likely double
>> > memory usage. Byte[] can contain utf-8 encoded
>> > contents, which are generally quite compact --
>> Strings
>> > (buffers, builders) consists of 16-bit character.
>> VTD
>> > would still have to tokenize (and read through)
>> the
>> > whole contents, and being about twice the size of
>> byte
>> > representation, would possibly be slower than
>> reading
>> > from a byte array (or even disk, depending on i/o
>> > speed).
>> >
>> > However; if you can afford to cache these docs in
>> > memory, you can probably afford to just cache the
>> > resulting VTD document object.
>> > After all, it adds only about 50% overhead; but
>> it's
>> > fully parsed and all, and thus efficient to
>> access.
>> >
>> > Finally, for these kinds of tasks, an actual
>> native
>> > XML database might make most sense. They store xml
>> > content efficiently in structured (not serialized)
>> > form. There are a few open source ones available
>> (like
>> > eXist).
>> >
>> > -+ Tatu +-
>> >
>> >
>> > __________________________________________________
>> > Do You Yahoo!?
>> > Tired of spam?  Yahoo! Mail has the best spam
>> protection around
>> > http://mail.yahoo.com
>> >
>> >
>>
> -------------------------------------------------------------------------
>> > Using Tomcat but need to do more? Need to support
>> web services, security?
>> > Get stuff done quickly with pre-integrated
>> technology to make your job
>> > easier
>> > Download IBM WebSphere Application Server v.1.0.1
>> based on Apache Geronimo
>> >
>>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> > _______________________________________________
>> > Vtd-xml-users mailing list
>> > Vtd...@li...
>> >
>>
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>> >
>>
>>
>>
>>
> -------------------------------------------------------------------------
>> Using Tomcat but need to do more? Need to support
>> web services, security?
>> Get stuff done quickly with pre-integrated
>> technology to make your job easier
>> Download IBM WebSphere Application Server v.1.0.1
>> based on Apache Geronimo
>>
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>> _______________________________________________
>> Vtd-xml-users mailing list
>> Vtd...@li...
>>
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>
>
>
>
>
> ____________________________________________________________________________________
> We have the perfect Group for you. Check out the handy changes to Yahoo! 
> Groups
> (http://groups.yahoo.com)
>
> 

Re: [Vtd-xml-users] vtd-xml 1.7 compiling

[Chinh H. <ho_...@ho...>]

For the second error, I included every file from the 1.7_c folder that I 
extracted from the downloaded ximpleware.  That means the l8.y and l8.l were 
included.


>From: "Jimmy Zhang" <cra...@co...>
>To: "Chinh Ho" <ho_...@ho...>
>CC: <vtd...@li...>
>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>Date: Tue, 31 Oct 2006 14:10:09 -0800
>
>For the first error, I think the file size has exceeded 2GB, which is why a 
>negative
>number is shown...
>
>for the second error, did you include "l8.y" and "l8.l" as part of the 
>project?
>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>To: <cra...@co...>
>Cc: <vtd...@li...>
>Sent: Monday, October 30, 2006 3:55 PM
>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>
>
>>Yes.  If you have the instruction, please let me know the link.
>>
>>Thanks
>>
>>
>>>From: "Jimmy Zhang" <cra...@co...>
>>>To: "Chinh Ho" <ho_...@ho...>
>>>CC: <vtd...@li...>
>>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>>Date: Thu, 26 Oct 2006 19:46:14 -0700
>>>
>>>do you still have the same issue compiling v 1.7??
>>>----- Original Message ----- From: "Jimmy Zhang" <cra...@co...>
>>>To: "Chinh Ho" <ho_...@ho...>
>>>Cc: <vtd...@li...>
>>>Sent: Wednesday, October 25, 2006 11:14 AM
>>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>>
>>>
>>>>The file size must have been greater than 2GB, as the size of file 
>>>>becomes
>>>>negative (> 0x7fffffff), have to look into the other problem
>>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>>To: <cra...@co...>
>>>>Cc: <vtd...@li...>
>>>>Sent: Wednesday, October 25, 2006 10:16 AM
>>>>Subject: Re: vtd-xml 1.7 compiling
>>>>
>>>>
>>>>>For 1.7_c_light, there is no errors.  However, when I run the benchmark 
>>>>>to
>>>>>open a 2 Gig xml file, it displayed "size of the file is -1841250597".
>>>>>That's an error.
>>>>>
>>>>>For 1.7_c, the errors are on the l8.* files.  For example, one of the
>>>>>errors is:
>>>>>
>>>>>l8.y(583) : error C2065: 'xpathInputPtr' : undeclared identifier
>>>>>
>>>>>
>>>>>>From: "Jimmy Zhang" <cra...@co...>
>>>>>>To: "Chinh Ho" <ho_...@ho...>
>>>>>>CC: <vtd...@li...>
>>>>>>Subject: Re: vtd-xml 1.7 compiling
>>>>>>Date: Wed, 25 Oct 2006 09:46:16 -0700
>>>>>>
>>>>>>What is the error message?
>>>>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>>>>To: <cra...@co...>; <vtd...@li...>
>>>>>>Sent: Wednesday, October 25, 2006 8:04 AM
>>>>>>Subject: vtd-xml 1.7 compiling
>>>>>>
>>>>>>
>>>>>>>Hi Jimmy,
>>>>>>>I downloaded 2 versions of the vtd-xml 1.7, ximpleware_1.7_c and
>>>>>>>ximpleware_1.7_c_light.  The 1.7_c is for Unix / Linux OS, isn't it? 
>>>>>>>The
>>>>>>>1.7_c_light is for Windows OS?
>>>>>>>
>>>>>>>I could not compile the 1.7_c under Windows.  I am using WinXP and
>>>>>>>MSVC2005.
>>>>>>>
>>>>>>>If both of them are for windows, how do I compile them?
>>>>>>>
>>>>>>>Do you have any instructions?  Readme?
>>>>>>>
>>>>>>>Thanks
>>>>>>>
>>>>>>>_________________________________________________________________
>>>>>>>Try Search Survival Kits: Fix up your home and better handle your 
>>>>>>>cash
>>>>>>>with Live Search!
>>>>>>>http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>_________________________________________________________________
>>>>>Try the next generation of search with Windows Live Search today!
>>>>>http://imagine-windowslive.com/minisites/searchlaunch/?locale=en-us&source=hmtagline
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>-------------------------------------------------------------------------
>>>>Using Tomcat but need to do more? Need to support web services, 
>>>>security?
>>>>Get stuff done quickly with pre-integrated technology to make your job 
>>>>easier
>>>>Download IBM WebSphere Application Server v.1.0.1 based on Apache 
>>>>Geronimo
>>>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>>>_______________________________________________
>>>>Vtd-xml-users mailing list
>>>>Vtd...@li...
>>>>https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>>>
>>>
>>>
>>
>>_________________________________________________________________
>>Stay in touch with old friends and meet new ones with Windows Live Spaces 
>>http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
>>
>>
>
>

_________________________________________________________________
Try Search Survival Kits: Fix up your home and better handle your cash with 
Live Search! 
http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline

Re: [Vtd-xml-users] design question/best practice

[Tatu S. <cow...@ya...>]

--- Jimmy Zhang <cra...@co...> wrote:

> Tatu, when detecting XML encoding types, which are
> currrently supported
> by Woodstox?? Examples below:
> 
> encoding="cp-1250"
> encoding="cp1250"
> encoding="windows-1250"
> encoding="win-1250"
> 
> It seems there can be many different types of
> encoding values...

Yes. Woodstox offers 2 levels of encoding support:
first one is "native" support, in which Woodstox
actually has its own decoder implementations, and then
fallback handling that uses JDK-provided decoders.
Set of natively supported decoders is quite small:
just ISO-Latin1 (8859-1), Ascii (7-bit), UTF-8, UTF-32
and UTF-16 (last one with partial native support).

But I do think JDK actually supports ones you listed;
about the only encoding I know that will not work is
IBM's EBCDIC, and I haven't received requests for
supporting any additional ones. So I assume either
people don't use other encodings, or JDK has support.

One page that's been useful when figuring out how
encodings are to work has been this one:

http://www.iana.org/assignments/character-sets

which is linked to from xml specification itself.

Hope this helps!

-+ Tatu +-

> Jimmy
> ----- Original Message ----- 
> From: "Tatu Saloranta" <cow...@ya...>
> To: <vtd...@li...>
> Sent: Monday, October 23, 2006 10:14 AM
> Subject: Re: [Vtd-xml-users] design question/best
> practice
> 
> 
> > --- Paul Tomsic <pt...@ya...> wrote:
> >
> >> well, by caching i mean in-memory.
> >> guess that's my question.  is it better to parse
> >> once,
> >> then store the xml as a stringBuffer or
> something?
> >
> > No, it's probably never good idea to use
> StringBuffer
> > (or builder) -- it will not have a parsed
> > representation, just a kind of serialization.
> There
> > are generally 3 steps in parsing:
> >
> > (a) Reading the data (from file, network)
> > (b) Decoding (byte->char)
> > (c) Tokenization (char[] -> events, or in case of
> > vtd-xml, int offsets + type)
> >
> > If you want to cache things in-memory, the most
> > compact presentation would be a raw byte array.
> That'd
> > get rid of (a). If you use StringBuffer (or char[]
> > etc), you get rid of (b), but most likely double
> > memory usage. Byte[] can contain utf-8 encoded
> > contents, which are generally quite compact --
> Strings
> > (buffers, builders) consists of 16-bit character.
> VTD
> > would still have to tokenize (and read through)
> the
> > whole contents, and being about twice the size of
> byte
> > representation, would possibly be slower than
> reading
> > from a byte array (or even disk, depending on i/o
> > speed).
> >
> > However; if you can afford to cache these docs in
> > memory, you can probably afford to just cache the
> > resulting VTD document object.
> > After all, it adds only about 50% overhead; but
> it's
> > fully parsed and all, and thus efficient to
> access.
> >
> > Finally, for these kinds of tasks, an actual
> native
> > XML database might make most sense. They store xml
> > content efficiently in structured (not serialized)
> > form. There are a few open source ones available
> (like
> > eXist).
> >
> > -+ Tatu +-
> >
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam
> protection around
> > http://mail.yahoo.com
> >
> >
>
-------------------------------------------------------------------------
> > Using Tomcat but need to do more? Need to support
> web services, security?
> > Get stuff done quickly with pre-integrated
> technology to make your job 
> > easier
> > Download IBM WebSphere Application Server v.1.0.1
> based on Apache Geronimo
> >
>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> > _______________________________________________
> > Vtd-xml-users mailing list
> > Vtd...@li...
> >
>
https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> > 
> 
> 
> 
>
-------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support
> web services, security?
> Get stuff done quickly with pre-integrated
> technology to make your job easier
> Download IBM WebSphere Application Server v.1.0.1
> based on Apache Geronimo
>
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
>
https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 



 
____________________________________________________________________________________
We have the perfect Group for you. Check out the handy changes to Yahoo! Groups 
(http://groups.yahoo.com)

Re: [Vtd-xml-users] design question/best practice

[Jimmy Z. <cra...@co...>]

Tatu, when detecting XML encoding types, which are currrently supported
by Woodstox?? Examples below:

encoding="cp-1250"
encoding="cp1250"
encoding="windows-1250"
encoding="win-1250"

It seems there can be many different types of encoding values...
Jimmy
----- Original Message ----- 
From: "Tatu Saloranta" <cow...@ya...>
To: <vtd...@li...>
Sent: Monday, October 23, 2006 10:14 AM
Subject: Re: [Vtd-xml-users] design question/best practice


> --- Paul Tomsic <pt...@ya...> wrote:
>
>> well, by caching i mean in-memory.
>> guess that's my question.  is it better to parse
>> once,
>> then store the xml as a stringBuffer or something?
>
> No, it's probably never good idea to use StringBuffer
> (or builder) -- it will not have a parsed
> representation, just a kind of serialization. There
> are generally 3 steps in parsing:
>
> (a) Reading the data (from file, network)
> (b) Decoding (byte->char)
> (c) Tokenization (char[] -> events, or in case of
> vtd-xml, int offsets + type)
>
> If you want to cache things in-memory, the most
> compact presentation would be a raw byte array. That'd
> get rid of (a). If you use StringBuffer (or char[]
> etc), you get rid of (b), but most likely double
> memory usage. Byte[] can contain utf-8 encoded
> contents, which are generally quite compact -- Strings
> (buffers, builders) consists of 16-bit character. VTD
> would still have to tokenize (and read through) the
> whole contents, and being about twice the size of byte
> representation, would possibly be slower than reading
> from a byte array (or even disk, depending on i/o
> speed).
>
> However; if you can afford to cache these docs in
> memory, you can probably afford to just cache the
> resulting VTD document object.
> After all, it adds only about 50% overhead; but it's
> fully parsed and all, and thus efficient to access.
>
> Finally, for these kinds of tasks, an actual native
> XML database might make most sense. They store xml
> content efficiently in structured (not serialized)
> form. There are a few open source ones available (like
> eXist).
>
> -+ Tatu +-
>
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
>
> -------------------------------------------------------------------------
> Using Tomcat but need to do more? Need to support web services, security?
> Get stuff done quickly with pre-integrated technology to make your job 
> easier
> Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
> http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

Re: [Vtd-xml-users] vtd-xml 1.7 compiling

[Jimmy Z. <cra...@co...>]

For the first error, I think the file size has exceeded 2GB, which is why a 
negative
number is shown...

for the second error, did you include "l8.y" and "l8.l" as part of the 
project?
----- Original Message ----- 
From: "Chinh Ho" <ho_...@ho...>
To: <cra...@co...>
Cc: <vtd...@li...>
Sent: Monday, October 30, 2006 3:55 PM
Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling


> Yes.  If you have the instruction, please let me know the link.
>
> Thanks
>
>
>>From: "Jimmy Zhang" <cra...@co...>
>>To: "Chinh Ho" <ho_...@ho...>
>>CC: <vtd...@li...>
>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>Date: Thu, 26 Oct 2006 19:46:14 -0700
>>
>>do you still have the same issue compiling v 1.7??
>>----- Original Message ----- From: "Jimmy Zhang" <cra...@co...>
>>To: "Chinh Ho" <ho_...@ho...>
>>Cc: <vtd...@li...>
>>Sent: Wednesday, October 25, 2006 11:14 AM
>>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>>
>>
>>>The file size must have been greater than 2GB, as the size of file 
>>>becomes
>>>negative (> 0x7fffffff), have to look into the other problem
>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>To: <cra...@co...>
>>>Cc: <vtd...@li...>
>>>Sent: Wednesday, October 25, 2006 10:16 AM
>>>Subject: Re: vtd-xml 1.7 compiling
>>>
>>>
>>>>For 1.7_c_light, there is no errors.  However, when I run the benchmark 
>>>>to
>>>>open a 2 Gig xml file, it displayed "size of the file is -1841250597".
>>>>That's an error.
>>>>
>>>>For 1.7_c, the errors are on the l8.* files.  For example, one of the
>>>>errors is:
>>>>
>>>>l8.y(583) : error C2065: 'xpathInputPtr' : undeclared identifier
>>>>
>>>>
>>>>>From: "Jimmy Zhang" <cra...@co...>
>>>>>To: "Chinh Ho" <ho_...@ho...>
>>>>>CC: <vtd...@li...>
>>>>>Subject: Re: vtd-xml 1.7 compiling
>>>>>Date: Wed, 25 Oct 2006 09:46:16 -0700
>>>>>
>>>>>What is the error message?
>>>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>>>To: <cra...@co...>; <vtd...@li...>
>>>>>Sent: Wednesday, October 25, 2006 8:04 AM
>>>>>Subject: vtd-xml 1.7 compiling
>>>>>
>>>>>
>>>>>>Hi Jimmy,
>>>>>>I downloaded 2 versions of the vtd-xml 1.7, ximpleware_1.7_c and
>>>>>>ximpleware_1.7_c_light.  The 1.7_c is for Unix / Linux OS, isn't it? 
>>>>>>The
>>>>>>1.7_c_light is for Windows OS?
>>>>>>
>>>>>>I could not compile the 1.7_c under Windows.  I am using WinXP and
>>>>>>MSVC2005.
>>>>>>
>>>>>>If both of them are for windows, how do I compile them?
>>>>>>
>>>>>>Do you have any instructions?  Readme?
>>>>>>
>>>>>>Thanks
>>>>>>
>>>>>>_________________________________________________________________
>>>>>>Try Search Survival Kits: Fix up your home and better handle your cash
>>>>>>with Live Search!
>>>>>>http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>_________________________________________________________________
>>>>Try the next generation of search with Windows Live Search today!
>>>>http://imagine-windowslive.com/minisites/searchlaunch/?locale=en-us&source=hmtagline
>>>>
>>>>
>>>
>>>
>>>
>>>-------------------------------------------------------------------------
>>>Using Tomcat but need to do more? Need to support web services, security?
>>>Get stuff done quickly with pre-integrated technology to make your job 
>>>easier
>>>Download IBM WebSphere Application Server v.1.0.1 based on Apache 
>>>Geronimo
>>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>>_______________________________________________
>>>Vtd-xml-users mailing list
>>>Vtd...@li...
>>>https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>>
>>
>>
>
> _________________________________________________________________
> Stay in touch with old friends and meet new ones with Windows Live Spaces 
> http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us
>
> 

Re: [Vtd-xml-users] vtd-xml 1.7 compiling

[Chinh H. <ho_...@ho...>]

Yes.  If you have the instruction, please let me know the link.

Thanks


>From: "Jimmy Zhang" <cra...@co...>
>To: "Chinh Ho" <ho_...@ho...>
>CC: <vtd...@li...>
>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>Date: Thu, 26 Oct 2006 19:46:14 -0700
>
>do you still have the same issue compiling v 1.7??
>----- Original Message ----- From: "Jimmy Zhang" <cra...@co...>
>To: "Chinh Ho" <ho_...@ho...>
>Cc: <vtd...@li...>
>Sent: Wednesday, October 25, 2006 11:14 AM
>Subject: Re: [Vtd-xml-users] vtd-xml 1.7 compiling
>
>
>>The file size must have been greater than 2GB, as the size of file becomes
>>negative (> 0x7fffffff), have to look into the other problem
>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>To: <cra...@co...>
>>Cc: <vtd...@li...>
>>Sent: Wednesday, October 25, 2006 10:16 AM
>>Subject: Re: vtd-xml 1.7 compiling
>>
>>
>>>For 1.7_c_light, there is no errors.  However, when I run the benchmark 
>>>to
>>>open a 2 Gig xml file, it displayed "size of the file is -1841250597".
>>>That's an error.
>>>
>>>For 1.7_c, the errors are on the l8.* files.  For example, one of the
>>>errors is:
>>>
>>>l8.y(583) : error C2065: 'xpathInputPtr' : undeclared identifier
>>>
>>>
>>>>From: "Jimmy Zhang" <cra...@co...>
>>>>To: "Chinh Ho" <ho_...@ho...>
>>>>CC: <vtd...@li...>
>>>>Subject: Re: vtd-xml 1.7 compiling
>>>>Date: Wed, 25 Oct 2006 09:46:16 -0700
>>>>
>>>>What is the error message?
>>>>----- Original Message ----- From: "Chinh Ho" <ho_...@ho...>
>>>>To: <cra...@co...>; <vtd...@li...>
>>>>Sent: Wednesday, October 25, 2006 8:04 AM
>>>>Subject: vtd-xml 1.7 compiling
>>>>
>>>>
>>>>>Hi Jimmy,
>>>>>I downloaded 2 versions of the vtd-xml 1.7, ximpleware_1.7_c and
>>>>>ximpleware_1.7_c_light.  The 1.7_c is for Unix / Linux OS, isn't it? 
>>>>>The
>>>>>1.7_c_light is for Windows OS?
>>>>>
>>>>>I could not compile the 1.7_c under Windows.  I am using WinXP and
>>>>>MSVC2005.
>>>>>
>>>>>If both of them are for windows, how do I compile them?
>>>>>
>>>>>Do you have any instructions?  Readme?
>>>>>
>>>>>Thanks
>>>>>
>>>>>_________________________________________________________________
>>>>>Try Search Survival Kits: Fix up your home and better handle your cash
>>>>>with Live Search!
>>>>>http://imagine-windowslive.com/search/kits/default.aspx?kit=improve&locale=en-US&source=hmtagline
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>_________________________________________________________________
>>>Try the next generation of search with Windows Live Search today!
>>>http://imagine-windowslive.com/minisites/searchlaunch/?locale=en-us&source=hmtagline
>>>
>>>
>>
>>
>>
>>-------------------------------------------------------------------------
>>Using Tomcat but need to do more? Need to support web services, security?
>>Get stuff done quickly with pre-integrated technology to make your job 
>>easier
>>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>>_______________________________________________
>>Vtd-xml-users mailing list
>>Vtd...@li...
>>https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>>
>
>

_________________________________________________________________
Stay in touch with old friends and meet new ones with Windows Live Spaces 
http://clk.atdmt.com/MSN/go/msnnkwsp0070000001msn/direct/01/?href=http://spaces.live.com/spacesapi.aspx?wx_action=create&wx_url=/friends.aspx&mkt=en-us