vtd-xml-users — This mailing list is for users or people interested in VTD-XML

[Vtd-xml-users] Bug? xpath parser doesn't accept non-ascii?

[Mark S. <ma...@Sc...>]

Hello,

This works:

ap.selectXPath("/purchaseOrder/shipTo[name='Ren']")

This does not:

ap.selectXPath("/purchaseOrder/shipTo[name='René']")

Any reason why non-ascii doesn't work?

Thanks.

-- 
http://www.ScheduleWorld.com/
Free Google Calendar synchronization with Outlook, Evolution,
cell phones, BlackBerry, PalmOS, Exchange, Mozilla, Thunderbird,
Pocket PC/Windows Mobile. Also sync tasks, notes and contacts!
WebDAV, vfreebusy, RSS, LDAP, iCalendar, iTIP, iMIP support.

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Jimmy Z. <cra...@co...>]

If the goal of unbinding is not for reuse, then it is reasonable that
there is no need for unbinding.. in other words, if you don't want
a bookmark object to bind to another VTDNav object, then
it is very likely that it is the end of the application, and those bookmarks
should be gc'ed ... any comment?

----- Original Message ----- 
From: "Rodrigo Cunha" <rn...@gm...>
To: "Jimmy Zhang" <cra...@co...>
Cc: <vtd...@li...>
Sent: Monday, April 16, 2007 2:14 AM
Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)


> Sorry about the delay but I've been sick with a nasty upper respiratory 
> tract infection ... back to work now!
>
> Yes, you're right, I can bind to another VTDNav object, but that sounds 
> like a kludge when the objective is really unbinding, and may imply 
> internal bufer allocation inside the bookmark object if the depth 
> increases.
>
>
> Jimmy Zhang wrote:
>> setCursorPosition() and getCursroPosition() have been added in CVS
>>
>> but the benefit of unbind() is still not clear
>>
>> if GC of VTDnav is the goal then one can simply call 
>> "setCursorPosition()" on
>> a different VTDNav object and the mission is accomplished without having 
>> a
>> unbind method for bookMark...
>>
>>
>> ----- Original Message ----- From: "Rodrigo Cunha" <rn...@gm...>
>> To: "Jimmy Zhang" <cra...@co...>
>> Sent: Thursday, April 05, 2007 2:41 AM
>> Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)
>>
>>
>>> Jimmy Zhang wrote:
>>>
>>> > What if a bookmark object is created, but not yet bind to a  non-null
>>> VTDNav object...
>>>
>>> That's for unbind, right? The unbind rationale is to allow GC of VTDNav 
>>> objects while still allowing easy reusing of bookmark. Obviously you 
>>> could bind them to a dummy VTDNav instead, but that's a kludge, and 
>>> requires more processing. Or you could allow binding to null, but that 
>>> could lead to unnoticed errors.
>>>
>>> > recordCursorPosition() and setCursorPosition() would throw
>>> NullPointerException.. would that bother you?
>>>
>>> No, that's what would happen really, and seems perfectly correct.
>>>
>>> -- 
>>> Rodrigo
>>>
>>>
>>
>>
>>
>
> 

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Rodrigo C. <rn...@gm...>]

Sorry about the delay but I've been sick with a nasty upper respiratory 
tract infection ... back to work now!

Yes, you're right, I can bind to another VTDNav object, but that sounds 
like a kludge when the objective is really unbinding, and may imply 
internal bufer allocation inside the bookmark object if the depth increases.


Jimmy Zhang wrote:
> setCursorPosition() and getCursroPosition() have been added in CVS
>
> but the benefit of unbind() is still not clear
>
> if GC of VTDnav is the goal then one can simply call 
> "setCursorPosition()" on
> a different VTDNav object and the mission is accomplished without 
> having a
> unbind method for bookMark...
>
>
> ----- Original Message ----- From: "Rodrigo Cunha" <rn...@gm...>
> To: "Jimmy Zhang" <cra...@co...>
> Sent: Thursday, April 05, 2007 2:41 AM
> Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)
>
>
>> Jimmy Zhang wrote:
>>
>> > What if a bookmark object is created, but not yet bind to a  non-null
>> VTDNav object...
>>
>> That's for unbind, right? The unbind rationale is to allow GC of 
>> VTDNav objects while still allowing easy reusing of bookmark. 
>> Obviously you could bind them to a dummy VTDNav instead, but that's a 
>> kludge, and requires more processing. Or you could allow binding to 
>> null, but that could lead to unnoticed errors.
>>
>> > recordCursorPosition() and setCursorPosition() would throw
>> NullPointerException.. would that bother you?
>>
>> No, that's what would happen really, and seems perfectly correct.
>>
>> -- 
>> Rodrigo
>>
>>
>
>
>

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Jimmy Z. <cra...@co...>]

setCursorPosition() and getCursroPosition() have been added in CVS

but the benefit of unbind() is still not clear

if GC of VTDnav is the goal then one can simply call "setCursorPosition()" 
on
a different VTDNav object and the mission is accomplished without having a
unbind method for bookMark...


----- Original Message ----- 
From: "Rodrigo Cunha" <rn...@gm...>
To: "Jimmy Zhang" <cra...@co...>
Sent: Thursday, April 05, 2007 2:41 AM
Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)


> Jimmy Zhang wrote:
>
> > What if a bookmark object is created, but not yet bind to a  non-null
> VTDNav object...
>
> That's for unbind, right? The unbind rationale is to allow GC of VTDNav 
> objects while still allowing easy reusing of bookmark. Obviously you could 
> bind them to a dummy VTDNav instead, but that's a kludge, and requires 
> more processing. Or you could allow binding to null, but that could lead 
> to unnoticed errors.
>
> > recordCursorPosition() and setCursorPosition() would throw
> NullPointerException.. would that bother you?
>
> No, that's what would happen really, and seems perfectly correct.
>
> --
> Rodrigo
>
> 

Re: [Vtd-xml-users] XMLModifier isn't quite what I need

[Jimmy Z. <cra...@co...>]

To "stamp"  new data onto those dummy elements, there must be enough 
space...
So it probably boils down to how much control you have when serializing the 
XML doc

----- Original Message ----- 
From: "Mark Swanson" <ma...@Sc...>
To: <vtd...@li...>
Sent: Sunday, April 08, 2007 10:55 PM
Subject: Re: [Vtd-xml-users] XMLModifier isn't quite what I need


>>
>> A quick summary: to tack extra content on to the end, just pre-serialize
>> a few dummy elemnts that can be modified without reparse...
>
> Yeah, I thought about that. But some law related to Murphy would bite me
> I'm sure.
>
> Cheers.
>
> -- 
> http://www.ScheduleWorld.com/
> Free Google Calendar synchronization with Outlook, Evolution,
> cell phones, BlackBerry, PalmOS, Exchange, Mozilla, Thunderbird,
> Pocket PC/Windows Mobile. Also sync tasks, notes and contacts!
> WebDAV, vfreebusy, RSS, LDAP, iCalendar, iTIP, iMIP support.
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share 
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

Re: [Vtd-xml-users] XMLModifier isn't quite what I need

[Mark S. <ma...@Sc...>]

> 
> A quick summary: to tack extra content on to the end, just pre-serialize
> a few dummy elemnts that can be modified without reparse...

Yeah, I thought about that. But some law related to Murphy would bite me 
I'm sure.

Cheers.

-- 
http://www.ScheduleWorld.com/
Free Google Calendar synchronization with Outlook, Evolution,
cell phones, BlackBerry, PalmOS, Exchange, Mozilla, Thunderbird,
Pocket PC/Windows Mobile. Also sync tasks, notes and contacts!
WebDAV, vfreebusy, RSS, LDAP, iCalendar, iTIP, iMIP support.

Re: [Vtd-xml-users] XMLModifier isn't quite what I need

[Jimmy Z. <cra...@co...>]

The step outlined will work as the most general purpose solution,
0. Create an XMLModifier
1. add an element
2. serialize all 100 megs
3. reparse the new output

One possible  alternative involves the use of  the overWrite feature...
and it is possible to disable/enable instead of remove/add elements to XML..
consider the following example,

<CATALOG>
    <CD>
        <TITLE>Empire Burlesque</TITLE>
        <ARTIST>Bob Dylan</ARTIST>
        <COUNTRY>USA</COUNTRY>
        <COMPANY>Columbia</COMPANY>
        <PRICE>10.90</PRICE>
        <YEAR>1985</YEAR>
     </CD>
     <CD>
        <TITLE>Still got the blues</TITLE>
        <ARTIST>Gary More</ARTIST>
        <COUNTRY>UK</COUNTRY>
        <COMPANY>Virgin redords</COMPANY>
        <PRICE>10.20</PRICE>
        <YEAR>1990</YEAR>
     </CD>
</CATALOG>

If it is serialized as


<CATALOG>
    <CD enable = '1'>
        <TITLE>Empire Burlesque</TITLE>
        <ARTIST>Bob Dylan</ARTIST>
        <COUNTRY>USA</COUNTRY>
        <COMPANY>Columbia</COMPANY>
        <PRICE>10.90</PRICE>
        <YEAR>1985</YEAR>
     </CD>
     <CD enable = '1'>
        <TITLE>Still got the blues</TITLE>
        <ARTIST>Gary More</ARTIST>
        <COUNTRY>UK</COUNTRY>
        <COMPANY>Virgin redords</COMPANY>
        <PRICE>10.20</PRICE>
        <YEAR>1990</YEAR>
     </CD>
</CATALOG>

We can simply disable it by setting "enable " to 0 to disable
an element

To enable an elements, we can *pre-serialize* some dummy
XML elements into the document whose initial value for
enable is set to zero. We also need to make sure that those
fields (e.g. title/artist/country/company) has long enough enough  to
insert new content (this is similar to define the width of a column
of a relational table), so an XML doc with dummy elements may
look like:

<CATALOG>
    <CD enable = '0'>
        <TITLE>                          </TITLE>
        <ARTIST>                       </ARTIST>
        <COUNTRY>                 </COUNTRY>
        <COMPANY>                </COMPANY>
        <PRICE>                         </PRICE>
        <YEAR>           </YEAR>
     </CD>
     <CD enable = '0'>
        <TITLE>                          </TITLE>
        <ARTIST>                       </ARTIST>
        <COUNTRY>                 </COUNTRY>
        <COMPANY>                </COMPANY>
        <PRICE>                         </PRICE>
        <YEAR>           </YEAR>
     </CD>
</CATALOG>


When parsing the document above, you have created VTD that will
not be changed even after inserting new byte content into the white
spaces...

also notice that adding "enable" attribute to an element is just one way of
disabling/enabling an element

A quick summary: to tack extra content on to the end, just pre-serialize
a few dummy elemnts that can be modified without reparse...



----- Original Message ----- 
From: "Mark Swanson" <ma...@Sc...>
To: <vtd...@li...>
Sent: Saturday, April 07, 2007 10:54 PM
Subject: [Vtd-xml-users] XMLModifier isn't quite what I need


> Hello,
>
> I would like to use XML for more than just an intermediary format. I'd
> like to use VTD to query/update/remove elements and attributes at will
> without having to resort to moving the XML into objects.
>
> Currently, if I have a hundred megs of data I have to do this to tack a
> new element on to the end of the document:
>
> 0. Create an XMLModifier
> 1. add an element
> 2. serialize all 100 megs
> 3. reparse the new output
>
> I wonder... 100% of my use cases currently would pass if I could just
> tack elements on to the end of the document. Would it be possible for
> this specific case to work easily? And by work, I mean basically add a
> VTDGen.parseAndAddToEndOfDocument(byte[] d); I'm hoping the index
> generation and data structure manipulation would be minimal for this
> special case.
>
> Thoughts?
>
>
> -- 
> http://www.ScheduleWorld.com/
> Free Google Calendar synchronization with Outlook, Evolution,
> cell phones, BlackBerry, PalmOS, Exchange, Mozilla, Thunderbird,
> Pocket PC/Windows Mobile. Also sync tasks, notes and contacts!
> WebDAV, vfreebusy, RSS, LDAP, iCalendar, iTIP, iMIP support.
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share 
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> 

[Vtd-xml-users] XMLModifier isn't quite what I need

[Mark S. <ma...@Sc...>]

Hello,

I would like to use XML for more than just an intermediary format. I'd 
like to use VTD to query/update/remove elements and attributes at will 
without having to resort to moving the XML into objects.

Currently, if I have a hundred megs of data I have to do this to tack a 
new element on to the end of the document:

0. Create an XMLModifier
1. add an element
2. serialize all 100 megs
3. reparse the new output

I wonder... 100% of my use cases currently would pass if I could just 
tack elements on to the end of the document. Would it be possible for 
this specific case to work easily? And by work, I mean basically add a 
VTDGen.parseAndAddToEndOfDocument(byte[] d); I'm hoping the index 
generation and data structure manipulation would be minimal for this 
special case.

Thoughts?


-- 
http://www.ScheduleWorld.com/
Free Google Calendar synchronization with Outlook, Evolution,
cell phones, BlackBerry, PalmOS, Exchange, Mozilla, Thunderbird,
Pocket PC/Windows Mobile. Also sync tasks, notes and contacts!
WebDAV, vfreebusy, RSS, LDAP, iCalendar, iTIP, iMIP support.

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Jimmy Z. <cra...@co...>]

What if a bookmark object is created, but not yet bind to 
a  non-null VTDNav object...

recordCursorPosition() and setCursorPosition() would throw
NullPointerException.. would that bother you?
----- Original Message ----- 
From: "Rodrigo Cunha" <rn...@gm...>
To: "Jimmy Zhang" <cra...@co...>
Sent: Wednesday, April 04, 2007 3:00 AM
Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)


> Jimmy Zhang wrote:
>> I just added getNav() and checked into CVS... anything else
>> you have in mind?
> 
> Yes:
> 
> unbind(), or bind(vn) accepting null, but I think unbind is better, 
> leads to less errors.
> 
> recordCursorPosition() with no parameters
> 
> equivalent to xpto.recordCursorPosition(xpto.getNav()); but more elegant 
> and faster, especially if directly implemented with less checks.
> 
> setCursorPosition() with no parameters
> equivalent to xpto.setCursorPosition(xpto.getNav()); but more elegant 
> and faster, especially if directly implemented with less checks.
> 
> 
> Cheers,
> 
> Rodrigo
>

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Jimmy Z. <cra...@co...>]

Cool!
----- Original Message ----- 
From: "Rodrigo Cunha" <rn...@gm...>
To: "Jimmy Zhang" <cra...@co...>
Cc: "Mark Swanson" <ma...@Sc...>; 
<vtd...@li...>
Sent: Thursday, March 29, 2007 3:10 AM
Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)


> Jimmy,
>
> getNavigator would be used in some of my code to avoid passing yet another 
> parameter to methods, I just don't use it right now because my bookmark 
> class does not keep the navigator reference. If it did I would use it 
> right away.
>
> Also the method is so simple no one would be confused. Why hide this 
> rather obvious feature?
>
> This works just like the Pattern class in Java: you create it using a 
> string, and later if you really want to get the string back you also can, 
> since it's probably there in the class anyway.  Only that is useless, and 
> this feature is useful.
>
> getNavigator: returns the navigator to witch the bookmark is bound, 
> otherwise returns null.
>
> I can test the implementation, sure. Just give me a few days :-)
>
> Also, I'll include the rather trivial getNavigator in the test, ok?
>
> Jimmy Zhang wrote:
>> Yes, BookMark will go into the next release as an experimental
>> feature ... with more user feedback/discussion, we can officially
>> introduce it later
>>
>> regarding getNavigator().. if  non-critical, I prefer to keep it
>> out for now... if we keep it in, then later decide to drop it,
>> it will break people's code... so it is easier to add a method than
>> taking it out... plus one less method makes the documentation shorter
>> and a bit less confusing ...
>>
>> regarding equals() and hashCode(), the methods are there, we
>> can alway change the implementation...
>>
>> In the meantime, could you give the implementation a bit testing?
>>
>>
>> ----- Original Message ----- From: "Rodrigo Cunha" <rn...@gm...>
>> To: "Jimmy Zhang" <cra...@co...>
>> Cc: "Mark Swanson" <ma...@Sc...>; 
>> <vtd...@li...>
>> Sent: Wednesday, March 28, 2007 3:16 AM
>> Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)
>>
>>
>>> Hi!
>>>
>>> The getNavigator() method might spare some parameter passing, since if 
>>> I'm gona pass a bookmark around to another object I'll need to pass the 
>>> navigator also, somehow. It's not critical, but since it can be done 
>>> trivially, why not? Also, being able to ask the bookmark "hey you, to 
>>> witch navigator do you belong?" seems the phylosophically correct thing 
>>> to do.
>>>
>>> equals() and hashCode() are religious issues, just keep them your way 
>>> :-)
>>>
>>> A hashCode function should ideally spread the input amongst all the 
>>> hashcodes space, since collections using hashcodes might, or might not, 
>>> use all bits in the hashcode. But just don't bother with that.
>>>
>>> Jimmy Zhang wrote:
>>>> Rodrigo, I just checked in a quick update into CVS
>>>>
>>>> Regarding some of prior comments, my questions are
>>>>
>>>> 1. When would a user call getNavigator() any way? I kinda had
>>>> a tough time coming up the scenario...
>>>>
>>>> 2. Regarding equals(), I think that the current bookMark implementation
>>>> should be fine in that the only time two bookMark instances are equal
>>>> is when they contains node position of the same VTDNav instance, and
>>>> the node indices are equal...
>>>>
>>>> 3.  I am still not sure about hashCode, what is the purpose of the 
>>>> spread?
>>>> For the current implementation, there will never be colision for the 
>>>> bookmarks
>>>> corresponding to the same document (the node index is ideal)...
>>>>
>>>> Let me know your thoughts..
>>>> Regards,
>>>> jimmy
>>>>
>>>
>>>
>>
>>
>>
>
> 

Re: [Vtd-xml-users] XPath doubt

[Fernando G. <fer...@gm...>]

Yes Jimmy, the missing "http://" was the fault. Well, at least, now I
understand better the behavior of XPath in VTD...

greetings,
Fernando

On 3/28/07, Jimmy Zhang <cra...@co...> wrote:
>
>  No problem, it gave me a chance to get to the code to reassure myself
> that it actually works :-)..
> Feel free if there is any question..
>
> ----- Original Message -----
> *From:* Fernando Gonzalez <fer...@gm...>
> *To:* vtd...@li...
> *Sent:* Wednesday, March 28, 2007 1:47 AM
> *Subject:* Re: [Vtd-xml-users] XPath doubt
>
> Thanks a lot for your help Jimmy. I cannot test right now the proposed
> solution, but I will do it this week. If it's just that, I'm sorry to have
> wasted your time...
>
> Fernando
>
> On 3/28/07, Jimmy Zhang <cra...@co...> wrote:
> >
> >  It seems that  "ap.declareXPathNameSpace("cite", "
> > www.opengeospatial.net/cite ");" should
> > have been   "ap.declareXPathNameSpace("cite", "*http://*www.opengeospatial.net/cite
> > ");"
> >
> > also ap.selectElementNS is not for XPath eval/selection, it is to
> > emulate DOM's
> > noteIterator inteface.
> >
> > Let me know if there is any question...
> >
> > ----- Original Message -----
> > *From:* Fernando Gonzalez <fer...@gm...>
> > *To:* vtd...@li...
> > *Sent:* Tuesday, March 27, 2007 1:55 AM
> > *Subject:* [Vtd-xml-users] XPath doubt
> >
> > Hi again,
> >
> > I'm having some problems using XPath expressions...
> > When I use the "/wfs:FeatureCollection/gml:featureMember" expression it
> > works fine.
> > But if I add one more element:
> > "/wfs:FeatureCollection/gml:featureMember/cite:Streams" it matches
> > nothing...
> > The most surprising (to me) is that if I use this
> > "/wfs:FeatureCollection/gml:featureMember/*" it matches the "cite:Streams"
> > elements in the XML.
> >
> > Am I missing something??
> >
> > By the way, I don't know what's the point of the "
> > AutoPilot.selectElementNS" and "AutoPilot.declareXPathNameSpace" methods
> > so maybe the problem is there. Where can I find some documentation about
> > them. I looked in the javadoc but it's not clear to me.
> >
> > Thanks in advance,
> >
> > This is the code and the file:
> >
> >         AutoPilot ap = new AutoPilot(vn);
> >         ap.selectElementNS("www.opengeospatial.net/cite ", "*");
> >         ap.declareXPathNameSpace("cite", "www.opengeospatial.net/cite");
> >         ap.selectElementNS(" http://www.opengis.net/gml", "*");
> >         ap.declareXPathNameSpace("gml", "http://www.opengis.net/gml");
> >         ap.selectElementNS(" http://www.opengis.net/wfs", "*");
> >         ap.declareXPathNameSpace("wfs", "http://www.opengis.net/wfs");
> >         ap.selectXPath
> > ("/wfs:FeatureCollection/gml:featureMember/cite:Streams");
> >         int result = -1;
> >         while ((result = ap.evalXPath()) != -1) {
> >             long l = vn.getElementFragment ();
> >             int tokenOffset = (int) (0x0000FFFF & l);
> >             int tokenLength = (int) ((0xFFFF0000 & l) >> 32);
> >
> >             System.out.println(new String(b, tokenOffset, tokenLength));
> >         }
> >
> >
> > I have the following XML file:
> >
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs "
> > xmlns:gml="http://www.opengis.net/gml" xmlns:cite="
> > http://www.opengeospatial.net/cite" xmlns:xsi="
> > http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
> > http://www.opengeospatial.net/cite http://www.refractions.net:8082/geoserver1.3.1/wfs/DescribeFeatureType?typeName=cite:Streams
> > http://www.opengis.net/wfs
> > http://www.refractions.net:8082/geoserver1.3.1/schemas/wfs/1.0.0/WFS-basic.xsd
> > ">
> >   <gml:boundedBy>
> >     <gml:Box>
> >       <gml:coordinates decimal="." cs="," ts=" ">- 0.0004,-0.0024 0.0036
> > ,0.0024</gml:coordinates>
> >     </gml:Box>
> >   </gml:boundedBy>
> >   <gml:featureMember>
> >     <cite:Streams fid="Streams.1">
> >       <cite:the_geom>
> >         <gml:MultiLineString>
> > ...
> >
> >
> >  ------------------------------
> >
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > opinions on IT & business topics through brief surveys-and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >
> >
> > ------------------------------
> >
> > _______________________________________________
> > Vtd-xml-users mailing list
> > Vtd...@li...
> > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> >
> >
>  ------------------------------
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> ------------------------------
>
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>
>

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Rodrigo C. <rn...@gm...>]

Jimmy,

getNavigator would be used in some of my code to avoid passing yet 
another parameter to methods, I just don't use it right now because my 
bookmark class does not keep the navigator reference. If it did I would 
use it right away.

Also the method is so simple no one would be confused. Why hide this 
rather obvious feature?

This works just like the Pattern class in Java: you create it using a 
string, and later if you really want to get the string back you also 
can, since it's probably there in the class anyway.  Only that is 
useless, and this feature is useful.

getNavigator: returns the navigator to witch the bookmark is bound, 
otherwise returns null.

I can test the implementation, sure. Just give me a few days :-)

Also, I'll include the rather trivial getNavigator in the test, ok?

Jimmy Zhang wrote:
> Yes, BookMark will go into the next release as an experimental
> feature ... with more user feedback/discussion, we can officially
> introduce it later
>
> regarding getNavigator().. if  non-critical, I prefer to keep it
> out for now... if we keep it in, then later decide to drop it,
> it will break people's code... so it is easier to add a method than
> taking it out... plus one less method makes the documentation shorter
> and a bit less confusing ...
>
> regarding equals() and hashCode(), the methods are there, we
> can alway change the implementation...
>
> In the meantime, could you give the implementation a bit testing?
>
>
> ----- Original Message ----- From: "Rodrigo Cunha" <rn...@gm...>
> To: "Jimmy Zhang" <cra...@co...>
> Cc: "Mark Swanson" <ma...@Sc...>; 
> <vtd...@li...>
> Sent: Wednesday, March 28, 2007 3:16 AM
> Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)
>
>
>> Hi!
>>
>> The getNavigator() method might spare some parameter passing, since 
>> if I'm gona pass a bookmark around to another object I'll need to 
>> pass the navigator also, somehow. It's not critical, but since it can 
>> be done trivially, why not? Also, being able to ask the bookmark "hey 
>> you, to witch navigator do you belong?" seems the phylosophically 
>> correct thing to do.
>>
>> equals() and hashCode() are religious issues, just keep them your way 
>> :-)
>>
>> A hashCode function should ideally spread the input amongst all the 
>> hashcodes space, since collections using hashcodes might, or might 
>> not, use all bits in the hashcode. But just don't bother with that.
>>
>> Jimmy Zhang wrote:
>>> Rodrigo, I just checked in a quick update into CVS
>>>
>>> Regarding some of prior comments, my questions are
>>>
>>> 1. When would a user call getNavigator() any way? I kinda had
>>> a tough time coming up the scenario...
>>>
>>> 2. Regarding equals(), I think that the current bookMark implementation
>>> should be fine in that the only time two bookMark instances are equal
>>> is when they contains node position of the same VTDNav instance, and
>>> the node indices are equal...
>>>
>>> 3.  I am still not sure about hashCode, what is the purpose of the 
>>> spread?
>>> For the current implementation, there will never be colision for the 
>>> bookmarks
>>> corresponding to the same document (the node index is ideal)...
>>>
>>> Let me know your thoughts..
>>> Regards,
>>> jimmy
>>>
>>
>>
>
>
>

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Jimmy Z. <cra...@co...>]

Yes, BookMark will go into the next release as an experimental
feature ... with more user feedback/discussion, we can officially
introduce it later

regarding getNavigator().. if  non-critical, I prefer to keep it
out for now... if we keep it in, then later decide to drop it,
it will break people's code... so it is easier to add a method than
taking it out... plus one less method makes the documentation shorter
and a bit less confusing ...

regarding equals() and hashCode(), the methods are there, we
can alway change the implementation...

In the meantime, could you give the implementation a bit testing?


----- Original Message ----- 
From: "Rodrigo Cunha" <rn...@gm...>
To: "Jimmy Zhang" <cra...@co...>
Cc: "Mark Swanson" <ma...@Sc...>; 
<vtd...@li...>
Sent: Wednesday, March 28, 2007 3:16 AM
Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)


> Hi!
>
> The getNavigator() method might spare some parameter passing, since if I'm 
> gona pass a bookmark around to another object I'll need to pass the 
> navigator also, somehow. It's not critical, but since it can be done 
> trivially, why not? Also, being able to ask the bookmark "hey you, to 
> witch navigator do you belong?" seems the phylosophically correct thing to 
> do.
>
> equals() and hashCode() are religious issues, just keep them your way :-)
>
> A hashCode function should ideally spread the input amongst all the 
> hashcodes space, since collections using hashcodes might, or might not, 
> use all bits in the hashcode. But just don't bother with that.
>
> Jimmy Zhang wrote:
>> Rodrigo, I just checked in a quick update into CVS
>>
>> Regarding some of prior comments, my questions are
>>
>> 1. When would a user call getNavigator() any way? I kinda had
>> a tough time coming up the scenario...
>>
>> 2. Regarding equals(), I think that the current bookMark implementation
>> should be fine in that the only time two bookMark instances are equal
>> is when they contains node position of the same VTDNav instance, and
>> the node indices are equal...
>>
>> 3.  I am still not sure about hashCode, what is the purpose of the 
>> spread?
>> For the current implementation, there will never be colision for the 
>> bookmarks
>> corresponding to the same document (the node index is ideal)...
>>
>> Let me know your thoughts..
>> Regards,
>> jimmy
>>
>
> 

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Rodrigo C. <rn...@gm...>]

Hi!

The getNavigator() method might spare some parameter passing, since if 
I'm gona pass a bookmark around to another object I'll need to pass the 
navigator also, somehow. It's not critical, but since it can be done 
trivially, why not? Also, being able to ask the bookmark "hey you, to 
witch navigator do you belong?" seems the phylosophically correct thing 
to do.

equals() and hashCode() are religious issues, just keep them your way :-)

A hashCode function should ideally spread the input amongst all the 
hashcodes space, since collections using hashcodes might, or might not, 
use all bits in the hashcode. But just don't bother with that.

Jimmy Zhang wrote:
> Rodrigo, I just checked in a quick update into CVS
>
> Regarding some of prior comments, my questions are
>
> 1. When would a user call getNavigator() any way? I kinda had
> a tough time coming up the scenario...
>
> 2. Regarding equals(), I think that the current bookMark implementation
> should be fine in that the only time two bookMark instances are equal
> is when they contains node position of the same VTDNav instance, and
> the node indices are equal...
>
> 3.  I am still not sure about hashCode, what is the purpose of the 
> spread?
> For the current implementation, there will never be colision for the 
> bookmarks
> corresponding to the same document (the node index is ideal)...
>
> Let me know your thoughts..
> Regards,
> jimmy
>

Re: [Vtd-xml-users] XPath doubt

[Fernando G. <fer...@gm...>]

Thanks a lot for your help Jimmy. I cannot test right now the proposed
solution, but I will do it this week. If it's just that, I'm sorry to have
wasted your time...

Fernando

On 3/28/07, Jimmy Zhang <cra...@co...> wrote:
>
>  It seems that  "ap.declareXPathNameSpace("cite", "
> www.opengeospatial.net/cite");" should
> have been   "ap.declareXPathNameSpace("cite", "*http://*
> www.opengeospatial.net/cite");"
>
> also ap.selectElementNS is not for XPath eval/selection, it is to emulate
> DOM's
> noteIterator inteface.
>
> Let me know if there is any question...
>
> ----- Original Message -----
> *From:* Fernando Gonzalez <fer...@gm...>
> *To:* vtd...@li...
> *Sent:* Tuesday, March 27, 2007 1:55 AM
> *Subject:* [Vtd-xml-users] XPath doubt
>
> Hi again,
>
> I'm having some problems using XPath expressions...
> When I use the "/wfs:FeatureCollection/gml:featureMember" expression it
> works fine.
> But if I add one more element:
> "/wfs:FeatureCollection/gml:featureMember/cite:Streams" it matches
> nothing...
> The most surprising (to me) is that if I use this
> "/wfs:FeatureCollection/gml:featureMember/*" it matches the "cite:Streams"
> elements in the XML.
>
> Am I missing something??
>
> By the way, I don't know what's the point of the "
> AutoPilot.selectElementNS" and "AutoPilot.declareXPathNameSpace" methods
> so maybe the problem is there. Where can I find some documentation about
> them. I looked in the javadoc but it's not clear to me.
>
> Thanks in advance,
>
> This is the code and the file:
>
>         AutoPilot ap = new AutoPilot(vn);
>         ap.selectElementNS("www.opengeospatial.net/cite ", "*");
>         ap.declareXPathNameSpace("cite", "www.opengeospatial.net/cite");
>         ap.selectElementNS(" http://www.opengis.net/gml", "*");
>         ap.declareXPathNameSpace("gml", "http://www.opengis.net/gml");
>         ap.selectElementNS(" http://www.opengis.net/wfs", "*");
>         ap.declareXPathNameSpace("wfs", "http://www.opengis.net/wfs");
>         ap.selectXPath
> ("/wfs:FeatureCollection/gml:featureMember/cite:Streams");
>         int result = -1;
>         while ((result = ap.evalXPath()) != -1) {
>             long l = vn.getElementFragment ();
>             int tokenOffset = (int) (0x0000FFFF & l);
>             int tokenLength = (int) ((0xFFFF0000 & l) >> 32);
>
>             System.out.println(new String(b, tokenOffset, tokenLength));
>         }
>
>
> I have the following XML file:
>
>
> <?xml version="1.0" encoding="UTF-8"?>
> <wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs " xmlns:gml="
> http://www.opengis.net/gml" xmlns:cite="http://www.opengeospatial.net/cite"
> xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance"
> xsi:schemaLocation="http://www.opengeospatial.net/cite
> http://www.refractions.net:8082/geoserver1.3.1/wfs/DescribeFeatureType?typeName=cite:Streams
> http://www.opengis.net/wfs
> http://www.refractions.net:8082/geoserver1.3.1/schemas/wfs/1.0.0/WFS-basic.xsd
> ">
>   <gml:boundedBy>
>     <gml:Box>
>       <gml:coordinates decimal="." cs="," ts=" ">- 0.0004,-0.0024 0.0036,
> 0.0024</gml:coordinates>
>     </gml:Box>
>   </gml:boundedBy>
>   <gml:featureMember>
>     <cite:Streams fid="Streams.1">
>       <cite:the_geom>
>         <gml:MultiLineString>
> ...
>
>
>  ------------------------------
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> ------------------------------
>
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>
>

Re: [Vtd-xml-users] Random Access Proposal (take 2)

[Jimmy Z. <cra...@co...>]

Rodrigo, I just checked in a quick update into CVS

Regarding some of prior comments, my questions are

1. When would a user call getNavigator() any way? I kinda had
a tough time coming up the scenario...

2. Regarding equals(), I think that the current bookMark implementation
should be fine in that the only time two bookMark instances are equal
is when they contains node position of the same VTDNav instance, and
the node indices are equal...

3.  I am still not sure about hashCode, what is the purpose of the spread?
For the current implementation, there will never be colision for the 
bookmarks
corresponding to the same document (the node index is ideal)...

Let me know your thoughts..
Regards,
jimmy


----- Original Message ----- 
From: "Rodrigo Cunha" <rn...@gm...>
To: "Jimmy Zhang" <cra...@co...>
Cc: "Mark Swanson" <ma...@Sc...>; 
<vtd...@li...>
Sent: Thursday, March 15, 2007 8:46 AM
Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)


> Jimmy, the API itself seems quite OK to me.
>
> I had made the decision not to keep a pointer to VN inside the context but 
> for extra safety this is both highly recomended and adds little extra fat, 
> so let's keep it your way. Ah... since you keep a pointer do VTDNav you 
> should add a method "getNavigator". This will spare some parameter passing 
> to functions that might use a Bookmark. It's functionally irrelevant but 
> convenient and might even add some efficiency.
>
> Now the inner workings:
>
> The implementation of both equals and hashcode could be different. Note 
> that in typical usage inside data structures my complex methods 
> assymptotically get as fast as yours, since they end up comparing 
> pre-calculated hashCodes, considering there are probably no collisions.
>
> Also, for maximum efficiency hashCode should be a spread function. Using 
> index numbers does not typically spread the hashCode amonsgst the 
> available 2^32 values, still this is a minor observation, don't bother 
> with that :-) And we don't have collisions, so it's quite ok.
>
> If Mark also agrees I think this new class is ready for prime time.
>
> Jimmy Zhang wrote:
>> hmm...interesting... "plup" sounds like a tennis racket hitting the 
>> ground...
>> it may work :-)
>>
>> Please take a look the file below and let me know what you think of it 
>> ...
>>
>> http://vtd-xml.cvs.sourceforge.net/*checkout*/vtd-xml/ximple-dev/com/ximpleware/BookMark.java
>>
>>
>> ----- Original Message ----- From: "Rodrigo Cunha" <rn...@gm...>
>> To: "Jimmy Zhang" <cra...@co...>
>> Cc: "Mark Swanson" <ma...@Sc...>; 
>> <vtd...@li...>
>> Sent: Wednesday, March 14, 2007 4:19 AM
>> Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)
>>
>>
>>> Yes, that would be a nice add :-)
>>>
>>> Wouldn't solve the exception handling or true random access issues 
>>> tought, but would be a nice addon.
>>>
>>> I propose "plup()" hehe
>>>
>>> Jimmy Zhang wrote:
>>>> Rodrigo, How about add a method (tentatively called set()) that 
>>>> basically
>>>> does pop() and push() in one shot , so your
>>>> example
>>>>
>>>> push();
>>>> // do something
>>>> pop();
>>>> push();
>>>> // do something
>>>> pop();
>>>> push();
>>>> // do something
>>>> pop();
>>>> push();
>>>> // do something
>>>> pop();
>>>>
>>>> becomes
>>>>
>>>> push();
>>>> // do something
>>>> set();
>>>> // do something
>>>> set();
>>>> // do something
>>>> set();
>>>> // do something
>>>> pop();
>>>>
>>>>
>>>> ----- Original Message ----- From: "Rodrigo Cunha" <rn...@gm...>
>>>> To: "Mark Swanson" <ma...@Sc...>; "Jimmy Zhang" 
>>>> <cra...@co...>
>>>> Cc: <vtd...@li...>
>>>> Sent: Tuesday, March 06, 2007 9:24 AM
>>>> Subject: Re: [Vtd-xml-users] Random Access Proposal (take 2)
>>>>
>>>>
>>>>> Hum, thanks for the hint also Mark. I normally use java.util 
>>>>> collections, but i'll give fastutils a look, since they might be 
>>>>> better for some of my problems.
>>>>>
>>>>> Jimmy, a context can be imported/exported as an array of integers, 
>>>>> this being a bit raw perhaps... and leaving the programmer with the 
>>>>> job of storing that array. I just created SimpleContext as a way of 
>>>>> abstracting that implementation detail, but since you like 
>>>>> performance, perhaps you could leave that raw. I still prefer the 
>>>>> abstracted version... besides, the internal representation might 
>>>>> change and the abstracted version would still work.
>>>>>
>>>>> So, yes, go ahead with my API even if you decide to change it later.
>>>>>
>>>>> Just as a little silly footnote I found my API is also useful in cases 
>>>>> where you have:
>>>>>
>>>>> push();
>>>>> // do something
>>>>> pop();
>>>>> push();
>>>>> // do something
>>>>> pop();
>>>>> push();
>>>>> // do something
>>>>> pop();
>>>>> push();
>>>>> // do something
>>>>> pop();
>>>>>
>>>>> Can be abreviated to:
>>>>>
>>>>> setCtxFromNav(ctx);
>>>>> // do something
>>>>> setNavFromCtx(ctx);
>>>>> // do something
>>>>> setNavFromCtx(ctx);
>>>>> // do something
>>>>> setNavFromCtx(ctx);
>>>>> // do something
>>>>> setNavFromCtx(ctx);
>>>>>
>>>>> This is both faster and more readable.
>>>>>
>>>>> It's also useful for things like:
>>>>>
>>>>> xpto = getComplexToFindTransmissionPath(VTDNav n, String s); // might 
>>>>> even use caching!
>>>>> push();
>>>>> nav.setNavFromCtx(xpto);
>>>>> // Do something
>>>>> pop();
>>>>>
>>>>> and also for exception handling:
>>>>>
>>>>> {
>>>>> setCtxFromNav(xpto);
>>>>> // do something complex and error-prone,
>>>>> // so I won't use the stack, but a few locally declared contexts 
>>>>> instead
>>>>> // oops! crash! go to handler!
>>>>> finally {
>>>>>    setNavFromCtx(xpto)
>>>>>    // we are now clean
>>>>> }
>>>>>
>>>>> So, it made using VTD much more enjoyable to me, even in rather 
>>>>> trivial situations.
>>>>>
>>>>> Mark Swanson wrote:
>>>>>> Jimmy Zhang wrote:
>>>>>>>
>>>>>>>> Just a FYI: I have cases where the key is an Integer, and cases 
>>>>>>>> where
>>>>>>>> it's a string.
>>>>>>>
>>>>>>> By Integer is it a java class? or just a primitive data type? Maybe 
>>>>>>> I can
>>>>>>> modify Rodrigo's class and put it into CVS so you guys can use 
>>>>>>> immediately...
>>>>>>> however, I can't guarantee that it will be included in the next 
>>>>>>> release...
>>>>>>> Would that work?
>>>>>>
>>>>>> Oh, I always use native ints and fastutil wherever possible.
>>>>>>
>>>>>> Just a thought: I use autojar on my code to build a tiny fastutil jar 
>>>>>> that just has the code I need. You could do the same thing to get 
>>>>>> excellent native collections instead of writing your own. I see you 
>>>>>> already wrote your own, but in case you need more.. Fastutil uses the 
>>>>>> LGPL.
>>>>>>
>>>>>> Cheers.
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>
>>
>>
>>
>
> 

Re: [Vtd-xml-users] XPath doubt

[Jimmy Z. <cra...@co...>]

It seems that  "ap.declareXPathNameSpace("cite", =
"www.opengeospatial.net/cite");" should
have been   "ap.declareXPathNameSpace("cite", =
"http://www.opengeospatial.net/cite");"


also ap.selectElementNS is not for XPath eval/selection, it is to =
emulate DOM's
noteIterator inteface.

Let me know if there is any question...
  ----- Original Message -----=20
  From: Fernando Gonzalez=20
  To: vtd...@li...=20
  Sent: Tuesday, March 27, 2007 1:55 AM
  Subject: [Vtd-xml-users] XPath doubt


  Hi again,

  I'm having some problems using XPath expressions...=20
  When I use the "/wfs:FeatureCollection/gml:featureMember" expression =
it works fine.=20
  But if I add one more element: =
"/wfs:FeatureCollection/gml:featureMember/cite:Streams" it matches =
nothing...=20
  The most surprising (to me) is that if I use this =
"/wfs:FeatureCollection/gml:featureMember/*" it matches the =
"cite:Streams" elements in the XML.

  Am I missing something??

  By the way, I don't know what's the point of the " =
AutoPilot.selectElementNS" and "AutoPilot.declareXPathNameSpace" methods =
so maybe the problem is there. Where can I find some documentation about =
them. I looked in the javadoc but it's not clear to me.

  Thanks in advance,

  This is the code and the file:

          AutoPilot ap =3D new AutoPilot(vn);
          ap.selectElementNS("www.opengeospatial.net/cite ", "*");
          ap.declareXPathNameSpace("cite", =
"www.opengeospatial.net/cite");
          ap.selectElementNS(" http://www.opengis.net/gml", "*");
          ap.declareXPathNameSpace("gml", "http://www.opengis.net/gml");
          ap.selectElementNS(" http://www.opengis.net/wfs", "*");
          ap.declareXPathNameSpace("wfs", "http://www.opengis.net/wfs"); =

          =
ap.selectXPath("/wfs:FeatureCollection/gml:featureMember/cite:Streams");
          int result =3D -1;
          while ((result =3D ap.evalXPath()) !=3D -1) {
              long l =3D vn.getElementFragment ();
              int tokenOffset =3D (int) (0x0000FFFF & l);
              int tokenLength =3D (int) ((0xFFFF0000 & l) >> 32);

              System.out.println(new String(b, tokenOffset, =
tokenLength));
          }


  I have the following XML file:


  <?xml version=3D"1.0" encoding=3D"UTF-8"?>
  <wfs:FeatureCollection xmlns:wfs=3D"http://www.opengis.net/wfs " =
xmlns:gml=3D"http://www.opengis.net/gml" =
xmlns:cite=3D"http://www.opengeospatial.net/cite" xmlns:xsi=3D" =
http://www.w3.org/2001/XMLSchema-instance" =
xsi:schemaLocation=3D"http://www.opengeospatial.net/cite =
http://www.refractions.net:8082/geoserver1.3.1/wfs/DescribeFeatureType?ty=
peName=3Dcite:Streams http://www.opengis.net/wfs =
http://www.refractions.net:8082/geoserver1.3.1/schemas/wfs/1.0.0/WFS-basi=
c.xsd">
    <gml:boundedBy>
      <gml:Box>
        <gml:coordinates decimal=3D"." cs=3D"," ts=3D" ">- =
0.0004,-0.0024 0.0036,0.0024</gml:coordinates>
      </gml:Box>
    </gml:boundedBy>
    <gml:featureMember>
      <cite:Streams fid=3D"Streams.1">
        <cite:the_geom>
          <gml:MultiLineString>
  ...





-------------------------------------------------------------------------=
-----


  =
-------------------------------------------------------------------------=

  Take Surveys. Earn Cash. Influence the Future of IT
  Join SourceForge.net's Techsay panel and you'll get the chance to =
share your
  opinions on IT & business topics through brief surveys-and earn cash
  =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV


-------------------------------------------------------------------------=
-----


  _______________________________________________
  Vtd-xml-users mailing list
  Vtd...@li...
  https://lists.sourceforge.net/lists/listinfo/vtd-xml-users

Re: [Vtd-xml-users] XPath doubt

[Jimmy Z. <cra...@co...>]

Can you try to compile and run the code attached in this email to see
if there is any issue?=20

/*
 * Created on Mar 27, 2007
 *
 * TODO To change the template for this generated file go to
 * Window - Preferences - Java - Code Style - Code Templates
 */
import com.ximpleware.*;
public class NSTest {
    public static void main(String s[]) {
        AutoPilot ap =3D new AutoPilot();
        ap.declareXPathNameSpace("ns1","ns1");
        ap.declareXPathNameSpace("ns2","ns2");
        ap.declareXPathNameSpace("ns3","ns3");
        String XML =3D "<ns1:good xmlns:ns1=3D'ns1'>\n"
                  +"<ns2:bad  xmlns:ns2=3D'ns2'>\n"
                  +"<ns3:ugly  xmlns:ns3=3D'ns3'/>\n"
                  +"</ns2:bad>\n"
        ;           +"</ns1:good>";
        try{
            ap.selectXPath("/ns1:good/ns2:bad/ns3:ugly");
            System.out.println(" expr =3D=3D=3D> " + =
ap.getExprString());
            VTDGen vg =3D new VTDGen();
            vg.setDoc(XML.getBytes());
            vg.parse(true);
            VTDNav vn =3D vg.getNav();
            ap.bind(vn);
            int i;
            while((i=3Dap. evalXPath())!=3D-1){
                System.out.println(" token name =
=3D=3D>"+vn.toString(i));
            }
        }catch(Exception e){
            System.out.println("exception =3D=3D>"+e);
        }
    }
}
  ----- Original Message -----=20
  From: Jimmy Zhang=20
  To: Fernando Gonzalez ; vtd...@li...=20
  Sent: Tuesday, March 27, 2007 11:14 AM
  Subject: [Norton AntiSpam] Re: [Vtd-xml-users] XPath doubt


  Will investigate and get back to you
    ----- Original Message -----=20
    From: Fernando Gonzalez=20
    To: vtd...@li...=20
    Sent: Tuesday, March 27, 2007 1:55 AM
    Subject: [Vtd-xml-users] XPath doubt


    Hi again,

    I'm having some problems using XPath expressions...=20
    When I use the "/wfs:FeatureCollection/gml:featureMember" expression =
it works fine.=20
    But if I add one more element: =
"/wfs:FeatureCollection/gml:featureMember/cite:Streams" it matches =
nothing...=20
    The most surprising (to me) is that if I use this =
"/wfs:FeatureCollection/gml:featureMember/*" it matches the =
"cite:Streams" elements in the XML.

    Am I missing something??

    By the way, I don't know what's the point of the " =
AutoPilot.selectElementNS" and "AutoPilot.declareXPathNameSpace" methods =
so maybe the problem is there. Where can I find some documentation about =
them. I looked in the javadoc but it's not clear to me.

    Thanks in advance,

    This is the code and the file:

            AutoPilot ap =3D new AutoPilot(vn);
            ap.selectElementNS("www.opengeospatial.net/cite ", "*");
            ap.declareXPathNameSpace("cite", =
"www.opengeospatial.net/cite");
            ap.selectElementNS(" http://www.opengis.net/gml", "*");
            ap.declareXPathNameSpace("gml", =
"http://www.opengis.net/gml");
            ap.selectElementNS(" http://www.opengis.net/wfs", "*");
            ap.declareXPathNameSpace("wfs", =
"http://www.opengis.net/wfs");=20
            =
ap.selectXPath("/wfs:FeatureCollection/gml:featureMember/cite:Streams");
            int result =3D -1;
            while ((result =3D ap.evalXPath()) !=3D -1) {
                long l =3D vn.getElementFragment ();
                int tokenOffset =3D (int) (0x0000FFFF & l);
                int tokenLength =3D (int) ((0xFFFF0000 & l) >> 32);

                System.out.println(new String(b, tokenOffset, =
tokenLength));
            }


    I have the following XML file:


    <?xml version=3D"1.0" encoding=3D"UTF-8"?>
    <wfs:FeatureCollection xmlns:wfs=3D"http://www.opengis.net/wfs " =
xmlns:gml=3D"http://www.opengis.net/gml" =
xmlns:cite=3D"http://www.opengeospatial.net/cite" xmlns:xsi=3D" =
http://www.w3.org/2001/XMLSchema-instance" =
xsi:schemaLocation=3D"http://www.opengeospatial.net/cite =
http://www.refractions.net:8082/geoserver1.3.1/wfs/DescribeFeatureType?ty=
peName=3Dcite:Streams http://www.opengis.net/wfs =
http://www.refractions.net:8082/geoserver1.3.1/schemas/wfs/1.0.0/WFS-basi=
c.xsd">
      <gml:boundedBy>
        <gml:Box>
          <gml:coordinates decimal=3D"." cs=3D"," ts=3D" ">- =
0.0004,-0.0024 0.0036,0.0024</gml:coordinates>
        </gml:Box>
      </gml:boundedBy>
      <gml:featureMember>
        <cite:Streams fid=3D"Streams.1">
          <cite:the_geom>
            <gml:MultiLineString>
    ...





-------------------------------------------------------------------------=
---


    =
-------------------------------------------------------------------------=

    Take Surveys. Earn Cash. Influence the Future of IT
    Join SourceForge.net's Techsay panel and you'll get the chance to =
share your
    opinions on IT & business topics through brief surveys-and earn cash
    =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV=20


-------------------------------------------------------------------------=
---


    _______________________________________________
    Vtd-xml-users mailing list
    Vtd...@li...
    https://lists.sourceforge.net/lists/listinfo/vtd-xml-users



-------------------------------------------------------------------------=
-----


  =
-------------------------------------------------------------------------=

  Take Surveys. Earn Cash. Influence the Future of IT
  Join SourceForge.net's Techsay panel and you'll get the chance to =
share your
  opinions on IT & business topics through brief surveys-and earn cash
  =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV


-------------------------------------------------------------------------=
-----


  _______________________________________________
  Vtd-xml-users mailing list
  Vtd...@li...
  https://lists.sourceforge.net/lists/listinfo/vtd-xml-users

Re: [Vtd-xml-users] XPath doubt

[Jimmy Z. <cra...@co...>]

Will investigate and get back to you
  ----- Original Message -----=20
  From: Fernando Gonzalez=20
  To: vtd...@li...=20
  Sent: Tuesday, March 27, 2007 1:55 AM
  Subject: [Vtd-xml-users] XPath doubt


  Hi again,

  I'm having some problems using XPath expressions...=20
  When I use the "/wfs:FeatureCollection/gml:featureMember" expression =
it works fine.=20
  But if I add one more element: =
"/wfs:FeatureCollection/gml:featureMember/cite:Streams" it matches =
nothing...=20
  The most surprising (to me) is that if I use this =
"/wfs:FeatureCollection/gml:featureMember/*" it matches the =
"cite:Streams" elements in the XML.

  Am I missing something??

  By the way, I don't know what's the point of the " =
AutoPilot.selectElementNS" and "AutoPilot.declareXPathNameSpace" methods =
so maybe the problem is there. Where can I find some documentation about =
them. I looked in the javadoc but it's not clear to me.

  Thanks in advance,

  This is the code and the file:

          AutoPilot ap =3D new AutoPilot(vn);
          ap.selectElementNS("www.opengeospatial.net/cite ", "*");
          ap.declareXPathNameSpace("cite", =
"www.opengeospatial.net/cite");
          ap.selectElementNS(" http://www.opengis.net/gml", "*");
          ap.declareXPathNameSpace("gml", "http://www.opengis.net/gml");
          ap.selectElementNS(" http://www.opengis.net/wfs", "*");
          ap.declareXPathNameSpace("wfs", "http://www.opengis.net/wfs"); =

          =
ap.selectXPath("/wfs:FeatureCollection/gml:featureMember/cite:Streams");
          int result =3D -1;
          while ((result =3D ap.evalXPath()) !=3D -1) {
              long l =3D vn.getElementFragment ();
              int tokenOffset =3D (int) (0x0000FFFF & l);
              int tokenLength =3D (int) ((0xFFFF0000 & l) >> 32);

              System.out.println(new String(b, tokenOffset, =
tokenLength));
          }


  I have the following XML file:


  <?xml version=3D"1.0" encoding=3D"UTF-8"?>
  <wfs:FeatureCollection xmlns:wfs=3D"http://www.opengis.net/wfs " =
xmlns:gml=3D"http://www.opengis.net/gml" =
xmlns:cite=3D"http://www.opengeospatial.net/cite" xmlns:xsi=3D" =
http://www.w3.org/2001/XMLSchema-instance" =
xsi:schemaLocation=3D"http://www.opengeospatial.net/cite =
http://www.refractions.net:8082/geoserver1.3.1/wfs/DescribeFeatureType?ty=
peName=3Dcite:Streams http://www.opengis.net/wfs =
http://www.refractions.net:8082/geoserver1.3.1/schemas/wfs/1.0.0/WFS-basi=
c.xsd">
    <gml:boundedBy>
      <gml:Box>
        <gml:coordinates decimal=3D"." cs=3D"," ts=3D" ">- =
0.0004,-0.0024 0.0036,0.0024</gml:coordinates>
      </gml:Box>
    </gml:boundedBy>
    <gml:featureMember>
      <cite:Streams fid=3D"Streams.1">
        <cite:the_geom>
          <gml:MultiLineString>
  ...





-------------------------------------------------------------------------=
-----


  =
-------------------------------------------------------------------------=

  Take Surveys. Earn Cash. Influence the Future of IT
  Join SourceForge.net's Techsay panel and you'll get the chance to =
share your
  opinions on IT & business topics through brief surveys-and earn cash
  =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV


-------------------------------------------------------------------------=
-----


  _______________________________________________
  Vtd-xml-users mailing list
  Vtd...@li...
  https://lists.sourceforge.net/lists/listinfo/vtd-xml-users

[Vtd-xml-users] XPath doubt

[Fernando G. <fer...@gm...>]

Hi again,

I'm having some problems using XPath expressions...
When I use the "/wfs:FeatureCollection/gml:featureMember" expression it
works fine.
But if I add one more element:
"/wfs:FeatureCollection/gml:featureMember/cite:Streams" it matches
nothing...
The most surprising (to me) is that if I use this
"/wfs:FeatureCollection/gml:featureMember/*" it matches the "cite:Streams"
elements in the XML.

Am I missing something??

By the way, I don't know what's the point of the "AutoPilot.selectElementNS"
and "AutoPilot.declareXPathNameSpace" methods so maybe the problem is there.
Where can I find some documentation about them. I looked in the javadoc but
it's not clear to me.

Thanks in advance,

This is the code and the file:

        AutoPilot ap = new AutoPilot(vn);
        ap.selectElementNS("www.opengeospatial.net/cite", "*");
        ap.declareXPathNameSpace("cite", "www.opengeospatial.net/cite");
        ap.selectElementNS("http://www.opengis.net/gml", "*");
        ap.declareXPathNameSpace("gml", "http://www.opengis.net/gml");
        ap.selectElementNS("http://www.opengis.net/wfs", "*");
        ap.declareXPathNameSpace("wfs", "http://www.opengis.net/wfs");
        ap.selectXPath
("/wfs:FeatureCollection/gml:featureMember/cite:Streams");
        int result = -1;
        while ((result = ap.evalXPath()) != -1) {
            long l = vn.getElementFragment();
            int tokenOffset = (int) (0x0000FFFF & l);
            int tokenLength = (int) ((0xFFFF0000 & l) >> 32);

            System.out.println(new String(b, tokenOffset, tokenLength));
        }


I have the following XML file:


<?xml version="1.0" encoding="UTF-8"?>
<wfs:FeatureCollection xmlns:wfs="http://www.opengis.net/wfs" xmlns:gml="
http://www.opengis.net/gml" xmlns:cite="http://www.opengeospatial.net/cite"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="
http://www.opengeospatial.net/cite
http://www.refractions.net:8082/geoserver1.3.1/wfs/DescribeFeatureType?typeName=cite:Streams
http://www.opengis.net/wfs
http://www.refractions.net:8082/geoserver1.3.1/schemas/wfs/1.0.0/WFS-basic.xsd
">
  <gml:boundedBy>
    <gml:Box>
      <gml:coordinates decimal="." cs="," ts=" ">-0.0004,-0.0024 0.0036,
0.0024</gml:coordinates>
    </gml:Box>
  </gml:boundedBy>
  <gml:featureMember>
    <cite:Streams fid="Streams.1">
      <cite:the_geom>
        <gml:MultiLineString>
...

Re: [Vtd-xml-users] Storing parsing info

[Fernando G. <fer...@gm...>]

Well, I don't need much of XPath. All XPath expressions I think I'm going to
do are the same: get the nth child of an element. I thought the XPath
evaluation was going to have a worse performance with the chunk approach.
However I obtained better results than I expected and those results are good
enough for me. Also keep in mind that my benchmark is not thorough. I tested
only a XPath expression with a unique 100Mb XML file.


On 3/21/07, Jimmy Zhang < cra...@co...> wrote:
>
>  Not a problem :-), one reason we are having this discussion is that the
> indexing feature (and
> VTD-XML itself) is so new and we have yet to understand the possiblities
> and design trade-offs
> ... Yes, I can see why tuning the optimum buffer size can potentially
> improve performance...
> but in general do you see any issue with the XPath evaluation ?
>
>
>
> ----- Original Message -----
>  *From:* Fernando Gonzalez <fer...@gm...>
> *To:* vtd...@li...
> *Sent:* Wednesday, March 21, 2007 3:14 AM
> *Subject:* Re: [Vtd-xml-users] Storing parsing info
>
> Sorry Jimmy, I misunderstood you. Please, forget the first paragraph of my
> last mail.
>
> yes, you're right, when you ask for a byte it may not be in the loaded
> chunk... so you have to load another chunk. Of course it's quite slower than
> the current solution. But I think there must be a buffer size that optimizes
> performance so the solution is only a bit slower without loading more than
> 20-30Mb in memory. Don't you think so?
>
> Anyway, the proposed changes don't force to use that approach. I'ts still
> possible to load the whole XML in memory, so the only side effect of the
> proposal is that the library is a bit more complex (there are some exception
> handling issues and the user have to provide a implementation of an
> interface instead of a byte[]) and the library is a bit slower (because of
> the added indirection level)
>
> Fernando
>
> On 3/21/07, Fernando Gonzalez <fer...@gm...> wrote:
> >
> > "if the chunks don't have what one is looking for, you will have to load
> > in another chunk... then another chunk.."
> >
> > Not exactly. If something asks for the the byte number 'x' I guess which
> > chunk the byte is in and I load only that chunk. Only if the information
> > asked for by the user is spread in two or more contiguous chunks will be
> > necessary to load more than one. The implementation can be seen in the "
> > org.ChunkByteBuffer" class in the "public byte byteAt(int streamIndex)"
> > method.
> >
> > About the alternative you propose. As I have told before, it's not a
> > good solution to remove or archive the original GML. Splitting wouldn't be
> > as bad as removal or archiving, but it would add some complexity to the
> > user. The user should keep track of the splitted GML files that form the
> > original GML file. The splitting could be implemented in a way that the user
> > doesn't notice it... while he doesn't try to access the file with another
> > application.
> >
> > Indeed, I think that, in the end, I'm doing something similar to
> > splitting since I logically split the GML file into several GML chunks.
> >
> > Fernando
> >
> >
> > On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
> > >
> > >  Ok, I see... it seems that you can be sure that the "chunks" of GML
> > > files contain what the user would need...
> > > But in general, if the chunks don't have what one is looking for, you
> > > will have to load in another chunk... then
> > > another chunk.. that could mean a lot of disk activities
> > > As an alternative, would it be possible to split GML into little
> > > chunks of well-formed GML files, then index
> > > them individually.
> > > So instead of dealing with 10 big GML files, split them into 100
> > > smaller GML files and the algorithm you describe
> > >  may still work..
> > >
> > > ----- Original Message -----
> > >  *From:* Fernando Gonzalez <fer...@gm...>
> > > *To:* vtd...@li...
> > >  *Sent:* Tuesday, March 20, 2007 2:39 AM
> > > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> > >
> > > On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
> > > >
> > > >  So what you are trying to accomplish is to load all the GML
> > > > docs into memory at once...
> > > > I guess you can simply index all those files to avoid parsing...
> > > > but I still don't seem to understand the benefits of read teh parse
> > > > info and a chunk of
> > > > the XML file..
> > > >
> > >
> > > Quite near. What I need is to access a random feature at any time with
> > > as a low cost as possible. That could be possible loading all the GML docs
> > > in memory but the GML files are very big so I cannot do it.
> > >
> > > As that solution wasn't suitable to my problem, I thought of opening
> > > one file each time (using buffer reuse) and then it came to my mind that I
> > > could save parsing time storing the parse info. As I told before I cannot
> > > delete the GML. Storing the GML twice will waste disk space. I'm talking
> > > about an environment where the user can have in his computer a lot of
> > > digital cartography. Disk space is quite a bottle neck. It could be valid,
> > > but storing only the parse info was so easy that I did it and I obtained a
> > > better solution (for my environment).
> > >
> > > There is a use case where the user doesn't work with the files
> > > directly, but with a spatial region. In this case, the GML files and other
> > > spatial data are "layers", so the user can work at the same time with a lot
> > > of files. These files can be in other formats than GML, satellite images,
> > > different raster or vectorial formats; and these can bring the system to a
> > > even more memory constrained situation. That's what lead me to load chunks
> > > of the GML file.
> > >
> > > The workflow is the following
> > > * I open a file with the chunk approach
> > > * I parse the file (loading it with the chunks approach takes a lot,
> > > but no problem)
> > > * I store the parse info
> > > The user asks for information:
> > > * I load the parse info
> > > * I load the chunk
> > > * I return the asked information
> > >
> > > I want to speed up the asking of information because the user can ask
> > > for a map image with 20 GML files, and the map code is something like this:
> > >
> > > for each gml file
> > >   guess what "features" are inside the map bounds (GML is indexed
> > > spatially previously)
> > >   get those features from the GML (random access) (load parse info +
> > > load chunk + return info)
> > >   draw the features on a image
> > > next gml file
> > >
> > > Maybe this will make things a bit clearer. This screenshot (http://www.gvsig.gva.es/fileadmin/conselleria/images/Documentacion/capturas/raster_shp_dgn_750.gif
> > > ) shows a program that uses the library. You can see on the left all
> > > the loaded (from the user point of view) files: four "dgn" files, one "shp"
> > > and seven "ecw" files. A lot of operations done in the map are done over
> > > *every* file listed on the left so I don't care how much time it takes to
> > > put all those files on the left (generating parse info, etc). I care how
> > > much time takes to read the information after they are loaded (again, from
> > > the user point of view).
> > >
> > > Well, I hope it's clear enough. Notice that I'm not proposing changing
> > > the way VTD-XML works but I'm proposing to add new ways.
> > >
> > > greetings,
> > > Fernando
> > >
> > >   ----- Original Message -----
> > > >  *From:* Fernando Gonzalez <fer...@gm...>
> > > >  *To:* vtd...@li...
> > > > *Sent:* Monday, March 19, 2007 2:56 AM
> > > > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> > > >
> > > > Well, jeje, the computer is new but I don't think my disk is so
> > > > fast. I think Java or the operating system has to cache something because
> > > > the first time I load the file it takes a bit more than 2 seconds and after
> > > > the first load, it only takes 300ms to read the file...
> > > > I have no experience on doing benchmarks and maybe I'm am missing
> > > > something. That's why I attached the program.
> > > >
> > > > "So if you can't delete the orginal XML files, can you compress them
> > > > and store them away (archiving)?"
> > > > I cannot delete nor archive the GML file because in this context it
> > > > won't be rare to be reading it from two different programs at the same
> > > > time... It's difficult to find an open source program that does everything
> > > > you need. For example, in a development context, there may be a map server
> > > > serving a map image based on a GML file while you are opening it to see some
> > > > data in it.
> > > >
> > > > "The other issue you raised is buffer reuse. To reuse internal
> > > > buffers of VTDGen, you can call setDoc_BR(...). But there is more
> > > > you can do...
> > > > you can in fact reuse the byte array containing the XML document."
> > > > Buffer reuse absolutly solves my memory constraints. But the problem
> > > > I see with buffer reuse is that it will force me to read and parse the whole
> > > > XML file every time the user ask for information on another XML file, won't
> > > > it? If I read the XML file by chunks and I store/read the parse information,
> > > > each time the user asks for information on another XML file I only have to
> > > > read the parse info and a chunk of the XML file.
> > > >
> > > > To show you my point of view:
> > > > The "user asking for another XML file" may be a map server that
> > > > reads some big GML files and draws its spatial information in a map image.
> > > > If each time the map server draws a GML file and "changes" to the next it
> > > > takes 2 seconds or so, the drawing of the map (all the GML files) takes too
> > > > much time.
> > > >
> > > > best regards,
> > > > Fernando
> > > >
> > > >
> > > > On 3/19/07, Jimmy Zhang <cra...@co... > wrote:
> > > > >
> > > > >
> > > > > What intrigues me with Fernando's test results is that it only
> > > > > takes 300ms to read a 100MB
> > > > > file? He got a super fast disk...
> > > > >
> > > > > ----- Original Message -----
> > > > >  *From:* Rodrigo Cunha <rn...@gm...>
> > > > > *To:* Jimmy Zhang <cra...@co...>
> > > > > *Cc:* Fernando Gonzalez <fer...@gm...> ; vtd...@li...
> > > > >
> > > > > *Sent:* Sunday, March 18, 2007 8:40 PM
> > > > > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> > > > >
> > > > > In fact the idea occured to me in the past also... but VTD is so
> > > > > fast reading large files anyway! With a fast processor I think we might be
> > > > > disk-limited rather than processor-limited. Still, if the code is made
> > > > > already, the option seems cute enought to keep :-)
> > > > >
> > > > > Since I mainly deal with large files requiring a lots of
> > > > > processing this has not been an issue. Others, in different environments,
> > > > > might disagree.
> > > > >
> > > > > Jimmy Zhang wrote:
> > > > >
> > > > > Fernando,  The option for storing VTD in a separate file  is open.
> > > > >
> > > > > I attached  the technical document from your last email, and am also
> > > > >
> > > > > interested in the suggestions/comments from the mailing list ...
> > > > >
> > > > >
> > > > >
> > > >  ------------------------------
> > > >
> > > >
> > > > -------------------------------------------------------------------------
> > > > Take Surveys. Earn Cash. Influence the Future of IT
> > > > Join SourceForge.net's Techsay panel and you'll get the chance to
> > > > share your
> > > > opinions on IT & business topics through brief surveys-and earn cash
> > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > > >
> > > >
> > > > ------------------------------
> > > >
> > > > _______________________________________________
> > > > Vtd-xml-users mailing list
> > > > Vtd...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> > > >
> > > >
> > > >
> > > > -------------------------------------------------------------------------
> > > > Take Surveys. Earn Cash. Influence the Future of IT
> > > > Join SourceForge.net's Techsay panel and you'll get the chance to
> > > > share your
> > > > opinions on IT & business topics through brief surveys-and earn cash
> > > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > > >
> > > > _______________________________________________
> > > > Vtd-xml-users mailing list
> > > > Vtd...@li...
> > > > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> > > >
> > > >
> > >  ------------------------------
> > >
> > >
> > > -------------------------------------------------------------------------
> > > Take Surveys. Earn Cash. Influence the Future of IT
> > > Join SourceForge.net's Techsay panel and you'll get the chance to
> > > share your
> > > opinions on IT & business topics through brief surveys-and earn cash
> > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > >
> > >
> > > ------------------------------
> > >
> > > _______________________________________________
> > > Vtd-xml-users mailing list
> > > Vtd...@li...
> > > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> > >
> > >
> >
>  ------------------------------
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> ------------------------------
>
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>
>

Re: [Vtd-xml-users] Storing parsing info

[Jimmy Z. <cra...@co...>]

Not a problem :-), one reason we are having this discussion is that the =
indexing feature (and
VTD-XML itself) is so new and we have yet to understand the possiblities =
and design trade-offs
... Yes, I can see why tuning the optimum buffer size can potentially =
improve performance...
but in general do you see any issue with the XPath evaluation ?


  ----- Original Message -----=20
  From: Fernando Gonzalez=20
  To: vtd...@li...=20
  Sent: Wednesday, March 21, 2007 3:14 AM
  Subject: Re: [Vtd-xml-users] Storing parsing info


  Sorry Jimmy, I misunderstood you. Please, forget the first paragraph =
of my last mail.

  yes, you're right, when you ask for a byte it may not be in the loaded =
chunk... so you have to load another chunk. Of course it's quite slower =
than the current solution. But I think there must be a buffer size that =
optimizes performance so the solution is only a bit slower without =
loading more than 20-30Mb in memory. Don't you think so?=20

  Anyway, the proposed changes don't force to use that approach. I'ts =
still possible to load the whole XML in memory, so the only side effect =
of the proposal is that the library is a bit more complex (there are =
some exception handling issues and the user have to provide a =
implementation of an interface instead of a byte[]) and the library is a =
bit slower (because of the added indirection level)=20

  Fernando


  On 3/21/07, Fernando Gonzalez <fer...@gm...> wrote:
    "if the chunks don't have what one is looking for, you will have to =
load in another chunk... then=20
    another chunk.."

    Not exactly. If something asks for the the byte number 'x' I guess =
which chunk the byte is in and I load only that chunk. Only if the =
information asked for by the user is spread in two or more contiguous =
chunks will be necessary to load more than one. The implementation can =
be seen in the " org.ChunkByteBuffer" class in the "public byte =
byteAt(int streamIndex)" method.

    About the alternative you propose. As I have told before, it's not a =
good solution to remove or archive the original GML. Splitting wouldn't =
be as bad as removal or archiving, but it would add some complexity to =
the user. The user should keep track of the splitted GML files that form =
the original GML file. The splitting could be implemented in a way that =
the user doesn't notice it... while he doesn't try to access the file =
with another application.=20

    Indeed, I think that, in the end, I'm doing something similar to =
splitting since I logically split the GML file into several GML chunks.

    Fernando




    On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
      Ok, I see... it seems that you can be sure that the "chunks" of =
GML files contain what the user would need...
      But in general, if the chunks don't have what one is looking for, =
you will have to load in another chunk... then
      another chunk.. that could mean a lot of disk activities
      As an alternative, would it be possible to split GML into little =
chunks of well-formed GML files, then index
      them individually.=20
      So instead of dealing with 10 big GML files, split them into 100 =
smaller GML files and the algorithm you describe
       may still work..
        ----- Original Message -----=20
        From: Fernando Gonzalez=20
        To: vtd...@li...=20
        Sent: Tuesday, March 20, 2007 2:39 AM
        Subject: Re: [Vtd-xml-users] Storing parsing info


        On 3/20/07, Jimmy Zhang <cra...@co...> wrote:=20
          So what you are trying to accomplish is to load all the GML =
docs into memory at once...
          I guess you can simply index all those files to avoid =
parsing...
          but I still don't seem to understand the benefits of read teh =
parse info and a chunk of
          the XML file..

        Quite near. What I need is to access a random feature at any =
time with as a low cost as possible. That could be possible loading all =
the GML docs in memory but the GML files are very big so I cannot do it. =


        As that solution wasn't suitable to my problem, I thought of =
opening one file each time (using buffer reuse) and then it came to my =
mind that I could save parsing time storing the parse info. As I told =
before I cannot delete the GML. Storing the GML twice will waste disk =
space. I'm talking about an environment where the user can have in his =
computer a lot of digital cartography. Disk space is quite a bottle =
neck. It could be valid, but storing only the parse info was so easy =
that I did it and I obtained a better solution (for my environment).

        There is a use case where the user doesn't work with the files =
directly, but with a spatial region. In this case, the GML files and =
other spatial data are "layers", so the user can work at the same time =
with a lot of files. These files can be in other formats than GML, =
satellite images, different raster or vectorial formats; and these can =
bring the system to a even more memory constrained situation. That's =
what lead me to load chunks of the GML file.

        The workflow is the following
        * I open a file with the chunk approach
        * I parse the file (loading it with the chunks approach takes a =
lot, but no problem)
        * I store the parse info=20
        The user asks for information:
        * I load the parse info
        * I load the chunk
        * I return the asked information

        I want to speed up the asking of information because the user =
can ask for a map image with 20 GML files, and the map code is something =
like this:

        for each gml file
          guess what "features" are inside the map bounds (GML is =
indexed spatially previously)
          get those features from the GML (random access) (load parse =
info + load chunk + return info)=20
          draw the features on a image
        next gml file

        Maybe this will make things a bit clearer. This screenshot =
(http://www.gvsig.gva.es/fileadmin/conselleria/images/Documentacion/captu=
ras/raster_shp_dgn_750.gif ) shows a program that uses the library. You =
can see on the left all the loaded (from the user point of view) files: =
four "dgn" files, one "shp" and seven "ecw" files. A lot of operations =
done in the map are done over *every* file listed on the left so I don't =
care how much time it takes to put all those files on the left =
(generating parse info, etc). I care how much time takes to read the =
information after they are loaded (again, from the user point of view).

        Well, I hope it's clear enough. Notice that I'm not proposing =
changing the way VTD-XML works but I'm proposing to add new ways.

        greetings,
        Fernando =20


            ----- Original Message -----=20
            From: Fernando Gonzalez=20
            To: vtd...@li...=20
            Sent: Monday, March 19, 2007 2:56 AM
            Subject: Re: [Vtd-xml-users] Storing parsing info


            Well, jeje, the computer is new but I don't think my disk is =
so fast. I think Java or the operating system has to cache something =
because the first time I load the file it takes a bit more than 2 =
seconds and after the first load, it only takes 300ms to read the =
file...=20
            I have no experience on doing benchmarks and maybe I'm am =
missing something. That's why I attached the program.

            "So if you can't delete the orginal XML files, can you =
compress them and=20
            store them away (archiving)?"

            I cannot delete nor archive the GML file because in this =
context it won't be rare to be reading it from two different programs at =
the same time... It's difficult to find an open source program that does =
everything you need. For example, in a development context, there may be =
a map server serving a map image based on a GML file while you are =
opening it to see some data in it.=20

            "The other issue you raised is buffer reuse. To reuse =
internal buffers of=20
            VTDGen, you can call setDoc_BR(...). But there is more you =
can do...
            you can in fact reuse the byte array containing the XML =
document."
            Buffer reuse absolutly solves my memory constraints. But the =
problem I see with buffer reuse is that it will force me to read and =
parse the whole XML file every time the user ask for information on =
another XML file, won't it? If I read the XML file by chunks and I =
store/read the parse information, each time the user asks for =
information on another XML file I only have to read the parse info and a =
chunk of the XML file.=20

            To show you my point of view:
            The "user asking for another XML file" may be a map server =
that reads some big GML files and draws its spatial information in a map =
image. If each time the map server draws a GML file and "changes" to the =
next it takes 2 seconds or so, the drawing of the map (all the GML =
files) takes too much time.=20

            best regards,
            Fernando



            On 3/19/07, Jimmy Zhang <cra...@co...> wrote:=20

              What intrigues me with Fernando's test results is that it =
only takes 300ms to read a 100MB
              file? He got a super fast disk...
                ----- Original Message -----=20
                From: Rodrigo Cunha=20
                To: Jimmy Zhang=20
                Cc: Fernando Gonzalez ; =
vtd...@li...=20
                Sent: Sunday, March 18, 2007 8:40 PM
                Subject: Re: [Vtd-xml-users] Storing parsing info


                In fact the idea occured to me in the past also... but =
VTD is so fast reading large files anyway! With a fast processor I think =
we might be disk-limited rather than processor-limited. Still, if the =
code is made already, the option seems cute enought to keep :-)

                Since I mainly deal with large files requiring a lots of =
processing this has not been an issue. Others, in different =
environments, might disagree.

                Jimmy Zhang wrote:=20
                  Fernando,  The option for storing VTD in a separate =
file  is open.=20
                  I attached  the technical document from your last =
email, and am also=20
                  interested in the suggestions/comments from the =
mailing list ...=20






--------------------------------------------------------------------


            =
-------------------------------------------------------------------------=

            Take Surveys. Earn Cash. Influence the Future of IT
            Join SourceForge.net's Techsay panel and you'll get the =
chance to share your
            opinions on IT & business topics through brief surveys-and =
earn cash
            =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV=20


--------------------------------------------------------------------


            _______________________________________________
            Vtd-xml-users mailing list
            Vtd...@li...
            https://lists.sourceforge.net/lists/listinfo/vtd-xml-users






          =
-------------------------------------------------------------------------=

          Take Surveys. Earn Cash. Influence the Future of IT
          Join SourceForge.net's Techsay panel and you'll get the chance =
to share your=20
          opinions on IT & business topics through brief surveys-and =
earn cash
          =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV
          _______________________________________________
          Vtd-xml-users mailing list
          Vtd...@li...
          https://lists.sourceforge.net/lists/listinfo/vtd-xml-users=20






------------------------------------------------------------------------


        =
-------------------------------------------------------------------------=

        Take Surveys. Earn Cash. Influence the Future of IT
        Join SourceForge.net's Techsay panel and you'll get the chance =
to share your
        opinions on IT & business topics through brief surveys-and earn =
cash
        =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV=20


------------------------------------------------------------------------


        _______________________________________________
        Vtd-xml-users mailing list
        Vtd...@li...
        https://lists.sourceforge.net/lists/listinfo/vtd-xml-users









-------------------------------------------------------------------------=
-----


  =
-------------------------------------------------------------------------=

  Take Surveys. Earn Cash. Influence the Future of IT
  Join SourceForge.net's Techsay panel and you'll get the chance to =
share your
  opinions on IT & business topics through brief surveys-and earn cash
  =
http://www.techsay.com/default.php?page=3Djoin.php&p=3Dsourceforge&CID=3D=
DEVDEV


-------------------------------------------------------------------------=
-----


  _______________________________________________
  Vtd-xml-users mailing list
  Vtd...@li...
  https://lists.sourceforge.net/lists/listinfo/vtd-xml-users

Re: [Vtd-xml-users] Storing parsing info

[Fernando G. <fer...@gm...>]

Sorry Jimmy, I misunderstood you. Please, forget the first paragraph of my
last mail.

yes, you're right, when you ask for a byte it may not be in the loaded
chunk... so you have to load another chunk. Of course it's quite slower than
the current solution. But I think there must be a buffer size that optimizes
performance so the solution is only a bit slower without loading more than
20-30Mb in memory. Don't you think so?

Anyway, the proposed changes don't force to use that approach. I'ts still
possible to load the whole XML in memory, so the only side effect of the
proposal is that the library is a bit more complex (there are some exception
handling issues and the user have to provide a implementation of an
interface instead of a byte[]) and the library is a bit slower (because of
the added indirection level)

Fernando

On 3/21/07, Fernando Gonzalez <fer...@gm...> wrote:
>
> "if the chunks don't have what one is looking for, you will have to load
> in another chunk... then another chunk.."
>
> Not exactly. If something asks for the the byte number 'x' I guess which
> chunk the byte is in and I load only that chunk. Only if the information
> asked for by the user is spread in two or more contiguous chunks will be
> necessary to load more than one. The implementation can be seen in the "
> org.ChunkByteBuffer" class in the "public byte byteAt(int streamIndex)"
> method.
>
> About the alternative you propose. As I have told before, it's not a good
> solution to remove or archive the original GML. Splitting wouldn't be as bad
> as removal or archiving, but it would add some complexity to the user. The
> user should keep track of the splitted GML files that form the original GML
> file. The splitting could be implemented in a way that the user doesn't
> notice it... while he doesn't try to access the file with another
> application.
>
> Indeed, I think that, in the end, I'm doing something similar to splitting
> since I logically split the GML file into several GML chunks.
>
> Fernando
>
>
> On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
> >
> >  Ok, I see... it seems that you can be sure that the "chunks" of GML
> > files contain what the user would need...
> > But in general, if the chunks don't have what one is looking for, you
> > will have to load in another chunk... then
> > another chunk.. that could mean a lot of disk activities
> > As an alternative, would it be possible to split GML into little chunks
> > of well-formed GML files, then index
> > them individually.
> > So instead of dealing with 10 big GML files, split them into 100 smaller
> > GML files and the algorithm you describe
> >  may still work..
> >
> > ----- Original Message -----
> >  *From:* Fernando Gonzalez <fer...@gm...>
> > *To:* vtd...@li...
> > *Sent:* Tuesday, March 20, 2007 2:39 AM
> > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> >
> > On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
> > >
> > >  So what you are trying to accomplish is to load all the GML docs into
> > > memory at once...
> > > I guess you can simply index all those files to avoid parsing...
> > > but I still don't seem to understand the benefits of read teh parse
> > > info and a chunk of
> > > the XML file..
> > >
> >
> > Quite near. What I need is to access a random feature at any time with
> > as a low cost as possible. That could be possible loading all the GML docs
> > in memory but the GML files are very big so I cannot do it.
> >
> > As that solution wasn't suitable to my problem, I thought of opening one
> > file each time (using buffer reuse) and then it came to my mind that I could
> > save parsing time storing the parse info. As I told before I cannot delete
> > the GML. Storing the GML twice will waste disk space. I'm talking about an
> > environment where the user can have in his computer a lot of digital
> > cartography. Disk space is quite a bottle neck. It could be valid, but
> > storing only the parse info was so easy that I did it and I obtained a
> > better solution (for my environment).
> >
> > There is a use case where the user doesn't work with the files directly,
> > but with a spatial region. In this case, the GML files and other spatial
> > data are "layers", so the user can work at the same time with a lot of
> > files. These files can be in other formats than GML, satellite images,
> > different raster or vectorial formats; and these can bring the system to a
> > even more memory constrained situation. That's what lead me to load chunks
> > of the GML file.
> >
> > The workflow is the following
> > * I open a file with the chunk approach
> > * I parse the file (loading it with the chunks approach takes a lot, but
> > no problem)
> > * I store the parse info
> > The user asks for information:
> > * I load the parse info
> > * I load the chunk
> > * I return the asked information
> >
> > I want to speed up the asking of information because the user can ask
> > for a map image with 20 GML files, and the map code is something like this:
> >
> > for each gml file
> >   guess what "features" are inside the map bounds (GML is indexed
> > spatially previously)
> >   get those features from the GML (random access) (load parse info +
> > load chunk + return info)
> >   draw the features on a image
> > next gml file
> >
> > Maybe this will make things a bit clearer. This screenshot (http://www.gvsig.gva.es/fileadmin/conselleria/images/Documentacion/capturas/raster_shp_dgn_750.gif
> > ) shows a program that uses the library. You can see on the left all the
> > loaded (from the user point of view) files: four "dgn" files, one "shp" and
> > seven "ecw" files. A lot of operations done in the map are done over *every*
> > file listed on the left so I don't care how much time it takes to put all
> > those files on the left (generating parse info, etc). I care how much time
> > takes to read the information after they are loaded (again, from the user
> > point of view).
> >
> > Well, I hope it's clear enough. Notice that I'm not proposing changing
> > the way VTD-XML works but I'm proposing to add new ways.
> >
> > greetings,
> > Fernando
> >
> >   ----- Original Message -----
> > >  *From:* Fernando Gonzalez <fer...@gm...>
> > >  *To:* vtd...@li...
> > > *Sent:* Monday, March 19, 2007 2:56 AM
> > > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> > >
> > > Well, jeje, the computer is new but I don't think my disk is so fast.
> > > I think Java or the operating system has to cache something because the
> > > first time I load the file it takes a bit more than 2 seconds and after the
> > > first load, it only takes 300ms to read the file...
> > > I have no experience on doing benchmarks and maybe I'm am missing
> > > something. That's why I attached the program.
> > >
> > > "So if you can't delete the orginal XML files, can you compress them
> > > and store them away (archiving)?"
> > > I cannot delete nor archive the GML file because in this context it
> > > won't be rare to be reading it from two different programs at the same
> > > time... It's difficult to find an open source program that does everything
> > > you need. For example, in a development context, there may be a map server
> > > serving a map image based on a GML file while you are opening it to see some
> > > data in it.
> > >
> > > "The other issue you raised is buffer reuse. To reuse internal buffers
> > > of VTDGen, you can call setDoc_BR(...). But there is more you can
> > > do...
> > > you can in fact reuse the byte array containing the XML document."
> > > Buffer reuse absolutly solves my memory constraints. But the problem I
> > > see with buffer reuse is that it will force me to read and parse the whole
> > > XML file every time the user ask for information on another XML file, won't
> > > it? If I read the XML file by chunks and I store/read the parse information,
> > > each time the user asks for information on another XML file I only have to
> > > read the parse info and a chunk of the XML file.
> > >
> > > To show you my point of view:
> > > The "user asking for another XML file" may be a map server that reads
> > > some big GML files and draws its spatial information in a map image. If each
> > > time the map server draws a GML file and "changes" to the next it takes 2
> > > seconds or so, the drawing of the map (all the GML files) takes too much
> > > time.
> > >
> > > best regards,
> > > Fernando
> > >
> > >
> > > On 3/19/07, Jimmy Zhang <cra...@co...> wrote:
> > > >
> > > >
> > > > What intrigues me with Fernando's test results is that it only takes
> > > > 300ms to read a 100MB
> > > > file? He got a super fast disk...
> > > >
> > > > ----- Original Message -----
> > > >  *From:* Rodrigo Cunha <rn...@gm...>
> > > > *To:* Jimmy Zhang <cra...@co...>
> > > > *Cc:* Fernando Gonzalez <fer...@gm...> ; vtd...@li...
> > > >
> > > > *Sent:* Sunday, March 18, 2007 8:40 PM
> > > > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> > > >
> > > > In fact the idea occured to me in the past also... but VTD is so
> > > > fast reading large files anyway! With a fast processor I think we might be
> > > > disk-limited rather than processor-limited. Still, if the code is made
> > > > already, the option seems cute enought to keep :-)
> > > >
> > > > Since I mainly deal with large files requiring a lots of processing
> > > > this has not been an issue. Others, in different environments, might
> > > > disagree.
> > > >
> > > > Jimmy Zhang wrote:
> > > >
> > > > Fernando,  The option for storing VTD in a separate file  is open.
> > > > I attached  the technical document from your last email, and am also
> > > >
> > > > interested in the suggestions/comments from the mailing list ...
> > > >
> > > >
> > > >
> > >  ------------------------------
> > >
> > >
> > > -------------------------------------------------------------------------
> > > Take Surveys. Earn Cash. Influence the Future of IT
> > > Join SourceForge.net's Techsay panel and you'll get the chance to
> > > share your
> > > opinions on IT & business topics through brief surveys-and earn cash
> > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > >
> > >
> > > ------------------------------
> > >
> > > _______________________________________________
> > > Vtd-xml-users mailing list
> > > Vtd...@li...
> > > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> > >
> > >
> > >
> > > -------------------------------------------------------------------------
> > > Take Surveys. Earn Cash. Influence the Future of IT
> > > Join SourceForge.net's Techsay panel and you'll get the chance to
> > > share your
> > > opinions on IT & business topics through brief surveys-and earn cash
> > >
> > > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > > _______________________________________________
> > > Vtd-xml-users mailing list
> > > Vtd...@li...
> > > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> > >
> > >
> >  ------------------------------
> >
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > opinions on IT & business topics through brief surveys-and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >
> >
> > ------------------------------
> >
> > _______________________________________________
> > Vtd-xml-users mailing list
> > Vtd...@li...
> > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> >
> >
>

Re: [Vtd-xml-users] Storing parsing info

[Fernando G. <fer...@gm...>]

"if the chunks don't have what one is looking for, you will have to load in
another chunk... then another chunk.."

Not exactly. If something asks for the the byte number 'x' I guess which
chunk the byte is in and I load only that chunk. Only if the information
asked for by the user is spread in two or more contiguous chunks will be
necessary to load more than one. The implementation can be seen in the "
org.ChunkByteBuffer" class in the "public byte byteAt(int streamIndex)"
method.

About the alternative you propose. As I have told before, it's not a good
solution to remove or archive the original GML. Splitting wouldn't be as bad
as removal or archiving, but it would add some complexity to the user. The
user should keep track of the splitted GML files that form the original GML
file. The splitting could be implemented in a way that the user doesn't
notice it... while he doesn't try to access the file with another
application.

Indeed, I think that, in the end, I'm doing something similar to splitting
since I logically split the GML file into several GML chunks.

Fernando


On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
>
>  Ok, I see... it seems that you can be sure that the "chunks" of GML files
> contain what the user would need...
> But in general, if the chunks don't have what one is looking for, you will
> have to load in another chunk... then
> another chunk.. that could mean a lot of disk activities
> As an alternative, would it be possible to split GML into little chunks of
> well-formed GML files, then index
> them individually.
> So instead of dealing with 10 big GML files, split them into 100 smaller
> GML files and the algorithm you describe
>  may still work..
>
> ----- Original Message -----
> *From:* Fernando Gonzalez <fer...@gm...>
> *To:* vtd...@li...
> *Sent:* Tuesday, March 20, 2007 2:39 AM
> *Subject:* Re: [Vtd-xml-users] Storing parsing info
>
> On 3/20/07, Jimmy Zhang <cra...@co...> wrote:
> >
> >  So what you are trying to accomplish is to load all the GML docs into
> > memory at once...
> > I guess you can simply index all those files to avoid parsing...
> > but I still don't seem to understand the benefits of read teh parse info
> > and a chunk of
> > the XML file..
> >
>
> Quite near. What I need is to access a random feature at any time with as
> a low cost as possible. That could be possible loading all the GML docs in
> memory but the GML files are very big so I cannot do it.
>
> As that solution wasn't suitable to my problem, I thought of opening one
> file each time (using buffer reuse) and then it came to my mind that I could
> save parsing time storing the parse info. As I told before I cannot delete
> the GML. Storing the GML twice will waste disk space. I'm talking about an
> environment where the user can have in his computer a lot of digital
> cartography. Disk space is quite a bottle neck. It could be valid, but
> storing only the parse info was so easy that I did it and I obtained a
> better solution (for my environment).
>
> There is a use case where the user doesn't work with the files directly,
> but with a spatial region. In this case, the GML files and other spatial
> data are "layers", so the user can work at the same time with a lot of
> files. These files can be in other formats than GML, satellite images,
> different raster or vectorial formats; and these can bring the system to a
> even more memory constrained situation. That's what lead me to load chunks
> of the GML file.
>
> The workflow is the following
> * I open a file with the chunk approach
> * I parse the file (loading it with the chunks approach takes a lot, but
> no problem)
> * I store the parse info
> The user asks for information:
> * I load the parse info
> * I load the chunk
> * I return the asked information
>
> I want to speed up the asking of information because the user can ask for
> a map image with 20 GML files, and the map code is something like this:
>
> for each gml file
>   guess what "features" are inside the map bounds (GML is indexed
> spatially previously)
>   get those features from the GML (random access) (load parse info + load
> chunk + return info)
>   draw the features on a image
> next gml file
>
> Maybe this will make things a bit clearer. This screenshot (
> http://www.gvsig.gva.es/fileadmin/conselleria/images/Documentacion/capturas/raster_shp_dgn_750.gif)
> shows a program that uses the library. You can see on the left all the
> loaded (from the user point of view) files: four "dgn" files, one "shp" and
> seven "ecw" files. A lot of operations done in the map are done over *every*
> file listed on the left so I don't care how much time it takes to put all
> those files on the left (generating parse info, etc). I care how much time
> takes to read the information after they are loaded (again, from the user
> point of view).
>
> Well, I hope it's clear enough. Notice that I'm not proposing changing the
> way VTD-XML works but I'm proposing to add new ways.
>
> greetings,
> Fernando
>
>   ----- Original Message -----
> > *From:* Fernando Gonzalez <fer...@gm...>
> >  *To:* vtd...@li...
> > *Sent:* Monday, March 19, 2007 2:56 AM
> > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> >
> > Well, jeje, the computer is new but I don't think my disk is so fast. I
> > think Java or the operating system has to cache something because the first
> > time I load the file it takes a bit more than 2 seconds and after the first
> > load, it only takes 300ms to read the file...
> > I have no experience on doing benchmarks and maybe I'm am missing
> > something. That's why I attached the program.
> >
> > "So if you can't delete the orginal XML files, can you compress them and store
> > them away (archiving)?"
> > I cannot delete nor archive the GML file because in this context it
> > won't be rare to be reading it from two different programs at the same
> > time... It's difficult to find an open source program that does everything
> > you need. For example, in a development context, there may be a map server
> > serving a map image based on a GML file while you are opening it to see some
> > data in it.
> >
> > "The other issue you raised is buffer reuse. To reuse internal buffers
> > of VTDGen, you can call setDoc_BR(...). But there is more you can do...
> > you can in fact reuse the byte array containing the XML document."
> > Buffer reuse absolutly solves my memory constraints. But the problem I
> > see with buffer reuse is that it will force me to read and parse the whole
> > XML file every time the user ask for information on another XML file, won't
> > it? If I read the XML file by chunks and I store/read the parse information,
> > each time the user asks for information on another XML file I only have to
> > read the parse info and a chunk of the XML file.
> >
> > To show you my point of view:
> > The "user asking for another XML file" may be a map server that reads
> > some big GML files and draws its spatial information in a map image. If each
> > time the map server draws a GML file and "changes" to the next it takes 2
> > seconds or so, the drawing of the map (all the GML files) takes too much
> > time.
> >
> > best regards,
> > Fernando
> >
> >
> > On 3/19/07, Jimmy Zhang <cra...@co...> wrote:
> > >
> > >
> > > What intrigues me with Fernando's test results is that it only takes
> > > 300ms to read a 100MB
> > > file? He got a super fast disk...
> > >
> > > ----- Original Message -----
> > > *From:* Rodrigo Cunha <rn...@gm...>
> > > *To:* Jimmy Zhang <cra...@co...>
> > > *Cc:* Fernando Gonzalez <fer...@gm...> ; vtd...@li...
> > >
> > > *Sent:* Sunday, March 18, 2007 8:40 PM
> > > *Subject:* Re: [Vtd-xml-users] Storing parsing info
> > >
> > > In fact the idea occured to me in the past also... but VTD is so fast
> > > reading large files anyway! With a fast processor I think we might be
> > > disk-limited rather than processor-limited. Still, if the code is made
> > > already, the option seems cute enought to keep :-)
> > >
> > > Since I mainly deal with large files requiring a lots of processing
> > > this has not been an issue. Others, in different environments, might
> > > disagree.
> > >
> > > Jimmy Zhang wrote:
> > >
> > > Fernando,  The option for storing VTD in a separate file  is open.
> > > I attached  the technical document from your last email, and am also
> > > interested in the suggestions/comments from the mailing list ...
> > >
> > >
> > >
> >  ------------------------------
> >
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > opinions on IT & business topics through brief surveys-and earn cash
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> >
> >
> > ------------------------------
> >
> > _______________________________________________
> > Vtd-xml-users mailing list
> > Vtd...@li...
> > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> >
> >
> >
> > -------------------------------------------------------------------------
> > Take Surveys. Earn Cash. Influence the Future of IT
> > Join SourceForge.net's Techsay panel and you'll get the chance to share
> > your
> > opinions on IT & business topics through brief surveys-and earn cash
> >
> > http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> > _______________________________________________
> > Vtd-xml-users mailing list
> > Vtd...@li...
> > https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
> >
> >
>  ------------------------------
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
>
> ------------------------------
>
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>
>

Re: [Vtd-xml-users] HowTo use Vtd for GML to Java Mapping?

[Jimmy Z. <cra...@co...>]

Exactly. Use XPath as a filter so that only the objects relevant to the
application logic are created ... this way the object creation cost can
be reduced significantly, making databind suitable for even very performance
demand apps....

----- Original Message ----- 
From: "Tatu Saloranta" <cow...@ya...>
To: <vtd...@li...>
Sent: Tuesday, March 20, 2007 11:25 AM
Subject: Re: [Vtd-xml-users] HowTo use Vtd for GML to Java Mapping?


> --- Fernando Gonzalez <fer...@gm...> wrote:
>
>> As long as I know about GML files... It only defines
>> data types to be used
>> on your application specific schema. If you have a
>> concrete
>> application-specific schema, congratulations. Use a
>> mapping tool like JAXB
>> or Castor or whatever. If you haven't, you can't map
>> because you don't have
>> a schema to map. In this point I agree with Jimmy
>
> Or equivalent object hierarchy... schema isn't
> strictly necessary, fortunatley.
> But object model has to exist and be (or made)
> compatible. JibX is another fine candidate.
>
>> that you should use XPath
>> to obtain the data of interest and I think VTD-XML
>> is very useful.
>
> I don't know anything about GML specifically, but
> point I tried to make is that the question was
> specifically (even if mistakenly) asking about
> xml/object mapping. Alternative to that may be using
> VTD-XML with xpath ("instead of object mapping, why
> not just directly exact data you need").
> Just want to keep terminology accurate -- otherwise
> users expectations may not be met (if they really have
> object model as was implied).
>
> -+ Tatu +-
>
>
>
>
> ____________________________________________________________________________________
> Bored stiff? Loosen up...
> Download and play hundreds of games for free on Yahoo! Games.
> http://games.yahoo.com/games/front
>
> -------------------------------------------------------------------------
> Take Surveys. Earn Cash. Influence the Future of IT
> Join SourceForge.net's Techsay panel and you'll get the chance to share 
> your
> opinions on IT & business topics through brief surveys-and earn cash
> http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
> _______________________________________________
> Vtd-xml-users mailing list
> Vtd...@li...
> https://lists.sourceforge.net/lists/listinfo/vtd-xml-users
>