exist-teixml — Discussions about eXist and TEI

[eXist-TEIXML] Starting out

[Gary B. <gar...@sy...>]

Hello,

I have a large TEI document (book) which needs to be put online navigable by chapter and sub-sections of some chapters.
Just throwing it out there to see if there is any general advice on how to proceed, any gotchas I need to look out for?

Thanks a lot,
Gary


GARY BROWNE | Development Programmer 
Library IT Services | Fisher Library F03 				 					
THE UNIVERSITY OF SYDNEY

T +61 2 9351 5946  | M +61 405 647 868  
E gar...@sy...  | W http://sydney.edu.au 
Sent from my plain old desktop computer.

CRICOS 00026A
This email plus any attachments to it are confidential. Any unauthorised use is strictly prohibited. If you receive this email in error, please delete it and any attachments.
Please think of our environment and only print this e-mail if necessary.

Re: [eXist-TEIXML] Rig-Veda in TEI

[Lou B. <lou...@gm...>]

If you make a new version, especially one in TEI XML, don't forget to
re-deposit it with the OTA!



On 17 August 2013 12:52, Claudius Teodorescu
<cla...@gm...>wrote:

> Great, thanks!
>
> Maybe I will apply on it the new module for NLP made by Wolfgang.
>
> Claudius
>
>
> On Sat, Aug 17, 2013 at 2:35 PM, Lou Burnard <lou...@gm...>wrote:
>
>> http://ota.ahds.ac.uk/desc/0589 is not in TEI but is freely downloadable
>> and could pretty easily be converted to XML
>>
>>
>>
>>
>> On 16 August 2013 15:32, Claudius Teodorescu <
>> cla...@gm...> wrote:
>>
>>> Hi,
>>>
>>> Thanks a lot for hints!
>>>
>>> Claudius
>>>
>>>
>>> On Fri, Aug 16, 2013 at 4:45 PM, Joe Wicentowski <jo...@gm...>wrote:
>>>
>>>> Hi Claudius,
>>>>
>>>> > Hi, Anyone knows about Rig-Veda in TEI?
>>>>
>>>> Are you looking for a TEI-encoded edition of the Rigveda?  I don't
>>>> know - googling yielded only this page with some other Sanskrit texts
>>>> encoded in TEI: http://indology.info/etexts/.  But the main TEI-L
>>>> mailing list is probably a better place to ask.  Searching the
>>>> archives, I see someone appears to have worked on it:
>>>> http://tei-l.markmail.org/thread/m4hkgskqxpyevu3j.  If you want to
>>>> post a message, the subscription link is at TEI-L:
>>>> http://listserv.brown.edu/archives/cgi-bin/wa?A0=TEI-L.  Another good
>>>> place is Digital Humanities Questions & Answers:
>>>> http://digitalhumanities.org/answers/.
>>>>
>>>> Good luck!
>>>> Joe
>>>>
>>>
>>>
>>>
>>> --
>>> http://kuberam.ro
>>> http://kuberam.ro/art
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>>> It's a free troubleshooting tool designed for production.
>>> Get down to code-level detail for bottlenecks, with <2% overhead.
>>> Download for free and get started troubleshooting in minutes.
>>>
>>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>>> _______________________________________________
>>> eXist-TEIXML mailing list
>>> eXi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>>>
>>>
>>
>
>
> --
> http://kuberam.ro
> http://kuberam.ro/art
>

Re: [eXist-TEIXML] Rig-Veda in TEI

[Claudius T. <cla...@gm...>]

Great, thanks!

Maybe I will apply on it the new module for NLP made by Wolfgang.

Claudius


On Sat, Aug 17, 2013 at 2:35 PM, Lou Burnard <lou...@gm...> wrote:

> http://ota.ahds.ac.uk/desc/0589 is not in TEI but is freely downloadable
> and could pretty easily be converted to XML
>
>
>
>
> On 16 August 2013 15:32, Claudius Teodorescu <
> cla...@gm...> wrote:
>
>> Hi,
>>
>> Thanks a lot for hints!
>>
>> Claudius
>>
>>
>> On Fri, Aug 16, 2013 at 4:45 PM, Joe Wicentowski <jo...@gm...>wrote:
>>
>>> Hi Claudius,
>>>
>>> > Hi, Anyone knows about Rig-Veda in TEI?
>>>
>>> Are you looking for a TEI-encoded edition of the Rigveda?  I don't
>>> know - googling yielded only this page with some other Sanskrit texts
>>> encoded in TEI: http://indology.info/etexts/.  But the main TEI-L
>>> mailing list is probably a better place to ask.  Searching the
>>> archives, I see someone appears to have worked on it:
>>> http://tei-l.markmail.org/thread/m4hkgskqxpyevu3j.  If you want to
>>> post a message, the subscription link is at TEI-L:
>>> http://listserv.brown.edu/archives/cgi-bin/wa?A0=TEI-L.  Another good
>>> place is Digital Humanities Questions & Answers:
>>> http://digitalhumanities.org/answers/.
>>>
>>> Good luck!
>>> Joe
>>>
>>
>>
>>
>> --
>> http://kuberam.ro
>> http://kuberam.ro/art
>>
>>
>> ------------------------------------------------------------------------------
>> Get 100% visibility into Java/.NET code with AppDynamics Lite!
>> It's a free troubleshooting tool designed for production.
>> Get down to code-level detail for bottlenecks, with <2% overhead.
>> Download for free and get started troubleshooting in minutes.
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
>> _______________________________________________
>> eXist-TEIXML mailing list
>> eXi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>>
>>
>


-- 
http://kuberam.ro
http://kuberam.ro/art

Re: [eXist-TEIXML] Rig-Veda in TEI

[Lou B. <lou...@gm...>]

http://ota.ahds.ac.uk/desc/0589 is not in TEI but is freely downloadable
and could pretty easily be converted to XML




On 16 August 2013 15:32, Claudius Teodorescu
<cla...@gm...>wrote:

> Hi,
>
> Thanks a lot for hints!
>
> Claudius
>
>
> On Fri, Aug 16, 2013 at 4:45 PM, Joe Wicentowski <jo...@gm...> wrote:
>
>> Hi Claudius,
>>
>> > Hi, Anyone knows about Rig-Veda in TEI?
>>
>> Are you looking for a TEI-encoded edition of the Rigveda?  I don't
>> know - googling yielded only this page with some other Sanskrit texts
>> encoded in TEI: http://indology.info/etexts/.  But the main TEI-L
>> mailing list is probably a better place to ask.  Searching the
>> archives, I see someone appears to have worked on it:
>> http://tei-l.markmail.org/thread/m4hkgskqxpyevu3j.  If you want to
>> post a message, the subscription link is at TEI-L:
>> http://listserv.brown.edu/archives/cgi-bin/wa?A0=TEI-L.  Another good
>> place is Digital Humanities Questions & Answers:
>> http://digitalhumanities.org/answers/.
>>
>> Good luck!
>> Joe
>>
>
>
>
> --
> http://kuberam.ro
> http://kuberam.ro/art
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>
>

Re: [eXist-TEIXML] Rig-Veda in TEI

[Claudius T. <cla...@gm...>]

Hi,

Thanks a lot for hints!

Claudius


On Fri, Aug 16, 2013 at 4:45 PM, Joe Wicentowski <jo...@gm...> wrote:

> Hi Claudius,
>
> > Hi, Anyone knows about Rig-Veda in TEI?
>
> Are you looking for a TEI-encoded edition of the Rigveda?  I don't
> know - googling yielded only this page with some other Sanskrit texts
> encoded in TEI: http://indology.info/etexts/.  But the main TEI-L
> mailing list is probably a better place to ask.  Searching the
> archives, I see someone appears to have worked on it:
> http://tei-l.markmail.org/thread/m4hkgskqxpyevu3j.  If you want to
> post a message, the subscription link is at TEI-L:
> http://listserv.brown.edu/archives/cgi-bin/wa?A0=TEI-L.  Another good
> place is Digital Humanities Questions & Answers:
> http://digitalhumanities.org/answers/.
>
> Good luck!
> Joe
>



-- 
http://kuberam.ro
http://kuberam.ro/art

Re: [eXist-TEIXML] Rig-Veda in TEI

[Joe W. <jo...@gm...>]

Hi Claudius,

> Hi, Anyone knows about Rig-Veda in TEI?

Are you looking for a TEI-encoded edition of the Rigveda?  I don't
know - googling yielded only this page with some other Sanskrit texts
encoded in TEI: http://indology.info/etexts/.  But the main TEI-L
mailing list is probably a better place to ask.  Searching the
archives, I see someone appears to have worked on it:
http://tei-l.markmail.org/thread/m4hkgskqxpyevu3j.  If you want to
post a message, the subscription link is at TEI-L:
http://listserv.brown.edu/archives/cgi-bin/wa?A0=TEI-L.  Another good
place is Digital Humanities Questions & Answers:
http://digitalhumanities.org/answers/.

Good luck!
Joe

[eXist-TEIXML] Rig-Veda in TEI

[Claudius T. <cla...@gm...>]

Hi, Anyone knows about Rig-Veda in TEI?

Thanks,
Claudius

-- 
http://kuberam.ro
http://kuberam.ro/art

Re: [eXist-TEIXML] Encoding dates for use in range index

[Peter S. <st...@we...>]

Am 04.04.2013 um 23:02 schrieb Christopher Thomson <chr...@ca...>:

> I also wonder whether this might be a good use for the map datatype?
> 
> let $pub-dates :=
> map {
>         "id1" = 1938,
>         "id2" = 1975,
>         "id3" = 1989
>        }
> 
> Each id would be a <biblStruct> xml:id.  The resulting map would be large, so I don't know if this is viable or not. But an advantage would be having the map functions to manipulate / manage it.

That's probably a very elegant solution with XQuery 3.0. Additionally you could store the map in a cache object so you would only need to create it once -- after every reboot or cache:clear().

Best Peter

-- 
Peter Stadler
Carl-Maria-von-Weber-Gesamtausgabe
Arbeitsstelle Detmold
Gartenstr. 20
D-32756 Detmold
Tel. +49 5231 975-665
Fax: +49 5231 975-668
stadler at weber-gesamtausgabe.de
www.weber-gesamtausgabe.de

Re: [eXist-TEIXML] Encoding dates for use in range index

[Christopher T. <chr...@ca...>]

Hi Peter,

Thanks for your reply.  I'd always envisioned constraining the date facets to year data, but had wanted to record the month and day when it was supplied as well.  It's really only an issue for articles published in newspapers, where the exact date is essential, but it would be handy if eXist's range index could work with xs:gYear values.

I had not thought beyond the range index as far as constructing my own index or concordance in the way you've done.  One complexity is that I have ~1300 TEI files (one for each author), and then many <biblStruct> elements within each TEI file (probably 20000+). However, I could match the xml:id for each <biblStruct> to a (publication) date similar to your approach. Thanks for sharing this.

I also wonder whether this might be a good use for the map datatype?

let $pub-dates :=
map {
         "id1" = 1938,
         "id2" = 1975,
         "id3" = 1989
        }

Each id would be a <biblStruct> xml:id.  The resulting map would be large, so I don't know if this is viable or not. But an advantage would be having the map functions to manipulate / manage it.

Thanks,
Chris

________________________________________
From: Peter Stadler [st...@we...]
Sent: 03 April 2013 03:13
To: Christopher Thomson
Cc: exist-teixml ‎[exi...@li...]‎
Subject: [SPAM: 11.000] Re: [eXist-TEIXML] Encoding dates for use in range index

Hi Chris,

I've been struggling with TEI dates for the same reasons as well: The TEI @when (as well as @from, @to et. al.) is a blend of several datatypes which makes it very hard to work with. My solution has been to a) constrain the possible datatypes to xs:gYear and xs:date in the schema and b) regularize the dates in a concordance file. This file simply lists all my document IDs together with its respective regularized date value (which is xs:date in my case).  That way I can perform all operations on that file only which is pretty fast. (That file should then be updated via e.g. triggers every time there are changes to your documents.)
But I think you can make your life much easier if you constrained your facets to years (which should be a sufficient granularity for bibliographies IMHO)?!
If you go down that road you probably won't need the util:index-keys approach but can simply do a distinct-values on your regularized year values.

Does that help?
All the best
Peter


Am 28.03.2013 um 01:57 schrieb Christopher Thomson <chr...@ca...>:

> Hi all,
>
> I'm working on obtaining data using util:index-keys in order to build a faceted search filter, as described in this tutorial: http://rvdb.wordpress.com/2010/10/06/mimicking-faceted-searching-in-exist/
>
> My question relates to dates.  I have been encoding dates using the TEI @when attribute.  However, I understand eXist's range index only works on valid xs:date attributes which need to be YYYY-MM-DD. The data is a humanities bibliography, so largely has publication dates which are year only, or sometimes year-month.  I can see a couple of options, but maybe there are others:
>
> 1) Index the @when as a string, ie <create qname="@when" type="xs:string"/>. This seems like a good option right now.  As my goal is to create a faceted search interface, I'm not sure if I lose much by recording dates as strings in the index. I want to be able to, say, narrow down search results by decade.  Any thoughts?  If I extract the date strings from my index and create facets, can I then cast them back to xs:date if that proves necessary?
>
> 2) Use a different attribute for non xs:date conforming date data, eg the TEI has @when-custom as an option for the date element.  The obvious downside to this is it puts my statistics for faceting into two places, ie @when and @when-custom.  The main use case is likely to be, 'Find all publications between these dates', so I suspect the data with year of publication only is going to be the most important to users.
>
> Does anyone have thoughts or advice on this?
>
> Thanks,
> Chris
>

--
Peter Stadler
Carl-Maria-von-Weber-Gesamtausgabe
Arbeitsstelle Detmold
Gartenstr. 20
D-32756 Detmold
Tel. +49 5231 975-665
Fax: +49 5231 975-668
stadler at weber-gesamtausgabe.de
www.weber-gesamtausgabe.de

Re: [eXist-TEIXML] Encoding dates for use in range index

[Peter S. <st...@we...>]

Hi Chris,

I've been struggling with TEI dates for the same reasons as well: The TEI @when (as well as @from, @to et. al.) is a blend of several datatypes which makes it very hard to work with. My solution has been to a) constrain the possible datatypes to xs:gYear and xs:date in the schema and b) regularize the dates in a concordance file. This file simply lists all my document IDs together with its respective regularized date value (which is xs:date in my case).  That way I can perform all operations on that file only which is pretty fast. (That file should then be updated via e.g. triggers every time there are changes to your documents.)
But I think you can make your life much easier if you constrained your facets to years (which should be a sufficient granularity for bibliographies IMHO)?!
If you go down that road you probably won't need the util:index-keys approach but can simply do a distinct-values on your regularized year values. 

Does that help?
All the best
Peter

 
Am 28.03.2013 um 01:57 schrieb Christopher Thomson <chr...@ca...>:

> Hi all,
> 
> I'm working on obtaining data using util:index-keys in order to build a faceted search filter, as described in this tutorial: http://rvdb.wordpress.com/2010/10/06/mimicking-faceted-searching-in-exist/
> 
> My question relates to dates.  I have been encoding dates using the TEI @when attribute.  However, I understand eXist's range index only works on valid xs:date attributes which need to be YYYY-MM-DD. The data is a humanities bibliography, so largely has publication dates which are year only, or sometimes year-month.  I can see a couple of options, but maybe there are others:
> 
> 1) Index the @when as a string, ie <create qname="@when" type="xs:string"/>. This seems like a good option right now.  As my goal is to create a faceted search interface, I'm not sure if I lose much by recording dates as strings in the index. I want to be able to, say, narrow down search results by decade.  Any thoughts?  If I extract the date strings from my index and create facets, can I then cast them back to xs:date if that proves necessary?
> 
> 2) Use a different attribute for non xs:date conforming date data, eg the TEI has @when-custom as an option for the date element.  The obvious downside to this is it puts my statistics for faceting into two places, ie @when and @when-custom.  The main use case is likely to be, 'Find all publications between these dates', so I suspect the data with year of publication only is going to be the most important to users.
> 
> Does anyone have thoughts or advice on this?
> 
> Thanks,
> Chris
> 

-- 
Peter Stadler
Carl-Maria-von-Weber-Gesamtausgabe
Arbeitsstelle Detmold
Gartenstr. 20
D-32756 Detmold
Tel. +49 5231 975-665
Fax: +49 5231 975-668
stadler at weber-gesamtausgabe.de
www.weber-gesamtausgabe.de

[eXist-TEIXML] Encoding dates for use in range index

[Christopher T. <chr...@ca...>]

Hi all,

I'm working on obtaining data using util:index-keys in order to build a faceted search filter, as described in this tutorial: http://rvdb.wordpress.com/2010/10/06/mimicking-faceted-searching-in-exist/

My question relates to dates.  I have been encoding dates using the TEI @when attribute.  However, I understand eXist's range index only works on valid xs:date attributes which need to be YYYY-MM-DD. The data is a humanities bibliography, so largely has publication dates which are year only, or sometimes year-month.  I can see a couple of options, but maybe there are others:

1) Index the @when as a string, ie <create qname="@when" type="xs:string"/>. This seems like a good option right now.  As my goal is to create a faceted search interface, I'm not sure if I lose much by recording dates as strings in the index. I want to be able to, say, narrow down search results by decade.  Any thoughts?  If I extract the date strings from my index and create facets, can I then cast them back to xs:date if that proves necessary?

2) Use a different attribute for non xs:date conforming date data, eg the TEI has @when-custom as an option for the date element.  The obvious downside to this is it puts my statistics for faceting into two places, ie @when and @when-custom.  The main use case is likely to be, 'Find all publications between these dates', so I suspect the data with year of publication only is going to be the most important to users.

Does anyone have thoughts or advice on this?

Thanks,
Chris

[eXist-TEIXML] WeGA eXist-TEI web app on GitHub

[Peter S. <st...@we...>]

Dear fellow TEI-eXistians,

it's my pleasure to announce that I finally pushed the source code of our XQUery'n'eXist-based web application 'WeGA-App' that powers weber-gesamtausgabe.d to GitHub: https://github.com/Edirom/WeGA-WebApp
It's at present designed to run with eXist 1.4.3 but I hope to make the transition as soon as the exciting 2.0 release is out!
And, be warned, it's of course highly idiosyncratic code that is tailored to our data but I hope it can nevertheless be helpful to spread the word of XQuery Web Apps!

I sincerely regret that I can't give you access to our whole corpus of TEI data but I added a few examples while the corresponding ODD files are already accessible at https://github.com/Edirom/WeGA-ODD

I'm looking forward to any question, comment, pull request, ...

Happy forking
and all the best
Peter


-- 
Peter Stadler
Carl-Maria-von-Weber-Gesamtausgabe
Arbeitsstelle Detmold
Gartenstr. 20
D-32756 Detmold
Tel. +49 5231 975-665
Fax: +49 5231 975-668
stadler at weber-gesamtausgabe.de
www.weber-gesamtausgabe.de

Re: [eXist-TEIXML] A branching problem

[Jens Ø. P. <oe...@gm...>]

Hi Peter,

Yes, this should return both a tei:placeName and a tei:persName. It does so in BaseX.

Jens

xquery version "1.0";

declare namespace tei="http://www.tei-c.org/ns/1.0";

let $input :=
<TEI xmlns="http://www.tei-c.org/ns/1.0">
    <teiHeader>
        <fileDesc>
            <titleStmt>
                <title>Title</title>
            </titleStmt>
            <publicationStmt>
                <p>Publication Information</p>
            </publicationStmt>
            <sourceDesc>
                <p>Information about the source</p>
            </sourceDesc>
        </fileDesc>
    </teiHeader>
    <text>
        <body>
            <div xml:id="d1e263" type="prose">
                <head>THE ENCHANTED CASTLE.</head>
                <p>There are warm days in <placeName corresp="london">London</placeName> when even a
                    window-box fails to charm, and one longs for the more open spaces of the
                    country. </p>
            </div>
            <div xml:id="d1e1806" type="review">
                <head rend="it">RUS IN URBE.</head>
                <p>"Sellinger's Bound," danced to an air that lulled <title>Titania</title> to sleep
                    all through the winter at the Savoy, was the most popular, with its ring of a
                    dozen dancers, hands joined, running together into the centre of their circle,
                    as if to honour some imaginary deity—possibly <persName corresp="sharp">Mr.
                            <name rend="sc">Cecil Sharp</name></persName>, director of the Society,
                    who has collected and revived the airs to which they dance.</p>
            </div>
        </body>
    </text>
</TEI>

let $corresp := ('sharp', 'london') 
	return
		$input//(tei:placeName, tei:persName)[@corresp=$corresp]

On Dec 21, 2012, at 5:49 PM, Peter Watson <pet...@ke...> wrote:

> Yes, I confirm that I could not get the alternatives to work in the XPath either.  Peter
> 
> On 21/12/2012 15:54, Jens Østergaard Petersen wrote:
>> 
>> On Dec 21, 2012, at 2:20 PM, Adam Retter <ad...@ex...> wrote:
>> 
>>> 
>>> Please excuse if I'm misunderstanding what you are trying to do. I assume you want to search both, persName and placeName, if both parameters are set? In this case you need to combine the two selections, e.g. with a union:
>>> 
>>> let $name-predicate := local:get-name-predicate(request:get-parameter('people',''))
>>> let $place-predicate := local:get-name-predicate(request:get-parameter('place',''))
>>> 
>>> 
>>> return
>>>     (if($name-predicate) then (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate]) else ())
>>> union
>>>     (if($place-predicate) then (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate]) else ())
>>> 
>>> Wolfgang
>>> 
>>> 
>>> I think you could also avoid the union altogether. Wolfgang, would this be more efficient? 
>>> 
>>> let $corresp := (local:get-name-predicate(request:get-parameter('people','')),  local:get-name-predicate(request:get-parameter('place,''))) return
>>>    collection('/db/punch/data')//tei:persName[@corresp=$corresp])
>>> 
>>> Thats the beauty of a language without null's where an empty sequence is a perfectly valid 'thing' to operate upon :-p
>> 
>> Well, then a sequence has to be introduced in the XPath,
>> 
>>  collection('/db/punch/data')//(tei:persName, tei:placeName)[@corresp=$corresp]
>> 
>> Strangely, eXist does not handle this correctly, ignoring all but the first item in the sequence ….
>> 
>> Jens
>> 
>>> 
>>> 
>>> -- 
>>> Adam Retter
>>> 
>>> eXist Developer
>>> { United Kingdom }
>>> ad...@ex...
>>> irc://irc.freenode.net/existdb
>>> ------------------------------------------------------------------------------
>>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>>> Remotely access PCs and mobile devices and provide instant support
>>> Improve your efficiency, and focus on delivering more value-add services
>>> Discover what IT Professionals Know. Rescue delivers
>>> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
>>> eXist-TEIXML mailing list
>>> eXi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>> 
>> 
>> 
>> ------------------------------------------------------------------------------
>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>> Remotely access PCs and mobile devices and provide instant support
>> Improve your efficiency, and focus on delivering more value-add services
>> Discover what IT Professionals Know. Rescue delivers
>> http://p.sf.net/sfu/logmein_12329d2d
>> 
>> 
>> _______________________________________________
>> eXist-TEIXML mailing list
>> eXi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
> 
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml

Re: [eXist-TEIXML] A branching problem

[Wolfgang M. <wol...@ex...>]

> Thank you all. I'm going to have to think about the answers. But what 
> I am finding is that where there is no $name-predicate it will not move 
> on and take up a $place-predicate that is present. I'm only looking for 
> one or the other, not both.

Aha, in this case your approach should work. Would it be possible that local:get-name-predicate() returns something (not '' or ()) even if the parameter is empty?

Without local:get-name-predicate, your example does return what I expect:

xquery version "3.0";
declare namespace tei="http://www.tei-c.org/ns/1.0";
let $name-predicate := request:get-parameter('people','')
let $place-predicate := request:get-parameter('place','#snelston')
let $selected-documents :=
if($name-predicate) then 
(collection('/db')//tei:persName[@corresp=$name-predicate])
else if($place-predicate) then 
(collection('/db')//tei:placeName[@corresp=$place-predicate])
else
()
return
$selected-documents



Wolfgang

Re: [eXist-TEIXML] A branching problem

[Peter W. <pet...@ke...>]

Yes, I confirm that I could not get the alternatives to work in the 
XPath either.  Peter

On 21/12/2012 15:54, Jens Østergaard Petersen wrote:
>
> On Dec 21, 2012, at 2:20 PM, Adam Retter <ad...@ex... 
> <mailto:ad...@ex...>> wrote:
>
>>
>>     Please excuse if I'm misunderstanding what you are trying to do.
>>     I assume you want to search both, persName and placeName, if both
>>     parameters are set? In this case you need to combine the two
>>     selections, e.g. with a union:
>>
>>     let $name-predicate :=
>>     local:get-name-predicate(request:get-parameter('people',''))
>>     let $place-predicate :=
>>     local:get-name-predicate(request:get-parameter('place',''))
>>
>>
>>     return
>>         (if($name-predicate) then
>>     (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate])
>>     else ())
>>     union
>>         (if($place-predicate) then
>>     (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate])
>>     else ())
>>
>>     Wolfgang
>>
>>
>>
>> I think you could also avoid the union altogether. Wolfgang, would 
>> this be more efficient?
>>
>> let $corresp := 
>> (local:get-name-predicate(request:get-parameter('people','')), 
>>  local:get-name-predicate(request:get-parameter('place,''))) return
>>  collection('/db/punch/data')//tei:persName[@corresp=$corresp])
>>
>> Thats the beauty of a language without null's where an empty sequence 
>> is a perfectly valid 'thing' to operate upon :-p
>
> Well, then a sequence has to be introduced in the XPath,
>
> collection('/db/punch/data')//(tei:persName, 
> tei:placeName)[@corresp=$corresp]
>
> Strangely, eXist does not handle this correctly, ignoring all but the 
> first item in the sequence ....
>
> Jens
>
>>
>>
>> -- 
>> Adam Retter
>>
>> eXist Developer
>> { United Kingdom }
>> ad...@ex... <mailto:ad...@ex...>
>> irc://irc.freenode.net/existdb <http://irc.freenode.net/existdb>
>> ------------------------------------------------------------------------------
>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>> Remotely access PCs and mobile devices and provide instant support
>> Improve your efficiency, and focus on delivering more value-add services
>> Discover what IT Professionals Know. Rescue delivers
>> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
>> eXist-TEIXML mailing list
>> eXi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
>
>
> _______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml

Re: [eXist-TEIXML] A branching problem

[Peter W. <pet...@ke...>]

Thank you all.  I'm going to have to think about the answers.  But what 
I am finding is that where there is no $name-predicate it will not move 
on and take up a $place-predicate that is present.  I'm only looking for 
one or the other, not both.  Peter

On 21/12/2012 12:22, Wolfgang Meier wrote:
> Hi Peter,
>> I'm trying to find a method of getting a program to branch depending on the parameter provided. After determining the selection, the following processes are identical. The following code will only provide data for $name-predicate. If I change the order, it works for a $place-predicate only.
> Isn't this what you would expect from an if statement? If $name-predicate is set, it will select the first clause and stop. You will always only get either the placeName or the persName elements returned.
>
> Please excuse if I'm misunderstanding what you are trying to do. I assume you want to search both, persName and placeName, if both parameters are set? In this case you need to combine the two selections, e.g. with a union:
>
> let $name-predicate := local:get-name-predicate(request:get-parameter('people',''))
> let $place-predicate := local:get-name-predicate(request:get-parameter('place',''))
>
>
> return
>      (if($name-predicate) then (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate]) else ())
> union
>      (if($place-predicate) then (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate]) else ())
>
> Wolfgang
>

Re: [eXist-TEIXML] A branching problem

[Jens Ø. P. <oe...@gm...>]

On Dec 21, 2012, at 11:54 AM, Peter Watson <pet...@ke...> wrote:

> Hi 
> 
> I'm trying to find a method of getting a program to branch depending on the parameter provided.  After determining the selection, the following processes are identical.  The following code will only provide data for  $name-predicate.  If I change the order, it works for a $place-predicate only. I presume from this that this particular use of a conditional clause is invalid.  What alternatives do I have for achieving the desired result? (Apart from separate XQueries which is where I am at the moment. I've also tried putting alternatives into the predicate which according to Presley p 46

I assume you mean Pris[cilla Walms]ley!

> does seem to be legitimate - see below, but that does not work either).

What is it you have found not to work?

Jens

> 
> Best wishes
> 
> Peter
> 
> let $name-predicate := local:get-name-predicate(request:get-parameter('people',''))
> let $place-predicate := local:get-name-predicate(request:get-parameter('place',''))
> let $selected-documents :=
> (if($name-predicate) then (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate]) 
> else if($place-predicate) then  (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate])
>   else ())
> return $selected-documents  
> 
> Alternative predicate:
> let $selected-documents := collection('/db/punch/data')//(if($name-predicate) then tei:persName[@corresp=$name-predicate] 
> else tei:placeName[@corresp=$place-predicate] )
> return $selected-documents  
> 
> Version: 2.0-tech-preview
> SVN Revision: 15855
> Build: 20120228
> 
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml

Re: [eXist-TEIXML] A branching problem

[Jens Ø. P. <oe...@gm...>]

On Dec 21, 2012, at 2:20 PM, Adam Retter <ad...@ex...> wrote:

> 
> Please excuse if I'm misunderstanding what you are trying to do. I assume you want to search both, persName and placeName, if both parameters are set? In this case you need to combine the two selections, e.g. with a union:
> 
> let $name-predicate := local:get-name-predicate(request:get-parameter('people',''))
> let $place-predicate := local:get-name-predicate(request:get-parameter('place',''))
> 
> 
> return
>     (if($name-predicate) then (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate]) else ())
> union
>     (if($place-predicate) then (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate]) else ())
> 
> Wolfgang
> 
> 
> I think you could also avoid the union altogether. Wolfgang, would this be more efficient? 
> 
> let $corresp := (local:get-name-predicate(request:get-parameter('people','')),  local:get-name-predicate(request:get-parameter('place,''))) return
>    collection('/db/punch/data')//tei:persName[@corresp=$corresp])
> 
> Thats the beauty of a language without null's where an empty sequence is a perfectly valid 'thing' to operate upon :-p

Well, then a sequence has to be introduced in the XPath,

	collection('/db/punch/data')//(tei:persName, tei:placeName)[@corresp=$corresp]

Strangely, eXist does not handle this correctly, ignoring all but the first item in the sequence ….

Jens

> 
> 
> -- 
> Adam Retter
> 
> eXist Developer
> { United Kingdom }
> ad...@ex...
> irc://irc.freenode.net/existdb
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d_______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml

Re: [eXist-TEIXML] A branching problem

[Adam R. <ad...@ex...>]

>
>
> Please excuse if I'm misunderstanding what you are trying to do. I assume
> you want to search both, persName and placeName, if both parameters are
> set? In this case you need to combine the two selections, e.g. with a union:
>
> let $name-predicate :=
> local:get-name-predicate(request:get-parameter('people',''))
> let $place-predicate :=
> local:get-name-predicate(request:get-parameter('place',''))
>
>
> return
>     (if($name-predicate) then
> (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate]) else
> ())
> union
>     (if($place-predicate) then
> (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate])
> else ())
>
> Wolfgang
>


I think you could also avoid the union altogether. Wolfgang, would this be
more efficient?

let $corresp := (local:get-name-predicate(request:get-parameter('people'
,'')),  local:get-name-predicate(request:get-parameter('place,''))) return
   collection('/db/punch/data')//tei:persName[@corresp=$corresp])

Thats the beauty of a language without null's where an empty sequence is a
perfectly valid 'thing' to operate upon :-p


-- 
Adam Retter

eXist Developer
{ United Kingdom }
ad...@ex...
irc://irc.freenode.net/existdb

Re: [eXist-TEIXML] A branching problem

[Wolfgang M. <wol...@ex...>]

Hi Peter,
> I'm trying to find a method of getting a program to branch depending on the parameter provided. After determining the selection, the following processes are identical. The following code will only provide data for $name-predicate. If I change the order, it works for a $place-predicate only.

Isn't this what you would expect from an if statement? If $name-predicate is set, it will select the first clause and stop. You will always only get either the placeName or the persName elements returned.

Please excuse if I'm misunderstanding what you are trying to do. I assume you want to search both, persName and placeName, if both parameters are set? In this case you need to combine the two selections, e.g. with a union:

let $name-predicate := local:get-name-predicate(request:get-parameter('people',''))
let $place-predicate := local:get-name-predicate(request:get-parameter('place',''))


return
    (if($name-predicate) then (collection('/db/punch/data')//tei:persName[@corresp=$name-predicate]) else ())
union
    (if($place-predicate) then (collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate]) else ())

Wolfgang

[eXist-TEIXML] A branching problem

[Peter W. <pet...@ke...>]

Hi

I'm trying to find a method of getting a program to branch depending on 
the parameter provided.  After determining the selection, the following 
processes are identical.  The following code will only provide data for  
$name-predicate.  If I change the order, it works for a $place-predicate 
only. I presume from this that this particular use of a conditional 
clause is invalid.  What alternatives do I have for achieving the 
desired result? (Apart from separate XQueries which is where I am at the 
moment. I've also tried putting alternatives into the predicate which 
according to Presley p 46 does seem to be legitimate - see below, but 
that does not work either).

Best wishes

Peter

let $name-predicate := 
local:get-name-predicate(request:get-parameter('people',''))
let $place-predicate := 
local:get-name-predicate(request:get-parameter('place',''))
let $selected-documents :=
(if($name-predicate) then 
(collection('/db/punch/data')//tei:persName[@corresp=$name-predicate])
else if($place-predicate) then 
(collection('/db/punch/data')//tei:placeName[@corresp=$place-predicate])
   else ())
return $selected-documents

Alternative predicate:
let $selected-documents := 
collection('/db/punch/data')//(if($name-predicate) then 
tei:persName[@corresp=$name-predicate]
else tei:placeName[@corresp=$place-predicate] )
return $selected-documents

  * Version: 2.0-tech-preview
  * SVN Revision: 15855
  * Build: 20120228

Re: [eXist-TEIXML] Large tables

[Wolfgang M. <wol...@ex...>]

> Thanks Joe, most helpful and interesting. Removing the brackets saved 
> nearly seven seconds in my old version of eXist down to 13.04 seconds. 
> There was a marginal improvement in 2.0RC to 22.56 seconds (22.76). 

This is interesting. The RC should certainly not be slower than the old tech preview. I'd love to test this myself, so I can fix this performance loss and maybe further help with optimizing the query. Would it be possible to have a runnable copy off list?

Wolfgang

Re: [eXist-TEIXML] Large tables

[Peter W. <pet...@ke...>]

Thanks Joe, most helpful and interesting.  Removing the brackets saved 
nearly seven seconds in my old version of eXist down to 13.04 seconds.  
There was a marginal improvement in 2.0RC to 22.56 seconds (22.76).  In 
this instance changing the order of the predicates marginally increased 
the time in both.  Separating the predicates brought the times down by 
about a second in both cases to 12.15 seconds (my old version) and 19.81 
second 2.0RC respectively.

With regard to saving my intermediate data so that it could be indexed, 
I've tried to hold off from that, as well as from a semi-permanent index 
of charter-person pairs as they seem to be the wrong direction to be 
going in the longer term.

Best wishes Peter

On 17/12/2012 13:52, Joe Wicentowski wrote:
> Hi Peter,
>
> The only opportunity for improving the performance of this code is in the line:
>
>    let $stats := $edges[(@source = $j/@id and @target = $i/@id)]
>
> You could see if Wolfgang's optimization tip, "Always process the most
> selective filter/expression first"
> (http://exist-db.org/exist/tuning.xml#d7751e583), could help.  Think
> about which of the two expressions in the predicate is more selective,
> and then place that first in the chain of predicates; it may or may
> not help to have two distinct predicates rather than the single
> "and"-joined predicate.  Also, the parentheses here are unnecessary,
> but I doubt if removing them will help with performance.
>
> Beyond this, I am intrigued by your reference to "intermediate
> documents [that] are not indexed".  That sounds like a potential
> bottleneck.  You might look into saving (caching) the intermediate
> documents to the database so they can benefit from indexing.  But I'd
> only resort to that if you confirm that there's no more performance to
> be squeezed out of your query.
>
> Joe
>
> On Mon, Dec 17, 2012 at 7:52 AM, Peter Watson
> <pet...@ke...> wrote:
>> Hi Wolfgang
>>
>> I've now installed 2.0RC and tested the query on that.  The time taken
>> by the latest version of eXist is about 3 seconds longer 22.76 seconds
>> as against 19.76 seconds for the earlier version of eXist I have been
>> using. Below is the code.  As far as I can see nearly all of the time is
>> taken up in the single line of code that constructs the table rows from
>> the edges/tuples. My first question has to be whether there are
>> alternative ways of tackling this that are more efficient?
>>
>> Best wishes  Peter
>>
>>
>> (: The input data consists of a vector of charters($sourcenodes) a
>> vector of the persons named in the charters($targetnodes) and the edges
>> or tuples that specify the relationship between a charter and a person.
>> These are intermediate documents and are not indexed.:)
>> declare function local:table-contents($edges, $sourcenodes,$targetnodes)
>> {
>> (:We proceed row by row ie person by person:)
>> for $i in $targetnodes
>> (:The next two lines extract the row labels from the vector of persons:)
>> let $row-label-1 := <td><a href ="{concat('namepage.xq?name=',
>> substring($i/@id/string(),2))}">{$i/@id/string()}</a></td>
>> let $row-label-2 := <td>{$i}</td>
>> (:We now extract the contents of the rows charter by charter:)
>> let $row-contents :=
>> for $j in $sourcenodes
>> (:This is the key line of code finding the edge, if it exists, that
>> matches the charter $j and person $i and placing it in its correct
>> position in the table:)
>> let $stats :=$edges[(@source = $j/@id and @target = $i/@id)]
>> (:The results are returned as html table cells :)
>> return <td align='center' >{if ($stats) then $stats/data else ('-')}</td>
>> return <tr>{$row-label-1} {$row-label-2} {$row-contents}</tr>
>> };
>>
>> On 16/12/2012 10:50, Peter Watson wrote:
>>> Thank you Wolfgang.  It was the functions panel that helped me confirm
>>> where I think the issue lies.  Anyway I'll update to 2.0RC and test the
>>> effect of that and report back.  Peter
>>>
>>> It might be helpful to look at the functions panel in the profiler. Can you identify further bottlenecks there or is the problem really in the comparisons indicated in the index usage panel? If value comparisons are the bottleneck there might be a bad optimization on the eXist side as well, which would be interesting to check.
>>>
>>> In any case I would recommend to first update to the 2.0RC (available via the sourceforge project page). A lot of issues have been fixed since the tech preview.
>>>
>>> If you can't figure out the bottleneck, I'll be happy to have a look at it if I had some sample data and queries (you can also send me something off list).
>>>
>>> Wolfgang
>>>
>>>
>>> Am Samstag, 15. Dezember 2012 um 06:42 schrieb Peter Watson:
>>>
>>>>> Hi
>>>>>
>>>>> I'm not sure where I should pose this question, but here goes. I'm
>>>>> producing cross-tabulation data from an XML TEI database using eXist 2.0
>>>>> tech preview version 15855. Using XQuery I produce data in the the form
>>>>> of .gml files for use in external programs such as nodeXL and matrix
>>>>> explorer. The data is in the form of nodes and edges/tuples for
>>>>> production of graphs. I can take this data and transform the
>>>>> edges/tuples directly into tables using XQuery and simpe html. These
>>>>> tables are effectively matrices say 25 x 100. My problem is that the
>>>>> process is quite slow computationally. Looking at the query profiler
>>>>> quite a lot of the time is spent making calls to the index for what are
>>>>> interim variables where no index is available. My question is where to
>>>>> look for information that will help me improve my code/approach or take
>>>>> any other measure that would speed up the process. I don't have a sense
>>>>> as to how large a computation this is in terms of XQuery/eXist. Does
>>>>> anyone have experience they could share?
>>>>>
>>>>> Best wishes
>>>>>
>>>>> Peter
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>>>>> Remotely access PCs and mobile devices and provide instant support
>>>>> Improve your efficiency, and focus on delivering more value-add services
>>>>> Discover what IT Professionals Know. Rescue delivers
>>>>> http://p.sf.net/sfu/logmein_12329d2d
>>>>> _______________________________________________
>>>>> eXist-TEIXML mailing list
>>>>> eXi...@li...  (mailto:eXi...@li...)
>>>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>>>> .
>>>>
>>> ------------------------------------------------------------------------------
>>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>>> Remotely access PCs and mobile devices and provide instant support
>>> Improve your efficiency, and focus on delivering more value-add services
>>> Discover what IT Professionals Know. Rescue delivers
>>> http://p.sf.net/sfu/logmein_12329d2d
>>> _______________________________________________
>>> eXist-TEIXML mailing list
>>> eXi...@li...
>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>>> .
>>>
>>
>> ------------------------------------------------------------------------------
>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>> Remotely access PCs and mobile devices and provide instant support
>> Improve your efficiency, and focus on delivering more value-add services
>> Discover what IT Professionals Know. Rescue delivers
>> http://p.sf.net/sfu/logmein_12329d2d
>> _______________________________________________
>> eXist-TEIXML mailing list
>> eXi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
> .
>

Re: [eXist-TEIXML] Large tables

[Joe W. <jo...@gm...>]

Hi Peter,

The only opportunity for improving the performance of this code is in the line:

  let $stats := $edges[(@source = $j/@id and @target = $i/@id)]

You could see if Wolfgang's optimization tip, "Always process the most
selective filter/expression first"
(http://exist-db.org/exist/tuning.xml#d7751e583), could help.  Think
about which of the two expressions in the predicate is more selective,
and then place that first in the chain of predicates; it may or may
not help to have two distinct predicates rather than the single
"and"-joined predicate.  Also, the parentheses here are unnecessary,
but I doubt if removing them will help with performance.

Beyond this, I am intrigued by your reference to "intermediate
documents [that] are not indexed".  That sounds like a potential
bottleneck.  You might look into saving (caching) the intermediate
documents to the database so they can benefit from indexing.  But I'd
only resort to that if you confirm that there's no more performance to
be squeezed out of your query.

Joe

On Mon, Dec 17, 2012 at 7:52 AM, Peter Watson
<pet...@ke...> wrote:
> Hi Wolfgang
>
> I've now installed 2.0RC and tested the query on that.  The time taken
> by the latest version of eXist is about 3 seconds longer 22.76 seconds
> as against 19.76 seconds for the earlier version of eXist I have been
> using. Below is the code.  As far as I can see nearly all of the time is
> taken up in the single line of code that constructs the table rows from
> the edges/tuples. My first question has to be whether there are
> alternative ways of tackling this that are more efficient?
>
> Best wishes  Peter
>
>
> (: The input data consists of a vector of charters($sourcenodes) a
> vector of the persons named in the charters($targetnodes) and the edges
> or tuples that specify the relationship between a charter and a person.
> These are intermediate documents and are not indexed.:)
> declare function local:table-contents($edges, $sourcenodes,$targetnodes)
> {
> (:We proceed row by row ie person by person:)
> for $i in $targetnodes
> (:The next two lines extract the row labels from the vector of persons:)
> let $row-label-1 := <td><a href ="{concat('namepage.xq?name=',
> substring($i/@id/string(),2))}">{$i/@id/string()}</a></td>
> let $row-label-2 := <td>{$i}</td>
> (:We now extract the contents of the rows charter by charter:)
> let $row-contents :=
> for $j in $sourcenodes
> (:This is the key line of code finding the edge, if it exists, that
> matches the charter $j and person $i and placing it in its correct
> position in the table:)
> let $stats :=$edges[(@source = $j/@id and @target = $i/@id)]
> (:The results are returned as html table cells :)
> return <td align='center' >{if ($stats) then $stats/data else ('-')}</td>
> return <tr>{$row-label-1} {$row-label-2} {$row-contents}</tr>
> };
>
> On 16/12/2012 10:50, Peter Watson wrote:
>> Thank you Wolfgang.  It was the functions panel that helped me confirm
>> where I think the issue lies.  Anyway I'll update to 2.0RC and test the
>> effect of that and report back.  Peter
>>
>> It might be helpful to look at the functions panel in the profiler. Can you identify further bottlenecks there or is the problem really in the comparisons indicated in the index usage panel? If value comparisons are the bottleneck there might be a bad optimization on the eXist side as well, which would be interesting to check.
>>
>> In any case I would recommend to first update to the 2.0RC (available via the sourceforge project page). A lot of issues have been fixed since the tech preview.
>>
>> If you can't figure out the bottleneck, I'll be happy to have a look at it if I had some sample data and queries (you can also send me something off list).
>>
>> Wolfgang
>>
>>
>> Am Samstag, 15. Dezember 2012 um 06:42 schrieb Peter Watson:
>>
>>>> Hi
>>>>
>>>> I'm not sure where I should pose this question, but here goes. I'm
>>>> producing cross-tabulation data from an XML TEI database using eXist 2.0
>>>> tech preview version 15855. Using XQuery I produce data in the the form
>>>> of .gml files for use in external programs such as nodeXL and matrix
>>>> explorer. The data is in the form of nodes and edges/tuples for
>>>> production of graphs. I can take this data and transform the
>>>> edges/tuples directly into tables using XQuery and simpe html. These
>>>> tables are effectively matrices say 25 x 100. My problem is that the
>>>> process is quite slow computationally. Looking at the query profiler
>>>> quite a lot of the time is spent making calls to the index for what are
>>>> interim variables where no index is available. My question is where to
>>>> look for information that will help me improve my code/approach or take
>>>> any other measure that would speed up the process. I don't have a sense
>>>> as to how large a computation this is in terms of XQuery/eXist. Does
>>>> anyone have experience they could share?
>>>>
>>>> Best wishes
>>>>
>>>> Peter
>>>>
>>>> ------------------------------------------------------------------------------
>>>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>>>> Remotely access PCs and mobile devices and provide instant support
>>>> Improve your efficiency, and focus on delivering more value-add services
>>>> Discover what IT Professionals Know. Rescue delivers
>>>> http://p.sf.net/sfu/logmein_12329d2d
>>>> _______________________________________________
>>>> eXist-TEIXML mailing list
>>>> eXi...@li...  (mailto:eXi...@li...)
>>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>>> .
>>>
>> ------------------------------------------------------------------------------
>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>> Remotely access PCs and mobile devices and provide instant support
>> Improve your efficiency, and focus on delivering more value-add services
>> Discover what IT Professionals Know. Rescue delivers
>> http://p.sf.net/sfu/logmein_12329d2d
>> _______________________________________________
>> eXist-TEIXML mailing list
>> eXi...@li...
>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>> .
>>
>
>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml

Re: [eXist-TEIXML] Large tables

[Peter W. <pet...@ke...>]

Hi Wolfgang

I've now installed 2.0RC and tested the query on that.  The time taken 
by the latest version of eXist is about 3 seconds longer 22.76 seconds 
as against 19.76 seconds for the earlier version of eXist I have been 
using. Below is the code.  As far as I can see nearly all of the time is 
taken up in the single line of code that constructs the table rows from 
the edges/tuples. My first question has to be whether there are 
alternative ways of tackling this that are more efficient?

Best wishes  Peter


(: The input data consists of a vector of charters($sourcenodes) a 
vector of the persons named in the charters($targetnodes) and the edges 
or tuples that specify the relationship between a charter and a person.  
These are intermediate documents and are not indexed.:)
declare function local:table-contents($edges, $sourcenodes,$targetnodes)
{
(:We proceed row by row ie person by person:)
for $i in $targetnodes
(:The next two lines extract the row labels from the vector of persons:)
let $row-label-1 := <td><a href ="{concat('namepage.xq?name=', 
substring($i/@id/string(),2))}">{$i/@id/string()}</a></td>
let $row-label-2 := <td>{$i}</td>
(:We now extract the contents of the rows charter by charter:)
let $row-contents :=
for $j in $sourcenodes
(:This is the key line of code finding the edge, if it exists, that 
matches the charter $j and person $i and placing it in its correct 
position in the table:)
let $stats :=$edges[(@source = $j/@id and @target = $i/@id)]
(:The results are returned as html table cells :)
return <td align='center' >{if ($stats) then $stats/data else ('-')}</td>
return <tr>{$row-label-1} {$row-label-2} {$row-contents}</tr>
};

On 16/12/2012 10:50, Peter Watson wrote:
> Thank you Wolfgang.  It was the functions panel that helped me confirm
> where I think the issue lies.  Anyway I'll update to 2.0RC and test the
> effect of that and report back.  Peter
>
> It might be helpful to look at the functions panel in the profiler. Can you identify further bottlenecks there or is the problem really in the comparisons indicated in the index usage panel? If value comparisons are the bottleneck there might be a bad optimization on the eXist side as well, which would be interesting to check.
>
> In any case I would recommend to first update to the 2.0RC (available via the sourceforge project page). A lot of issues have been fixed since the tech preview.
>
> If you can't figure out the bottleneck, I'll be happy to have a look at it if I had some sample data and queries (you can also send me something off list).
>
> Wolfgang
>
>
> Am Samstag, 15. Dezember 2012 um 06:42 schrieb Peter Watson:
>
>>> Hi
>>>
>>> I'm not sure where I should pose this question, but here goes. I'm
>>> producing cross-tabulation data from an XML TEI database using eXist 2.0
>>> tech preview version 15855. Using XQuery I produce data in the the form
>>> of .gml files for use in external programs such as nodeXL and matrix
>>> explorer. The data is in the form of nodes and edges/tuples for
>>> production of graphs. I can take this data and transform the
>>> edges/tuples directly into tables using XQuery and simpe html. These
>>> tables are effectively matrices say 25 x 100. My problem is that the
>>> process is quite slow computationally. Looking at the query profiler
>>> quite a lot of the time is spent making calls to the index for what are
>>> interim variables where no index is available. My question is where to
>>> look for information that will help me improve my code/approach or take
>>> any other measure that would speed up the process. I don't have a sense
>>> as to how large a computation this is in terms of XQuery/eXist. Does
>>> anyone have experience they could share?
>>>
>>> Best wishes
>>>
>>> Peter
>>>
>>> ------------------------------------------------------------------------------
>>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>>> Remotely access PCs and mobile devices and provide instant support
>>> Improve your efficiency, and focus on delivering more value-add services
>>> Discover what IT Professionals Know. Rescue delivers
>>> http://p.sf.net/sfu/logmein_12329d2d
>>> _______________________________________________
>>> eXist-TEIXML mailing list
>>> eXi...@li...  (mailto:eXi...@li...)
>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>> .
>>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml
> .
>