Re: [eXist-TEIXML] Large tables

[Peter W. <pet...@ke...>]

Hi Wolfgang

I've now installed 2.0RC and tested the query on that.  The time taken 
by the latest version of eXist is about 3 seconds longer 22.76 seconds 
as against 19.76 seconds for the earlier version of eXist I have been 
using. Below is the code.  As far as I can see nearly all of the time is 
taken up in the single line of code that constructs the table rows from 
the edges/tuples. My first question has to be whether there are 
alternative ways of tackling this that are more efficient?

Best wishes  Peter


(: The input data consists of a vector of charters($sourcenodes) a 
vector of the persons named in the charters($targetnodes) and the edges 
or tuples that specify the relationship between a charter and a person.  
These are intermediate documents and are not indexed.:)
declare function local:table-contents($edges, $sourcenodes,$targetnodes)
{
(:We proceed row by row ie person by person:)
for $i in $targetnodes
(:The next two lines extract the row labels from the vector of persons:)
let $row-label-1 := <td><a href ="{concat('namepage.xq?name=', 
substring($i/@id/string(),2))}">{$i/@id/string()}</a></td>
let $row-label-2 := <td>{$i}</td>
(:We now extract the contents of the rows charter by charter:)
let $row-contents :=
for $j in $sourcenodes
(:This is the key line of code finding the edge, if it exists, that 
matches the charter $j and person $i and placing it in its correct 
position in the table:)
let $stats :=$edges[(@source = $j/@id and @target = $i/@id)]
(:The results are returned as html table cells :)
return <td align='center' >{if ($stats) then $stats/data else ('-')}</td>
return <tr>{$row-label-1} {$row-label-2} {$row-contents}</tr>
};

On 16/12/2012 10:50, Peter Watson wrote:
> Thank you Wolfgang.  It was the functions panel that helped me confirm
> where I think the issue lies.  Anyway I'll update to 2.0RC and test the
> effect of that and report back.  Peter
>
> It might be helpful to look at the functions panel in the profiler. Can you identify further bottlenecks there or is the problem really in the comparisons indicated in the index usage panel? If value comparisons are the bottleneck there might be a bad optimization on the eXist side as well, which would be interesting to check.
>
> In any case I would recommend to first update to the 2.0RC (available via the sourceforge project page). A lot of issues have been fixed since the tech preview.
>
> If you can't figure out the bottleneck, I'll be happy to have a look at it if I had some sample data and queries (you can also send me something off list).
>
> Wolfgang
>
>
> Am Samstag, 15. Dezember 2012 um 06:42 schrieb Peter Watson:
>
>>> Hi
>>>
>>> I'm not sure where I should pose this question, but here goes. I'm
>>> producing cross-tabulation data from an XML TEI database using eXist 2.0
>>> tech preview version 15855. Using XQuery I produce data in the the form
>>> of .gml files for use in external programs such as nodeXL and matrix
>>> explorer. The data is in the form of nodes and edges/tuples for
>>> production of graphs. I can take this data and transform the
>>> edges/tuples directly into tables using XQuery and simpe html. These
>>> tables are effectively matrices say 25 x 100. My problem is that the
>>> process is quite slow computationally. Looking at the query profiler
>>> quite a lot of the time is spent making calls to the index for what are
>>> interim variables where no index is available. My question is where to
>>> look for information that will help me improve my code/approach or take
>>> any other measure that would speed up the process. I don't have a sense
>>> as to how large a computation this is in terms of XQuery/eXist. Does
>>> anyone have experience they could share?
>>>
>>> Best wishes
>>>
>>> Peter
>>>
>>> ------------------------------------------------------------------------------
>>> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
>>> Remotely access PCs and mobile devices and provide instant support
>>> Improve your efficiency, and focus on delivering more value-add services
>>> Discover what IT Professionals Know. Rescue delivers
>>> http://p.sf.net/sfu/logmein_12329d2d
>>> _______________________________________________
>>> eXist-TEIXML mailing list
>>> eXi...@li...  (mailto:eXi...@li...)
>>> https://lists.sourceforge.net/lists/listinfo/exist-teixml
>> .
>>
> ------------------------------------------------------------------------------
> LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
> Remotely access PCs and mobile devices and provide instant support
> Improve your efficiency, and focus on delivering more value-add services
> Discover what IT Professionals Know. Rescue delivers
> http://p.sf.net/sfu/logmein_12329d2d
> _______________________________________________
> eXist-TEIXML mailing list
> eXi...@li...
> https://lists.sourceforge.net/lists/listinfo/exist-teixml
> .
>