## Re: [saxon] How to measure precisely the performance of some F & Ofunctions as implemented in Saxon 9?

 Re: [saxon] How to measure precisely the performance of some F & Ofunctions as implemented in Saxon 9? From: Dimitre Novatchev - 2008-07-01 01:01:33 ```Thank you for the detailed reply. Somehow I managed to produce a transformation1, which evaluates only: count(Seq) Seems by using some tricks I was able to confuse Saxon's optimizer and the transformation took the expected comparatively long time. I already have a transformation2, which contains: some-seq-returning-function() count(Seq) Then I subtract the time it takes for transformation1 from the time it takes for transformation2. Thus on a slightly faster computer the original xslt application (transformation2) performed in 49947ms. Then the one below (transformation1) took 10916ms. Can we conclude that the processing that is present in transformation2 and not in transformation1 (mostly related to the insert-before() function) is resposible for the difference between the two times: 39031ms ? While it seems logical, do you think such approach to timing is correct? The stripped down transformation (transformation1) is: -- Cheers, Dimitre Novatchev --------------------------------------- Truly great madness cannot be achieved without significant intelligence. --------------------------------------- To invent, you need a good imagination and a pile of junk ------------------------------------- Never fight an inanimate object ------------------------------------- You've achieved success in your field when you don't know whether what you're doing is work or play On Mon, Jun 30, 2008 at 12:40 AM, Michael Kay wrote: > As you observe, operations like insert-before() are pipelined, which means > they are interleaved with the operations that produce their input and > consume their output, and this makes it very difficult to isolate them from > a performance perspective. In fact, the only way you can measure the > performace of insert-before() is to break the pipeline and force both the > input and the output to be held as sequence extents in memory; but if you do > that, then you are not measuring the function itself, you are measuring the > cost of allocating the memory for these intermediate results. I think this > really is a case where you can't measure it without changing it. > > And the fact is, that when things are nicely pipelined the cost of > insert-before() is microscopically small. The real costs of your application > is not the sum of the costs of the functions it calls, but the cost of > organizing the data for those functions to operate on and passing the data > from one to another. > > I think a better approach is to compare the costs of two different ways of > achieving some higher-level task. > > Michael Kay > http://www.saxonica.com/ > >> -----Original Message----- >> From: saxon-help-bounces@... >> [mailto:saxon-help-bounces@...] On Behalf >> Of Dimitre Novatchev >> Sent: 30 June 2008 05:01 >> To: Mailing list for the SAXON XSLT and XQuery processor >> Subject: [saxon] How to measure precisely the performance of >> some F & Ofunctions as implemented in Saxon 9? >> >> Hi, >> >> I am trying to measure the performance of some F & O >> functions as implemented in Saxon 9. >> >> There are a number of challenges here: >> 1. Saxon optimizes too cleverly and we have to be sure a >> function was evaluated N times and not only once. >> >> 2. Lazy evaluation may cause a function to be evaluated >> only as far as it is possible to produce the required result. >> For example, if a sequence is to be produced and then its >> first item displayed, the evaluation of the function may stop >> immediately after it produces the first item of the sequence. >> >> 3. Appending a sequence to another may have significantly >> different performance than prepending it. >> >> Below is a complete example. My question is: >> >> Am I doing this in a right (wrong) way and is there a more >> precise way to cause the complete evaluation of a specific function? >> >> In particular, I would like to have avoided the need of >> >> >> >> in >> >> > "insert-before(\$vSeq3, 1, \$vSeq4)"/> >> >> >> >> which at this time seems to be the only way to make sure that >> the application of the insert-before() function has been >> completely evaluated. >> >> The clearly takes some time itself, and >> I would like to know how much this time is. >> >> Any recommendation/advice would be highly appreciated. >> >> >> >> >> >> Saxon 9.0.0.2J from Saxonica >> Java version 1.6.0_05 >> Stylesheet compilation time: 469 milliseconds Processing >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> Building tree for >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 >> milliseconds Tree size: 3 nodes, 0 characters, 0 attributes >> Execution time: 93859 milliseconds Memory used: 59029320 >> NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs >> ------------------------------- >> Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> Building tree for >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 >> milliseconds Tree size: 3 nodes, 0 characters, 0 attributes >> Execution time: 93437 milliseconds Memory used: 52309520 >> NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs >> ------------------------------- >> Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> Building tree for >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 >> milliseconds Tree size: 3 nodes, 0 characters, 0 attributes >> Execution time: 90610 milliseconds Memory used: 52560448 >> NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs >> ------------------------------- >> *** Average execution time over 3 runs: 92635ms >> >> >> >> > xmlns:xsl="http://www.w3.org/1999/XSL/Transform"; >> xmlns:xs="http://www.w3.org/2001/XMLSchema"; >> xmlns:f="http://fxsl.sf.net/"; >> exclude-result-prefixes="xsl xs f" >> > >> >> >> > select="1 to 10000"/> >> >> > select="for \$i in \$vSeq1 >> return \$i+5" >> /> >> >> >> > select="if(. mod 2 eq 0) >> then \$vSeq1 >> else \$vSeq2 >> "/> >> > select="if(. mod 2 eq 1) >> then \$vSeq1 >> else \$vSeq2 >> "/> >> > "insert-before(\$vSeq3, 1, \$vSeq4)"/> >> >> >> >> >> >> >> >> -- >> Cheers, >> Dimitre Novatchev >> --------------------------------------- >> Truly great madness cannot be achieved without significant >> intelligence. >> --------------------------------------- >> To invent, you need a good imagination and a pile of junk >> ------------------------------------- >> Never fight an inanimate object >> ------------------------------------- >> You've achieved success in your field when you don't know >> whether what you're doing is work or play >> >> -------------------------------------------------------------- >> ----------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for just about >> anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> saxon-help mailing list archived at >> http://saxon.markmail.org/ saxon-help@... >> https://lists.sourceforge.net/lists/listinfo/saxon-help > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > saxon-help mailing list archived at http://saxon.markmail.org/ > saxon-help@... > https://lists.sourceforge.net/lists/listinfo/saxon-help > ```

 [saxon] How to measure precisely the performance of some F & O functions as implemented in Saxon 9? From: Dimitre Novatchev - 2008-06-30 04:01:11 ```Hi, I am trying to measure the performance of some F & O functions as implemented in Saxon 9. There are a number of challenges here: 1. Saxon optimizes too cleverly and we have to be sure a function was evaluated N times and not only once. 2. Lazy evaluation may cause a function to be evaluated only as far as it is possible to produce the required result. For example, if a sequence is to be produced and then its first item displayed, the evaluation of the function may stop immediately after it produces the first item of the sequence. 3. Appending a sequence to another may have significantly different performance than prepending it. Below is a complete example. My question is: Am I doing this in a right (wrong) way and is there a more precise way to cause the complete evaluation of a specific function? In particular, I would like to have avoided the need of in which at this time seems to be the only way to make sure that the application of the insert-before() function has been completely evaluated. The clearly takes some time itself, and I would like to know how much this time is. Any recommendation/advice would be highly appreciated. Saxon 9.0.0.2J from Saxonica Java version 1.6.0_05 Stylesheet compilation time: 469 milliseconds Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml Building tree for file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 milliseconds Tree size: 3 nodes, 0 characters, 0 attributes Execution time: 93859 milliseconds Memory used: 59029320 NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs ------------------------------- Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml Building tree for file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 milliseconds Tree size: 3 nodes, 0 characters, 0 attributes Execution time: 93437 milliseconds Memory used: 52309520 NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs ------------------------------- Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml Building tree for file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 milliseconds Tree size: 3 nodes, 0 characters, 0 attributes Execution time: 90610 milliseconds Memory used: 52560448 NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs ------------------------------- *** Average execution time over 3 runs: 92635ms -- Cheers, Dimitre Novatchev --------------------------------------- Truly great madness cannot be achieved without significant intelligence. --------------------------------------- To invent, you need a good imagination and a pile of junk ------------------------------------- Never fight an inanimate object ------------------------------------- You've achieved success in your field when you don't know whether what you're doing is work or play ```
 Re: [saxon] How to measure precisely the performance of some F & Ofunctions as implemented in Saxon 9? From: Michael Kay - 2008-06-30 07:40:10 ```As you observe, operations like insert-before() are pipelined, which means they are interleaved with the operations that produce their input and consume their output, and this makes it very difficult to isolate them from a performance perspective. In fact, the only way you can measure the performace of insert-before() is to break the pipeline and force both the input and the output to be held as sequence extents in memory; but if you do that, then you are not measuring the function itself, you are measuring the cost of allocating the memory for these intermediate results. I think this really is a case where you can't measure it without changing it. And the fact is, that when things are nicely pipelined the cost of insert-before() is microscopically small. The real costs of your application is not the sum of the costs of the functions it calls, but the cost of organizing the data for those functions to operate on and passing the data from one to another. I think a better approach is to compare the costs of two different ways of achieving some higher-level task. Michael Kay http://www.saxonica.com/ > -----Original Message----- > From: saxon-help-bounces@... > [mailto:saxon-help-bounces@...] On Behalf > Of Dimitre Novatchev > Sent: 30 June 2008 05:01 > To: Mailing list for the SAXON XSLT and XQuery processor > Subject: [saxon] How to measure precisely the performance of > some F & Ofunctions as implemented in Saxon 9? > > Hi, > > I am trying to measure the performance of some F & O > functions as implemented in Saxon 9. > > There are a number of challenges here: > 1. Saxon optimizes too cleverly and we have to be sure a > function was evaluated N times and not only once. > > 2. Lazy evaluation may cause a function to be evaluated > only as far as it is possible to produce the required result. > For example, if a sequence is to be produced and then its > first item displayed, the evaluation of the function may stop > immediately after it produces the first item of the sequence. > > 3. Appending a sequence to another may have significantly > different performance than prepending it. > > Below is a complete example. My question is: > > Am I doing this in a right (wrong) way and is there a more > precise way to cause the complete evaluation of a specific function? > > In particular, I would like to have avoided the need of > > > > in > > "insert-before(\$vSeq3, 1, \$vSeq4)"/> > > > > which at this time seems to be the only way to make sure that > the application of the insert-before() function has been > completely evaluated. > > The clearly takes some time itself, and > I would like to know how much this time is. > > Any recommendation/advice would be highly appreciated. > > > > > > Saxon 9.0.0.2J from Saxonica > Java version 1.6.0_05 > Stylesheet compilation time: 469 milliseconds Processing > file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml > Building tree for > file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml > using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 > milliseconds Tree size: 3 nodes, 0 characters, 0 attributes > Execution time: 93859 milliseconds Memory used: 59029320 > NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs > ------------------------------- > Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml > Building tree for > file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml > using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 > milliseconds Tree size: 3 nodes, 0 characters, 0 attributes > Execution time: 93437 milliseconds Memory used: 52309520 > NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs > ------------------------------- > Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml > Building tree for > file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml > using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 > milliseconds Tree size: 3 nodes, 0 characters, 0 attributes > Execution time: 90610 milliseconds Memory used: 52560448 > NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs > ------------------------------- > *** Average execution time over 3 runs: 92635ms > > > > xmlns:xsl="http://www.w3.org/1999/XSL/Transform"; > xmlns:xs="http://www.w3.org/2001/XMLSchema"; > xmlns:f="http://fxsl.sf.net/"; > exclude-result-prefixes="xsl xs f" > > > > > select="1 to 10000"/> > > select="for \$i in \$vSeq1 > return \$i+5" > /> > > > select="if(. mod 2 eq 0) > then \$vSeq1 > else \$vSeq2 > "/> > select="if(. mod 2 eq 1) > then \$vSeq1 > else \$vSeq2 > "/> > "insert-before(\$vSeq3, 1, \$vSeq4)"/> > > > > > > > > -- > Cheers, > Dimitre Novatchev > --------------------------------------- > Truly great madness cannot be achieved without significant > intelligence. > --------------------------------------- > To invent, you need a good imagination and a pile of junk > ------------------------------------- > Never fight an inanimate object > ------------------------------------- > You've achieved success in your field when you don't know > whether what you're doing is work or play > > -------------------------------------------------------------- > ----------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for just about > anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > saxon-help mailing list archived at > http://saxon.markmail.org/ saxon-help@... > https://lists.sourceforge.net/lists/listinfo/saxon-help ```
 Re: [saxon] How to measure precisely the performance of some F & Ofunctions as implemented in Saxon 9? From: Dimitre Novatchev - 2008-07-01 01:01:33 ```Thank you for the detailed reply. Somehow I managed to produce a transformation1, which evaluates only: count(Seq) Seems by using some tricks I was able to confuse Saxon's optimizer and the transformation took the expected comparatively long time. I already have a transformation2, which contains: some-seq-returning-function() count(Seq) Then I subtract the time it takes for transformation1 from the time it takes for transformation2. Thus on a slightly faster computer the original xslt application (transformation2) performed in 49947ms. Then the one below (transformation1) took 10916ms. Can we conclude that the processing that is present in transformation2 and not in transformation1 (mostly related to the insert-before() function) is resposible for the difference between the two times: 39031ms ? While it seems logical, do you think such approach to timing is correct? The stripped down transformation (transformation1) is: -- Cheers, Dimitre Novatchev --------------------------------------- Truly great madness cannot be achieved without significant intelligence. --------------------------------------- To invent, you need a good imagination and a pile of junk ------------------------------------- Never fight an inanimate object ------------------------------------- You've achieved success in your field when you don't know whether what you're doing is work or play On Mon, Jun 30, 2008 at 12:40 AM, Michael Kay wrote: > As you observe, operations like insert-before() are pipelined, which means > they are interleaved with the operations that produce their input and > consume their output, and this makes it very difficult to isolate them from > a performance perspective. In fact, the only way you can measure the > performace of insert-before() is to break the pipeline and force both the > input and the output to be held as sequence extents in memory; but if you do > that, then you are not measuring the function itself, you are measuring the > cost of allocating the memory for these intermediate results. I think this > really is a case where you can't measure it without changing it. > > And the fact is, that when things are nicely pipelined the cost of > insert-before() is microscopically small. The real costs of your application > is not the sum of the costs of the functions it calls, but the cost of > organizing the data for those functions to operate on and passing the data > from one to another. > > I think a better approach is to compare the costs of two different ways of > achieving some higher-level task. > > Michael Kay > http://www.saxonica.com/ > >> -----Original Message----- >> From: saxon-help-bounces@... >> [mailto:saxon-help-bounces@...] On Behalf >> Of Dimitre Novatchev >> Sent: 30 June 2008 05:01 >> To: Mailing list for the SAXON XSLT and XQuery processor >> Subject: [saxon] How to measure precisely the performance of >> some F & Ofunctions as implemented in Saxon 9? >> >> Hi, >> >> I am trying to measure the performance of some F & O >> functions as implemented in Saxon 9. >> >> There are a number of challenges here: >> 1. Saxon optimizes too cleverly and we have to be sure a >> function was evaluated N times and not only once. >> >> 2. Lazy evaluation may cause a function to be evaluated >> only as far as it is possible to produce the required result. >> For example, if a sequence is to be produced and then its >> first item displayed, the evaluation of the function may stop >> immediately after it produces the first item of the sequence. >> >> 3. Appending a sequence to another may have significantly >> different performance than prepending it. >> >> Below is a complete example. My question is: >> >> Am I doing this in a right (wrong) way and is there a more >> precise way to cause the complete evaluation of a specific function? >> >> In particular, I would like to have avoided the need of >> >> >> >> in >> >> > "insert-before(\$vSeq3, 1, \$vSeq4)"/> >> >> >> >> which at this time seems to be the only way to make sure that >> the application of the insert-before() function has been >> completely evaluated. >> >> The clearly takes some time itself, and >> I would like to know how much this time is. >> >> Any recommendation/advice would be highly appreciated. >> >> >> >> >> >> Saxon 9.0.0.2J from Saxonica >> Java version 1.6.0_05 >> Stylesheet compilation time: 469 milliseconds Processing >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> Building tree for >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 >> milliseconds Tree size: 3 nodes, 0 characters, 0 attributes >> Execution time: 93859 milliseconds Memory used: 59029320 >> NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs >> ------------------------------- >> Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> Building tree for >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 >> milliseconds Tree size: 3 nodes, 0 characters, 0 attributes >> Execution time: 93437 milliseconds Memory used: 52309520 >> NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs >> ------------------------------- >> Processing file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> Building tree for >> file:/C:/Program%20Files/Java/jre1.6.0_05/bin/marrowtr.xml >> using class net.sf.saxon.tinytree.TinyBuilder Tree built in 0 >> milliseconds Tree size: 3 nodes, 0 characters, 0 attributes >> Execution time: 90610 milliseconds Memory used: 52560448 >> NamePool contents: 11 entries in 11 chains. 7 prefixes, 7 URIs >> ------------------------------- >> *** Average execution time over 3 runs: 92635ms >> >> >> >> > xmlns:xsl="http://www.w3.org/1999/XSL/Transform"; >> xmlns:xs="http://www.w3.org/2001/XMLSchema"; >> xmlns:f="http://fxsl.sf.net/"; >> exclude-result-prefixes="xsl xs f" >> > >> >> >> > select="1 to 10000"/> >> >> > select="for \$i in \$vSeq1 >> return \$i+5" >> /> >> >> >> > select="if(. mod 2 eq 0) >> then \$vSeq1 >> else \$vSeq2 >> "/> >> > select="if(. mod 2 eq 1) >> then \$vSeq1 >> else \$vSeq2 >> "/> >> > "insert-before(\$vSeq3, 1, \$vSeq4)"/> >> >> >> >> >> >> >> >> -- >> Cheers, >> Dimitre Novatchev >> --------------------------------------- >> Truly great madness cannot be achieved without significant >> intelligence. >> --------------------------------------- >> To invent, you need a good imagination and a pile of junk >> ------------------------------------- >> Never fight an inanimate object >> ------------------------------------- >> You've achieved success in your field when you don't know >> whether what you're doing is work or play >> >> -------------------------------------------------------------- >> ----------- >> Check out the new SourceForge.net Marketplace. >> It's the best place to buy or sell services for just about >> anything Open Source. >> http://sourceforge.net/services/buy/index.php >> _______________________________________________ >> saxon-help mailing list archived at >> http://saxon.markmail.org/ saxon-help@... >> https://lists.sourceforge.net/lists/listinfo/saxon-help > > > ------------------------------------------------------------------------- > Check out the new SourceForge.net Marketplace. > It's the best place to buy or sell services for > just about anything Open Source. > http://sourceforge.net/services/buy/index.php > _______________________________________________ > saxon-help mailing list archived at http://saxon.markmail.org/ > saxon-help@... > https://lists.sourceforge.net/lists/listinfo/saxon-help > ```
 Re: [saxon] How to measure precisely the performance of some F &Ofunctions as implemented in Saxon 9? From: Michael Kay - 2008-07-01 15:32:28 ```> Then I subtract the time it takes for transformation1 from > the time it takes for transformation2. Subtractive measurement is often a useful technique. > > Thus on a slightly faster computer the original xslt application > (transformation2) performed in 49947ms. > > Then the one below (transformation1) took 10916ms. Can we > conclude that the processing that is present in > transformation2 and not in > transformation1 (mostly related to the insert-before() > function) is resposible for the difference between the two > times: 39031ms ? The other possibility is that the main difference between the two cases is that a different optimization strategy has been used in the two cases. It's very hard to isolate such factors. Studying the -explain output can help. Michael Kay http://www.saxonica.com/ ```