Thanks for the advice. I now have streaming queries working as I want them.

 

However I found that the extra FLWOR does make a difference. These queries are streaming:

 

for $x in saxon:stream(doc('uri')/*/*) return string($x)

 

for $x in saxon:stream(doc('uri')/*/*) return $x

 

for $x in saxon:stream(doc('uri')/*/*) return $x/*

 

but these are not:

 

saxon:stream(doc('uri')/*/*)/string()

 

saxon:stream(doc('uri')/*/*)

 

saxon:stream(doc('uri')/*/*)/*

 

Where those three run out of memory in roughly 10 seconds, the following takes 160 seconds

before running out of memory, too:

 

for $x in saxon:stream(doc('uri')/*/*)/* return $x

 

My test code is shown below. Tested on Saxon-EE 9.5.1.4J.

 

Best regards

Gunther

 

 

       @Test(dataProvider="streamingQueries")

       public void testStreamingSaxon(String query, long limit) throws Exception {

              XQueryExecutable xqueryExecutable = xqueryCompiler.compile(query);

              XQueryEvaluator xqueryEvaluator = xqueryExecutable.load();

              xqueryEvaluator.setURIResolver(new StandardURIResolver() {

                     private static final long serialVersionUID = 7097605951872171305L;

 

                     @Override

                     public Source resolve(String href, String base) throws XPathException {

                           return new StreamSource(new InputStream() {

                                  private static final String intro =

                                         "<song>\n";

                                  private static final String chorus =

                                         "  <chorus>\n" +

                                         "    <line>Let me take you down</line>\n" +

                                         "    <line>Cause I'm going to Strawberry Fields</line>\n" +

                                         "    <line>Nothing is real</line>\n" +

                                         "    <line>And nothing to get hung about</line>\n" +

                                         "    <line>Strawberry Fields forever</line>\n" +

                                         "  </chorus>\n";

                                  private int i = - intro.length();

                                 

                                  @Override

                                  public int read() throws IOException {

                                         if (i++ < 0) {

                                                return intro.charAt(i + intro.length() - 1);

                                         }

                                         else {

                                                i %= chorus.length();

                                                return chorus.charAt(i);

                                         }

                                  }

                           });

                     }

              });

 

              long start = System.currentTimeMillis();

              long count = 0;

              long nextStop = 1;

              for (Iterator<XdmItem> xdmItemIterator = xqueryEvaluator.iterator(); xdmItemIterator.hasNext(); ) {

                     xdmItemIterator.next();

                     if (++count == nextStop) {

                           long elapsed = System.currentTimeMillis() - start;

                           System.out.println("t(" + count + "): " + elapsed + " msec");

                           if (count >= limit)

                                  break;

                           nextStop <<= 1;

                     }

              }

       }

 

 

From: Michael Kay [mailto:mike@saxonica.com]
Sent: Donnerstag, 20. Februar 2014 19:33
To: Mailing list for the SAXON XSLT and XQuery processor
Subject: Re: [saxon] Question on streaming via s9api

 

 

saxon:stream() is a pseudo-function; its effect depends on the form of the expression in its argument, not just on the value of the expression, and the simplest rule is that the expression must be a call on doc() followed by something that looks like an XSLT pattern. So having /string() on the end will spoil it. You don't need a FLWOR: the following should work equally well:

 

saxon:stream(doc('" + input + "')/*/*)/string()

 

 


Software AG – Sitz/Registered office: Uhlandstraße 12, 64297 Darmstadt, Germany – Registergericht/Commercial register: Darmstadt HRB 1562 - Vorstand/Management Board: Karl-Heinz Streibich (Vorsitzender/Chairman), Dr. Wolfram Jost, Arnd Zinnhardt; - Aufsichtsratsvorsitzender/Chairman of the Supervisory Board: Dr. Andreas Bereczky - http://www.softwareag.com