Menu
▾
▴
Re: [dotNetRDF-Develop] Stackoverflow calling QueryWithResultGraph()
|
From: Rob V. <rv...@do...> - 2014-02-07 09:29:55
|
No problem Should you succeed in nailing down the cause (whether it is dotNetRDF or otherwise) if you could let us know that would be much appreciated Thanks, Rob From: Rob Nicholls <ro...@co...> Date: Thursday, 6 February 2014 16:31 To: Rob Vesse <rv...@do...>, 'dotNetRDF Developer Discussion and Feature Request' <dot...@li...> Subject: RE: [dotNetRDF-Develop] Stackoverflow calling QueryWithResultGraph() > Thanks for your help Rob, > > Yes pretty much where we got to. It¹s totally sporadic, and hard to > reproduce. It works sometimes, and not others it either always works, or > always stackoverflows, on multiple environments (irrelevant of build > settings). > > We first used a named graph, and now we don¹t. Left the queries (several > hundred of them) using named graphs as doesn¹t (seem to, we¹re still learning) > make any difference as you mention. > > dotNetRDF > pulled via nuget > Version: 1.0.3.0 > Description: API for manipulating RDF data (.Net 4.0) > > Based on it probably being a environment or config related issue, and before > you waste any more time, I¹ll try some more debugging off your code base > (rather than release assemblies) and double checking our dependency graph to > make it¹s not something else impacting. > > The triple store is over a gigabyte I believe, so will try and avoid sending > that over for the time being. > > Many thanks. > > > > From: Rob Vesse [mailto:rv...@do...] > Sent: 06 February 2014 15:16 > To: dotNetRDF Developer Discussion and Feature Request > Cc: Rob Nicholls > Subject: Re: [dotNetRDF-Develop] Stackoverflow calling QueryWithResultGraph() > > > So I am pretty stumped on this as I've been completely unable to reproduce > this after several hours of trying. > > > > I checked whether you could easily hit a StackOverflow with the RDF/XML parser > and you really can't. I wrote unit tests which call the parser from 5,000 > levels deep of recursion and did not get a StackOverflow so I don't see how > the data alone could cause the issue. > > > > I've also put the data you provided into a local instance of Fuseki and ran > the query against it using SparqlRemoteEndpoint.QueryWithResultGraph() and I > cannot produce a StackOverflow. > > > > Part of my problem is that I just don't seem to have quite the same setup as > you, your query uses named graphs but your data does not include them so I had > to modify the query slightly. Yet I don't see how that would make any > difference especially since the CONSTRUCT template remains identical between > your query and mine so the data received should be identical. > > > > You state that you've reproduced this in a console app so could you please > provide your console app that reproduces the issue along with your raw data > (assuming a TDB database zipping up the TDB folder would provide this) that I > can run under the debugger on my system to see what is going on? > > Failing that can you run under the debugger and capture what the stack trace > is at the point where the StackOverflowException gets thrown? As otherwise I > can't really do any further useful debugging on this. > > > > Also can you please confirm again exactly the build of dotNetRDF being used? > If you can look at the properties for the dotNetRDF dependency in VS and look > at the Version and the Description strings and provide that information, the > Description is particularly relevant because that will tell me exactly which > of the core library builds you are using. > > > > Thanks, > > > > Rob > > > > From: Rob Vesse <rv...@do...> > Reply-To: dotNetRDF Developer Discussion and Feature Request > <dot...@li...> > Date: Thursday, 6 February 2014 11:12 > To: Rob Nicholls <ro...@co...> > Cc: dotNetRDF Developer Discussion and Feature Request > <dot...@li...> > Subject: SPAM-LOW: Re: [dotNetRDF-Develop] Stackoverflow calling > QueryWithResultGraph() > > >> >> Thanks for the additional information, the RDF/XML is perfectly valid as you >> noted. >> >> >> >> That error is not really more descriptive, turning on the HTTP Full Debugging >> consumes the response stream so the calling code then has an already closed >> stream which causes different errors depending on the calling code. I've >> changed this behaviour in our default branch so that it'll now throw an error >> informing you that the response stream was consumed and not available to >> calling code. >> >> >> >> The RDF/XML parser code does recurse heavily so I'm wondering if this file is >> just under the stack limit when parsed in the rdfEditor but when called via >> the SparqlRemoteEndpoint which itself has several levels of recursion tips it >> over the edge. I'm going to take the data you provided and debug this >> further and will get back to you when I know more >> >> >> >> Rob >> >> >> >> From: Rob Nicholls <ro...@co...> >> Date: Monday, 3 February 2014 13:46 >> To: Rob Vesse <rv...@do...> >> Cc: 'dotNetRDF Developer Discussion and Feature Request' >> <dot...@li...> >> Subject: RE: [dotNetRDF-Develop] Stackoverflow calling QueryWithResultGraph() >> >> >>> >>> Ahh, missed that. Thanks. >>> >>> Ok, now throwing a more descriptive error; XML parser, root element is >>> missing. >>> >>> Output attached. >>> >>> I¹ve checked the RDF with your RDFeditor validator, no problems. >>> >>> Is it feasibly a memory issue, based on the size of the RDF/XML? (I¹ve >>> stripped the code right back to a simple console app using dotNetRDF same >>> result) Or maybe I¹m missing something obvious? >>> >>> /Rob >>> >>> >>> From: Rob Vesse [mailto:rv...@do...] >>> Sent: 30 January 2014 19:16 >>> To: Rob Nicholls >>> Cc: dotNetRDF Developer Discussion and Feature Request >>> Subject: Re: [dotNetRDF-Develop] Stackoverflow calling >>> QueryWithResultGraph() >>> >>> >>> Ok, that's useful information. Specific version of Fuseki should not be an >>> issue since I don't see anything in the query that would trigger any known >>> bugs in Fuseki/TDB/ARQ >>> >>> >>> >>> Would be interesting to see what format Fuseki is returning the data in when >>> called from dotNetRDF. Could you run the query with HTTP Debugging turned >>> on as detailed at [1] and provide the HTTP traces this produces to the >>> console? >>> >>> >>> >>> The full HTTP trace could be particularly useful in determining if there is >>> something in the data serialisation of the results of this query that >>> triggers a SO in one of the dotNetRDF parsers. >>> >>> >>> >>> Thanks, >>> >>> >>> >>> Rob >>> >>> >>> >>> [1]: >>> https://bitbucket.org/dotnetrdf/dotnetrdf/wiki/HowTo/Debug%20HTTP%20Communic >>> ation.wiki#!debugging-http-communication >>> >>> >>> >>> >>> >>> From: Rob Nicholls <ro...@co...> >>> Date: Thursday, 30 January 2014 10:26 >>> To: Rob Vesse <rv...@do...>, 'dotNetRDF Developer Discussion and >>> Feature Request' <dot...@li...> >>> Subject: RE: [dotNetRDF-Develop] Stackoverflow calling >>> QueryWithResultGraph() >>> >>> >>>> >>>> dotnetRDF library to query (via .NET4.5 build) >>>> Jena Fuseki for the Triple Store and Endpoint (I forget specific versions, >>>> but will check if helpful) >>>> mmx is our Dataset >>>> >>>> Calling code is basically (omitted the noise): >>>> >>>> _endpoint = new SparqlRemoteEndpoint(³http://jena:3030/mmx/query²); >>>> IGraph graph = _endpoint.QueryWithResultGraph(thequery); // ß Stackoverflow >>>> here >>>> >>>> Where thequery is: >>>> >>>> >>>> 1 PREFIX dc: <http://purl.org/dc/elements/1.1/> >>>> 2 PREFIX exif: <http://www.w3.org/2003/12/exif/ns/> >>>> 3 PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema# >>>> <http://www.w3.org/2000/01/rdf-schema> > >>>> 4 PREFIX mmx: <http://data.sciencemuseum.org.uk/def/ontology/> >>>> 5 PREFIX foaf: <http://xmlns.com/foaf/0.1/> >>>> 6 PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns# >>>> <http://www.w3.org/1999/02/22-rdf-syntax-ns> > >>>> 7 PREFIX skos: <http://www.w3.org/2004/02/skos/core# >>>> <http://www.w3.org/2004/02/skos/core> > >>>> 8 PREFIX dcterms: <http://purl.org/dc/terms/> >>>> 9 >>>> 10 CONSTRUCT >>>> 11 { ?ass mmx:relatedArchiveDocument >>>> <http://id.sciencemuseum.org.uk/archivedocument/smad-110001337> . >>>> 12 ?ass rdf:type mmx:Asset . >>>> 13 ?ass dcterms:hasVersion ?ver . >>>> 14 ?ver ?pred2 ?obj2 . >>>> 15 ?ass ?p2 ?o2 .} >>>> 16 WHERE >>>> 17 { { GRAPH ?g >>>> 18 { ?ass mmx:relatedArchiveDocument >>>> <http://id.sciencemuseum.org.uk/archivedocument/smad-110001337> . >>>> 19 ?ass dcterms:hasVersion ?ver . >>>> 20 ?ver ?pred2 ?obj2 >>>> 21 } >>>> 22 } >>>> 23 UNION >>>> 24 { GRAPH ?g >>>> 25 { ?ass mmx:relatedArchiveDocument >>>> <http://id.sciencemuseum.org.uk/archivedocument/smad-110001337> . >>>> 26 ?ass ?p2 ?o2 >>>> 27 } >>>> 28 } >>>> 29 } >>>> >>>> Thought I¹d exclude the setup and query as, as mentioned, the query does >>>> work (and did, only recently started throwing the exception hence I >>>> wondered if it was data, or maybe encoding, related based on how dotnetRDF >>>> is parsing). >>>> >>>> The attached triples were the result of the construct from above, not the >>>> entire triple store. >>>> >>>> This all works fine for small result sets, it¹s just this particular query >>>> returns 360 relatedArchiveDocument¹s and the library doesn¹t seem to like >>>> it. >>>> >>>> If it¹s nothing obvious, I will grab your source code and try and debug >>>> further can¹t tell much of what is happening within >>>> QueryWithResultGraph() atm. >>>> >>>> /Rob >>>> >>>> >>>> >>>> From: Rob Vesse [mailto:rv...@do...] >>>> Sent: 30 January 2014 17:12 >>>> To: dotNetRDF Developer Discussion and Feature Request >>>> Cc: ro...@co... >>>> Subject: Re: [dotNetRDF-Develop] Stackoverflow calling >>>> QueryWithResultGraph() >>>> >>>> >>>> Rob >>>> >>>> >>>> >>>> You didn't include the query or any context of how you invoke the query so >>>> we can't really offer any attempt at diagnosis. Please provide a minimal >>>> complete example. >>>> >>>> >>>> >>>> Are the results you provided the results of a successful run of the >>>> specific query or a dump of the entire test dataset? Nothing looks >>>> untoward with the data so likely some interaction with the specific query. >>>> >>>> >>>> >>>> QueryWithResultGraph() implies use of SparqlRemoteEndpoint (whether >>>> directly/indirectly) to query a remote service so can you provide the code >>>> that invokes the query plus the specific problematic query. The only kind >>>> of query that springs to mind that could easily lead to a StackOverflow >>>> would be a complex property path but without seeing the query I can't tell, >>>> it may well be something innocuous hitting a rare corner case in our SPARQL >>>> engine. >>>> >>>> >>>> >>>> Also what can you tell us about the set up of that remote service since you >>>> imply you control that as well. Is the remote server using dotNetRDF to >>>> host the data, if so what's the setup? >>>> >>>> >>>> >>>> Or if it isn't under your control can you tell us anything about it e.g. >>>> endpoint, server software if known etc >>>> >>>> >>>> >>>> Thanks, >>>> >>>> >>>> >>>> Rob Vesse >>>> >>>> >>>> >>>> From: Rob Nicholls <ro...@co...> >>>> Reply-To: dotNetRDF Developer Discussion and Feature Request >>>> <dot...@li...> >>>> Date: Thursday, 30 January 2014 05:44 >>>> To: <dot...@li...> >>>> Subject: [dotNetRDF-Develop] Stackoverflow calling QueryWithResultGraph() >>>> >>>> >>>>> >>>>> Hi all, >>>>> >>>>> I¹m receiving consistent stack overflows from QueryWithResultGraph(query) >>>>> when using a specific query against our triple store. The query runs fine >>>>> (within ms) and returns results ok. >>>>> >>>>> Results returned from the triple store are in the attached file >>>>> (queryresults.txt), not a huge amount of data to justify killing a 12GB >>>>> web server, and it does occasionally function correctly on my developer >>>>> machine (never when running the debugger though). >>>>> >>>>> Can anyone offer any advice? >>>>> Is this perhaps an issue with the data itself causing some kind of >>>>> circular reference? Although doesn¹t explain why it occasionally works. >>>>> >>>>> Using: >>>>> dotnetRdf 1.0.3 / .NET4.5 >>>>> >>>>> Many thanks, >>>>> Rob >>>>> >>>>> cogapp >>>>> >>>>> >>>>> Rob Nicholls >>>>> Senior Programmer >>>>> +44 1273 821600 >>>>> www.cogapp.com <http://www.cogapp.com/> >>>>> >>>>> Cogapp Brighton >>>>> 21-33 Dyke Road, Brighton, BN1 3FE. UK >>>>> >>>>> Cogapp New York >>>>> 641 Lexington Avenue, 15th Floor, New York, NY 10022, USA >>>>> >>>>> News >>>>> Pocahontas' portrait and letters from George Washington online: a >>>>> responsive website for the Virginia Historical Society >>>>> <http://www.vahistorical.org/> , just launched. >>>>> >>>>> For regular Cogapp news, please follow us on Twitter >>>>> <https://twitter.com/cogapp> . >>>>> >>>>> -------------------------------------------------------------------------- >>>>> ---- WatchGuard Dimension instantly turns raw network data into actionable >>>>> security intelligence. It gives you real-time visual feedback on key >>>>> security issues and trends. Skip the complicated setup - simply import a >>>>> virtual appliance and go from zero to informed in seconds. >>>>> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clkt >>>>> rk_______________________________________________ dotNetRDF-develop >>>>> mailing list dot...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/dotnetrdf-develop >> ----------------------------------------------------------------------------- >> - Managing the Performance of Cloud-Based Applications Take advantage of what >> the Cloud has to offer - Avoid Common Pitfalls. Read the Whitepaper. >> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk_ >> ______________________________________________ dotNetRDF-develop mailing list >> dot...@li... >> https://lists.sourceforge.net/lists/listinfo/dotnetrdf-develop |
