Menu
▾
▴
Re: [Exist-open] JSON to XML
|
From: Adam R. <ad...@ex...> - 2012-11-10 14:27:06
|
Ah! I now see that there is a bug in eXist-db's XQuery 3.0 fn:analyze-string function. As I did the first implementation of this I will try and fix it... I have spent an hour or so looking at it, and I can see the issue but not why it occurs, I will try looking for inspiration in other implementations... On 9 November 2012 22:42, Joe Wicentowski <jo...@gm...> wrote: > Hi Adam, > > > What about John Snelson's approach? > > http://john.snelson.org.uk/parsing-json-into-xquery > > > > The best thing about it being that you done need to write any code as > > its already available here: > > > http://xqilla.hg.sourceforge.net/hgweb/xqilla/xqilla/raw-file/6458513c94c0/src/functions/XQillaModule.xq > > Thanks for the pointer! It sounds like a promising approach. But > does it work for you? I get a mysterious "String index out of range: > -2" error [1]. My steps to reproduce are to save Snelson's module > into my database (I'm running trunk, updated this morning) as > xqilla-json.xq. Then I save some JSON [2] into the database. Then I > run this script in eXide: > > ---- > xquery version "3.0"; > > import module namespace > xqilla="http://xqilla.sourceforge.net/Functions" at > "xmldb:exist:///db/apps/twitter/xqilla-json.xq"; > > let $json-filename := '/db/apps/twitter/home-timeline.json' > let $json-binary := util:binary-doc($json-filename) > let $json := util:binary-to-string($json-binary) > return > xqilla:parse-json($json) > ---- > > eXide returns a "Compilation Error: String index out of range: -2" > error, and exist.log has the entry below. Based on the exist.log > entry, I can't tell if the problem is in Snelson's code or in the > fn:analyze-string() function in eXist-db. > > Do you have any ideas? If I can get this to work in eXist-db, I think > lots of people will find it useful. > > Thanks, > Joe > > > [1] error from exist.log > > 2012-11-09 17:30:33,684 [eXistThread-108] ERROR (XQueryServlet.java > [process]:564) - String index out of range: -2 > java.lang.StringIndexOutOfBoundsException: String index out of range: -2 > at java.lang.String.substring(String.java:1937) > at > org.exist.xquery.functions.fn.FunAnalyzeString.analyzeString(FunAnalyzeString.java:114) > at > org.exist.xquery.functions.fn.FunAnalyzeString.eval(FunAnalyzeString.java:86) > at org.exist.xquery.BasicFunction.eval(BasicFunction.java:68) > at > org.exist.xquery.InternalFunctionCall.eval(InternalFunctionCall.java:55) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71) > at org.exist.xquery.PathExpr.eval(PathExpr.java:264) > at org.exist.xquery.ForExpr.eval(ForExpr.java:189) > at org.exist.xquery.LetExpr.eval(LetExpr.java:187) > at org.exist.xquery.LetExpr.eval(LetExpr.java:187) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153) > at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) > at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:324) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:229) > at > org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:74) > at org.exist.xquery.DynamicNameCheck.eval(DynamicNameCheck.java:64) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:211) > at org.exist.xquery.LetExpr.eval(LetExpr.java:142) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.UserDefinedFunction.eval(UserDefinedFunction.java:153) > at > org.exist.xquery.DynamicCardinalityCheck.eval(DynamicCardinalityCheck.java:74) > at org.exist.xquery.DynamicTypeCheck.eval(DynamicTypeCheck.java:61) > at org.exist.xquery.FunctionCall.evalFunction(FunctionCall.java:324) > at org.exist.xquery.FunctionCall.eval(FunctionCall.java:229) > at > org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:57) > at > org.exist.xquery.DebuggableExpression.eval(DebuggableExpression.java:64) > at org.exist.xquery.LetExpr.eval(LetExpr.java:189) > at org.exist.xquery.LetExpr.eval(LetExpr.java:187) > at org.exist.xquery.LetExpr.eval(LetExpr.java:187) > at org.exist.xquery.BindingExpression.eval(BindingExpression.java:156) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71) > at org.exist.xquery.PathExpr.eval(PathExpr.java:264) > at org.exist.xquery.AbstractExpression.eval(AbstractExpression.java:71) > at org.exist.xquery.XQuery.execute(XQuery.java:255) > at org.exist.xquery.XQuery.execute(XQuery.java:202) > at org.exist.http.servlets.XQueryServlet.process(XQueryServlet.java:488) > at org.exist.http.servlets.XQueryServlet.doPost(XQueryServlet.java:194) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:755) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:598) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:486) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:542) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:224) > at org.eclipse.jetty.server.Dispatcher.forward(Dispatcher.java:98) > at org.exist.http.urlrewrite.Forward.doRewrite(Forward.java:47) > at > org.exist.http.urlrewrite.XQueryURLRewrite.doRewrite(XQueryURLRewrite.java:552) > at > org.exist.http.urlrewrite.XQueryURLRewrite.service(XQueryURLRewrite.java:336) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:848) > at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:598) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1367) > at > de.betterform.agent.web.filter.XFormsFilter.doFilter(XFormsFilter.java:173) > at > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1338) > at > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119) > at > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:499) > at > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233) > at > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065) > at > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413) > at > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192) > at > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999) > at > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117) > at > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149) > at > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111) > at org.eclipse.jetty.server.Server.handle(Server.java:350) > at > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454) > at > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:900) > at > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:954) > at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:851) > at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:235) > at > org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:77) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:606) > at > org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:46) > at > org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:603) > at > org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:538) > at java.lang.Thread.run(Thread.java:662) > > > [2] some sample json retrieved via twitter API - saved as > /db/apps/twitter/home-timeline.json. > > [ > { > "created_at": "Fri Nov 09 22:21:06 +0000 2012", > "id": 267029071874121729, > "id_str": "267029071874121729", > "text": "Supreme Court To Weigh Constitutionality Of Voting > Rights Act http://t.co/9nLEVUGv", > "source": "<a href=\"http://twitterfeed.com\" > rel=\"nofollow\">twitterfeed<\/a>", > "truncated": false, > "in_reply_to_status_id": null, > "in_reply_to_status_id_str": null, > "in_reply_to_user_id": null, > "in_reply_to_user_id_str": null, > "in_reply_to_screen_name": null, > "user": { > "id": 5392522, > "id_str": "5392522", > "name": "NPR News", > "screen_name": "nprnews", > "location": "Washington, DC", > "description": "News. Arts & Life. Music. Everything and > more from NPR.", > "url": "http://www.npr.org", > "entities": { > "url": {"urls": [{ > "url": "http://www.npr.org", > "expanded_url": null, > "indices": [ > 0, > 18 > ] > }]}, > "description": {"urls": []} > }, > "protected": false, > "followers_count": 1320915, > "friends_count": 76198, > "listed_count": 38369, > "created_at": "Sun Apr 22 05:10:15 +0000 2007", > "favourites_count": 0, > "utc_offset": -18000, > "time_zone": "Eastern Time (US & Canada)", > "geo_enabled": false, > "verified": true, > "statuses_count": 46455, > "lang": "en", > "contributors_enabled": false, > "is_translator": false, > "profile_background_color": "9AE4E8", > "profile_background_image_url": > " > http://a0.twimg.com/profile_background_images/414161733/nprnews_background.jpg > ", > "profile_background_image_url_https": > " > https://si0.twimg.com/profile_background_images/414161733/nprnews_background.jpg > ", > "profile_background_tile": false, > "profile_image_url": > "http://a0.twimg.com/profile_images/1796148436/nprnews_icon_normal.jpg", > "profile_image_url_https": > "https://si0.twimg.com/profile_images/1796148436/nprnews_icon_normal.jpg", > "profile_banner_url": > "https://si0.twimg.com/brand_banners/nprnews/1328071709/live", > "profile_link_color": "0000FF", > "profile_sidebar_border_color": "87BC44", > "profile_sidebar_fill_color": "E0FF92", > "profile_text_color": "000000", > "profile_use_background_image": true, > "default_profile": false, > "default_profile_image": false, > "following": true, > "follow_request_sent": null, > "notifications": null > }, > "geo": null, > "coordinates": null, > "place": null, > "contributors": null, > "retweet_count": 16, > "entities": { > "hashtags": [], > "urls": [{ > "url": "http://t.co/9nLEVUGv", > "expanded_url": "http://n.pr/WMMNfe", > "display_url": "n.pr/WMMNfe", > "indices": [ > 62, > 82 > ] > }], > "user_mentions": [] > }, > "favorited": false, > "retweeted": false, > "possibly_sensitive": false > }, > { > "created_at": "Fri Nov 09 22:19:18 +0000 2012", > "id": 267028619241607169, > "id_str": "267028619241607169", > "text": "Rock legend? RT @yamainu: Searching Mercury in JSTOR: > planet, element, god...? #chs12", > "source": "<a href=\"http://www.hootsuite.com\" > rel=\"nofollow\">HootSuite<\/a>", > "truncated": false, > "in_reply_to_status_id": null, > "in_reply_to_status_id_str": null, > "in_reply_to_user_id": null, > "in_reply_to_user_id_str": null, > "in_reply_to_screen_name": null, > "user": { > "id": 6899432, > "id_str": "6899432", > "name": "Krista", > "screen_name": "digipub", > "location": "Chicago, Illinois", > "description": "A completely and totally unofficial > tweetstream from the University of Chicago Press Digital Publishing > Office. Electronic publishing--140 characters at a time.", > "url": "http://press.uchicago.edu", > "entities": { > "url": {"urls": [{ > "url": "http://press.uchicago.edu", > "expanded_url": null, > "indices": [ > 0, > 25 > ] > }]}, > "description": {"urls": []} > }, > "protected": false, > "followers_count": 419, > "friends_count": 419, > "listed_count": 36, > "created_at": "Mon Jun 18 20:52:42 +0000 2007", > "favourites_count": 5, > "utc_offset": -21600, > "time_zone": "Central Time (US & Canada)", > "geo_enabled": true, > "verified": false, > "statuses_count": 892, > "lang": "en", > "contributors_enabled": false, > "is_translator": false, > "profile_background_color": "8B542B", > "profile_background_image_url": > " > http://a0.twimg.com/profile_background_images/276961021/Phoenix_72lrg3_1000b.jpg > ", > "profile_background_image_url_https": > " > https://si0.twimg.com/profile_background_images/276961021/Phoenix_72lrg3_1000b.jpg > ", > "profile_background_tile": true, > "profile_image_url": > "http://a0.twimg.com/profile_images/1088777565/3fc563f_normal.jpg", > "profile_image_url_https": > "https://si0.twimg.com/profile_images/1088777565/3fc563f_normal.jpg", > "profile_link_color": "9D582E", > "profile_sidebar_border_color": "D9B17E", > "profile_sidebar_fill_color": "EADEAA", > "profile_text_color": "333333", > "profile_use_background_image": true, > "default_profile": false, > "default_profile_image": false, > "following": true, > "follow_request_sent": null, > "notifications": null > }, > "geo": null, > "coordinates": null, > "place": null, > "contributors": null, > "retweet_count": 0, > "entities": { > "hashtags": [{ > "text": "chs12", > "indices": [ > 79, > 85 > ] > }], > "urls": [], > "user_mentions": [{ > "screen_name": "yamainu", > "name": "yamainu", > "id": 14055151, > "id_str": "14055151", > "indices": [ > 16, > 24 > ] > }] > }, > "favorited": false, > "retweeted": false > } > ] > -- Adam Retter eXist Developer { United Kingdom } ad...@ex... irc://irc.freenode.net/existdb |
![http://a0.twimg.com/profile_images/1796148436/nprnews_icon_normal.jpg",](http://a0.twimg.com/profile_images/1796148436/nprnews_icon_normal.jpg",){kind=link}
![https://si0.twimg.com/profile_images/1796148436/nprnews_icon_normal.jpg",](https://si0.twimg.com/profile_images/1796148436/nprnews_icon_normal.jpg",){kind=link}
![http://a0.twimg.com/profile_images/1088777565/3fc563f_normal.jpg",](http://a0.twimg.com/profile_images/1088777565/3fc563f_normal.jpg",){kind=link}
![https://si0.twimg.com/profile_images/1088777565/3fc563f_normal.jpg",](https://si0.twimg.com/profile_images/1088777565/3fc563f_normal.jpg",){kind=link}