> I can't comment in detail, but I think that having the ability to
> query specific SPARQL endpoints and map the results to property values
> would be a killer feature.

> There is so much data on the semantic web, being able to navigate it
> in my wiki and being able to augment my wiki's content with it would
> be fantastic.

At a minimum, we're currently working on updating the SparqlExtension to function with SMW 1.6.  With this extension we already allow querying external endpoints (we need to update the documentation), and it's possible with it to map results to property values as you mention.  As a caveat, we've tested this with Joseki and Sesame, although not yet with 4Store and Virtuoso.

Markus brings up a lot of good points, and below are a few responses to these, based on our experience with the SparqlExtension:

> * some mechanism to deal with slow and dead SPARQL services to ensure
> that they do not delay page display; maybe some asynchronous result
> loading + caching;

We haven't had a problem with dead SPARQL endpoints.  For example, when the CIA World Factbook has been down, no data gets returned, and the page loads quickly.  Slow services are another issue though, and we haven't tackled it yet, and asynchronous result loading is a great idea.  To help things out, we currently use a Squid server to cache both incoming and outgoing query results, and we have code that clears the query results in the cache whenever the page containing those queries is purged.

> * some mechanism to configure available SPARQL services to avoid people
> having to enter the full URL when sending a query (this is also a
> possible security issue);

As for security, is the main concern about the update URL?  I assume that the query URL should be ok to be public knowledge.

For the rest of Markus' points (SMW data items, #ask parsers, existing query printers, etc.) the bottom line is that we haven't set up the SparqlExtension to be tightly integrated with the SMW code yet, and this is an area for improvement.  

> Alternatively, one can of course
> simply connect SMW to an RDF store and let this store provide the SPARQL
> endpoint. This seems more viable but then SPARQL is not a standard
> functionality of SMW.

Is the main concern about having something that just works out of the box with minimal installation hassle?  Could there be an option for a kind of "expert functionality" in the interim where you can rely on the functionality of the RDF store while the rest of the features mentioned are being implemented?

Regards,

Chris


From: Dan Bolser <dan.bolser@gmail.com>
To: Markus Krötzsch <markus@semantic-mediawiki.org>
Cc: Chris Davis <c_b_dvs@yahoo.com>; "semediawiki-devel@lists.sourceforge.net" <semediawiki-devel@lists.sourceforge.net>
Sent: Monday, September 5, 2011 2:44 PM
Subject: Re: [SMW-devel] SPARQL queries and SMW 1.6

I can't comment in detail, but I think that having the ability to
query specific SPARQL endpoints and map the results to property values
would be a killer feature.

There is so much data on the semantic web, being able to navigate it
in my wiki and being able to augment my wiki's content with it would
be fantastic.

I'll try looking at the code, but I've a lot to learn.


Cheers,
Dan.

On 26 August 2011 17:56, Markus Krötzsch <markus@semantic-mediawiki.org> wrote:
> On 26/08/11 15:31, Chris Davis wrote:
>> Am I correct in observing that SMW 1.6 does not support sparql queries
>> directly included in wiki pages, but only allows for ask queries which
>> are then translated into sparql syntax?
>>
>> The reason for the question is that I'm one of the developers of the
>> SparqlExtension, and would like to migrate to SMW 1.6. However, I've
>> become somewhat addicted to/dependent on the more advanced features of
>> sparql such as federated queries, type conversions, custom functions,
>> etc (see http://enipedia.tudelft.nl/ for what this enables).
>>
>> Is this something that will be eventually integrated in the core code,
>> or am I better off for the moment setting up an extension that enables
>> this more advanced functionality with SMW 1.6?
>
> We recently discussed that this would be nice to have in the core, but
> that some more work is needed to get it there. There are certainly
> various parties who would be interested in this feature, but someone
> needs to take the lead in implementing what remains to be done to get
> this into SMW. Below are some facts to summarise the status.
>
>
> What SMW 1.6 provides is support for SPARQL to the extent needed to have
> a SPARQL-based backend for query answering. So the following functional
> components are in SMW:
>
> * a SPARQL communication abstraction, similar to MW's DataBase
> * SPARQL result parser for the standard XML result format
> * Some basic mechanism for interpreting SPARQL elements (esp. URIs) as
> WikiPages; could be extended to wrap literals into accoding SMW data
> items as well
> * SMW-data to RDF translation
> * #ask to SPARQL translation that works with the SMW-data to RDF
> translation in the sense that the results over SPARQL/RDF should be the
> same as over SQL
>
>
> What is missing to support #sparql-ike queries to external SPARQL
> services from the wiki:
>
> * adaptor code that allows all existing query printers to act on SPARQL
> results thus reeived (probably implement a "virtual" store that only
> retrieves data that came from one SPARQL query);
> * add an #ask-like parser function to make this available to users
>
> Asynchronous retrieval and caching seem to be the hardest problems here.
> Caching could possibly be built into the existing SPARQL communication
> abstraction layer.
>
>
> What is missing to have SMW itself provide a SPARQL service to the
> outside world in general:
>
> * a parser for SPARQL queries (could be a library),
> * code to translate SPARQL queries into SQL queries over SMW's database
>
> The second item is a lot of work. Alternatively, one can of course
> simply connect SMW to an RDF store and let this store provide the SPARQL
> endpoint. This seems more viable but then SPARQL is not a standard
> functionality of SMW.
>
>
> Regards,
>
> Markus
>
>
> ------------------------------------------------------------------------------
> EMC VNX: the world's simplest storage, starting under $10K
> The only unified storage solution that offers unified management
> Up to 160% more powerful than alternatives and 25% more efficient.
> Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev
> _______________________________________________
> Semediawiki-devel mailing list
> Semediawiki-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/semediawiki-devel
>