Re: [Bigdata-developers] When can I see inference results

[Bryan T. <br...@sy...>]

Antoni,

Incremental inference is not supported in scale-out (in the
JiniFederation).  It requires a stable (unchanging) view of the committed
triples in order to do truth maintenance.  This is at odds with the
concept that updates could originate from multiple concurrent clients.  In
order to do truth maintenance, we would have to impose a constraint that
all updates flow through a single point.

The JiniFederation does support database at once closure.  So you can load
large data sets and (re-) compute their closure.  You can also arrange to
coordinate the read-behind view used by the clients for query such that
they only see those points when the closure is complete.

The HAJournalServer (replication cluster) does support incremental truth
maintenance.  This is basically the RWStore mode journal using a low-level
write replication protocol to mirror the state of the database onto a
highly available quorum of machines.  The replication cluster is designed
to provide high availability and to scale-out the capability for
concurrent query.  However, it does not shard the data itself so its data
scale is limited by the capacity of a single machine.

In terms of the NSS REST API, operations should be done by the time the
HTTP response is received.  All writes should be stable on the disk.
However, per above, the database is not computing inferences incrementally
during updates on the JiniFederation.

If your inferences can be completed entirely within the scope of the
triples that you are asserting, then you can partition the inference
problem.  I.e., sometimes you can do the inference before you load the
data.  However, if you need inferences that runs across all the data in
the database and you need to do low latency updates with incremental truth
maintenance on those inferences, then you can not do that on the
JiniFederation.  The HAJournalServer provides a solution that scales query
horizontally and data up to 50,000,000,000 triples/quads.  If you need
more data scale, then you could use property paths (recently committed to
the development branch) or possibly integrate a higher level reasoner into
the query engine.  

Please tell us a little bit more about what you are trying to achieve here.

Thanks,
Bryan

On 2/7/13 4:41 AM, "Antoni Mylka" <ant...@ba...> wrote:

>Hi,
>
>Is inference done asynchronously in Bigdata?
>
>In my integration tests I see spurious failures:
>
>1. I run code that does SPARQL UPDATE queries. It uses SPARQLRepository
>or direct, handcrafted invocations of the REST API. The production code
>uses HTTP Client connection pooling and it's difficult to say how many
>physical connections it uses and when they stop.
>2. When that code finishes, I create a new SPARQLRepository and use it to
>see if content of the repository meets my expectations.
>3. The test code sometimes succeeds, sometimes fails.
>
>The NanoSparqlServer wikipage states that "Mutation operations are ACID
>against a standalone database and shard-wise ACID against a bigdata
>federation." Is inference done within the write transaction, or does the
>ACID only refer to explicit triples?
>
>Obviously there may be 1001 other reasons for the test failures I'm
>seeing. If you clearly confirm that when Bigdata says it's done - it
>means it's done I can look further.
>
>Another idea for investigation would be to check how HTTPClient
>connection pooling influences Bigdata. Does it use Keep-Alive and does
>this mean that the write transaction isn't actually committed until
>HTTPClient chooses to evict a connection from the pool.
>
>BTW, we're using a .war from rev 6717 deployed in Tomcat.
>
>--
>Antoni Myłka
>Software Engineer
>
>basis06 AG, Birkenweg 61, CH-3013 Bern - Fon +41 31 311 32 22
>http://www.basis06.ch - source of smart business
>
>--------------------------------------------------------------------------
>----
>Free Next-Gen Firewall Hardware Offer
>Buy your Sophos next-gen firewall before the end March 2013
>and get the hardware for free! Learn more.
>http://p.sf.net/sfu/sophos-d2d-feb
>_______________________________________________
>Bigdata-developers mailing list
>Big...@li...
>https://lists.sourceforge.net/lists/listinfo/bigdata-developers