Re: [iris-reasoner-support] Are there more advanced IRelation implementations?

[Barry B. <bar...@st...>]

Hi Martin,

Thanks for the encouraging words, but sorry to hear that you have run in 
to memory problems. You are not alone in this any we have work scheduled 
for improving the memory consumption (there are a number of tricks for 
this) and also possibly implementing a database backed relation class.

However, currently we are snowed under with non-IRIS related tasks and 
must postpone things for a short time.

Regarding the memory consumption, IRIS is not very sophisticated and 
could easily be modified so that:

1. every term is unique
2. every unique combination of terms (i.e. a tuple) is also unique

This might slow things down a little bit, but should save a lot of 
memory too, since really all a datalog evaluation does is recombine 
existing terms in different ways.

So yes, we plan for IRIS to be able to manage evaluations involving 
millions of facts, but some more work needs to be done.

Thanks again for your interest,

Barry Bishop
Senior Scientific Programmer
Semantic Technology Institute (STI)
University of Innsbruck, Austria
-----------------------------------
E-Mail: bar...@st...
Tel:    +43 512 507 96873
-----------------------------------

Martin Bravenboer wrote:
> Hi all,
> 
> I started using IRIS because it's very user-friendly and supports a
> lot of datatypes. The educational value of IRIS is definitely great.
> 
> I have been running very small experiments until now, but in the long
> run my Datalog programs need to scale to relations with millions of
> tuples. I do not exactly expect IRIS to handle that, but I was hoping
> that I could at least keep using IRIS for medium-size problems with
> relations of a few 100k and compare the results to the results of
> other Datalog engines.
> 
> Unfortunately my medium-sized tests, with 200k to 600k input facts all
> result in out of memory errors for Datalog programs that are not
> entirely trivial (i.e. involving a few joins). I've set the Java heap
> size to 2048m. The biggest issues seems to be the SimpleRelation.
> Java's stack trace usually shows an out of memory error while copying
> arrays as a result of operations on SimpleRelation.
> 
> So, I'm  wondering, do you have any other implementations around for
> IRelation that you haven't included in IRIS? Maybe backed up by a
> database?
> 
> Do you have any general advice for reducing the memory consumption?
> 
> Maybe you could also say something about the scale you want to support
> with IRIS?
> 
> Thanks!
> 
> Cheers,