[Bigdata-developers] bigdata(r) snapshot release: 0.83.2

[Mike P. <mi...@sy...>]

We've just released a new version of Bigdata(r).  This is a bigdata(r) snapshot release.  This release is capable of loading 1B triples in under one hour on a 15 node cluster and has been used to load up to 13B triples on the same cluster.  JDK 1.6 is required.

See [1] for instructions on installing bigdata(r), [2] for the javadoc and [3] and [4] for news, questions, and the latest developments.  For more information about SYSTAP, LLC and bigdata(r), see [5].

Please note that we recommend checking out the code from SVN using the tag for this release.  The code will build automatically under eclipse.  You can also build the code using the ant script.  The cluster installer requires the use of the ant script.  You can checkout this release from the following URL:

https://bigdata.svn.sourceforge.net/svnroot/bigdata/branches/BIGDATA_RELEASE_0_83_2

New features:

- This release provides a bug fix for issue#118.  Upgrade to this release is advised.  See https://sourceforge.net/apps/trac/bigdata/ticket/118 for details.

- Inlining XSD numerics, xsd:boolean, or custom datatype extensions into the statement indices.  Inlining provides a smaller footprint and faster queries for data using XSD numeric datatypes.  In order to introduce inlining we were forced to make a change in the physical schema for the RDF database which breaks binary compatibility for existing stores.  The recommended migration path is to export the data and import it into a new bigdata instance.

- Refactor of the dynamic sharding mechanism for higher performance.

- The SparseRowStore has been modified to make Unicode primary keys decodable by representing Unicode primary keys using UTF8 rather than Unicode sort keys.  This change also allows the SparseRowStore to work with the JDK collator option which embeds nul bytes into Unicode sort keys.  This change breaks binary compatibility, but there is an option for historical compatibility.

The roadmap for the next releases include:

- Query optimizations;

- Support for high-volume analytic query workloads and SPARQL aggregations;

- High availability for the journal and the cluster;

- Simplified deployment, configuration, and administration for clusters.

For more information, please see the following links:

[1] http://bigdata.wiki.sourceforge.net/GettingStarted
[2] http://www.bigdata.com/bigdata/docs/api/
[3] http://sourceforge.net/projects/bigdata/
[4] http://www.bigdata.com/blog
[5] http://www.systap.com/bigdata.htm

About bigdata:

Bigdata(r) is a horizontally-scaled, general purpose storage and computing fabric for ordered data (B+Trees), designed to operate on either a single server or a cluster of commodity hardware. Bigdata(r) uses dynamically partitioned key-range shards in order to remove any realistic scaling limits - in principle, bigdata(r) may be deployed on 10s, 100s, or even thousands of machines and new capacity may be added incrementally without requiring the full reload of all data. The bigdata(r) RDF database supports RDFS and OWL Lite reasoning, high-level query (SPARQL), and datum level provenance.

---
Mike Personick
SYSTAP, LLC.
801.328.3945 (office)
801.243.3678 (mobile)
801.938.5320 (skype)
mi...@sy...<mailto:mi...@sy...>