[Semediawiki-user] SMW 3.0, Elasticsearch, and more ...

[James HK <jam...@gm...>]

Hi,

This email is to inform that we do not expect to make a 3.0 release in
Q1/2018 for reasons I will explain hereafter.

Over the years we have seen users asking for a scalability solution in
connection with Semantic MediaWiki especially from enterprise and
wiki-farms users. Last month I made an effort to investigate how
Elasticsearch (aka ES) could be integrate with SMW by moving the query
answering from SQL to ES and hereby allowing us to facilitate on its
scalability architecture [1] and combine structured and text query
execution.

The [2] PR introduces an interface to interact, update, and query ES
by forming a bridge between SMW and ES without the need to make any
infrastructure or DB related changes to achieve a coherent
integration.

After consulting with Karsten, he quickly provided an ES instance for
the sandbox [3] and for several weeks we have been running [2] as
drop-in replacement for the query answering to evaluate and showcase
its integration. As a user, you are ask to validate the interface by
creating queries that work with the current SQLStore but do not with
the ElasticStore [2]. We know [4] that for all existing test cases we
gathered over the years, they pass and we are interested in those that
don't.

A side from adding [2] there are other bigger and smaller features [5]
planned to be part of 3.0 but the documentation is yet missing and
this should be resolved before announcing a new release.

Furthermore, any major release (as indicated by changing from 2 to 3)
opens an opportunity to introduce breaking changes and remove
deprecated functions [0] but this as well remains an unresolved task.

When can a release be expected? Well, it more or less depends on
community involvement especially on the part of updating the
documentation because as I outlined in [2], [2] requires additional
effort and I will spend my volunteer time mostly on this task so it
can be completed within Q2/2018.

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/1014
[1] https://www.elastic.co/guide/en/elasticsearch/guide/current/_scale_horizontally.html
[2] https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/3054
[3] https://sandbox.semantic-mediawiki.org
[4] https://travis-ci.org/SemanticMediaWiki/SemanticMediaWiki/jobs/348737223
[5] https://github.com/SemanticMediaWiki/SemanticMediaWiki/blob/master/docs/RELEASE-NOTES.md

Cheers