[Roundup-devel] Handling FTS5 missing from sqlite

[John P. R. <ro...@cs...>]

Hi all:

Over on the users list, Tonu is trying to install Roundup
2.3.0. However his SQLite library (on RedHat 7) is built without full
text search version 5 (FTS5) support.

As a result it crashes when trying to use SQLite as the back end.

Should the next release of Roundup support a SQLite library with
missing FTS5? FTS5 has been in the standard amalgamation of SQLite
since Oct 2015. Using FTS5 is optional as we have the native
(internal) FTS mechanism, plus xapian and whoosh.

I could wrap the SQL call that fails to create the FTS5 table in a
try/except block. Then check for a missing FTS5 in the except block
and report a suitable warning.

This should permit Roundup to initialize with the SQLite
backend. However, it will crash at runtime when a user makes a full
text search request if the admin configured it to use native-fts. Not
a great experience.

Also it makes upgrading the Roundup schema more difficult. A schema
version is a simple integer. Schema version 7 (or newer) is expected
to include the FTS5 table. So we could have a schema version 7+
without the FTS table. This may require additional code and runtime to
handle the variation.

I can add a test in the configuration module to exit nicely on startup
when native-fts is used with non FTS5 SQLite library. This would make
Roundup exit cleanly before the user can discover the issue.

We already provide a method that performs a sanity check for xapian's
indexer language. It also validates the indexer language for postgres
against a hard coded list (so I don't need to do a remote SQL call to
get the language list). So this test would go in the same method.

When running under roundup-admin it looks like the configuration test
are only run on startup. I assume CGI will run the test on every
request.

However I am not sure what happens with wsgi or other deployment
methods. Will they pay the cost of checking for the misconfig?

Does anybody have any thoughts on which of these options I should use:

  1) do nothing (well maybe handle the crash a little better with
     a definitive diagnosis). Roundup > 2.1.0 will not work with a
     SQLite library missing FTS5 support.

  2) capture and report the failure to set up an FTS5 table so the
     tracker will initialize and run. However if the admin
     misconfigures the indexer to use native-fts the tracker will
     crash when a user tries to use it.

  3) implement the capture and report from #2 plus a test in
     configuration.py to validate that:

        database-config: sqlite
        indexer-config: native-fts
        sqlite: must support fts5

     otherwise we exit on startup.

Thoughts? I am leaning toward #1 as I don't want to start dealing with
divergent schemas and the possible can of worms that may start.

Anybody have a strong counter-argument, or idea to handle schema
issues?

Have a great week.
--
				-- rouilj
John Rouillard
===========================================================================
My employers don't acknowledge my existence much less my opinions.