Re: [chessx-developer] ChessX & SCID

[Shivkumar S. <ssh...@gm...>]

An interesting proposal. I agree with easy to implement (though leveldb and similar frameworks have a simple api too) and it would be fast to use for simple operations, but not with speed for complex operations or portability necessarily.

If you have multiple indexes, one needs to support multiple key value entries and aggregate across them. This gets more into leveldb territory. Similar problems arise when the database is too big to fit into RAM with all position indexes. In this case, leveldb will load what is necessary into RAM and smartly switch data in and out of RAM. With QT serialization, we have to figure out how to do this efficiently by coding it ourselves if the data does not fit into RAM. On portability, I was thinking of using this format outside chessx and the QT framework itself.

All that being said, for small databases (and some large databases with a single index) this can work. For large databases requiring multiple indexes that don't fit into RAM, its more challenging. Just my 2 cents.

Shiv
 
On Nov 21, 2012, at 1:21 PM, James Coons <ja...@co...> wrote:

> The fastest and easiest and most portable format for a binary ChessX Database IMO would be to simply use the built in Qt Serialization framework.
> 
> James Coons
> 
> 
> On Nov 21, 2012, at 2:57 PM, Shivkumar Shivaji <ssh...@gm...> wrote:
> 
>> 
>>> 
>>> Long ago Rico Zenklusen worked on a native database for ChessX as an
>>> alternative to SCIDdb (which is unusable in ChessX and never gained
>>> any momentum outside SCID). You might find some traces in the source
>>> code, and Rico is still listed as a contributor. Unfortunately, this
>>> native ChessX format had a number of issues. The biggest one was that
>>> it used multiple files to encode a single db. This is the exact same
>>> problem with ChessBase databases. IMHO a format more compact than PGN
>>> would be very useful for exchanging databases among users.
>> 
>> Interesting. If we had to do it again, I would recommend using something like Google's leveldb or similar and not come up with a new file storage format. One still needs to create/support a schema when using the leveldb 'format', but that is a much easier problem to solve in my mind at least. Speed, good API support (in multiple programming languages), compression, reliability, recovery etc will all be provided for free when using a popular open source key value data format. 
>> 
>> Shiv
>> 
>> 
>> ------------------------------------------------------------------------------
>> Monitor your physical, virtual and cloud infrastructure from a single
>> web console. Get in-depth insight into apps, servers, databases, vmware,
>> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
>> Pricing starts from $795 for 25 servers or applications!
>> http://p.sf.net/sfu/zoho_dev2dev_nov
>> _______________________________________________
>> chessx-developer mailing list
>> che...@li...
>> https://lists.sourceforge.net/lists/listinfo/chessx-developer
> 
> 
> ------------------------------------------------------------------------------
> Monitor your physical, virtual and cloud infrastructure from a single
> web console. Get in-depth insight into apps, servers, databases, vmware,
> SAP, cloud infrastructure, etc. Download 30-day Free Trial.
> Pricing starts from $795 for 25 servers or applications!
> http://p.sf.net/sfu/zoho_dev2dev_nov
> _______________________________________________
> chessx-developer mailing list
> che...@li...
> https://lists.sourceforge.net/lists/listinfo/chessx-developer