Menu
▾
▴
[groonga-talk:330] [ANN] Groonga 6.0.8
|
From: Kentaro H. <ha...@cl...> - 2016-08-29 03:05:40
|
Groonga 6.0.8 has been released! http://groonga.org/docs/news.html#release-6-0-8 Install: http://groonga.org/docs/install.html Charcteristics: http://groonga.org/docs/characteristic.html The topics in this release: * Limitation about the maximum number record is relaxed * Supported valid lexicon check on setting index sources * Changed to report error when column value set is failed #### Limitation about the maximum number record is relaxed A table had the following limitations. * The maximum one key size: 4KiB * The maximum total size of keys: 4GiB * The maximum number of records: 268,435,455 (more than 268 million) In the past, the maximum number of records is announced as 268 million. With recent research, it turns out that there is a capability to store more records than ever. Here is the updated limitation about the maximum number of records: * TABLE_NO_KEY: 1,073,741,815 (2^30 - 9) (1,073 millions) * TABLE_HASH_KEY: 536,870,912 (2^29) (536 millions) * TABLE_PAT_KEY: 1,073,741,823 (2^30 - 1) (1,073 millions) * TABLE_DAT_KEY: 268,435,455 (2^28 - 1) (268 millions) Keep in mind that these limitations may vary depending on conditions. For example, you need to use small size type for key to store many records. Because the maximum total size of keys limitation is exceeded before the maximum number of records limitation is exceeded. If you use UInt64 (8byte) type and store 2^29 records, total key size is 4GiB (= 8 * (2^29)). You can't add more records. You need to choose decreasing key size (e.g. UInt32) or using KEY_LARGE and TABLE_HASH_KEY to store more records. #### Supported valid lexicon check on setting index sources In this release, invalid lexicon is treated as an error. Let's try sample schema which contains an invalid case. table_create Tags TABLE_HASH_KEY ShortText table_create Sites TABLE_HASH_KEY ShortText column_create Sites tags COLUMN_VECTOR Tags table_create TagsIndex TABLE_HASH_KEY ShortText column_create TagsIndex sites_tags COLUMN_INDEX Sites tags In the previous versions of Groonga, there is no error report. $ groonga -n testdb/db < sample.grn [[0,1472192178.94475,0.006346225738525391],true] [[0,1472192178.951148,0.006434917449951172],true] [[0,1472192178.957618,0.009174346923828125],true] [[0,1472192178.966841,0.006294727325439453],true] [[0,1472192178.973179,0.01138973236083984],true] With Groonga 6.0.8, valid lexicon check reports an invalid error. $ groonga -n testdb/db < sample.grn [[0,1472192179.028527,0.00267481803894043],true] [[0,1472192179.031259,0.002692937850952148],true] [[0,1472192179.033985,0.004066228866577148],true] [[0,1472192179.03809,0.002942562103271484],true] [[-22,1472192179.041083,0.01088976860046387, "[column][index][source] index table must equal to source type: <TagsIndex> -> <Tags>: index-column:<TagsIndex.sites_tags> sourc",[["grn_obj_set_info_source_invalid_lexicon_error","db.c",8418]]],false] #### Changed to report error when column value set is failed In this release, when column value set is failed, it is reported as error. Here is the sample data and schema. age column stores the value of Int32. but it try to load string value. table_create Users TABLE_NO_KEY column_create Users age COLUMN_SCALAR Int32 load --table Users [ {"age": "invalid number!"} ] In the previous versions, there is no error. $ groonga testdb/db < invalid.test [[0,1472195681.942045,0.006659030914306641],true] [[0,1472195681.948765,0.007688283920288086],true] [[0,1472195681.956529,0.002389430999755859],1] [[0,1472195681.958963,0.0009322166442871094],[[[1],[["_id","UInt32"],["age","Int32"]],[1,0]]]] With Groonga 6.0.8, it fails to load. $ groonga -n testdb/db < invalid.test [[0,1472195948.997128,0.005559444427490234],true] [[0,1472195949.002747,0.007435083389282227],true] [[-22,1472195949.01023,0.000843048095703125, "<Users.age>: failed to cast to <Int32>: <\"invalid number!\">", [["grn_obj_set_value_column_fix_size","db.c",7520]]],1] [[0,1472195949.011117,0.000377655029296875],[[[1],[["_id","UInt32"],["age","Int32"]],[1,0]]]] You can notice mismatch of data types when loading. ## Improvements * [object_list] Supported to show more properties such as value_size and n_elements in metadata. * Supported operator per selector. This change enables to choose correct index for selector. It means that between() chooses index for range search, in_values() chooses index for equality comparison. [GitHub#589] [Reported by Naoya Murakami] * [debian] Changed to use nginx log reopen feature instead of Groonga's [log_reopen] command because log_reopen command works only for one worker. On the contrast, nginx log reopen feature works for multiple workers. * [table_copy] Added table_copy command which copies specified table. * [column_copy] Supported to cast A table record to B table record. It fixes a case that column_copy failure in the previous version. Note that both tables must support key. * [column_copy] Supported reference type vector. * [admin] Supported no response error case. It fixes the problem that "Loading..." message will remain displayed. * [groonga executable file][http] Supported to return 400 Bad Request against not implemented function. * [groonga-httpd] Supported to return body on failure. * [groonga-httpd] Supported to load large data as stream. * Supported sequential search by _key when it is specified as a source of index column. * Supported to report index information on resolving by accessor for data column if log level is equal to info. * Added valid lexicon check on setting index sources. If users create wrong index for sources by [column_create], this check shows details. * [Limitations] Updated description about table limitations. ## Fixes * [column_create] Fixed a bug that buffer overflow occurs on logging. * Fixed to output response even when critical level error. * Fixed to ensure clearing output buffer for each grn_ctx_send. This change solves sometimes response may broken. [GitHub#330] * [fuzzy_search] Fixed a bug that prefix_match_size option returns wrong node. It causes a problem that fuzzy_search returns wrong edit distance. [GitHub#590] [Patch by Naoya Murakami] * [load] Changed to report error when column value set is failed. It enables you to notice mismatch between type of column and actual value. * [groonga-httpd] Fixed a bug that wrong HTTP status is set on success. * [fuzzy_search][in_values] Fixed to resolve record id correctly on sequential search. [GitHub#591,#592,#593] [Patch by Naoya Murakami] ## Thanks * Naoya Murakami -- Kentaro Hayashi <ha...@cl...> |
