Search didn't work for multi-byte character title/description

[yeahy]

When search for multi-byte characters title/description, always get "No results found for xxx".

Existing photos title:
äüö.jpg,
大.jpg

Search string 1: äüö
Search result: No results found for äüö

Search string 2: 大
Search result: No results found for 大

[jankoprowski]

I check by my own. I can't find any solution anywhere:
http://forums.mysql.com/read.php?107,277354,277354#msg-277354
http://forum.php.pl/MySQL_Znaki__and22823_w_zapytaniu_match_against_t128413.html

[andy_st]

Hi jankoprowski,

I think MySQL fulltext search should work with multi-byte Unicode characters just fine, at least with MySQL 5 and later versions.
It should certainly work with Cyrillic, and there might (still) be some problems with Han characters (Chinese, Japanese, Korean, ...), where MySQL at least in prior version had problems because it wouldn't tokenize words correctly.

Please have a look at some of the discussion comments at:
http://dev.mysql.com/doc/refman/5.1/en/fulltext-search.html

It'd be great if you could experiment a bit with your G3 / MySQL installation to find out what parameters are needed to make it work and we can then try to configure most of it at installation / runtime, and document the rest.

Generally, I'd look in 3 places:

• MySQL server settings, i.e. there are fulltext (ft) settings which can be configured in your MySQL settings file. A restart of the MySQL server is necessary when changing these. And you might have to rebuild your fulltext index as well after changing these settings.
• MySQL connection settings. When gallery 3 connects to the MySQL server, the connection has some default values for character encoding and the like. After creating the connection, the connection can be configured. The character encoding should be utf8.
• MySQL table column collation. We need utf8_unicode_ci columns for item title, description, etc.

Please investigate and experiment the issue. I'm looking forward to hearing back from you such that we can ensure that search in Gallery 3 works just fine with multi-byte characters.

[jankoprowski]

Ok :) I try... But I'am really trying (a lot) and I still get nothing. I try on two other Unix system (until now i check this on ma xampp installation).

Greetings from Poland !

[jankoprowski]

Fulltext searching for given characters start working after set ft_min_word_len to 1.

SHOW VARIABLES LIKE 'ft_min_word_len';
ft_min_word_len 1

We set this variable adding in section [mysqld] to my.cnf

[mysqld]
ft_min_word_len=1

and restarting the server. My other configuration around this problem:

my.cnf
[mysqld]
character-set-server=utf8
collation-server=utf8_general_ci

SHOW VARIABLES LIKE "character_set%";
character_set_client	utf8
character_set_connection	utf8
character_set_database	utf8
character_set_filesystem	binary
character_set_results	utf8
character_set_server	utf8
character_set_system	utf8

But I think the most significant in this case is ft_min_word_len value.
One more think. After change this value we must reindex table. For example by:

REPAIR TABLE table_name QUICK;

[andy_st]

I'd imagine ft_min_word_len=1 is quite expensive (space and time).

It looks like ft_min_word_len is the number of UTF-8 code points (~ characters).

Is there a way to tell MySQL that ft_min_word_len should be counting in bytes, not characters? That way, we could set it to 3 and avoid indexing Latin script words of length 1-2.

Either way, I guess we should add your recommendations to Gallery 3's documentation. It's a MySQL configuration setting which we should recommend for users with CJK content.

[jankoprowski]

I can't find any additional informations about another solution. Recommendation in documentation or even some kind of warning/information in database installator section sounds good. I found one workaround propositionŁ

http://bytes.com/topic/mysql/answers/77599-problem-ft_min_word_len

but I don't know is this worth to implement. Especially at this stage of project.

[dentizm]

This is a great inspiring article.I am pretty much pleased with your good work.mantolama,
dış cephe mantolama,mantolama malzemeleri,yalıtım,ısı yalıtımı,çatı,
çatı tadilatı,izolasyon,mantolama fiyatları,ısı yalıtım malzemeleri, You put really very helpful information. best regards.