Re: [CLucene-dev] help ! - using LanguageBasedAnalyzer/CJKTokenizer returns wrong result

[Veit J. <nun...@go...>]

Hi Vivien,

sorry, it took some time. Maybe this is an encoding issue. What
encoding do your source code files have? I got the same result, when
the encoding is UTF-8. But CLucene works internally with UTF-16 and in
this case handles the string and the query as an UTF-16 encoded
string. Thus, the created index is more or less useless.

But if I transcode the string from UTF-8 to UTF-16 before I put it
into the index, the index looks fine
and I get the correct number of results.

Kind regards,

Veit