#174 MySQL Embedded server arguments encoding

[WhiteWind]

Because of this line
https://sourceforge.net/p/zeoslib/code-0/HEAD/tree/tags/7.2.1-rc/src/plain/ZPlainMySqlDriver.pas#l744
it's impossible to use non-ascii characters in --datadir server argument

[marsupilami79]

Hello WhiteWind,

I checked the MySQL 5.7 documentation for the C API - which is what the Zeos Source code refers to. I couldn't find any information regarding on how to use Unicode with this API - for example - if it always expects UTF8 - which it surely doesn't.

The function mysql_server_init is declared as:

int mysql_server_init(int argc, char **argv, char **groups)

Everything is declared as Char here which under Windows usually means that we have to convert everything to the ANSI codepage. Do you know any different information or information on how to use UTF8 with that API?

With best regards,

Jan

[WhiteWind]

There is function mysql_set_character_set, but it requires established connection.
SInce mysql_library_init(mysql_server_init) is the very first function to be called, I suppose there is no way to change encoding for this function.

More than that, this function expects the same argv as main function. In windows, main receives ANSI argv, and wmain receives unicode argv. Mysql library doesn't nave unicode version of mysql_library_init, so it should expect ANSI charset.

[marsupilami79]

So it seems that this is a MySQL / MariaDB API limitation. One option might be to go to the MySQL forums and ask there if there is a possibility to get the embedded server to use UTF8 for this call. Also I am not sure wether it is possible for a program to somehow change its ANSI Codepage to UTF8 which probably also would solve this problem if it is done before loading the MySQL embedded server. I have to admit that I am not sure how Delphis RTL and other libraries would react to that kind of tweak - if it even is possible...

[WhiteWind]

I do not think that ANSI variants of WinAPI functions are able to handle UTF-8 at all.
So the fix is straightforward: replace UTF8String() with AnsiString(). I've applied this change and it works.

[marsupilami79]

That makes sense. I will push that to the source code. I raised the question in the MySQL forums nonetheless if there is some way to tell libmysql and libmysqld that argv is UFT8 encoded...

[marsupilami79]

Closing this bug because the only way to transport information seems to be to use ANSI strings. On Windows this totally blocks the usage of UTF8. On other Systems your mileage might vary...

[WhiteWind]

Sorry, I have not tested this fix properly.
It appears that at least in Delphi XE7, PAnsiChar(AnsiString()) produces utf8 encoded string.
Proper fix should be:

//var S: AnsiString;
for i := 0 to ServerArgsLen - 1 do
begin
  S := AnsiString(TmpList[i]);
  ServerArgs[i] := {$IFDEF WITH_STRNEW_DEPRECATED}AnsiStrings.{$ENDIF}StrNew(PAnsiChar(S));
end;

And you can remove the IFDEFs and code for non-unicode versions, since this code is backwards-compatible

[marsupilami79]

Hello WhiteWind,

I hope you still foolow on to this topic. Honestly I seem to have different results. The following procedure will give me a 1 as a result which is totally expected since my system uses WIN1252 as its ANSI codepage:

procedure TForm1.Button3Click(Sender: TObject);
var
  X: String;
  Y: RawByteString;
begin
  X := 'ö';
  Y := AnsiString(X);
  ShowMessage(IntToStr(Length(Y)));
end;

whereas the following gives a 2 - as I would expect for UTF8:

procedure TForm1.Button3Click(Sender: TObject);
var
  X: String;
  Y: RawByteString;
begin
  X := 'ö';
  Y := UTF8Encode(X);
  ShowMessage(IntToStr(Length(Y)));
end;

What do you think?

Jan

[WhiteWind]

Well, I can not reproduce my issue. It probably was some Delphi bug.
AnsiString(S) implicitly calls procedure _LStrFromUStr(var Dest: _AnsiStr; const Source: UnicodeString; CodePage: Word);
And in case of StrNew(PAnsiChar(AnsiString(TmpList[i]))) CodePage was $FDE9 (Utf8), but now it's $0 (Default System encoding).
Let's close this issue again

[marsupilami79]

Ok -> closed.