Re: [Firebird-net-provider] Re: Character set support

[<car...@te...>]

Hello:

( i have encoded the email with UTF-8 for better representation of 
special characters )

> Thanks very much for your sample, i'm have done a little C# app
> with the same steps :), give me some time for test it (i can send it 
> to you if you want).

I get the same results as with ISQL sample, but some comments.

1. Select with Win1251 test.

Field C_WIN1251 description sent by the server:

    sqlType     453
    sqlScale    0
    sqlSubType  52      <--- Character set WIN1251
    sqlLen      4     
    sqlInd      0
    sqlName     "C_WIN1251"
    relName     "T1"
    ownerName   "SYSDBA"
    aliasName   "C_WIN1251"

Field C_UTF8 description sent by the server:

    sqlType     453
    sqlScale    0
    sqlSubType  3       <--- Character set UNICODE_FSS
    sqlLen      12      <--- Gives 12 bytes but data is being sent using 
WIN1251
    sqlInd      0
    sqlName     "C_UTF8"
    relName     "T1"
    ownerName   "SYSDBA"
    aliasName   "C_UTF8"

Data for fields is being sent using the connection caracter set:

Bytes for "эле "

    C_WIN1251 = {253, 235, 229, 32}
    C_UTF8    = {235, 235, 229, 32, 32, 32, 32, 32, 32, 32, 32, 32}

String representation:

    C_WIN1251 = "эле "
    C_UTF8    = "эле         "


2. Select with None test ( this is going to use ISO8859_1 that is my 
default caracter
set in .NET ).

Field C_WIN1251 description sent by the server:

    sqlType     453
    sqlScale    0
    sqlSubType  52     <--- Character set WIN1251
    sqlLen      4     
    sqlInd      0
    sqlName     "C_WIN1251"
    relName     "T1"
    ownerName   "SYSDBA"
    aliasName   "C_WIN1251"

Field C_UTF8 description sent by the server:

    sqlType     453
    sqlScale    0
    sqlSubType  3       <--- Character set UNICODE_FSS
    sqlLen      12
    sqlInd      0
    sqlName     "C_UTF8"
    relName     "T1"
    ownerName   "SYSDBA"
    aliasName   "C_UTF8"

Data for fields is being sent using the connection caracter set:

Bytes for "эле ":

    C_WIN1251 = {253, 235, 229, 32}
    C_UTF8    = {209, 141, 208, 187, 208, 181, 32, 32, 32, 32, 32, 32}

String representation:

    C_WIN1251 = "эле "
    C_UTF8    = "эле      "
               
In this case seems to be using the character set of each field, i have 
added a change into the .net data provider for encode strings with field 
character set when NONE is used as default character set, for being sure 
about this issue.

3. Select with UnicodeFss test

This test fails.





After view the results of the test i have made some changes into the 
.NET provider source code
(at this moment only in my local tree)

1. GdsInetReader.cs

Encode strings with field character set when NONE is used as default 
character set.

2. GdsField.cs

New property CharCount that will be used to retrieve the length in 
characters of character fields instead of use SqlLen that is the length 
in bytes.

3. GdsInetReader.cs

If the length of the readed data (in characters) is greater than the 
CharCount of the field, return only CharCount characters ( this will be 
made using the Charset information for the current field ).

4. FbDataReader.cs:

In the GetSchema method save in the ColumnSize field the real number of 
characters for character fields, instead of the length in bytes.



A thing i have noticed ( but i'm not sure if it's a problem or not ) is 
that in charsets.h there are two charsets defined that doesn't appear in 
the RDB$CHARACTER_SETS system table:

#define   CS_JIS_0208            7    /* JIS 0208; 1990   */
#define   CS_UNICODE_UCS2        8    /* UNICODE v 1.10   */



Thanks very much for your test case, very helpful :D

Comments are wellcome :D




--
Best regards

Carlos Guzmán Álvarez
Vigo-Spain