#114 Truncate system field value with multibyte character set on Firebird

[Tsutomu Hayashi]

Hi all

I got rdb$relations from firebird server 1.0/1.5/2.0/2.1/2.5 then rdb$relation_name was truncated about 1/character set byte nums.

Ex. I connected sjis_0208 database with ZConnection codepage=unicode_fss, I get rdb$relation_name max 10 character. "RDB$INDEX_SEGMENTS" returns "RDB$INDEX_".

This is deadly bug, in ZeosLib for multibyte country developers.

I guess this truncation is caused by mis handling character set byte length.

best regards,

[EgonHugeist]

Hi,

did you test this with latest version of trunk or \testing-7.2 from SVN?
There was already a ticket: https://sourceforge.net/p/zeoslib/tickets/97/

Don't know if this is related? ):

Michael

[Tsutomu Hayashi]

Hi Michael

Thnks for your reply!

I tested snapshot build, but this problem is not resolved.

I attached simple test program with Delphi XE4.

Simply, open ZQuery1, displayed grid and check rdb$relation_name.

Originaly "rdb$index_segments" is displayed "rdb$index_". This problem will solve to create field component using field list.

But, using system table without created field component, user missing real field name from "rdb$relation_name". This problem is affected all system tables.

So system tables are using unicode_fss we know, but, rdb$fields.rdb$character_lenght is set to null. You must specify field on user tables and on system tables.

best regards,

[EgonHugeist]

Since i'm not on a MBCS encoded system (win1252) i think i can't test your app. In addition i quickly checked the Zip and the db is missing ):

A second advice may help:
It might be a complete different issue with the Display-Width of the fields..

Could you test to enable the option:

TZDataSet.Options := [doNoAlignDisplayWidth]

Michael

[Tsutomu Hayashi]

Hi Micahel

Happy new year!

Sorry, I can't attache my database. This is about system tables then you can test any databases you created.

BTW

I check these confusions. Now I am using 7.1.2-stable, Interbase6ResultSet written bellow.

ColumnDisplaySize := MaxLenghtBytes div ConSettings.ClientCodePage^.CharWidth;

Now, 7.2.0-testing is fixed below

ColumnDisplaySize := MaxLenghtBytes div ConSettings.ClientCodePage^.CharWidth;

But but, ColumnDisplaySize( = Column char size) := MaxLenghtBytes(= RDB$CHARACTER_LENGTH) div Column.RDB$BYTES_PER_CHARACTER)

This is first confusion. You must use ColumnRDB$BYTES_PER_CHARACTER instead of ConSettings.ClientCodePage^.CharWidth.

A field is set by default character set so it is sjis_0208, but another field is set by ASCII, if so.

Second,

The system tables starting with RDB$... are created by UNICODE_FSS character set.

But but, their any field have no RDB$CHARACTER_LENGTH. These RDB$CHARACTER_LENGHT is set by NULL. I don't know what reason.

So, system tables have real length in their RDB$FIELD_LENGTH. These are not divided by ConSettings.ClientCodePage^.CharWidth.

Conclusion,

You must fix this issue for getting metadata, to use RDB$CHARACTER_LENGTH instead of RDB$FIELD_LENGTH, but if RDB$CHARACTER_LENGTH is NULL then you must use RDB$FIELD_LENGHT so on.

Not use divide with ClientCodePage^.CharWidth.

best, regards,

[Tsutomu Hayashi]

See this,

https://sourceforge.net/p/firebird/mailman/message/28611193/

http://tracker.firebirdsql.org/browse/JDBC-220

[EgonHugeist]

Happy new year!

Sure you're on 7.2?

Current DisplayWdth alignment after fixing #97 looks like:

    if FieldSqlType in [stString, stUnicodeString] then
    begin
      MaxLenghtBytes := FIZSQLDA.GetIbSqlLen(I);
      CP := GetIbSqlSubType(I);
      if (CP = ConSettings^.ClientCodePage^.ID) or //avoid the loops if we allready have the info's we need
         (CP > High(FCodePageArray)) then //spezial case for collations like PXW_INTL850 which are nowhere to find in docs
        //see test Bug#886194, we retrieve 565 as CP...
        ZCodePageInfo := ConSettings^.ClientCodePage
      else
        //see: http://sourceforge.net/p/zeoslib/tickets/97/
        ZCodePageInfo := FPlainDriver.ValidateCharEncoding(CP); //get column CodePage info
      ColumnCodePage := ZCodePageInfo^.CP;
      Precision := GetFieldSize(ColumnType, ConSettings, MaxLenghtBytes,
        ZCodePageInfo^.CharWidth, @ColumnDisplaySize, True);
    end
    else
      if FieldSqlType in [stAsciiStream, stUnicodeStream] then
        ColumnCodePage := ConSettings^.ClientCodePage^.CP
      else
      begin
        ColumnCodePage := zCP_NONE;
        if FieldSQLType = stBytes then
        begin
          MaxLenghtBytes := FIZSQLDA.GetIbSqlLen(I);
          Precision := MaxLenghtBytes;
        end
        else
          Signed := FieldSqlType in [stShort, stSmall, stInteger, stLong];
      end;

To my knowledge i don't need the metainformations: Opening a connection with a different CS, FB returns the global CS for all fields except CS_NONE(my findings) or am i wrong?

Note i have no XE4 so open your test-application fails. ): Could you use Lazarus or a older delphi?

Michael

[Tsutomu Hayashi]

Hi Michael

Firebird can have one or more character set in a database. For example,

Database default character set is SJIS_0208,

create table t1(c1 varchar(10) character set ascii, c2 varchar(10));

Table t1 created and character set of column c1 is ascii, but character set of column c2 is sjis_0208.

Then Firebird returns convert to client chatacter set bind isc_dpb_lc_ctype.

if I set lc_ctype=sjis_0208, then Firebird returns c1 ASCII --> UTF8 --> SJIS_0208, c2 SJIS_0208 --> UTF8 --> SJIS0208. This mechanism implemented from version 2.0.

Before version 1.5 not implemented temporary toranslation to UTF8, c1 ASCII --> SJIS_0208, c2 SJIS_0208 --> SJIS_0208(no translation).

BTW

I talked about your getting metadata informations. Can you look system table RDB$FIELDS, and check the record RDB$FIELD_NAME is RDB$FIELD_NAME, column RDB$CHARACTER_LENGTH is NULL, but RDB$CHARACTER_SET_ID is 3 and RDB$FIELD_LENGTH is 31.

But, ordinary about user defined fields, for example RDB$CHARACTER_LENGTH is 10 and RDB$CHARACTER_SET_ID is 3, then RDB$FIELD_LENGTH is 30.

So, no system tables have display width, equal character length, in own RDB$CHARACTER_LENGTH. But in system tables have no character length, then use field length as it is.

See jaybird discussion.

best,

[EgonHugeist]

On my side it works.
May i point you to:
"
Could you test to enable the option:

TZDataSet.Options := [doNoAlignDisplayWidth]
" Again?

Which doesn't mean you're wrong. Which MetaData do you mean exactly?

[EgonHugeist]

push or we'll close the ticket...

[Tsutomu Hayashi]

Sory! I tested with 7.2.0-beta, it seems working fine. Thanks!