[Firebird-devel] Problem ... looking for suggestions. LONG

[Ann W. H. <aha...@ib...>]

I've been looking at a problem 'til I'm cross-eyed and it's
time for some help.  My copy of Firebird and earlier versions
of both Firebird and InterBase get a "wrong length record (183)"
error on the second or third record in the RDB$FUNCTIONS table
of one particular V5.6 database.  The certified InterBase does
not, though I can't find any changes on sourceforge that might
account for the difference.  Nor does inspecting the record
suggest what might be wrong with it (other than typical harrison
muddleheadedness).

There is a database available as a zip file from ftp at
firebird.netfrastructure.com  - anonymous login, get
the filed called e1gdb.zip.  It should work on either Windows
or Linux.  It's an ODS 9 database, of course.

Here are things I would like:

   Could someone who has a built copy of InterBase Open Edition
   try to backup this database using that engine?

   Could someone who has a recent Firebird build (blush) try
   the same?

   Could someone figure out what's going on?

Regards,

Ann

The rest of this might be of interest to people thinking about
my third request.


Here's some more detailed information:

The message comes from VIO.c, but is likely to have been caused
by an error in SQZ.c.   This is the routine in VIO.

For those who don't know, a tdbb is a thread specific database
block, a pool is an area where space can be allocated for a specific
function and an rpb is a record parameter block.  Once a record has
been lifted off page, it travels around in an rpb.  Active and inactive
refer to whether the page with the record is locked.

The record in question is not a back version, or a fragmented record,
so this code should execute it's simplest path - essentially just
getting the bits and decompressing them.

void VIO_data (
     TDBB        tdbb,
     register RPB        *rpb,
     BLK                 pool)
{
/**************************************
  *
  *      V I O _ d a t a
  *
  **************************************
  *
  * Functional description
  *      Given an active record parameter block, fetch the full record.
  *
  *      This routine is called with an active RPB and exits with
  *      an INactive RPB.  Yes, Virginia, getting the data for a
  *      record means losing control of the record.  This turns out
  *      to matter a lot.
  **************************************/
FMT     format;
REC     record, prior;
ULONG   back_page;
USHORT  length, back_line;
UCHAR   *tail, *tail_end, differences [MAX_DIFFERENCES];

SET_TDBB (tdbb);


/* If we're not already set up for this format version number, find
    the format block and set up the record block.  This is a performance
    optimization. */

record = VIO_record (tdbb, rpb, NULL_PTR, pool);
format = record->rec_format;

/* If the record is a delta version, start with data from prior record. */

prior = rpb->rpb_prior;

if (prior)
     {
     tail = differences;
     tail_end = differences + sizeof (differences);
     if (prior != record)
         MOVE_FASTER (prior->rec_data, record->rec_data, format->fmt_length);
     }
else
     {
     tail = record->rec_data;
     tail_end = tail + record->rec_length;
     }

/* Set up prior record point for next version */

rpb->rpb_prior = (rpb->rpb_b_page && (rpb->rpb_flags & rpb_delta)) ? record 
: 0;

/* Snarf data from record */

tail = SQZ_decompress (rpb->rpb_address, rpb->rpb_length, tail, tail_end);

if (rpb->rpb_flags & rpb_incomplete)
     {
     back_page = rpb->rpb_b_page;
     back_line = rpb->rpb_b_line;
     while (rpb->rpb_flags & rpb_incomplete)
         {
         DPM_fetch_fragment (tdbb, rpb, LCK_read);
         tail = SQZ_decompress (rpb->rpb_address, rpb->rpb_length, tail, 
tail_end);
         }
     rpb->rpb_b_page = back_page;
     rpb->rpb_b_line = back_line;
     }

/* unlock the page */

CCH_RELEASE (tdbb, &rpb->rpb_window);

/* If this is a delta version, apply changes */

if (prior)
     length = SQZ_apply_differences (record, differences, tail);
else
     length = tail - record->rec_data;

if (format->fmt_length != length)
     BUGCHECK (183); /* msg 183 wrong record length */

rpb->rpb_address = record->rec_data;
rpb->rpb_length = format->fmt_length;
}


The problem, more than likely, is in SQZ - squeeze.  Here's
SQZ decompress.  As you know, compression is a run length
encoding using a byte length.  The values from 1 - 127 indicate
that the bytes that follow are to be copied directly to the
output.  The values from -3 to -128 indicate that the byte
that follows is to be replicated (-1 times )that many times.


SCHAR *SQZ_decompress (
     SCHAR       *input,
     USHORT      length,
     SCHAR       *output,
     SCHAR       *output_end)
{
/**************************************
  *
  *      S Q Z _ d e c o m p r e s s
  *
  **************************************
  *
  * Functional description
  *      Decompress a compressed string into a buffer.  Return the address
  *      where the output stopped.
  *
  **************************************/
SCHAR   *last, c;
SSHORT  l;

last = input + length;

while (input < last)
     {
     l = *input++;
     if (l < 0)
         {
         c = *input++;

         if ((output -l) > output_end)
                 BUGCHECK (179); /* msg 179 decompression overran buffer */
         memset (output, (UCHAR) c, (-1*l) );
         output -= l;
         }
     else
         {
         if ((output + l) > output_end)
                 BUGCHECK (179); /* msg 179 decompression overran buffer */
         MOVE_FAST(input,output,l);
         output +=l;
         input +=l;
         }
     }

if (output > output_end)
     BUGCHECK (179); /* msg 179 decompression overran buffer */

return output;
}

And, as if this weren't enough, here is a dump of the page with the
error.  I believe that the problem is with the second record on page
(1) but it may be with the third (2).  My dump program may have problems...
who doesn't?

214     DATA PAGE        checksum 12345  generation 1
         Relation: 14,   Sequence: 0, Count: 26,
          Flags:

         0 - offset: 4016, length: 79, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 66)
         01 80 fd 00 0e F_DEFAULTFLOATef  fd 00 e1  f5 00 10 0e 00 Fre
         eUDFLib.dll80 00 91 00 0d Default_Floatee  fc 00

         Decompressed format: (length 370)
         80 00 00 00 F_DEFAULTFLOAT                 00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 0e 00
         FreeUDFLib.dll00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 Default_Float
                  00 00 00 00

         1 - offset: 3924, length: 90, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 77)
         01 80 fd 00 14 F_AGEINDAYSTHRESHOLDf5  fd 00 e1  f5 00 10 0e
         00 FreeUDFLib.dll80 00 91 00 12 AgeInDaysThresholdf3  fc 00

         Decompressed format: (length 370)
         80 00 00 00 F_AGEINDAYSTHRESHOLD           00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 0e 00
         FreeUDFLib.dll00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 AgeInDaysThreshold
                  00 00 00 00

         2 - offset: 3860, length: 61, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 48)
         01 80 fd 00 05 FLOORe6  fd 00 e1  f5 00 08 06 00 ib_udf80 00
         89 00 0c IB_UDF_floored  fc 00

         Decompressed format: (length 370)
         80 00 00 00 FLOOR                          00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 06 00
         ib_udf00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 IB_UDF_
         floor                   00 00 00 00


And finally, here's a dump of the same page, created by the V6 engine,
which works perfectly.

214     DATA PAGE        checksum 12345  generation 1
         Relation: 14,   Sequence: 0, Count: 26,
          Flags:

         0 - offset: 4016, length: 79, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 66)
         01 80 fd 00 0e F_DEFAULTFLOATef  fd 00 e1  f5 00 10 0e 00 Fre
         eUDFLib.dll80 00 91 00 0d Default_Floatee  fc 00

         Decompressed format: (length 370)
         80 00 00 00 F_DEFAULTFLOAT                 00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 0e 00
         FreeUDFLib.dll00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 Default_Float
                  00 00 00 00

         1 - offset: 3924, length: 90, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 77)
         01 80 fd 00 14 F_AGEINDAYSTHRESHOLDf5  fd 00 e1  f5 00 10 0e
         00 FreeUDFLib.dll80 00 91 00 12 AgeInDaysThresholdf3  fc 00

         Decompressed format: (length 370)
         80 00 00 00 F_AGEINDAYSTHRESHOLD           00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 0e 00
         FreeUDFLib.dll00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 AgeInDaysThreshold
                  00 00 00 00

         2 - offset: 3860, length: 61, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 48)
         01 80 fd 00 05 FLOORe6  fd 00 e1  f5 00 08 06 00 ib_udf80 00
         89 00 0c IB_UDF_floored  fc 00

         Decompressed format: (length 370)
         80 00 00 00 FLOOR                          00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 06 00
         ib_udf00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 IB_UDF_
         floor                   00 00 00 00

         3 - offset: 3788, length: 72, expanded data length: 370
         trans: 1, format: 0

         Compressed format: (length 59)
         01 80 fd 00 0a F_STRTOINTeb  fd 00 e1  f5 00 10 0e 00 FreeUDF
         Lib.dll80 00 91 00 0a Str_To_Inteb  fc 00

         Decompressed format: (length 370)
         80 00 00 00 F_STRTOINT                     00 00 00
                               00 00 00 00 00 00 00 00 00 00 00 0e 00
         FreeUDFLib.dll00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 Str_To_Int
                  00 00 00 00


Regards,

Ann
www.ibphoenix.com
We have answers.