Menu

#24 number_display incorrect for signed EBCDIC fields

Release 4.4.6
closed
None
2015-06-09
2015-03-25
RHarris
No

In COBOL, for a DISPLAY data item, the sign is recorded in the high-order half-byte (nibble) of the last full byte of the field. The sign position contains the hexadecimal value F if the number is unsigned, the value C if the number is positive and D if the number is negative.

This code for number_display seems to work properly
EBCDIC_File.number_display:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
def number_display( buffer, attr ):
"""Extract a numeric field's value."""
last_digit = bytes( [(buffer[-1] & 0x0F) | 0xF0] )
sign = '-' if buffer[-1] >> 4 == 0xD else ''
text, size = EBCDIC_File.decoder(last_digit if len(buffer) == 1 else buffer[:-1] + last_digit)
return Character_File.number_display( sign+text, attr )

and Character_File.number_display:

def number_display( buffer, attr ):
"""Extract a numeric field's value."""
final, alpha, length, scale, precision, signed, dec_sign = attr.size_scale_precision
sign=-1 if (buffer[0] == '-' and signed) else 1
try:
display=buffer.strip()
if precision != 0 and dec_sign =='V':
display= display[:-precision]+"."+display[-precision:]
return decimal.Decimal(sign) * decimal.Decimal( display )
except Exception:
Character_File.log.debug( "Can't process {0!r} from {1!r}".format(display,buffer) )
raise
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Discussion

  • Steven F. Lott

    Steven F. Lott - 2015-05-06

    Important: The field in question is defined as a PIC S9.
    The raw EBCDIC value is C4

    The field is explicitly signed.

     
  • RHarris

    RHarris - 2015-05-06

    The above code has been tested with real data and properly handles PIC S9 with the converted data passing cross-validation.

    C4 (or D4) gets converted to F4 by last_digit

     
  • Steven F. Lott

    Steven F. Lott - 2015-05-06
    • status: open --> pending
    • assigned_to: Steven F. Lott
    • Milestone: Release 4.4 --> Release 4.4.6
     
  • Steven F. Lott

    Steven F. Lott - 2015-06-09

    Change Handling of 1-byte values.

     
  • Steven F. Lott

    Steven F. Lott - 2015-06-09
    • status: pending --> closed
     

Log in to post a comment.