#16 ODO SetSizeAndOffset

[RHarris]

When parsing ODO defined dataset a handful of records (most parse fine) cause the exception:
/stingray/cobol/defs.py", line 1236, in setSizeAndOffset
aDDE.totalSize = aDDE.size * aDDE.occurs.number(aRow)
/stingray/cobol/defs.py", line 779, in number
value= aRow.cell( self.attr ).to_int()
/stingray/cell.py", line 367, in to_int
raise ValueError( self.value )
ValueError: None

catching/handling the exception in value= aRow.cell( self.attr ).to_int() in defs.py:OccursDependingOn with
except:
print('EXCEPTION: {!s}'.format(aRow.cell( self.attr ).raw))
value= 0

Shows that each time the exception is triggered, the value is b'\x00'

root cause of the problem is undetermined, catching/handling it in this manner might simply be obscuring the underlying issue, however there is some issue here that needs to be handled/resolved.

[Steven F. Lott]

What's the ODO definition? And the definition on which it depends? An OCCURS DEPENDING ON depends on a field which doesn't have a valid value, that independent value is creating ErrorCell.

Improved logging of ErrorCell creation might be helpful.

[RHarris]

The section that is throwing the exception (occasionally) is:
018500 04 CHD-NO-DEL-ITEMS PIC S9(4)V COMP.
...
0260000 03 CHD-DELINQUENT-ITEMS.
260100 04 CHD-DEL-ITEM OCCURS 1 TO 180 TIMES
260200 DEPENDING ON CHD-NO-DEL-ITEMS.
260300 05 CHD-DEL-INDEX PIC 9(4) COMP.
260400 05 CHD-DEL-AMT PIC S9(15)V99 COMP-3.

changed statement to:
print('EXCEPTION: {!s} {!s} {!s} {!s}'.format(aRow.cell( self.attr ).raw, self.name, self.limit, self.refers_to, self.attr))

EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.
EXCEPTION: b'\x00' CHD-NO-DEL-ITEMS 180 04 CHD-NO-DEL-ITEMS PIC S9(4)V USAGE COMP.

that's the total number of records with issues after processing a few thousand records.

Awaiting additional clarification on potential cause of this condition, but it apparently exists in the real world.

[Steven F. Lott]

http://www-01.ibm.com/support/docview.wss?uid=swg1PI07821

"The behavior is undefined if the value of the object is
outside of the range integer-1 through integer-2"

The official rules are that there are no official rules.

I suspect that this happens when the CHD-DELINQUENT-ITEMS is irrelevant for those 24 records. Perhaps they're "headers" or "trailers" or in some way another subclass that doesn't meaningfully have any items.

The fix, it appears, is to be more graceful about handling this and avoiding the ValueError exception in the first place.

[Steven F. Lott]

The exception shows a single byte value (b'\x00') for a two-byte COMP field. How is this possible?

Answer.

rstrip() in COBOL_File.row_get(). While it seems like stripping trailing spaces is a good idea, it only works int he limited case of handling DISPLAY data, and even then it's inappropriate because the COBOL_File class doesn't have that kind of responsibility.

[RHarris]

After limited testing of the current code and the issue appears to be resolved.