404040 and 4040404 Junk Pattern skip

[Govindu Narendr]

Dear Bruce Martin,
Need your help to understand when there is junk data which is like 40404 and 404040 pattern we need to catch this data and through the nulls , could you please help to understand where can we catch this .

Regards
Naren

[Bruce Martin]

x'40' is the space char in Ebcdic; so

if (! "".equals(line).getFieldValue("field-name").asString())

should work in this case. In general there will probably be a field-value that tells you wether a field is used or not. Once you understand how the copybook works you may not need to do space checks.

There are also isLowValues() and isHighValues() tests.

---

While many cobol Copybooks are simple and straightforward. This is not always the case (particularly for old systems). For some Cobol Copybooks you need to know the Rules
associated with the copybook to understand what is going on. Unfortunately the Cobol
itself does not tell you this. In a lot of case an experienced Cobol programmer could look at the copybook and tell you what is going on. But in others in falls in General knowledge.

you may have

     .....
     05 Client-type.
     05 Standard-Client
           07 ...
            .....
     05 Reinsured-client.

Unfortunately it is more likely to be

    05 Client-type.
     05 redf00101-blah-blah-St-Clt.
           07 ...
            .....
     05 redf00101-blah-blah-Ri-clt.

in which case you need to know the abreviations used

[Govindu Narendr]

public static String getDecimal(final byte[] record, final int start, final int fin) {
method ret field giving 4040404040404c,0c40404040 and 404040404040 patterns are coming instead blanks those are actually junk values.PLease help.

[Bruce Martin]

By the sounds of it, the field is not being initialised and it is picking up
whatever was there before the record was created (not really the correct term for cobol).

You really need to talk to the Cobol programmers about the field. They should
know when the field has valid data (they may have look at the program though).
The cobol programmers would face a similar problem; they would know when to access
the field and when not to.

There will be some way to determine if the field has valid data or is junk.
Check the flags / 88 levels in the Cobol Copybook. One thing they will not be
doing in Cobol is validating the field.

The other alternative would be update the Cobol program and initialise the
record/field to hex zero's. This would work if there are no redefines.

If you show me the Copybook / file, I might be able to figure it out
(if the file is reasonably simple).

[Govindu Narendr]

Hi Bruce,space byte is 64 and 40 number also byte 64 is coming because of that we are not able to catch space byte and if we catch we are loosing values which has 40 .please help.
Regards
Naren

[Bruce Martin]

I do not understand the last comment. Can you expand on a bit / give me a Cobol picture / raw data

Note: In ebcdic Space=x'40'= decimal 64.

[Saif]

Hi Bruce,

I am tagging to this thread as I have a question regarding the 404040 pattern.

The sytem that I am getting the data from sometimes does not initialise the comp-3 field resulting in spaces. So, when I read them through JRecord, I get 40404 which as you indicated is space char in Ebcdic

PIC 9V9(3) COMP-3. = 404.04
PIC 9(8) COMP-3. = 4040404040
PIC 9(8) COMP-3. = 4040404040
PIC S9(7)V99 COMP-3. = 40404040.4
PIC V9(3) COMP-3. = 4.04

Unfortunately, I do not have control over the cobol side of things nor can I get anyone to fix it at their end. Is there anything I can do in the java side to take care of this issue.

[Bruce Martin]

At the moment try using the asHex() method is probably as easy as anything;

 if (isSpaces(line.getFieldValue(...)) { ... }

 boolean isSpaces(IFieldValue fv) {
     char c = fv.asHex().toCharArray();

     for (int i =0; i < c.length; i += 2) {
             if (c[i] != '4' || c[i+1] != '0') {
                 return false;
             }
     }
     return true;
 }

alternatively casting back to line could be used:

if (line instanceof Line) {
     byte[] bytes = ((Line) line).getFieldBytes(...);

     for (byte b : bytes) {
         if (b != 64 /* x'40' */) { return false; }
     }
     return true;
}
throw error

not very elagant but it should work, I will update JRecord with a spaces test but that that will not be released for a while

[Bruce Martin]

I have updated JRecord Source in subversion. This update includes

• A lot of updated (mostly related to RecordEditor / ReCsvEditor)
• Add isSpaces test to IFieldValue interface.

I am not in a possition to do a JRecord release at the moment. If you want to apply my changes to you JRecord Code, the relavent changes are in

• IFieldValue
• FieldValue
• FieldValueLine

The Tests are:

• TestFieldValue
• TestFieldValueLine

[Saif]

Hi Bruce,

Thank you very much ..For now, I have added your initial solution and hopefully that should solve my problem.

 private boolean isSpaces(IFieldValue fv) {
     char[] c = fv.asHex().toCharArray();

     for (int i =0; i < c.length; i += 2) {
             if (c[i] != '4' || c[i+1] != '0') {
                 return false;
             }
     }
     return true;
 }
 --------------------while defining columns
    switch(fieldDetail.getType()) {
        case 31 :
        case 33 : 
            ColObj.setPacked(true);
    }   
---------------------while reading values
    if(ColObj.isPacked())
        if(isSpaces(fieldValue))
            val ="";

[Saif]

Hi Bruce,

Sorry to tag this question in the same thread.

Some data from source (Cobol) has invalid data with NUL as part of the string. Find attched capture.jpg. This causes a problem when I try to insert the data in the downstream system.

Is there anything I can do in the java side to take care of this issue. Thank you

[Bruce Martin]

I presume it is lowvalue (x'00') in which case you can use the isLowValues() test:

     IFieldValue fvMyField = line.getFieldValue("My-Field);
     if (fvMyField.isLowValues) {
          // process invalid field
     } else {
          // process the field
     }