Need help on REDEFINE copybooks

[Govindu Narendr]

Hi team,
I have a copybook with REDEFINE blocks and have binary data mainfrane file but not able to read REDEFINE fields properly.Below is my copybook.If TRANS-PROD-TRNTYP record is DR,CR,IPD need to get below fields based in TRANS-PROD-TRNTYP field.Please help ASAP.

Ex: if TRANS-PROD-TRNTYP=='CR' then
get TRANS-CREDIT-DETLS

Attached copybook and binary file for testing.Please help me to post the sample code.


[Bruce Martin]

The first thing, In the binrary file you sent me, the first 2 records are 1528 bytes long,
the third is not !!! ===> Not Fixed-Width.

I would guess the file was a VB file on the mainframe and the
file was transfered to the PC without the Record-Descriptor-Word (basically the record length).
This needs to be sorted out first. You need to retransfer the file with the
RDW (Record-Descriptor-Word (basically the record length))

With regards the testing you can do

    String transProdTrnType = line.getFieldValue("TRANS-PROD-TRNTYP").asString();

    if ("CR".equals(transProdTrnType)) {
       int trnCrDteTrn = line.getFieldValue("TRN-CR-DTE-TRN").asInt();
       int trnCrDteVal = line.getFieldValue("TRN-CR-DTE-VAL").asInt();
       long trnCrAmt = line.getFieldValue("TRN-CR-AMT").asLong();
       ....
    } else if ("DR".equals(transProdTrnType)) {
       // Get the DR fields
       int trnDrDteTrn = line.getFieldValue("TRN-DR-DTE-TRN").asInt();
       int trnDrDteVal = line.getFieldValue("TRN-DR-DTE-VAL").asInt();
       long trnDrAmt   = line.getFieldValue("TRN-DCR-AMT").asLong();
       ....
    } else if ("IPD".equals(transProdTrnType)) {
       // Get the IPD fields
       ....
    }  ....

Tedious I know but not much else to do.

Note: deleted the data file from the forum incase it had sensitive data in it.

Note: have added a Cobol-Field-Name class generated in the RecordEditor. Hopefully you will find it useful ~ you can use IDE's completion.

---

Finally I will have another look at it in the morning

[Govindu Narendr]

Thanks Bruce Martin for quick response and please help what VB file organization optino should Iuse.

[Bruce Martin]

You need to get the VB file first; when you do just set the file-organisation like

          .setFileOrganization(Constants.IO_VB)

or possibly

          .setFileOrganization(Constants.IO_VB_DUMP)

---

You could try the latest version of the RecordEditor (https://sourceforge.net/projects/record-editor/files/Test/)

In the cobol-import section, you enter a Cobol Copybook and a Cobol-Data-File; It should be able to work out the File-Structure (for VB files). There is an now an option to generate a JRecord-IOBuilder + Cool-Field-name class.

[Govindu Narendr]

We extracted binary file from Mainframe system with RWD and use same configuratioin but getting error : java.io.IOException: Invalid Record Descriptor word at line 1
at net.sf.JRecord.ByteIO.VbByteReader.read(VbByteReader.java:123)
at net.sf.JRecord.IO.LineReaderWrapper.readImplementation(LineReaderWrapper.java:68)
at net.sf.JRecord.IO.AbstractLineReader.read(AbstractLineReader.java:148)
at net.sf.JRecord.zExamples.cobol.iobuilder.AmsReceipt06.<init>(AmsReceipt06.java:78)
at net.sf.JRecord.zExamples.cobol.iobuilder.AmsReceipt06.main(AmsReceipt06.java:47)

Please help and attached java & binary files.

[Bruce Martin]

The file aqtrans pir2ub is exactly the same as the previous file aqtrans pir2u. You can see it in the file lengths - hence the same result

[Bruce Martin]

I get failure on Record 25. The first check byte was 1 (it should be 0 for VB but it can have other values for VS / VBS files).

Some one needs to check exctly what the File Attrributes are on the Mainframe.
You may well have VS / VBS (variable blocked spanned) or one of the old file Formats.

If it is one of the old formats, some one will need to:

• Copy the file to a FB/VB file
• Download the new file.

[Govindu Narendr]

The file is actually VBS and I downloaded as a VB file with below Connect:Direct script.

step1 copy from (DSN="CAST.BIP.INTF.ONLINE"
snode DISP=(SHR) DCB=(LRECL=30000,RECFM=VBS)
)
to (DSN="/FCSIT/fcsg/feeds/staging/mainframe/CAST.BKUP.ONLINE"
pnode SYSOPTS=":datatype=binary" DISP=(RPL) DCB=(LRECL=29996,RECFM=VB,BLKSIZE=30000)
)

and attached Data Set Information

[Bruce Martin]

After looking at the second file, I thought it would be VBS.

Looking at the file, it still has valid SDW's (VBS control words) instead of a RDW's.

I would suggest trying the copy and browsing the file in ISPF before transfering.

If neccessary I can look at a VBS reader.

Will check in the morning

---

For my reference:

SDW for VBS ( Variable Blocked Spanned ) Records

The four bytes preceding the logical record is the Segment Descriptor Word.

Bytes Description

1-2 Length of the logical record (plus length of the four-byte Descriptor Word)

3 Segment Control Codes
Value Relative position of segment
00 A complete logical record
01 The first segment of a multiple segments record
02 The last segment of a multiple segments record
03 A middle segment of a multiple segments record

4 Low value, ( as usual ) reserved for future use

Read more: http://ibmmainframes.com/about57510.html#ixzz46SaQJU6z

[Govindu Narendr]

Please help for VBS reader.

[Bruce Martin]

I will add a VBS reader to JRecord, but rember

1. you will probably be the only user of it - so less testing
2. I will be writing it based on the documentation I havew seen; so I may miss something

---

You should be able to copy the VBS file to a VB file using a Batch Sort job (or may be IDCAMS would be better ???) on the mainframe. This would be a safer option !!!

[Govindu Narendr]

Hi Martin,thanks for your help creating API for VBS files and I will check with check with maimframe team for second options also.

[Bruce Martin]

1. I have uploaded a version of JRecord that has a VBS IO : https://sourceforge.net/projects/jrecord/files/jrecord/Version_0.80.8i/JRecord_upgrade_22_april_2016.7z/download
2. See attached my test code, it includes a generated access class

Basically to use a VBS do:

    iobuilder = CobolIoProvider.getInstance()
            .newIOBuilder(cpybookFileName, ICopybookDialects.FMT_MAINFRAME)
                    .setFont("CP037")
                    .setFileOrganization(Constants.IO_VBS)
                    .setSplitCopybook(CopybookLoader.SPLIT_REDEFINE);

[Bruce Martin]

Found a mistake in VBS testing; I have just uploaded another:

• https://sourceforge.net/projects/jrecord/files/jrecord/Version_0.80.8i/JRecord_upgrade_22_april_2016.7z/download

so If you have downloaded the previous version, download again

[Govindu Narendr]

HI Bruce Martin, Intial testing done and working as expected and let you know once end-to-end testing completed.

Thank you so much for your help.And need to understand what is the class used for generating POJO objects by providning copy book.

[Bruce Martin]

The Program to generate the "Pojo" is available at:

• at https://sourceforge.net/projects/jrecord/files/jrecord_CodeGen/
• The source is also in JRecord at Source/OtherSource/JRecord_CodeGen/

The program basically

1. Reads a Cobol-Copybook and Creates an Extended Layout (or Schema)
2. Reads a properties file which has a list of Velocity Templates + options for the Templates
3. Generates the Templates from 2.

The "Pojo" I supplied was from a Velocity-Template I knocked up fairly quickly.
I should be able to add it to CodeGen but it still requires quite a bit of
Work.

The CodeGen has it own Pojo implementation that works slightly differently
but uses the same interfaces I supplied.
It could be used if you are carefull of its current limitations

• Single Record Files only
• Only single byte files (i.e. EBCDIC or ASCII) no double byte files (UTF-6 / 16).
• Issues with some (but not all) of generated code with Redefines.

---

Logic of Pojo's is basically

• I${RedorcName}Pojo - Interface defining Getters / Setters. There are 2 implementing classes

  • ${RedorcName}JR - Version backed by an Array of Bytes and uses JRecord code to get/set the fields
  • ${RedorcName}Pojo - Version using basic Java objects and contains no reference to JRecord.

• Code${RedorcName} - Static class that will convert a ${RedorcName}JR to/from a ${RedorcName}Pojo.
  This class would need to be modified for Redefines.

---

It just needs uninterupted time, testing and documentation.

---

I am intereseted is in finding out

• What code users are writing and what the purpose is. For example if people
  are loading data into Hadoop, a Generic Cobol --> Avro would be usefull.
• What generated code would support that.

[Govindu Narendr]

Thanks Bruce Martin,providing very good information.And VBS structure working fine.

[Govindu Narendr]

Please find attached copybook and binary files

[Brian Reynolds]

Can you post your code that calls JRecord please?

[Govindu Narendr]

public class AmsReceipt06 {

public static void main(String[] args) throws Exception {
    new AmsReceipt06();
}

//  private static final String copybookFileName = "AmsReceipt.cbl";
private static final String copybookFileName = "aqtrans.txt";

//private LayoutDetail schema;
private final ICobolIOBuilder iobuilder;

private AmsReceipt06() throws Exception {
    String copybookFileame = this.getClass().getResource(copybookFileName).getFile();
    iobuilder = CobolIoProvider.getInstance()
                        .newIOBuilder(copybookFileame,  
                                                ICopybookDialects.FMT_MAINFRAME).setFont("CP037")
                                .setFileOrganization(Constants.IO_FIXED_LENGTH)
                                .setSplitCopybook(CopybookLoader.SPLIT_REDEFINE);

    readFile();
}

private void readFile() throws IOException, RecordException {
//      AbstractLineReader r = iobuilder.newReader("C:\\Software\\cobol\\Ams_Receipt_05AUG08190103.txt");
    AbstractLineReader r = iobuilder.newReader("C:\\Software\\cobol\\aqtrans pir2u");
    LayoutDetail schema = iobuilder.getLayout();
    AbstractLine l;
    String id;

    IFieldDetail recordType = schema.getGroupField("TRANS-RECORD","TRANS-BASIC-TRNTYP");

      System.out.println("Recordtype : "+recordType.getType());

    while ((l = r.read()) != null) {
        id = l.getFieldValue(recordType).asString();
    System.out.println(id);
    }

    r.close();
}

/**
 * This method adds a Record-Selection-Test to a Record, three times.
 * The 3 tests will be will be combined with a boolean AND operator   
 *  
 * @param extlayoutCBL Group or parent record
 * @param recordName name of the record to be update
 * @param fieldName Field to Test
 * @param value value to be tested
 */
@SuppressWarnings("unused")
private static void addFieldTest3times(ExternalRecord extlayoutCBL, String recordName, String fieldName, String value) {
    int idx = findRecordIndex(extlayoutCBL, recordName);

    if (idx < 0) {
        System.out.println("Record " + recordName + " was not found");
    } else {
        // This add a test to the record three times.
        // The 3 tests will be joined with a logical AND
        // Obviously this would only make sense if the tests where different.
        // I did not have any sensible examples handy
        extlayoutCBL.getRecord(idx).addTstField(fieldName, value);
        extlayoutCBL.getRecord(idx).addTstField(fieldName, value);
        extlayoutCBL.getRecord(idx).addTstField(fieldName, value);
    }
}

private static int findRecordIndex(ExternalRecord extlayoutCBL, String recordName) {
    int ret = -1;

    for (int i = 0; i < extlayoutCBL.getNumberOfRecords(); i++) {
        if (recordName.equalsIgnoreCase(extlayoutCBL.getRecord(i).getRecordName())) {
            ret = i;
            break;
        }
    }

    return ret;
}

}

[Govindu Narendr]

This code same as AmsReceipt06.java example frm Jrecord