COBOL to CSV conversion

[RAJKUMAR VISWANATHAN]

Hi Bruce - I am trying to use JRecord to convert a Mainframe file (transferred to linux as binary) that has COMP and COMP-3 fields to a csv ascii file. Within my corporate environment I don't have access to cobl2csv.jar. I only have access to cb2xml.jar and Jrecord.jar. How can I use these 2 Jar's to convert the file to csv? Appreciate your help. Thanks!

[Bruce Martin]

You will need to work out the parameters to use for the IoBuilder. The RecordEditor can help with this, see https://sourceforge.net/p/jrecord/wiki/Generate%20Code%20for%20Binary%20Cobol%20file/

But here is a basic example of the logic (I will do an enhanced version in the morning)

        try {
            ICobolIOBuilder iob = JRecordInterface1.COBOL
                                        .newIOBuilder(copybookName)
                                            .setFont("cp037")                                   // US EBCDIC
                                            .setFileOrganization(IFileStructureConstants.IO_FIXED_LENGTH_RECORDS);  
            AbstractLineReader reader = iob.newReader(dataFile);

            AbstractLine line = reader.read();
            if (line != null) {
                String sep = "";
                FieldIterator fieldIterator = line.getFieldIterator(0);
                for (AbstractFieldValue fv : fieldIterator) {
                    System.out.print(sep + fv.getFieldDetail().getName());
                    sep = ",";
                }
                System.out.println();
                while (line != null) {
                    sep = "";
                    for (AbstractFieldValue fv : fieldIterator) {
                        System.out.print(sep + fv.asString());
                        sep = ",";
                    }
                    line = reader.read();
                }
            }

            reader.close();
        } catch (Exception e) {
            System.out.println("~~> " + e.getMessage());
            System.out.println();

            e.printStackTrace();
        }

[Bruce Martin]

Here is an updated (and better version) version using JRecord's CSV writer:

        try {
            ICobolIOBuilder iob = JRecordInterface1.COBOL
                                        .newIOBuilder(copybookName)
                                            .setFont("cp037")                                   // US EBCDIC
                                            .setFileOrganization(IFileStructureConstants.IO_FIXED_LENGTH_RECORDS);  
            AbstractLineReader reader = iob.newReader(dataFile);
            ICsvIOBuilder csvOBuilder = JRecordInterface1.CSV.newIOBuilder(",", "\"");
            IDefineCsvFields csvFields = csvOBuilder.defineFields();

            AbstractLine line = reader.read();
            if (line != null) {
                FieldIterator fieldIterator = line.getFieldIterator(0);
                for (AbstractFieldValue fv : fieldIterator) {
                    csvFields.addCsvField(fv.getFieldDetail().getName(), Type.ftChar, 0);
                }
                csvOBuilder = csvFields.endOfRecord();

                ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
                AbstractLineWriter csvWriter = csvOBuilder.newWriter(byteArrayOutputStream);
                while (line != null) {
                    AbstractLine csvLine = csvOBuilder.newLine();
                    int fldNum = 0;
                    for (AbstractFieldValue fv : fieldIterator) {
                        csvLine.getFieldValue(0, fldNum++).set(fv.asString());
                    }
                    csvWriter.write(csvLine);
                    line = reader.read();
                }
                csvWriter.close();
                System.out.println(byteArrayOutputStream.toString());
            }

            reader.close();

        } catch (Exception e) {
            System.out.println("~~> " + e.getMessage());
            System.out.println();

            e.printStackTrace();
        }

[RAJKUMAR VISWANATHAN]

Thanks for your response Bruce. My input file is Variable Block, with multiple 01 levels. There is a field-type that will help use the right 01 level. How should I handle it?

[Bruce Martin]

For Multiple records you will need .setSplitCopybook(CopybookLoader.SPLIT_01_LEVEL) in the ioBuilder.

The line.getFieldIterator(0) takes either a record-index or record-name as a parameter

So

            ICobolIOBuilder iob = JRecordInterface1.COBOL
                                        .newIOBuilder(copybookName)
                                        .setSplitCopybook(CopybookLoader.SPLIT_01_LEVEL)
                                            .setFont("cp037")                    // US EBCDIC
                                            .setFileOrganization(IFileStructureConstants.IO_FIXED_LENGTH_RECORDS); 

To determine the the Record-Type you can use:

• Java code - i.e. test the field in java
• Use the .setRecordSelection to tell JRecord how to distinguish the record-type
• Use RecordDecider to tell JRecord how to distinguish the record-type

[RAJKUMAR VISWANATHAN]

So, the file I am reading is a fixed length file (i had wrongly mentioned earlier that it is VB). I have multiple 01 level records with the first one being basic structure (say first 100 bytes) and the subsequent 01 levels are to be applied based on the record-type field that is in the basic structure. Kind of like below.

01 BASIC-STRUC pic x(100).
        05  BASIC-STRUC-FIELD1 pic x(10).
        05  RECORD-TYPE.    pic s9(01)
                88 REC-TYPE1   value +1.
                88 REC-TYPE2.  value +2.
        05 Filler          pic x(89).
 01  REC-TYPE1.
        05 FILLER pic x(100)
        05 REC-TYPE1-FIELD1 pic s9(10) comp-3.
        05 REC-TYPE1-FIELD2 pic x(44).
  01  REC-TYPE2.
        05 Filler pic x(100).
        05 REC-TYPE2-FIELD1   pic x(20).
        05 REC-TYPE2-FIELD2    pic x(30).

How do I iterate through each record?

Thanks!

[RAJKUMAR VISWANATHAN]

Fo the record selection, I am doing
if (line.getFieldValue("Basic-struc-field2") ..asInt() == 1 ) {
}
I know I can also use a case switch stmt.

The issue I am facing is, I am able to successfully read the first record, but not able to read the subsequent records as I am getting the following error when again doing line.getFieldValue("Basic-struc-field2).

net.sf.JRecord.RecordException. Basic-struc-field2: Invalid Zoned Decimal:

I am not using the FieldIterator to read the field values. Instead I am using like shown above - line.getFieldValue("field-name"). Maybe I should use the FieldIterator? If so, how do I assign the record-index/record-name?

[Bruce Martin]

I suggest trying to view the file in the RecordEditor.

See https://stackoverflow.com/questions/45794642/how-do-you-edit-a-binary-mainframe-file-in-the-recordeditor-using-a-cobol-copybo

Also the USB version of the RecordEditor does not need to be installed on a computer.

[RAJKUMAR VISWANATHAN]

Hi Bruce, at this point I just need to know how to iterate through each of the records in the file. Like I said, I am able to read the first record. It is the subsequent records that I am having trouble with. Should I used the Field Iterator or line.getFieldValue("Field-Name")? I am currently using the line.getFieldValue("Field-Name").

Thanks for your help!

[Bruce Martin]

The basic logic should be:

       AbstractLine saleRecord;

        ICobolIOBuilder iob = JRecordInterface1.COBOL
                                        .newIOBuilder(copybookName)
                                            .setFont("cp037")                                   // US EBCDIC
                                            .setFileOrganization(IFileStructureConstants.IO_FIXED_LENGTH_RECORDS);  
        AbstractLineReader reader = iob.newReader(salesFile);

        while ((saleRecord = reader.read()) != null) {

        }

If the lines are not correctly formatted then either:

• The wrong **FileOrganisation is being used
• There is problems with the file.

[RAJKUMAR VISWANATHAN]

Thanks Bruce. So, every time it iterates through the "while" loop and does reader.read(), does it read each record from the input file? How does it know how many bytes to read for each record? From the copybook in .newIOBuilder(copybookName)?

[RAJKUMAR VISWANATHAN]

I got it to work. Basically, the copy book I was given for the data was wrong.

Also, looks like in case of multiple 01 level records (each 01 of different length), the application takes the length of the longest record and applies that when reading the input file.

Thanks for all your help Bruce! Your app is very helpful!

[RAJKUMAR VISWANATHAN]

Sine my input is a Variable Block file with each record of different length, am I to use .setFileOrganization(Constants.IO_VB)? When I do that, I am getting ArrayIndexOutOfBoundsException when I try to check the field to determine the Record-Type (to determine which 01 level to use).

[Bruce Martin]

.setFileOrganization(Constants.IO_VB) is correct, the problem is almost certainly the file transfer.

On the mainframe a VB file consists of

<record-Length-1><record-data-1>
<record-Length-2><record-data-2>
<record-Length-3><record-data-3>

By default some file transfer options will drop the record-lengths when transferring a file. This is what has probably happened. There is an option (quite possibly called RDW) to transfer both the record-length and the Data.

If you supply a Sample file to the CodeGen option in the RecordEditor, it will look for Record-Lengths (Record-descriptor-word)