EXTFH and NOT OPTIONAL MF Compability

[Anonymous]

Hello. New here. We're investigating an external 3rd party COBOL ISAM handler linked with callfh functionality. This library is fully functional with MF COBOL (and ACU and RM) and behaves according to established EXTFH standards.

Good news - Our example code runs perfectly under the BDB ISAM handler. Our library links and successfully creates files under our file system. Testing with OPEN OUTPUT, files are created under our file handler library and records added.

The unknown - Our simple example fails with error 23 when linked with callfh. as no records are added to table.

The first problem we noticed is that NOT OPTIONAL on OPEN I-O does not trigger a file create. If we first create a file loaded with records using OPEN OUTPUT, then running the example with OPEN I-O we get successful read of records.

Looking at GNUCOBOL source we noticed handling of NOT OPTIONAL doesn't seem complete for starters

In FCD

        unsigned char   otherFlags;                     /* miscellaneous flags */
#define OTH_OPTIONAL            0x80
#define OTH_NOT_OPTIONAL        0x20
#define OTH_EXTERNAL            0x10
#define OTH_DOLSREAD            0x08
#define OTH_NODETECTLOCK        0x04
#define OTH_MULTI_REEL          0x02
#define OTH_LINE_ADVANCE        0x01

From MF documentation:

fcd-other-flags pic x comp-x    
Bit 7   OPTIONAL file (OPEN INPUT)
Bit 6   IGNORELOCK=ON
Bit 5   Not OPTIONAL (OPEN I/O and EXTEND)
Bit 4   Filename is EXTERNAL
Bit 2   NODETECTLOCK directive enabled
Bit 1   Multiple reel file (record sequential)
Bit 0   Line advancing file (record sequential)

That implies Bit 5 should be explicitly set when NOT OPTIONAL is specified with OPEN I-O or OPEN EXTEND. And, Bit 7 is for OPTIONAL and OPEN INPUT. This is what it seems is happening when our driver receives the FCD information, it is not what we expect and we only do what the FCD instructs.

We find this in fileio.c:copy_fcd_to_file

        if((fcd->otherFlags & OTH_OPTIONAL))
                f->flag_optional = 1;
        else
                f->flag_optional = 0;

We don't see OTH_NOT_OPTIONAL handling at all. (Then there's the -foptional-file flag consideration as well which is confusing in this regard and makes no difference with our library testing).

Is this support still lacking? Or are we missing something? Is there someone we should be directly talking to about complete EXTFH support such as MF provides? (We'd love to be working under GNUCOBOL! )

[Simon Sobisch]

Thanks for your interest. @rjnorman74 possibly has some insights.
In any case: what version of GnuCOBOL do you currently run?

[Anonymous]

Thanks! This is testing with latest V3..1.2

I tried to pull current trunk and it compiled fine, however, the resulting cobc failed to compile our example. But that's life at the edge of a trunk.

/tmp/cob29970_0.c:712:32: error: incompatible types when assigning to type ‘unsigned char[4]’ from type ‘unsigned char *’
   h_CUSTOMER_FILE->file_status = h_CUSTOMER_FILE_status;

[Anonymous]

As a ping I was able to test with the 4.0 early dev build and same results. I do see your extfh work has been abstracted out in the source which is encouraging!

[Anonymous]

I should clarify - Same results as original post in that the extfh interface isn't behaving as our driver expects. Table is not created on OPEN I-O with SELECT NOT OPTIONAL clause.

GnuCOBOL 4.0 built and compiled cleanly (environment issue on my end with the trunk build before --- sorry!)

[Simon Sobisch]

Yes, the extfh implementation is split-out but will stay sync'd to 3.x (so far it contains one unmerged changed in 4.x which is about handling of composite keys).
In any case you either dive into the C source-code to check or need to wait for @rjnorman74 to investigate, which is much more likely if you create a minimal program to reproduce this (which at least would make it clear how the test is done).

BTW: I highly suggest to register/login, otherwise a post of yours may stay "for days" until a human took the time to manually moderate your posts.

[Anonymous]

"NOT OPTIONAL" is this equivalent to Micro Focus "NOOPTIONAL FILE" ?

[Brian Tiffin]

Short answer: yes.

Longer answer, not sure. That seems to make all files non optional in terms of INPUT and IO modes, even if OPTIONAL is explicit in the SELECT? Or, I'm reading that entry as you need to use NOOPTIONAL-FILE to stick with Standard (OPTIONAL needs to be explicit) and implied with MF will all SELECT for input phrases being OPTIONAL by default?

Cheers,
Blue

[Anonymous]

As I remember it NOOPTIONAL-FILE (the default) will not create a file on an OPEN INPUT clause failing because the daateset was not found.
OPTIONAL-FILE will create a file and raise a '05' status code if the DSN associated with the OPEN is not found.

Obviously with NOOPTIONAL-FILE in effect a status code '35' will be raised on a data set not found at OPEN INPUT.

As I recall use of Micro Focus (MF) EXTFH referencing a SELECT CLAUSE and FD will populate the FCD with the NOOPTIONAL-FILE bit set ON.

Regarding ISAM / VSAM, OPEN I-O (or extend) I always do a test for Status Code '35', if '35' was raised I issue an OPEN OUTPUT, CLOSE, and OPEN I-O. On the mainframe (IBM) IDCAMS creates the VSAM dataset - I do not think you can OPEN an OUTPUT VSAM data set without IDCAMS services. - I think ?

[Simon Sobisch]

Just to drop that note here, too: Ron will have a look at this sometime the next weeks.

[Craig Bailey]

Thanks. I'm the OP now registered. Attached (I hope it goes through) is a primitive example (Sorry, I'm far from a COBOL dev). It behaves differently between the default DB handler and ExtFH interface. To compile with our external handler we're using this:

cobc -x -fcallfh=myhandler test.cbl myhandler.so

I also see inconsistent behavior in ExtFH after a close and re-open of the table. The table is not created the second time (after it's closed and removed) and then an infinite READ loop with no error. Should be self explanatory from comments and output. Again, we think its just a few bits that aren't passed down in the FCD options. I tried to modify source, but probably missed a few strategic locations.

Unfortunately, I'm not at liberty to publicly disclose our external file handler. (I'm happy to have that discussion offline if that helps with testing.) --Cheers

[Craig Bailey]

Here is my simple example.

[Craig Bailey]

This second example is probably a better one. The first was too "clean".

[Simon Sobisch]

@cwbaileymo is this still open with a GC 3.2 snapshot? Then we should have a look at fixing this after the rc is out.

[Craig Bailey]

From testing today's current 3.2 build from 23 Oct I see the same behavior as I recall from before. The default handler runs as expected. A compiled in EXTFH handler behaves as I recall.

cobc -x -fcallfh=handler test2.cbl libhandler.so