#1246 CBL_WRITE_FILE causes unbreakable loop

[Alexander v.Essen]

This simple test program causes a CBL_WRITE_FILE to a byte-stream file to loop indefinitely.
Obviously dynamic access with byte-stream files is impossible.
I tried input-output mode (3) as well with the same result.
I suspect more than one call to "CBL_WRITE_FILE" when the offset is higher than the file length
is causing this erronious behavior.
The program works well from step 1 to step 8, the following write gives the problem.
The test program simply uses calls to the CBL_ byte-stream routines in a sequential manner:

1. create a file
2. close the file
3. open the same file in write mode
4. write a string with 10 byte length
5. close the file again
6. open the same file in read mode
7. read the string with 10 byte length
8. close the file again
9. open the same file in write mode
10. attempt to write a string with 20 byte length appending after the 10 byte
11. This is where the problem starts:
12. the program loops forever, ctrl-c does not break or interrupt the execution, the file is filled with hex 00 until the disk runs over...
13. task manager is irresponsive, even if the task (Windows command processor or the Command Window or bstest.exe) is ended, the file is steadily increasing in size!
14. only working solution: restart of windows asap before the disk runs over...

15. problem: this produces multy-gigabyte-files within a few seconds!!!

16. close the file again (never reached!)

Included bstst.zip with cobol source, exe, various descriptions and screenshots

[Arnold Trembley]

I do not think this is a true bug, because I don't believe the CBL routines were used properly. I will attach images of the GnuCOBOL programmer's guide pages.

In order to APPEND records to a file using CBL_WRITE_FILE, I believe that at Step 9 above you must open the file in input/output mode, then perform CBL_READ_FILE with the FLAG argument set to 128 to request the size of the file in bytes. That should be returned in the OFFSET parameter if CBL_READ_FILE completes successfully.

The size of the file should be retained in OFFSET for the the CBL_WRITE_FILE in order to write another record (or series of bytes) after the previous end of file.

This is similar to APPEND mode using COBOL verbs with Sequential files (either line sequential or binary sequential).

[Alexander v.Essen]

Hi Arnold, you must be joking if this is not a "real" bug. Are there any unreal ones or perhaps surreal... now I am joking. But a routine that can compromise a whole OS should not be a bug? That´s what a really unbreakable loop can do if it continues writing until the disk is full - just imagine it happens on the system partition...
Even if the programming technique should be not appropriate a program loop shout be able to be escaped by the user / programmer and not write and write and write continuously on ONE CBL_WRITE_FILE statement. Or should it? Remember: not even the Windows Task Manager can end this program! So it´s not a bug? A very big and dangerous one, I would say!
COBOL (and the included subroutines) is designed as an error-tolerant system, so a behaviour like this should not be possibel!
I read the manual very well. The passages about CBL_CREATE_FILE, CBL_OPEN_FILE, CBL_READ_FILE, CBL_WRITE_FILE and CBL_CLOSE_FILE are almost burnt into my memory, so often did I read them. By the way: I could not read a single line about the necessity to close / reopen the file to switch the access mode.
From the very moment Micro Focus introduced these byte-stream routines in the first compiler versions I used them extensively. To be sure I studied some old sources and found repetetive CBL_CLOSE_FILE / CBL_OPEN_FILE´s were NOT NECESSARY. I have not a single program compiled with Micro Focus Products where there is more than one OPEN necessary. In IO-Mode (also they used 3!) I switched back and forth beween READ, WRITE as I pleased. Without any CLOSE / OPEN inbetween!
This shows clearly the routines in GNUCOBOL are definitely different, if they make strange programming techniques necessary. I will try your suggestion tomorrow but I am sceptical it will work. I had a module in work which gave me the same trouble using IO Mode (3), that´s why I wrote this little test program.
Another illogic thing: why is the FIRST CBL_WRITE_FILE beyond the EOF boundary of the file working without problem? Only the following ones lead to the problem!
As long as this problem is not fixed I will not use these routines any more and I can recommmend this to anybody else.
Sorry for the long sermon, but I would not know how to express this in shorter ways...
Greetings from the Philippines,
Alex
Included a sample showing the behaviour under Micro Focus Netexpress 3.1

[Eugenio Di Lorenzo]

For better understanding and clarity.
Why are these routines used ?
Couldn't it be possible to use a standard COBOL syntax for sequential files ?

[Alexander v.Essen]

These built-in subroutines were introduced by Micro Focus a long time ago.
They handle very efficiently byte-stream input-output. Files can be created, written and read with any offset and buffer length. Data can be binary or with any other (text) content. I used them a lot to build my own tools like logfile handling, message file modules and the necessary maintainance programs, modules for multi-language handling of screen-io, print spoolers, etc, etc.
During the time we were still fighting with file sizes relative and indexed files were much larger. I usually used null-string termination to create files with much smaller sizes than the standard COBOL runtime could offer. So these routines are somehow on a lower level.
Only the ones built into GNUCOBOL seem to be faulty.
I am just running some tests to prove it.

[Alexander v.Essen]

Problem recognized but not solved.

The main reason for this bug is the documentation. I am referring to the GnuCOBOL 3.2 Programmer´s Reference.
8.2.44. CBL READ FILE
The offset argument (PIC X(8) USAGE COMP-X) defines the location in the file of the first
byte to be read. The first byte of a file is byte offset 0 and MUST be preset to zero for first
use.
This is correct. But in
8.2.50. CBL WRITE FILE is stated:
This routine writes nbytes of data from buffer to the byte-stream file defined by handle
starting at byte number offset within the file.
The handle argument (PIC X(4) USAGE COMP-X) must have been populated by a prior call
to CBL OPEN FILE. The offset argument (PIC X(4) USAGE COMP-X) defines the location
in the file of the first byte to be written to. The first byte of a file is byte offset 0.
This is NOT correct. The offset must be declared as PIC X(8) USAGE COMP-X.
Of course CBL_READ_FILE and CBL_WRITE_FILE can share the same offset variable.
There still remains the question how a wrongful DATA DECLARATION can cause endless and unbreakable loops. In my opinion this is bug and a hefty one.
So, at least the Programmer´s Reference and the Programmer´s Guide have to be updated.
They are anyway largely redundand in wide parts and should be merged into one publication.
The description should be also amended, because the behaviour of these routines is far different from the one the Micro Focus routines had and have. These byte-stream routines need a completely different and far more complicated programming technique which should be described in detail or at least with some example programs. I could do this, but I don´t know how to offer my help and I don´t know who is responsible for the documentation.
But the bug is still waiting for the next programmer who uses a wrong declaration.
Would it not be better to check the length of the arguments in these built-in subroutines to avoid these problems in the future?
Greetings, Alex

[Simon Sobisch]

I'll inspect both the "unlimited write" issue and the possibility to check the size when called from COBOL (those entry points actually can be called from C as well).

Given the wrong call (and the right size in your Micro Focus Netexpress 3.1 example) I'd guess that sample works correctly with GnuCOBOL, right?

[Alexander v.Essen]

I did not try this example written under Micro Focus Netexpress, but I am sure it will NOT work that way. What I found is the GnuCOBOL routines necessitate a completeley different programming technique with close / open if you want to change the access mode. With Netexpress one could use mode 3 (input-output) and read / write at one´s disgretion. Only the offsets / lengths had to be adjusted.
Actually what Arnold Trembley suggested is the right thing: before any write a read with 128 in the flag has to be done. If the offset is declared as x(8) comp-x the correct file length is returned. After this a write could follow.
What is also a bit disturbing: I used only write mode (mode 2) for the write and it "killed" the data written beforehand, they were overwritten with hex 00. Something like that should aslo never happen. You see: one must be very careful using these routines!
I will continue with my tests today and see if I can use the routines for my little project.
Should it work the way I expect I might contribute is as an example.
Greetings, Alex

[Vincent (Bryan) Coen]

You are correct the offset is x(8) and this is shown in contrib (Cobol Tools) printcbl which is also used in cobxref.

This module has been in use for some considerable time and I am sorry I did not spot this error.
I am guessing that the doc was changed with out noticing it and or comparing with my existing code base but I blame age (79). There again I no doubt assumed that the call params would be the same for both READ and WRITE although printcbl only reads files to a depth of 10 and the compiler only goes (or did) to a depth of 5.

Manual has been corrected and issued dated today.

[Alexander v.Essen]

Thank you!
But I cannot understand why you closed the issue - there are still some unsolved problems. Not only the documentation is affected.
See the post and my answer to Simon Sobich.
This routine is still buggy in my opinion.
Greetings, Alex

[Eugenio Di Lorenzo]

What was fixed in the PG manual ?
Is there anything specific in the changelog ?

[Vincent (Bryan) Coen]

See previous remarks

[Eugenio Di Lorenzo]

I'd already read them, but unfortunately, I still couldn't identify the changes within the PG.
Even looking at the character's change log doesn't reveal what was changed.

[Alexander v.Essen]

Byte Stream File Tests finished successfully:

After using the correct definitions for the offset (i.e. pic x(8) comp-x instead of pic x(4) comp-x) the byte-stream routines passed almost all my tests.

If you want to update the documentation you are free to use this text. I guess some of these informations might help others.

A byte-stream file opened in input-output mode (3) can be used with true dynamic access.
After creating a file with CBL_CREATE_FILE and closing it with CBL_CLOSE_FILE the file can be opened in input-output mode (3) for writing with CBL_WRITE_FILE and reading with CBL_READ_FILE in any kind of sequence with any offset and buffer length. The same definitions for offset, length, flags and buffer can be shared by these routines.
For retrieving the file size a CBL_READ_FILE can be used with the read-flag set to 128. In this case the buffer is not filled, instead the file size is returned in the offset variable. This procedure can be used to append additional records to the file, addressing the space immediately after the last record.
To read from the file and fill the buffer with the content of the file from the given offset with the given length the read-flag has to be set to zero.
Records can be written even outside the file boundary. In this case a CBL_WRITE_FILE will fill gaps in the file with the necessary number of hex 00 bytes. If the file is 100 byte long and the CBL_WRITE_FILE is executed with an offset of 120 the 20 bytes between offset 100 and 120 are filled with hex 00, after this the contents of the buffer are written in the given length.
The gaps filled with hex 00 can be addressed like any other space in the file.
Updates can be executed with CBL_READ_FILE and CBL_WRITE_FILE in input-output open-mode (3). The buffer can be altered and rewritten at the same position, as long as offset and length are not changed.
The special register RETURN-CODE usually returns the known codes for error-evaluation in the COBOL file system, i.e. 35 for the attempt to open a non-existent file, zero for a successful operation, 30 for permanent i-o error, etc. Only exception is -1 which means one of the parameters is wrong.
The only problem encountered during read tests is the file system return code for EOF, code 10. Repeated CBL_READ_FILE in input-output mode (3) never returned any file error. Even if the offset of the read went far beyond the file boundary the RETURN-CODE was always zero. In input mode (1) the EOF condition was only returned on an empty file (length 0 bytes). It is good advice to use a counter or intercept the offset to work around this problem.
Conclusion: with the exception of the EOF problem the byte-stream file routines can be used like the ones introduced once by Micro Focus.