#23 Suppress when spaces

GC 2.1
accepted
Joe Robbins
None
1
2015-06-12
2013-01-22
Luke Smith
No

Please add SUPPRESS WHEN SPACES to the ALTERNATE RECORD KEY IS ... clause.

     ALTERNATE RECORD KEY IS ... 
         SUPPRESS WHEN [ SPACES  ]
                       [ ZEROS   ]
                       [ ALL "single character literal" ]

This would improve performance for those that use alternate keys. Suppress when ZEROS or ALL literal would be nice to have (another vendor has them). Suppress when SPACES is mandatory in my book.

Discussion

  • Luke Smith
    Luke Smith
    2013-01-31

    Early RM-Cobol (back when it was re-packaged as TI-990 Cobol) always suppressed alternate keys when spaces. There was no way to write out an alternate key when it was spaces. This feature is needed to accommodate early RM-Cobol programs.

     
  • Luke Smith
    Luke Smith
    2014-06-13

    I've done some work on this that looks promising.

     
  • Simon Sobisch
    Simon Sobisch
    2014-06-13

    Hm, although I've read the MF docs for SUPPRESS WHEN ... I'm still unsure what this is doing. Please give us a code sample that highlights the usage (together with the test entries that should be added to the test suite).
    If you've got a working patch for current svn version (gnu-cobol-2.0 branch), please post it I'll review it. If it's not yet working we may could give some help.

    Simon

     
    Last edit: Simon Sobisch 2014-10-17
    • Joe Robbins
      Joe Robbins
      2014-06-14

      Simon: I have implemented SUPPRESS in fileio rewrite. I'll get back to you.

      Joe Robbins

       
      Last edit: Joe Robbins 2014-06-18
  • Luke Smith
    Luke Smith
    2014-06-18

    I've also got suppress working under 2.0. I wrote a test program that tests:
    - rewrite a blank alternate key as non-blank
    - rewrite a non-blank alternate key as blank
    - delete a non-blank alternate key
    - suppress works correctly in all these cases.

    I was thinking the test program could also be a training program on alternate keys. So, with a bit of polishing up I've come up with the attached. I would like to check it in to the contributions area.

    If you want to try the program without suppress. then comment out the suppress clause.

    The suppress clause is useful for building an indexed file where most of the alternate keys are not used. For example, if someone had a garden club of 70,007 members, and an alternate key indicating which ones were officers of the club. With only 7 officers the alternate key would have 70,000 blank keys. I believe MF blows up when there are more than 65,536 duplicates. Suppress avoids that error by not writing the alternate key. So the alternate key file only has 7 records, not 70,007.

     
    Last edit: Luke Smith 2014-06-18
    Attachments
  • Simon Sobisch
    Simon Sobisch
    2014-08-16

    Hi Luke,

    feel free to commit it into the contrib repository.

    Please check if it works as expected with the branch for fileio-rewrite, too. SPLIT-KEYs for primary keys are currently unsupported, I've added the missing parts to SUPPRESS [cobc side, didn't tested the result in the physical file, yet (I guess in your sample when reading the file from the beginning to the end with the alternate key and SUPPRESS there should be an EOF after reading the 7 officers, shouldn't it?)]. Please check the existing test programs for SUPPRESS (tests/testsuite.src/*_file.at) and report back if you think it is tested well enough.

    Simon

     
    • Luke Smith
      Luke Smith
      2014-08-24

      Yes, the reading by the secondary key should get an EOF after 7 records in that example.

      I'll be happy to check in the test program, and add some to testsuite. But Joe has brought up some issues I think should be addressed first.

       
      • Simon Sobisch
        Simon Sobisch
        2014-08-26

        contrib repo can be filled right now, under samples I guess ;-)
        For the testsuite: please have a look at the test in fileio-rewrite and check if you think there are missing bits. It's good to have a quick test base, even if it breaks the test suite in some configurations.

        The issues Joe has brought up are already part solved:
        Ron will have a look for patching VBISAM to support SPARSE KEYs in the D-ISAM way
        Joe will throw an "SPARSE KEYS not supported in C-ISAM" error (for now with VB-ISAM, too)
        * Joe will add SPARSE KEYs for D-ISAM (you can get a time limited free evaluation only version - this is enough for implementation and testing)

        Simon

         
  • Simon Sobisch
    Simon Sobisch
    2014-08-16

    • assigned_to: Simon Sobisch
    • Group: 0.20 --> GC 2.0
     
  • Simon Sobisch
    Simon Sobisch
    2014-08-16

    If possible I'll try to merge the necessary changes from branch fileio-rewrite to GC 2.0, together with SPLIT-KEY support.

    Simon

     
  • Joe Robbins
    Joe Robbins
    2014-08-17

    Some observations on SUPRESS WHEN ...

    1. It would be great if this feature could be implemented so that porting COBOL sources that use it is made as painless as possible.

    2. Considerations such as maximun permitted duplicates, size of B-TREE tables, disk space occupied, etc are probably less important than they used to be because ISAM architectures have advanced, E.G. 32-bit integer.

    3. Implementing this for GNU-COBOL is relatively easy for builds that use BDB for ISAM support. This is because when a row is added to a table, client code is called-back to extract each key and return it to BDB for indexing. For "sparse" keys, the client code simply returns DB_DONOTINDEX to indicate there is no key-value for this index for this record.

    4. However, for GNU-COBOL that uses VBISAM for ISAM support, the picture is more complicated. When inserting a new row using VBISAM, the record is passed and VBISAM itself extracts the keys (using key-descriptors supplied when the file was opened). There doesn't seem to be any provision for "sparse" keys. (I see under enhanced features on the commercialised VBISAM - VSAMEX[TREME] - that it supports sparse keys.)

    5. The simplest strategy is to permit ./configure for SUPPRESS WHEN only if --with-indexed = bdb.

    6. A second approach would be to generate a modified version of VBISAM to support sparse keys. (Perhaps setting iNParts = 0 in the key-descriptor if the key-value is empty before calling iswrite() ???)

    7. A third strategy that springs to mind is (i) to tell VBISAM that the index may contain duplicates (irrespective of the declaration in COBOL source). (ii) Pass records unchanged to VBISAM. (iii) Code a masquerade in fileio-isam-xisam.c that skips records when the active KEY returns a "sparse" value. Thus those records are not seen by the COBOL code. (iv) Some additional work would be necessary. For example: if the key in COBOL source is NOT "duplicates allowed" then on WRITE/REWRITE fileio would need to read for the key-value and reject the update if a record with the same key-value already exists. This would need file-locking (to prevent a race with a second process that was attempting to do the same thing) - and could be inefficient. This is a non-trivial task.

    Luke: you say in your posting: "I've also got suppress working under 2.0". Was this based on BDB and/or VBISAM? Or did you take a different approach?

    Regards: Joe Robbins

     
    Last edit: Joe Robbins 2014-08-17
    • Simon Sobisch
      Simon Sobisch
      2014-08-18

      I've checked DISAM and the solution there looks like NULLKEYS.

      Quoted from their evaluation package read/nullkeys.ref

      the default value used to determine if a given key is null is zero,
      but this can be modified by adding the value, left shifted by eight,
      to the key type field.

      the following code constructs a null key index in which a character
      field containing all spaces will be considered null:

            key->k_flags = ISDUPS + NULLKEY;
            key->k_nparts = 1;
            key->k_part[0].kp_type = CHARTYPE + ( 32 << 8 );
            key->k_part[0].kp_start = 10;       
            key->k_part[0].kp_leng = 10;
      

      null key indexes are not cisam compatible.

      If this works fine we should try to get the same feature into VBISAM (even if it's main goal is to be a free CISAM alternative, a patch would be accepted). I explicit invite Ron to take part on the discussion here (he can test both DISAM and VBISAM).

      Simon

      BTW: I'm very sure VSAMEX[TREME] has nothing to do with VBISAM. Do you have any information that shows this opinion being wrong?

       
      • Simon Sobisch
        Simon Sobisch
        2014-08-18

        @Joe:
        As CISAM doesn't support this the parser should error out with something like "Compiler not configured for SPARSE KEYS (SUPPRESS WHEN is only available with BDB, DISAM). Did you already added something like this?

        Simon

         
    • Luke Smith
      Luke Smith
      2014-08-24

      I only tested my changes with bdb. I checked the 2.0 branch and found that the changes I made were similar.

      I was really hoping to switch to vbisam some day. Let's hope that it can be patched for this.

       
  • Simon Sobisch
    Simon Sobisch
    2014-08-18

    I'll ask the D-ISAM folks if there is a solution on their side. At least in old MF docs MF states that SPARSE KEYS aren't supported for C-ISAM files. VBISAM can be tweaked, patches are welcome :-)

    Simon

     
  • Simon Sobisch
    Simon Sobisch
    2014-08-28

    • status: open --> accepted
    • assigned_to: Simon Sobisch --> Joe Robbins
    • Group: GC 2.0 --> GC 2.1
     
    Last edit: Simon Sobisch 2014-08-28
  • Simon Sobisch
    Simon Sobisch
    2014-08-28

    Moved to GC 2.1 (where the merge of fileio-rewrite is targeted) and assigned to Joe.

    Simon

     
  • Alan Jump
    Alan Jump
    2014-10-16

    No joy under a freshly-compiled 2.1 (gcc 4.8.2, Kubuntu 14.04.1).

    altkey.cob: 25: Error: syntax error, unexpected SUPPRESS, expecting SEQUENTIAL
    
     
    • Simon Sobisch
      Simon Sobisch
      2014-10-17

      2.1 is the target version. As there is currently no 2.1 working branch (the 2.0 isn't finished yet) you cannot test this with 2.1.

      If you need the support or simply want to test it checkout the fileio-rewrite branch where it's implemented (this branch will be merged to 2.1 after 2.0 release). But it will only worked when configured with bdb (being one of the reasons why it's not in 2.0).

      I've just added SUPPRESS clause as PENDING in 2.0 branch with [r428], this allows altkey.cob to be at least compiled with GC (without SUPPRESS clause working, of course).

      Simon

       

      Related

      Commit: [r428]

      • Alan Jump
        Alan Jump
        2014-10-17

        My bad...sometimes going with the bleeding-edge means all the holes
        haven't been filled yet.

         
        Last edit: Simon Sobisch 2015-02-18