#23 Suppress when spaces

GC 3.x
needs-merge
None
5 - default
20 hours ago
2013-01-22
Luke Smith
No

Please add SUPPRESS WHEN SPACES to the ALTERNATE RECORD KEY IS ... clause.

     ALTERNATE RECORD KEY IS ... 
         SUPPRESS WHEN [ SPACES  ]
                       [ ZEROS   ]
                       [ ALL "single character literal" ]

This would improve performance for those that use alternate keys. Suppress when ZEROS or ALL literal would be nice to have (another vendor has them). Suppress when SPACES is mandatory in my book.

Related

Discussion: Alternate key suppress when space

Discussion

  • Luke Smith

    Luke Smith - 2013-01-31

    Early RM-Cobol (back when it was re-packaged as TI-990 Cobol) always suppressed alternate keys when spaces. There was no way to write out an alternate key when it was spaces. This feature is needed to accommodate early RM-Cobol programs.

     
  • Luke Smith

    Luke Smith - 2014-06-13

    I've done some work on this that looks promising.

     
  • Simon Sobisch

    Simon Sobisch - 2014-06-13

    Hm, although I've read the MF docs for SUPPRESS WHEN ... I'm still unsure what this is doing. Please give us a code sample that highlights the usage (together with the test entries that should be added to the test suite).
    If you've got a working patch for current svn version (gnu-cobol-2.0 branch), please post it I'll review it. If it's not yet working we may could give some help.

    Simon

     
    Last edit: Simon Sobisch 2014-10-17
    • Joe Robbins

      Joe Robbins - 2014-06-14

      Simon: I have implemented SUPPRESS in fileio rewrite. I'll get back to you.

      Joe Robbins

       
      Last edit: Joe Robbins 2014-06-18
  • Luke Smith

    Luke Smith - 2014-06-18

    I've also got suppress working under 2.0. I wrote a test program that tests:
    - rewrite a blank alternate key as non-blank
    - rewrite a non-blank alternate key as blank
    - delete a non-blank alternate key
    - suppress works correctly in all these cases.

    I was thinking the test program could also be a training program on alternate keys. So, with a bit of polishing up I've come up with the attached. I would like to check it in to the contributions area.

    If you want to try the program without suppress. then comment out the suppress clause.

    The suppress clause is useful for building an indexed file where most of the alternate keys are not used. For example, if someone had a garden club of 70,007 members, and an alternate key indicating which ones were officers of the club. With only 7 officers the alternate key would have 70,000 blank keys. I believe MF blows up when there are more than 65,536 duplicates. Suppress avoids that error by not writing the alternate key. So the alternate key file only has 7 records, not 70,007.

     
    Last edit: Luke Smith 2014-06-18
  • Simon Sobisch

    Simon Sobisch - 2014-08-16

    Hi Luke,

    feel free to commit it into the contrib repository.

    Please check if it works as expected with the branch for fileio-rewrite, too. SPLIT-KEYs for primary keys are currently unsupported, I've added the missing parts to SUPPRESS [cobc side, didn't tested the result in the physical file, yet (I guess in your sample when reading the file from the beginning to the end with the alternate key and SUPPRESS there should be an EOF after reading the 7 officers, shouldn't it?)]. Please check the existing test programs for SUPPRESS (tests/testsuite.src/*_file.at) and report back if you think it is tested well enough.

    Simon

     
    • Luke Smith

      Luke Smith - 2014-08-24

      Yes, the reading by the secondary key should get an EOF after 7 records in that example.

      I'll be happy to check in the test program, and add some to testsuite. But Joe has brought up some issues I think should be addressed first.

       
      • Simon Sobisch

        Simon Sobisch - 2014-08-26

        contrib repo can be filled right now, under samples I guess ;-)
        For the testsuite: please have a look at the test in fileio-rewrite and check if you think there are missing bits. It's good to have a quick test base, even if it breaks the test suite in some configurations.

        The issues Joe has brought up are already part solved:
        Ron will have a look for patching VBISAM to support SPARSE KEYs in the D-ISAM way
        Joe will throw an "SPARSE KEYS not supported in C-ISAM" error (for now with VB-ISAM, too)
        * Joe will add SPARSE KEYs for D-ISAM (you can get a time limited free evaluation only version - this is enough for implementation and testing)

        Simon

         
  • Simon Sobisch

    Simon Sobisch - 2014-08-16
    • assigned_to: Simon Sobisch
    • Group: 0.20 --> GC 2.0
     
  • Simon Sobisch

    Simon Sobisch - 2014-08-16

    If possible I'll try to merge the necessary changes from branch fileio-rewrite to GC 2.0, together with SPLIT-KEY support.

    Simon

     
  • Joe Robbins

    Joe Robbins - 2014-08-17

    Some observations on SUPRESS WHEN ...

    1. It would be great if this feature could be implemented so that porting COBOL sources that use it is made as painless as possible.

    2. Considerations such as maximun permitted duplicates, size of B-TREE tables, disk space occupied, etc are probably less important than they used to be because ISAM architectures have advanced, E.G. 32-bit integer.

    3. Implementing this for GNU-COBOL is relatively easy for builds that use BDB for ISAM support. This is because when a row is added to a table, client code is called-back to extract each key and return it to BDB for indexing. For "sparse" keys, the client code simply returns DB_DONOTINDEX to indicate there is no key-value for this index for this record.

    4. However, for GNU-COBOL that uses VBISAM for ISAM support, the picture is more complicated. When inserting a new row using VBISAM, the record is passed and VBISAM itself extracts the keys (using key-descriptors supplied when the file was opened). There doesn't seem to be any provision for "sparse" keys. (I see under enhanced features on the commercialised VBISAM - VSAMEX[TREME] - that it supports sparse keys.)

    5. The simplest strategy is to permit ./configure for SUPPRESS WHEN only if --with-indexed = bdb.

    6. A second approach would be to generate a modified version of VBISAM to support sparse keys. (Perhaps setting iNParts = 0 in the key-descriptor if the key-value is empty before calling iswrite() ???)

    7. A third strategy that springs to mind is (i) to tell VBISAM that the index may contain duplicates (irrespective of the declaration in COBOL source). (ii) Pass records unchanged to VBISAM. (iii) Code a masquerade in fileio-isam-xisam.c that skips records when the active KEY returns a "sparse" value. Thus those records are not seen by the COBOL code. (iv) Some additional work would be necessary. For example: if the key in COBOL source is NOT "duplicates allowed" then on WRITE/REWRITE fileio would need to read for the key-value and reject the update if a record with the same key-value already exists. This would need file-locking (to prevent a race with a second process that was attempting to do the same thing) - and could be inefficient. This is a non-trivial task.

    Luke: you say in your posting: "I've also got suppress working under 2.0". Was this based on BDB and/or VBISAM? Or did you take a different approach?

    Regards: Joe Robbins

     
    Last edit: Joe Robbins 2014-08-17
    • Simon Sobisch

      Simon Sobisch - 2014-08-18

      I've checked DISAM and the solution there looks like NULLKEYS.

      Quoted from their evaluation package read/nullkeys.ref

      the default value used to determine if a given key is null is zero,
      but this can be modified by adding the value, left shifted by eight,
      to the key type field.

      the following code constructs a null key index in which a character
      field containing all spaces will be considered null:

            key->k_flags = ISDUPS + NULLKEY;
            key->k_nparts = 1;
            key->k_part[0].kp_type = CHARTYPE + ( 32 << 8 );
            key->k_part[0].kp_start = 10;       
            key->k_part[0].kp_leng = 10;
      

      null key indexes are not cisam compatible.

      If this works fine we should try to get the same feature into VBISAM (even if it's main goal is to be a free CISAM alternative, a patch would be accepted). I explicit invite Ron to take part on the discussion here (he can test both DISAM and VBISAM).

      Simon

      BTW: I'm very sure VSAMEX[TREME] has nothing to do with VBISAM. Do you have any information that shows this opinion being wrong?

       
      • Simon Sobisch

        Simon Sobisch - 2014-08-18

        @Joe:
        As CISAM doesn't support this the parser should error out with something like "Compiler not configured for SPARSE KEYS (SUPPRESS WHEN is only available with BDB, DISAM). Did you already added something like this?

        Simon

         
    • Luke Smith

      Luke Smith - 2014-08-24

      I only tested my changes with bdb. I checked the 2.0 branch and found that the changes I made were similar.

      I was really hoping to switch to vbisam some day. Let's hope that it can be patched for this.

       
  • Simon Sobisch

    Simon Sobisch - 2014-08-18

    I'll ask the D-ISAM folks if there is a solution on their side. At least in old MF docs MF states that SPARSE KEYS aren't supported for C-ISAM files. VBISAM can be tweaked, patches are welcome :-)

    Simon

     
  • Simon Sobisch

    Simon Sobisch - 2014-08-28
    • status: open --> accepted
    • assigned_to: Simon Sobisch --> Joe Robbins
    • Group: GC 2.0 --> GC 2.1
     
    Last edit: Simon Sobisch 2014-08-28
  • Simon Sobisch

    Simon Sobisch - 2014-08-28

    Moved to GC 2.1 (where the merge of fileio-rewrite is targeted) and assigned to Joe.

    Simon

     
  • Alan Jump

    Alan Jump - 2014-10-16

    No joy under a freshly-compiled 2.1 (gcc 4.8.2, Kubuntu 14.04.1).

    altkey.cob: 25: Error: syntax error, unexpected SUPPRESS, expecting SEQUENTIAL
    
     
    • Simon Sobisch

      Simon Sobisch - 2014-10-17

      2.1 is the target version. As there is currently no 2.1 working branch (the 2.0 isn't finished yet) you cannot test this with 2.1.

      If you need the support or simply want to test it checkout the fileio-rewrite branch where it's implemented (this branch will be merged to 2.1 after 2.0 release). But it will only worked when configured with bdb (being one of the reasons why it's not in 2.0).

      I've just added SUPPRESS clause as PENDING in 2.0 branch with [r428], this allows altkey.cob to be at least compiled with GC (without SUPPRESS clause working, of course).

      Simon

       

      Related

      Commit: [r428]

      • Alan Jump

        Alan Jump - 2014-10-17

        My bad...sometimes going with the bleeding-edge means all the holes
        haven't been filled yet.

         
        Last edit: Simon Sobisch 2015-02-18
  • Simon Sobisch

    Simon Sobisch - 2016-03-31
    • assigned_to: Joe Robbins --> Simon Sobisch
    • Priority: 1 --> 5 - default
     
  • Simon Sobisch

    Simon Sobisch - 2016-03-31

    I've changed the owner as a check against RW branch is necessary - Roin implemented this (not only for BDB) if I remember this correctly.

     
  • Luke Smith

    Luke Smith - 2017-12-06

    I really want to get this in GC 2.3. I must have this. Every time I download a new GC I have to put this back in. This is getting very tedious. Especially since I want to test pre-release versions.

    I hope to have some time to work on this by March 2018. At least for db. And make it error for vbisam and others if they still doesn't support.

    Unless I hear any objections, I will check it in for the 2.3 release.

     
    • Simon Sobisch

      Simon Sobisch - 2017-12-06

      No need for any changes by you, Ron did the work already in reportwriter branch, see [abdd342f].

      It won't be in GnuCOBOL 2.3 (and I'll start a discussion with the devs now if we actually see a reason for this version as trunk is switched to 3.0 already and includes the complete REPORTWRITER module now - which is enough stuff to directly release 3.0-rc1 instead of 2.3-rc1 (which I wanted to do this weekend).

       

      Related

      Discussion: abdd342f


      Last edit: Simon Sobisch 2017-12-06
  • Simon Sobisch

    Simon Sobisch - 20 hours ago
    • status: accepted --> needs-merge
     

Log in to post a comment.