vbcheck , duplicate multi index with prim key

2008-08-12
2013-04-29
  • Hi,

    A VB-Isam file has for example 15 indices.  Prim-key index = index 1 , position in the file = (0,5)
    Index 15 is "duplicate" multi index, and the second part = prim-key
    In every such cases the following output is like this.

    an@oslo:~/vbisam-2.0/bin> vbcheck  ~/DP0/TEDAPPL/K
    Processing: /home/DP0/TEDAPPL/K
    Table node size: 4096 bytes
    Index 1: ISNODUPS
    Part 1: 0,5,CHARTYPE
    Index 2: ISDUPS
    Part 1: 5,12,CHARTYPE
    Index 3: ISDUPS
    Part 1: 17,12,CHARTYPE
    Index 4: ISDUPS
    Part 1: 29,10,CHARTYPE
    Index 5: ISDUPS
    Part 1: 39,3,CHARTYPE
    Index 6: ISDUPS
    Part 1: 63,35,CHARTYPE
    Index 7: ISDUPS
    Part 1: 168,35,CHARTYPE
    Index 8: ISDUPS
    Part 1: 203,35,CHARTYPE
    Index 9: ISDUPS
    Part 1: 373,3,CHARTYPE
    Index 10: ISDUPS
    Part 1: 377,15,CHARTYPE
    Index 11: ISDUPS
    Part 1: 432,3,CHARTYPE
    Index 12: ISDUPS
    Part 1: 435,5,CHARTYPE
    Index 13: ISDUPS
    Part 1: 969,3,CHARTYPE
    Index 14: ISDUPS
    Part 1: 992,1,CHARTYPE
    Part 2: 63,35,CHARTYPE
    Index 15: ISDUPS
    Part 1: 168,35,CHARTYPE
    Part 2: 0,5,CHARTYPE
    Index is out of order!
    Rebuilding index free list
    Segmentation fault

    The next run of vbcheck for this file shows this:

    an@oslo:~/vbisam-2.0/bin> vbcheck  ~/DP0/TEDAPPL/K
    Processing: /home/DP0/TEDAPPL/K
    Table node size: 4096 bytes
    Index 1: ISNODUPS
    Part 1: 0,5,CHARTYPE
    Index 2: ISDUPS
    Part 1: 5,12,CHARTYPE
    Index 3: ISDUPS
    Part 1: 17,12,CHARTYPE
    Index 4: ISDUPS
    Part 1: 29,10,CHARTYPE
    Index 5: ISDUPS
    Part 1: 39,3,CHARTYPE
    Index 6: ISDUPS
    Part 1: 63,35,CHARTYPE
    Index 7: ISDUPS
    Part 1: 168,35,CHARTYPE
    Index 8: ISDUPS
    Part 1: 203,35,CHARTYPE
    Index 9: ISDUPS
    Part 1: 373,3,CHARTYPE
    Index 10: ISDUPS
    Part 1: 377,15,CHARTYPE
    Index 11: ISDUPS
    Part 1: 432,3,CHARTYPE
    Index 12: ISDUPS
    Part 1: 435,5,CHARTYPE
    Index 13: ISDUPS
    Part 1: 969,3,CHARTYPE
    Index 14: ISDUPS
    Part 1: 992,1,CHARTYPE
    Part 2: 63,35,CHARTYPE
    Index 15: ISDUPS
    Part 1: 168,35,CHARTYPE
    Part 2: 0,5,CHARTYPE
    Rebuilding index free list
    Segmentation fault

    If the primkey does not become part of the multi-key than no errormessages appear.

    The testfile was created with "iswrite" without errormessages.

    Error in vbcheck ??

    Johann

     
    • Hi Johann,

      It's been quite a while since I 'played' with the VBISAM code in any great depth.  However, since I need to actually USE VBISAM in a project soon, I may as well refresh myself...
      It appears (from other posts) that you have DISAM too (I assume this is the Byte Designs code?).  A quick way to determine whether vbcheck is the REAL culprit would be to try dcheck (DISAM) or bcheck (CISAM) on the same file.
      If these products don't 'complain' at the file, then the problem must be limited to vbcheck.  If they DO complain, then it's more likely that I have problems with BOTH vbcheck *AND* the VBISAM library code itself.
      Is it possible for you to check this for me using bcheck / dcheck?
      Also, does this problem happen on an 'empty' ISAM file or only when the file has been populated with some data?
      Perhaps if you're able to send me a 'sample' VBISAM file, I can delve into the issue

       
    • Roger While
      Roger While
      2008-08-28

      Should be fixed in current VBISAM2 tarball.
      (Note you need to recreate the ISAM file)
      Trevor, it seems the problem is with isaddindex (or it may be a general problem).
      The file is opened for exclusive access (correctly and checked for in isaddindex).
      However, when we have exclusive access, we bypass the node update in iVBExit.
      For isaddindex, I added a call to iVBForceExit before the call to iVBExit which
      would appear to resolve the reported problem.
      I am not sure if there is a general problem with exclusive mode.

      Roger

       
      • OK, I _think_ I understand this issue...
        For some *SILLY* reason, I appear to have thought that the dictionary node of the .idx file (Which is node#1, the very FIRST node in the file...  WHY did RDS not start numbering from ZERO like every other system in the world???) never needs to be updated if the user has the file exclusively opened.
        In practice, the dictionary node can be updated with ANY 'table modifying' function.  This even includes issetunique() and isuniqueid().
        Rather than inserting 'special case' code into each table modifying function to explicitly call iVBForceExit(), I would think that maintaining a global DictionaryNodeIsDirty variable or even perhaps removing the conditionality of the DictionaryNode update altoegther such that it updates the node within iVBExit() even if the table was ISEXCLLOCK isopen()'d.

        Update:
        It appears that I already maintain a flag that determines if the 'in-memory' copy of the dictionary is 'dirty'.  Therefore, the obvious 'cure' seems to be removing the code that prematurely returns from iVBExit() function if, and only if, the file is open with ISEXCLLOCK.

        Do I have any votes left???

         
      • Finally I understood it. My vbisam-2.0 tarball was too old, I knew not that one must check regularly with sim-basic.de. ( :-) ) . With the newest tarball (from yesterday) runs vbcheck without errors. Perhaps Roger should announce here a new version of tarball. Thanks also for isdi_xxx functions. Super work.

        Johann

         
    • Roger While
      Roger While
      2008-08-28

      Re: Update
      Hmmm.
      Note that the early exclusive return is also done in iVBEnter.

      Roger

       
      • From what I can remember, the iVBEnter function does not modify the table in any way whatsoever.
        My 'opinion' is that if a table is exclusively isopen()'d, then it it should be impossible for ANY other process to instantiate a 'table modifying' function.  Furthermore, the way I have interpretted CISAM / DISAM is that an ISEXCLLOCK open should actively inhibit any other process from even OPENING the table.
        If you look closely at the isopen() code, you'll notice that it grabs a comprehensive lock [iVBFileOpenLock (2)] if the mode includes ISEXCLLOCK.  Since this lock is mutually exclusive with the normal per-process type lock applied with iVBFileOpenLock (1), it is reasonable to expect that NO other process can open open the same table once it has been opened with ISEXCLLOCK.  It is equally reasonable to expect that if some other process already has the table open, then the ISEXCLUSIVE open will fail.
        My conclusion therefore, is that it remains 'safe' to 'early exit' from iVBEnter if, and only if, the file has been exclusively opened.
        The remaining question is to ascertain what the CORRECT value for sFlags.iIsDictLocked should be...
        Since ISEXCLLOCK means that we are certain that NO other process has this table open, I believe that the value should be set to 0x01 (see isinternal.h).

        Sound fair?

         
    • Roger While
      Roger While
      2008-08-28

      Hmm. OK. but we need to make sure that iIsDictLocked is set to 0x02 in
      appropriate functions such as isaddindex, etc.

      Just seen a problem with iVBFileOpenLock (with mode 2) -

              case    2:
                      iLockType = VBWRLOCK;
                      iResult = iVBBlockRead (iHandle, TRUE, (off_t) 1, cVBNode [0]);
                      memcpy ((void *) &psVBFile [iHandle]->sDictNode, (void *) cVBNode [0], sizeof (struct DICTNODE));
                      break;

              default:
                      return (EBADARG);
              }

              iResult = iVBLock (psVBFile [iHandle]->iIndexHandle, tOffset, 1, iLockType);
              if (iResult != 0)
                      return (EFLOCKED);

      This looks wrong. We should first try the lock and then do the read.
      The result from the read is overwritten and if it fails we scribble rubbish into the dictnode.

      Roger

       
      • I accept and agree that a tiny window exists in iVBFileOpenLock() during which some other process can screw up our dictionary BEFORE we've acquired the needed lock...
        The iVBBlockRead() needs to be moved down to somewhere AFTER the lock has been successfully acquired.  (It should perhaps remain conditional on iMode == 2 to avoid taking a performance hit though)

        Technically speaking, there are other (less serious) race conditions present too...
        For example:
        Imagine ProcessA has begun an isbuild() call but not completed it...
        ProcessB now comes along and attempts to isopen() the same table before ProcessA has been able to apply an exclusive lock.
        If this is a problem to you (and by 'you', I mean ANYONE reading this thread), then feel free to 'fix' it.  It's not a problem to me in practice, and as far as I know, CISAM and DISAM both suffer from the same 'issue'.
        There are other, more serious, issues in CISAM that I am aware of, transaction processing [isbegin(), iscommit() and isrollback()] in particular.

        As for the iIsDictLocked = 0x02 code, I think your example might be a wrong?  In the version of code _I_ am looking at, the variable is correctly set to correctly include the 0x02 mask within isaddindex().  (However, keep in mind that my local code snapshot has been modified a little recently).
        The general point you raise is very pertinent though...  ANY code that changes the in-memory copy of the dictionary node needs to set the 0x02 bit of iIsDictLocked so that iVBExit() knows that it needs to be rewritten to disk.
        A 'lazy' fix would be to unconditionally rewrite the dictionary node within iVBExit(), but this is obviously quite a performance hit to take.
        Another option would be to rewrite the node EVERY time we change one of the constiutent fields.  (Even WORSE from a performance perspective!)

        Once iVBEnter() has been successfully called, we have confidence that NO other process is going to modify the table.  (That's the primary PURPOSE of iVBEnter() / iVBExit() in the first place!).  Therefore, I can read the dictionary node into memory in iVBEnter() and allow wholesale changes to that in-memory copy until iVBExit() is called.  Before I release the lock on the file inhibiting other process 'modification access', I rewrite the dictionary node (conditional upon whether iIsDictNode telling me that it's been changed).

        For a scary few seconds, I had a horrible thought that there might be a window of opportunity for some other process only READING data from the table to be horribly confused by working from the 'on disk' copy of the dictionary node while it's being modified by our application.  However, I just checked the relevant isread() and isstart() etc functions and they have appropriate calls to iVBEnter() / iVBExit().
        The iVBEnter() function has two modes of operation...
        1:
        If the VBISAM library function needs to make a CHANGE to the index file, the second argument to iVBEnter() will be TRUE.  This will apply a WRITE lock.  If ANY other process is inside a iVBEnter() / iVBExit() pair, the local process will BLOCK until they have all exitted.  ANY other process attempting to acquire a lock on this region will NOT succeed until we have made the call to our corresponding iVBExit ()
        It is noteworthy to mention that the MAJORITY of VBISAM function calls need to modify the index file.
        2:
        If the VBISAM library function does NOT need to make any changes to the index file, the second argument to iVBEnter() will be FALSE.  This will apply a READ lock.  An arbitrary number of processes may all acquire a READ lock on the SAME region of the file at the same time.  What this guarantees is that the index file will remain 'static' for the duration of the current iVBEnter() / iVBExit() pair of calls. As mentioned above, if some process wishes to modify the index file, it will try to acquire a WRITE lock and this will BLOCK that process until we are done.  The implicit serialization of locking calls performed in the kernel means that the writing process WILL acquire it's lock even if the system is ridiculously loaded.
        The only VBISAM library calls that use the FALSE argument to iVBEnter() are isread() and isstart().  Luckily for us, these two functions normally comprise the majority of operations.

         
    • Roger While
      Roger While
      2008-08-28

      Re: fileopenlock.
      Yep, I have changed my copy to do the lock first and then, for mode == 2,
      do the read.

      Re: dictlocked = 0x02
      Yep, it certainly is in isadd(del)index. I was just saying that other places that do modification should be checked that they also do this.
      I will take out the exclusive check from iVBExit, revert my isadd(del)index change and do some more testing.

      Re: isbuild/isopen race.
      Yep, weird but true. I have never understood why isbuild is allowed
      without ISEXCLLOCK. Seems a contradiction in terms.

      Roger

       
    • Roger While
      Roger While
      2008-08-28

      Trevor,
      I can't seem to find any call to fileopenlock that does an
      unlock.
      ie. No call iVBFileOpenLock (iHandle, 0);
      (except in isrecover and vbIndexEdit which are not relevant here)

      Can you check in your local copy.

      Roger

       
      • Hmmmm, same here...
        (No iVBFileOpenLock(iHandle, 0) calls directly inside the library).
        I guess this means that EVERY file you've ever isopen()d with VBISAM is STILL locked!!! [Wicked grins]

        Technically speaking, it's apparent that I'm relying on the fact that the underlying O/S is unable to retain a lock once the associated handle is finally closed.  The associated unlocks are therefore implicit rather than explicit.
        I'm also fully aware that the 'virus from Redmond' releases such locks asynchronously (potentially HOURS after the file was closed if the virus hasn't already BSOD'd by then).
        In a weird and twisted sort of way, I'm going to call it a FEATURE rather than a bug. (Trev looks in mirror... Weird / Twisted? Yep, that's an accurate description)
        If you wish to disagree and want to amend your local copy, please remember that in SOME cases (transaction processing), the files MUST remain open and locked even though isclose() has been called. (i.e. The lock should be removed just prior to the actual call to close(2) the file)

         
    • Roger While
      Roger While
      2008-08-29

      Actually, I think we do have to do something at isclose time especially if the file has an exclusive lock.
      Excerpt from C-ISAM doc -
      ****
      If you open or build your file with exclusive locking, no other program can
      access the file until you close it with the isclose function call. This is the only way to remove an exclusive lock.
      ****

      However, it is not clear from the isclose description that this is so within a transaction. Go figure. (ie. Does it demote the lock to a shared lock?)

      In any case, outside of a transaction, we should/must release the locks at isclose time. When ttansaction processing, then we should/must do it at iscommit/isrollback time.

      Roger

       
      • Roger,

        Maybe I am missing something?  Or maybe we just disgree?
        Here's how I see things:
        If VBISAM is NOT operating in 'transactional mode', then any calls made to isclose() will release any and all locks held.  This is because the low level call to close(2) implicitly releases any held locks.  The fact that this might NOT release the locks synchronously on a Wind-Blows syste, is, in my opinion, a BONUS!
        Let's now examine what happens when we find ourselves within an isbegin() + isrollback() / isommit() phase...
        If we call isclose(), we can NOT close(2) the O/S files (.idx and .dat) simply because we must MAINTAIN any locks held.  The actual close(2) call is intentionally postponed until the time of the corresponding iscommit() / isrollback() call.  Furthermore, if we end up calling isrollback(), we need to be able to 'un do' everything that we have done since we called isbegin().  Let's assume that the table was isopen()'d using ISEXCLLOCK before isbegin() was called.  We then call isbegin(), do some stuff and then isclose() the table.  Finally, we call isrollback() for whatever reason.
        After we return from the isrollback() call, the system *SHOULD* have the table open in ISEXCLLOCK mode just as it was when isbegin() was called!
        In order to guarantee the ability to do that, we must HOLD the ISEXCLLOCK for the duration of the transaction.
        (To put that another way, I believe we cannot ever LOWER the level of exclusivity while we're within a transacttion.  we can only increment it.)

        Perhaps I've 'missed' something?

         
    • Roger While
      Roger While
      2008-08-31

      Yep, that is a fair assessment.
      However, for the non-transaction case, we should release the file lock
      maybe in close2, maybe lower where the actual close is.

      The question here is what exactly the file state should be -
      a) after iscommit
      b) after isrollback

      This is not readily clear to me from the doc.
      It also is not clear what the situation is/should be when calls
      are made before the isbegin.

      Incidentally, here are latest release notes for C-ISAM -
      http://publib.boulder.ibm.com/epubs/html/c2399240.html

      Interesting is this -
      *************
      The isrelease() Function
      ------------------------
      After releasing all locks on records, as described on
      page 8-51, isrelease() also unlocks the open file
      descriptor that its argument specifies.
      *************

      Also note the log file format (This has been like described since 7.25 released sometime in 2000)

      Note there has not been an update to the C-ISAM programmers reference (pdf) since 2001.

      Roger

       
    • Roger While
      Roger While
      2008-08-31

      Trevor,
      In isopen, this looks to me to be wrong -

              if (iVBInTrans == VBNOTRANS)
              {      
                      iVBTransOpen (iHandle, pcFilename);
                      psVBFile [iHandle]->sFlags.iTransYet = 1;
              }

      I think the test should be != or?

      Roger