Does close() also imply that sync() is called

Kevin Day
2005-08-19
2013-06-03
  • Kevin Day
    Kevin Day
    2005-08-19

    Does anyone know if calling close() on a file stream or RAF implicitly calls GetFD().sync() ?

    I'm wondering if there is a potential chink in the transaction armor during shutdown of jdbm record file...

    I know in NTFS, disk writes are cached by the operating system.  I'm assuming that getFS().sync() actually forces the writes to disk.  My strong suspicion is that close() does NOT do this...

    Unless I had a very clear guarantee that close() also causes a sync(), then I think the close() in the record file should explicitly call sync().

    I know that the transaction manager does this - but if you are running without transcations then you could wind up with a corrupted file - even if though you closed it.

    Am I off here?

    Thanks,

    - K

     
    • Alex Boisvert
      Alex Boisvert
      2005-08-19

      I don't believe close() implies a FD sync.

      I guess the question should be whether JDBM should do a sync by default or leave it to the application to do it (for those applications that need it).

      I would be OK with a solution where JDBM would sync by default with an option to not do it (following the principle of least damage).

      alex

       
      • Bryan Thompson
        Bryan Thompson
        2005-08-22

        I think that I am seeing a problem that may be linked to the failure to sync to disk.  I am running tests which close and re-open the record manager in order to verify that data has been coherently persisted.  If I want to try a sync(), where would it go in the jdbm code?

        The exception that I am seeing is below.  Since it is trying to read past the EOF, I find this suggestive that perhaps not all data has been flushed to disk.

        java.io.EOFException
            at java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:2635)
            at java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:867)
            at jdbm.btree.BPage.deserialize(BPage.java:967)
            at jdbm.recman.BaseRecordManager.fetch(BaseRecordManager.java:341)
            at jdbm.recman.CacheRecordManager.fetch(CacheRecordManager.java:304)
            at jdbm.containers.LogicalRecordManager.fetch(LogicalRecordManager.java:578)
            at jdbm.btree.BTree.getRoot(BTree.java:533)
            at jdbm.btree.BTree.find(BTree.java:409)
            at org.CognitiveWeb.generic.store.jdbm.JDBMObjectManager.getPropertyClass(JDBMObjectManager.java:473)
            ... 19 more

         
        • Kevin Day
          Kevin Day
          2005-08-22

          The sync() call you want is in the RecordFile class.  I'll be adding some code to call this automatically in the RecordFile.close() method (currently, only commit() gets called, and if you have transactions disabled, this does not call sync).

          However, from what you are describing, it sounds like maybe you are forgetting to call close() on your record manager before the app closes...  sync() should only be necessary if you absolutely want to the data to get to disk (And not the operating system's file cache).

          If you are closing the record manager properly, and the OS isn't losing it's connection to disk (or crashing), then that last call to sync() should not be required.

          My interest in sync() is for storing data onto a network connection that could be terminated without warning.  If that happens, the app could terminate properly, but still lose data.

          If you are writing to a local disk, it would be difficult for this failure mode to exhibit itself.

          - K

           
        • Bryan Thompson
          Bryan Thompson
          2005-08-22

          Hello,

          I *think* that the right place for the sync is line 313
          of jdbm.recman.RecordFile.

          >        sync();        // BBT 8/22/2005
                  file.close();
                  file = null;

          Can anyone confirm this?

          -bryan

           
          • Kevin Day
            Kevin Day
            2005-08-22

            Yes.  That's the correct place.

            However, it would be better to only call sync if transactions are disabled (it gets called by the transaction manager already if they are not disabled), and to provide an option in the record manager, etc... to allow users to turn this sync off if they don't like it.

            This should be the default behavior, though.

            If you'd like, I can include this in my release that adds auto-commit when transactions are disabled.

            Like I said, though, I don't think this will have an impact on the error you are seeing.  If you call close(), all data will be written to the OS's disk cache (if not the disk)...

            - K