#159 Reload cached data sets when updated

2.7.3
open
opendkim (98)
5
2013-09-30
2012-12-18
Daniel Black
No

A lot of database backends are file types. It would be good if a lookup tested if its stat mtime on the file had changed since opendkim was started and reload this before performing the reset of the lookup.

Changing these files on the fly is useful rather than shutting down the opendkim service.

Discussion

1 2 > >> (Page 1 of 2)
  • Sites for which these data sets change frequently tend to use non-trivial data files like db:, dsn:, ldap:, memcache:, etc. Is there really a use case where file: data sets change often, where moving to one of the other formats would not be a better choice?

     
    • summary: database file types to reload if updates --> Reload cached data sets when updated
    • status: open --> pending
     
  • Daniel Black
    Daniel Black
    2012-12-18

    I was trying the sleepycat db: and it didn't seem to be picking up changes.

    was loading in changes with db4.8_load -t hash -T test.db on a domain db:/etc/test.db map with opendkim running.

    I will check again.

     
  • Daniel Black
    Daniel Black
    2012-12-18

    • status: pending --> open
     
    • priority: 5 --> 6
     
  • There are only two data sets that are file-backed which we control, namely file: and refile:. I have a simple "reopen" patch in test now. It would probably work for Sleepycat databases too, but I haven't tried it; then again, it shouldn't be necessary for them.

    It looks like db_load opens an existing database and modifies it in-place, which is what we want. I don't know why this wouldn't be seeing new or modified entries due to db_load since we do keep the descriptor open, meaning we're reading from the same file db_load updated.

    I'll do some tests myself.

     
  • I can see what you're seeing. The inode isn't changing, and a reopen does pick up the new records. This leads me to believe that Sleepycat does some caching. We need to figure out if the cache is tunable or can be manually flushed.

     
  • Posted a question to the Sleepycat forum. We'll see if I get a reply back.

     
    • milestone: --> 2.7.3
    • assigned_to: nobody --> cm-msk
     
    • priority: 6 --> 5
     
1 2 > >> (Page 1 of 2)