Re: [Crm114-general] *.css race conditions? - was: New Rev with Paolo's Fix and More

[Richard E. <re...@ya...>]

On Thu, Feb 12, 2004 at 11:59:11AM -0500, Raul Miller wrote:
> On Thu, Feb 12, 2004 at 11:39:55AM -0500, Richard Ellis wrote:
> > Again, lockfile does this, minus the "cmd cmdargs" part.
> ...
> > It locks multiple files, in order, and can be asked to retry a
> > limited number of times, and to abort and "break" the lock after a
> > certain time limit.
> 
> The other difference between lockfile any my hypothetical locker is that
> lockfile doesn't create files, it locks existing files.

>From lockfile manpage:

  DESCRIPTION
         lockfile can be used  to  create  one  or  more  semaphore
         files.

It does create the "lock" files.  If the file does not exist, it is
created, and lockfile exits with success.  If the file exists,
lockfile waits for it to disappear (be removed) and then exits with
success.  That's all it does, but that's sufficient to serialize
parallel access to some other shared resource.  It's the old standard
"dot-file locking" system where everyone agrees to check for the
".lock" file first, and only proceed if no ".lock" file is found,
after creating one of course.

Is it perfect, granted, no, it's large granularity, only one person
can be using the protected resource, the whole protected resource, at
once.  For finer grained graunularity (lock offset X to Y in
spam.css) requires that the locking code be moved into crm as part of
it's handling of the .css files.  But really, I doubt we need that
finer grained locking.  We just need to make sure that when crm
process X has mmapped the .css files to change them, that no other
crm process can jump in behind it's back (uniprocess task switch, SMP
parallel, etc.) and change them before it gets a chance to write it's
changes back out to disk.

We essentially do this inside mailfilter

lockfile ".crm.modify.css.lock"
  mmap spam.css for modify
  modify mmap'ed copy
  mflush map
rm -f ".crm.modify.css.lock"

And we have prevented parallel crm processes from stepping on each
other's toes.

> >          Lockfile is NFS-resistant and eight-bit clean.
> 
> Hmm... I don't know if it's possible to get reliable locking at all
> on NFS.  NFS-resistant might be better than what I can do, but the
> only really general technique I know of is to make sure that you
> don't have name space collisions at all for writes from independent
> nfs clients.

Correct, NFS presents a whole slew of other problems related to
controlling access to shared resources.  That's the reason for
lockfile's presence in procmail, as a single central point to place
all of that NFS intelligence, so there's only one code base to maintain. 
While still allowing it "the intelligence" to be useful to shell
scripts and the like.

All I'm advocating is, instead of trying to reinvent a new whiz-bang
wheel, we should at least consider resuing someone else's lengthy
effort at producing a pretty good whiz-bang wheel.  It will work for
what we need, and it already includes well over ten years worth of
work on making it NFS resistant.  Chances are, any new whiz-bang
wheel we'd create would have to eventually relearn all the lessons
that lockfile has had to learn along the way.  

And we don't have to "fork" lockfile either.  Just distribute the
necessary files (unchanged from procmail) to compile it in a subdir
in the crm tarball.  Then, if the procmail maintainers ever update
lockfile (to learn a new lesson), we just copy it (again unchanged)
into crm, and we gain from their effort.