Re: [Lse-tech] [PATCH] Process Notification (task_notifiers)

[Matt H. <mat...@us...>]

On Mon, 2005-10-10 at 22:21 -0500, Jack Steiner wrote:
> On Mon, Oct 10, 2005 at 03:37:08PM -0700, Chandra Seetharaman wrote:
> > Jack,
> > 
> > Forewarning: I am looking to get the least common functionality
> > separated.
> >  
> > It looks simpler than the earlier version and has less overhead. But,
> > not everybody that would use this functionality would need the callout
> > list to be maintained in the task and having the overhead of copying it
> > over to a new process. For example, CKRM and Process Event Notifier
> > would use this feature but don't need it to be maintained per task.
> 
> Interesting. That points out our different environments. For many of the
> HPC workloads that use the PAGG/pnotify/task-notifiers, keeping the lists
> in the task_struct is a performance win because only a small percentage
> of tasks actually have non-null lists. And most that do are long 
> running tasks. A global list adds more overhead & has locking issues.. 
> 
> Am I making a mistake trying to create a single mechanism that all can
> use? Do we need different but similar mechanisms? Current proposals include:
> 
> 	- task_notifiers: per-task list, no locks, restricted to own-task attachment
> 	- pnotify/PAGG: per-task list, locks, no restrictions on task attachment
> 	- global list: nothing in task_struct, global lock, no restrictions on task attachment
> 
> Hmmmm....

	Unfortunately nothing else I've thought of so far beats calling the
code directly -- which avoids locking and per-task data structures.

	Might it be possible to manage a global task_notifier list without a
lock? Perhaps using RCU? In the case of RCU I think we'd have to
disallow self-removal within notifier functions on the global list.
Addition and removal would take place at module load/unload time and be
protected with a spinlock while the read and calling of the notifier
function would happen within an RCU read lock. However, I'm always
skeptical of my understanding of RCU's (mis)uses so I'm not sure if this
would work correctly. Comments?

Cheers,
	-Matt Helsley

The patch I've trimmed from this reply is at:
http://marc.theaimsgroup.com/?l=lse-tech&m=112869558116290&q=raw