[Lse-tech] Re: [PATCH] O(1) scheduler set_cpus_allowed for non-current tasks

[Ingo M. <mi...@el...>]

On Thu, 21 Feb 2002, Paul Jackson wrote:

> Could you, or some other kind soul who understands this, to explain
> why the following alternative for migrating proceses currently running
> on some other cpu wouldn't have been better (simpler and sufficient):
>
>     - add another variable to the task_struct, say "eviction_notice"
>     - when it comes time to tell a process to migrate, set this
> 	eviction_notice (and then continue without waiting)
>     - rely on code that fires each time slice on the cpu currently
>         hosting the evicted process to notice the eviction notice and
> 	serve it (migrate the process instead of giving it yet another
> 	slice).

well, how do you migrate the task from an IRQ context? This is the
fundamental question. A task can only be removed from the runqueue if it's
not running currently. (the only exception is schedule() itself.)

even if it worked, this adds a 1/(2*HZ) average latency to the migration.
I wanted to have something that is capable of migrating tasks 'instantly'.
(well, as 'instantly' as soon the migrated task can be preempted.) I also
wanted to have a set_cpus_allowed() interface that guarantees that the
task has migrated to the valid set.

i'm doing one more variation wrt. migration threads: instead of the
current method of waking up the target CPU's migration thread and 'kicking
off' the migrated task from its current CPU, it's much cleaner to do the
following:

 - every migration thread has maximum priority.
 - in set_cpus_allowed() we wake up the migration thread on the *source*
   CPU.
 - the migration thread preempts the migrated thread on the source CPU -
   and it can just remove the migrated task from the local runqueue. (if
   the task is still there at that point.)

this is even simpler and does not change p->state in any way.

	Ingo