I don't recall all the details but here is the kind of stacktrace we got:
Looking at what is going on in add_event_entry(), I don't see anything but
a freed event_buffer as the root cause of the problem.
On Thu, Oct 8, 2009 at 7:20 PM, Robert Richter <robert.richter@...> wrote:
> On 09.09.09 15:02:33, Stephane Eranian wrote:
>> Looking at the 2.6.31-rc9 code, it appears there is a race condition
>> in the event_buffer
>> cleanup code path (shutdown). This could lead to kernel panic as some
>> CPUs may be
>> operating on the event buffer AFTER it has been freed. The attached
>> patch solves the
>> problem and makes sure CPUs check if the buffer is not NULL before
>> they access it
>> as some may have been spinning on the mutex while the buffer was being freed.
>> Signed-off-by: David Rientjes <rientjes@...>
>> Signed-off-by: Stephane Eranian <eranian@...>
> I was looking for the root cause for the event buffer access after
> freed and could not find it. Also I could not trigger the bug by
> stressing a system with shutdowns during sampling.
> The code looks good:
> The event buffer is freed in free_event_buffer() which is only called
> (with exception of error handlers during startup) in
> oprofile_shutdown(). oprofile_shutdown() is locked by the start_mutex
> and is calling sync_stop() before freeing event_buffer.
> The event_buffer is used in add_event_entry() that is called (in the
> end) only from sync_buffer() and oprofile_put_buff(). Your use case is
> probably only sync_buffer(), that is called from the notifiers or the
> per cpu workqueues. All notifiers and workqueues are shutdown cleanly
> in sync_stop() with blocking unregistration and
> flush_scheduled_work(). Since the start_mutex is set, no buffer syncs
> will be scheduled anymore.
> So, the question is, when does this happen? Do you have a stacktrace
> or could you trigger this again with WARN_ON(!event_buffer)?
> Advanced Micro Devices, Inc.
> Operating System Research Center
> email: robert.richter@...
Stephane Eranian | EMEA Software Engineering
Google France | 38 avenue de l'Opéra | 75002 Paris
Tel : +33 (0) 1 42 68 53 00
This email may be confidential or privileged. If you received this
communication by mistake, please
don't forward it to anyone else, please erase all copies and
attachments, and please let me know that
it went to the wrong person. Thanks