[Valgrind-users] Performance debugging of Linux process scheduling

[Alireza H. <al...@cs...>]

I am developing a high-performance multi-threaded application on the
Linux platform. Ideally it is expected that all similar threads run
with the same speed. However, there are some situation which One
thread may execute while another is waiting for some other system
latency. One example of system latency is timer interrupt handlers
(LOC) which periodically runs on all CPU cores.

I am looking for a low overhead way to perform system performance
debugging and detect such a situations. More specifically, I need to
know WHY the application threads does not run with ideal speed and
which system task is responsible for that wait state.

I think there are some performance statistics about process scheduling
in /proc/PID/sched likenr_wakeups. However, I am more interested to
know WHY rather than How Many.

I used to run ftrace while my application is running to monitor Linux
kernel function activities and detect execution of any unusual tasks
or interrupts. That was how I detect timer interrupt handlers put my
application threads in wait state. However, this method has very high
performance overhead and its hard to interpret the output.

Ideally, I can put my application threads to specific CPU cores. Then
I need a performance monitoring tool which tells me what other
processes has been executed on these CPU cores during the execution of
my application.

Does Valgrind a tool to help me out ?

Thanks