Re: [Valgrind-developers] Data tracking tool

[Julian S. <js...@ac...>]

>  I used this tool to evaluate the amount of data
> transfer between different functions if they were
> executed on , say, different processors.  I was
> wondering if there has been a talk about such a tool
> and if  it would be useful to other people. Of course,
> the code is not release quality yet but I can work on

This sounds similar to a tool discussed a couple of years
back; it came to be referred to Bandsaw.  The idea is
essentially the same, except that we thought it would be valuable
for knowing the amount of data transferred even within the
same process.  I think Jeremy made a partial prototype 
implementation at one stage, but it went no further.

So I'd be interested to see some example results from your 
tool.  Can you send some?  I'd be interested to see if a
tool along these lines can be made to work well, since if
it could, it would be useful.

J

----------

Some comments from long ago:

[...]
All these things gave me an idea.  I can't easily show you where 
you made redundant copies, but I can easily enough show you how 
data flows from place to place in your program through memory.
Consider this:

               int a[];
               register int sum;

   line A:     for (i = 0; i < 10*1000*1000; i++) 
                  a[i] = ... whatever ...

   (in some other place in the code)
   line B:     for (i = 0; i < 10*1000*1000; i++) 
                  sum += a[i];

Whenever data is written to memory, the tool writes into shadow memory
the program location doing the write.  When data is read, the tool
knows the location doing the read, and by inspecting shadow memory it
knows who put that data there.  It can therefore increment a counter
tracking the volume of data transferred across this (source, destination)
pair.  For the above fragment, assuming it was executed once, it
would say

   40 MB transferred from (file.c line A) to (file.c line B)

Imagine now doing this across a whole program, and having a GUI
tool like kcachegrind to show the results.  The tool shows each line
of code and graphically illustrates (possibly using width of arrows
or something) the locations to which this line sent data, or received
data, and the quantity of data transferred.

In short, the tool considers memory as a channel for transferring 
data between different parts of your program, and shows exactly
how you are using that (large but finite) bandwidth.