From: "John Levon" <levon@...>
Sent: Monday, March 04, 2002 11:31 PM
> On Mon, Mar 04, 2002 at 04:52:22PM -0500, Phillip Ezolt wrote:
> > Hi All,
> Hi Phillip.
[DCPI label - Epoch]
> > I thought that oprofile had this feature and called them "sessions" (a
> > much more sensible name, IMHO.) Unfortunately, I can't figure out
> > anyway to specify a new session.
> Currently a new session is started when some parameters change, between
> individual runs of the daemon.
> Adding in an epoch feature would be pretty easy I think, and another
> good idea.
We just need to add the notion of named session. I don't see exactly
the difference between labels and epochs. Is labels just a trick to avoid
restart a different epochs ?
> > 3) How does oprofile deal with modules on ramdisks?
> We (quickly) ignore any such samples - there is no (automatic) way to
> find the binary. But I don't think there's any real requirement for us
> to have access to the binary whilst the daemon is running; currently we
> just map a sample file based on the size of the binary, when in fact we
> could be truncating the sample file to a larger size when we get an out
> of bounds sample (I think - Phil E. ?)
A) growing and remapping the samples files would work. There is
also a related problem in post-profile code. (nr samples calculation)
> > 4) Can sample totals be stored at the beginning of the sample files?
> We've been planning something similar for a while, the post-profile
> tools are WAY too slow.
> In the future, we'd like to able to freeze a profile state into a
> processed form, for quick analysis without all the trashing of pages.
> > I believe that this is because it is mmaping all of the files in the
> > sample directory. This isn't necessary for op_time. It only needs
> > to know totals.
B) At my eyes that's a problem in the linux kernel: when we mmap
or read a sparse file the hole are stored in the cache and so
on clobber all memory with zeroed blocks. try with a "little
$ du -h /var/opd/samples
and look the cached field...
> > 5) Can the sample files be compressed?
> > Your sample files compress enormously.
> > -rw-r--r-- 1 ezolt csdpg 23296560 Mar 4 16:36 red-carpet
> > -rw-r--r-- 1 ezolt csdpg 37487 Mar 4 16:37 red-carpet.gz
> try a stat :)
> Unless you're storing your sample files on vfat or something, they
> should be sparse files and take up very little actual disk space.
C) Even when comparing it to du -h result the compression ratio
is impressive. That's recall to me we don't *greatly* discourage
the use of a filesystem w/o sparse files support (vfat and
network fs ?) in the documentation.
> > By using zlib to read and write them, you could dramatically reduce
> > the amount of I/O necessary to total all of the samples.
> This is pretty difficult compared to the current method of just
> incrementing a counter in a file-backed mmap page anyway :)
D) I think Philipp means: using for post-profile the compressed
files will give a great improvement and, at least at a memory
use point of view, its a real improvement. I've think in the past
to compress file when creating a new session but I often
use post-profile tools on a running session...
John, we have discuss in the past of the sparse file format.
I think we must (see A to D point) eject it even if this increase
(moderately) the overhead of oprofile.
The best promising data struct, at my eyes, to store samples
is an in-memory B tree build inside a growable mmaped file
(one tree per image) with little size page (probably less than 16
entry by page)
Philipp you have work with DCPI: what was roughly the overhead
of DCPI ?
> > 6) Please keep the columns consistent/spread-sheet friendly.
I'm amazing than nobody, myself too, have never complain about
the output format
> > The column layout of the output from op_time and oprofpp differ. If one
> > were to write a GUI that sits on top of oprofile, it would be easier to
> > parse the output if it was identical in all cases.
plain op_time (ie w/o -l option) have not yet a columned output. I have
started an oprofpp like GUI tools a few days ago but I don't rely on the
output of post-profile tools, I build it on the top of container used
to store samples in post-profile tools.
For now I've planned only two view, one grouping oprofpp -l/-L
capabilities with a tree view of samples and one another to see
graphically hotspot in code. I'm interested by idea/help on what
to implement in GUI post-profile tools. Because we are near a
release period I prefer to not open a branch in cvs and wait a few
day before committing the GUI.
> > 7) Any plan on supporting the Pentium IV counters?
> > Self Explanatory.
> lack of test machines :/
> It wouldn't be too hard ... maybe one day I will write some support for
> the basic mechanism "blind", as at least a starting point for an
> interested developer with a P4 machine.
> PEBS support is a bigger task.
we can perhaps take a more precise eip value from the PEBS. I hope
this eip doesn't suffer from the irq latency problem.
It will be nice also to make port of oprofile to other architecture.