From: Rob F. <rj...@cs...> - 2006-02-28 03:48:08
|
> > On Mon, Feb 27, 2006 at 05:53:44PM -0600, Maynard Johnson wrote: > >> Maynard Johnson wrote: >> Ping. Has anyone had any time to look at this? John, did I answer your >> questions with the revisions I made? > > I just don't have the time to look at this right now :( > > john > > > I've been meaning to send a reply to this, to the discussion about the API (C: yes, C++: a nightmare of compatibility issues), and reply to a message from John with questions about about design decisions I described in my Dec 21 2005 message to this list. I will try to discuss this at length next week. Unfortunately, I've been swamped in other stuff and will be for at least another week. In the interest of timeliness, though, here are some short comments: My impression, admittedly with only a brief scan, is that your format would be fine for high level reports, it will be way too verbose for reporting detailed profiles of really big programs. For example, in <Event Event_ID="PM_INST_FROM_L35_GRP7" unit_mask="0x0" setup_count="60000" sample_count="35" /> everything except the sample count is redundant wrt every other line that reports the same event. We defined a very lean format for our use. See http://hipersoft.rice.edu/hpctoolkit/documentation/HPCToolkit/doc/dtd_profile.html for a description of an older version. Our approach is not to extract everything to XML. Even with the lean XML format for profile data, there is still a substantial dilation in the size of the data. We don't want to do this to every executable image on thousands of nodes of a huge cluster. Hence, we wrote a tool that extracts detailed profiles only for designated images. Regards, Rob Fowler |
From: Dave N. <dc...@us...> - 2006-04-27 23:06:39
Attachments:
oprof.4.xsd
example.4.xml
|
Hi. I am planning on implementing the -X option to opreport to generate XML. I have been working with Maynard Johnson's XML schema and he and I have reworked some aspects of the schema. I plan to start with the patch that Junichi Uekawa posted on the mailing list on 02-06-06 (Re: How to auto-process opreport output?) We have done some restructuring of the original XML schema that Maynard posted to this list and I think we have addressed some of the XML size issues raised by some developers as well as added some new elements for scenarios that the initial schema did not handle. Any comments on the revised schema would be welcome. The example.4.xml is the XML associated with a profiling run using --separate=thread, and opreport -X --details --debug-info. If opreport were run without --details or --debug-info then the SymbolData elements would not contain the symbolic information, and the Symbol elements would not contain any Line elements, only a single SampleData element containing the summary information for that Symbol. |
From: Maynard J. <may...@us...> - 2006-03-01 15:19:19
|
Rob Fowler wrote: > >> >> On Mon, Feb 27, 2006 at 05:53:44PM -0600, Maynard Johnson wrote: >> >>> Maynard Johnson wrote: >>> Ping. Has anyone had any time to look at this? John, did I answer >>> your questions with the revisions I made? >> >> >> I just don't have the time to look at this right now :( >> >> john >> >> >> > > > I've been meaning to send a reply to this, to the discussion > about the API (C: yes, C++: a nightmare of compatibility issues), > and reply to a message from John with questions about about design > decisions I described in my Dec 21 2005 message to this list. I > will try to discuss this at length next week. > > Unfortunately, I've been swamped in other stuff and will It must be contagious. The swamp is up to my chin, too. :-) > be for at least another week. In the interest of timeliness, > though, here are some short comments: > > My impression, admittedly with only a brief scan, is that your > format would be fine for high level reports, it will be way too > verbose for reporting detailed profiles of really big programs. > For example, in > <Event Event_ID="PM_INST_FROM_L35_GRP7" unit_mask="0x0" > setup_count="60000" sample_count="35" /> > everything except the sample count is redundant wrt every other This was simply a mistake in the hand-coding of the example XML. The schema defines Event such that unit_mask, setup_count, and sample_count are optional and should only be inserted into the resultant XML instance as appropriate. I concede the point that including the entire string of the event ID for each sample would be unnecessary. This could be easily fixed by including an 'id' attribute in Event. Then, in the singleton SampleInfo defined for the entire profile, the Event elements would have their 'id' attribute set to some unique value. Then the SampleInfo elements representing actual samples could use the 'id' to refer to the appropriate pre-defined Event. I'll work on this change (as soon as I come up for air) and post the revised XML schema. > line that reports the same event. > > We defined a very lean format for our use. See > http://hipersoft.rice.edu/hpctoolkit/documentation/HPCToolkit/doc/dtd_profile.html > > for a description of an older version. Your very lean approach to element and attribute naming makes it difficult for me to say how close we are conceptually. But I suspect that your PGM element is very close to my Module element. The element and attribute names I chose for the proposed schema are more or less self-describing. But while such verbose names may be useful for discussion, you make a good point that this would result in unnecessarily large XML instances. Once we agree on structure, the names should definitely be abbreviated. > > Our approach is not to extract everything to XML. Even > with the lean XML format for profile data, there is still > a substantial dilation in the size of the data. We don't > want to do this to every executable image on thousands of > nodes of a huge cluster. Hence, we wrote a tool > that extracts detailed profiles only for designated images. My intention with the proposed schema design is that it would be flexible enough to include as little or as much as what the user asks for in their profile spec passed into opreport. Thanks for the constructive comments. Regards, -Maynard > > Regards, > Rob Fowler > > > ------------------------------------------------------- > This SF.Net email is sponsored by xPML, a groundbreaking scripting > language > that extends applications into web and mobile media. Attend the live > webcast > and join the prime developer group breaking into this new coding > territory! > http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642 > _______________________________________________ > oprofile-list mailing list > opr...@li... > https://lists.sourceforge.net/lists/listinfo/oprofile-list |
From: Junichi U. <da...@ne...> - 2006-03-06 14:40:45
|
Hi, > > Our approach is not to extract everything to XML. Even > > with the lean XML format for profile data, there is still > > a substantial dilation in the size of the data. We don't > > want to do this to every executable image on thousands of > > nodes of a huge cluster. Hence, we wrote a tool > > that extracts detailed profiles only for designated images. > > My intention with the proposed schema design is that it would be > flexible enough to include as little or as much as what the user asks > for in their profile spec passed into opreport. > > Thanks for the constructive comments. I have not yet had the time to fully review your schema, but I have my concerns that a full XML output might be too big in size for practical usage. Do you have an example XML output for a system under load, or an estimate over size? XML processing isn't free, if it's going to take ~10 seconds on the latest hardware, this design might be a bit of a problem, at least for my use of using it as a interface between emacs and oprofile. regards, junichi -- dancer@{debian.org,netfort.gr.jp} Debian Project |
From: Maynard J. <may...@us...> - 2006-03-06 17:48:38
|
Junichi Uekawa wrote: > Hi, > > >>>Our approach is not to extract everything to XML. Even >>>with the lean XML format for profile data, there is still >>>a substantial dilation in the size of the data. We don't >>>want to do this to every executable image on thousands of >>>nodes of a huge cluster. Hence, we wrote a tool >>>that extracts detailed profiles only for designated images. >> >>My intention with the proposed schema design is that it would be >>flexible enough to include as little or as much as what the user asks >>for in their profile spec passed into opreport. >> >>Thanks for the constructive comments. > > > I have not yet had the time to fully review your schema, but I have my > concerns that a full XML output might be too big in size for practical That's why many of the elements are optional. So, for example, say you run the profiler without separate=thread, and then you invoke opreport with a profile spec to restrict ouptut to only certain binary images. Then the resultant XML generated would contain one Module element per requested binary image, plus only the SymbolData elements (i.e., "samples") corresponding to the specified binary images. Does this answer your concern, or do you see something in my proposed schema that would preclude an optimized output? I appreciate your time in looking at this. Regards, -Maynard > usage. Do you have an example XML output for a system under load, or > an estimate over size? > > XML processing isn't free, if it's going to take ~10 seconds on the > latest hardware, this design might be a bit of a problem, at least for > my use of using it as a interface between emacs and oprofile. > > regards, > junichi |