From: John L. <le...@mo...> - 2006-02-21 14:30:46
|
On Mon, Feb 13, 2006 at 02:58:45PM -0600, Yeh, Jason wrote: > This mail includes the first revision of the previous api patch. I try > to incorporate as much suggestions from John and Ralf as possible. > However, there are still few things not completed: I think we should seriously consider making it a C API (needs to link libstdc++ still, but using C makes it a lot more accessible). Since the API shouldn't expose *any* structures, this shouldn't be too difficult, we just pass out foo.c_str() etc. to iterator callbacks > 1. What's the preferable way to handle errors within Oprofile library? > Would throwing exceptions be preferred than returning error code/failure > or vice versa? Error codes. Given my extremely limited time, would it be feasible to work up API documentation first? IE exactly what functions are exposed to the users, examples etc. This would be much easier to review than actual code right now... cheers john |
From: Yeh, J. <jas...@am...> - 2006-02-13 22:57:07
|
In addition to the missing feature I mentioned in my previous my. I neglected to mention that both call graph data and session enumeration is still not implemented in the library yet. The next version should include both features. regards, Jason |
From: Yeh, J. <jas...@am...> - 2006-02-15 21:01:39
|
Hi, This email includes a brief description of the Oprofile API patch send few days earlier. The usage of the proposed Oprofile API is revolved around the class currently named op_summary. A typical sequence of using the API would be similar to: a)Create an instance of op_summary. b)If desired, retrieve available events through the event begin iterator. c)If desired, retrieve available session through the session begin iterator. d)Create an instance of struct op_profile_spec and set the desired spec option. Details of each each option will be explained in later sections. e)Create an instance of op_summary::op_separate_options and set the desired separate options. f)Set the options of op_summary via calling op_summary member function set_profile_spec(...) g)At this moment, profiling data can be retrieved via getting begin iterator for summary, symbols, and profile classes. h)Go back to set d), with different options if different result is needed. By default, all profiling data is merged. Instance of op_separate_options can be ORed with any combination of separate_cpu, separate_lib, separate_tid, separate_tgid, and separate_unitmask. op_profile_spec can be used to specify specific information to be retrieved. a) image_name: List of executable full paths of the binary the user want to retrieve data from. Corresponds to opreport command line option "image:<...>,<....>,..." b) lib_image: List of full paths of libraries the user want to retrieve data from. c) session: List of session that the Oprofile pp should include. Corresponds to option "session:<...>,<...>,..." d) tgid: List of interested tgid. Corresponds to option "tgid:<...>,<...>...". e) tid, List of interested tid. Corresponds to option "tid:<...>,<...>...". f) count: List of event counter count. Corresponds to option "count:<...>,<...>..." g) cpu: Retrieve only data collection on this list of cpus. Corresponds to option "cpu:<...>,<...>..." h) event_name: Return only data of this list of events. Corresponds to option "event:<...>,<...>..." Jason |
From: Yeh, J. <jas...@am...> - 2006-02-21 17:20:53
|
John, I will work on a generic document describing the API and send it out this week. As for the C API, I have no problem with it. It would just be a thin layer on top of the C++ code. I would like some clarification though. I am not sure the meaning of the term iterator in the C API context. Can you give me a concrete example or maybe a project, code segment that demonstrate it? =20 thanks, Jason -----Original Message----- From: John Levon [mailto:le...@mo...]=20 Sent: Tuesday, February 21, 2006 8:32 AM To: Yeh, Jason Cc: opr...@li... Subject: Re: OProfile API discussion -- CodeAnalyst On Mon, Feb 13, 2006 at 02:58:45PM -0600, Yeh, Jason wrote: > This mail includes the first revision of the previous api patch. I try > to incorporate as much suggestions from John and Ralf as possible. > However, there are still few things not completed: I think we should seriously consider making it a C API (needs to link libstdc++ still, but using C makes it a lot more accessible). Since the API shouldn't expose *any* structures, this shouldn't be too difficult, we just pass out foo.c_str() etc. to iterator callbacks > 1. What's the preferable way to handle errors within Oprofile library? > Would throwing exceptions be preferred than returning error code/failure > or vice versa?=20 Error codes. Given my extremely limited time, would it be feasible to work up API documentation first? IE exactly what functions are exposed to the users, examples etc. This would be much easier to review than actual code right now... cheers john |
From: John L. <le...@mo...> - 2006-02-21 17:53:24
|
On Tue, Feb 21, 2006 at 11:20:02AM -0600, Yeh, Jason wrote: > As for the C API, I have no problem with it. It would just be a thin > layer on top of the C++ code. I would like some clarification though. > I am not sure the meaning of the term iterator in the C API context. > Can you give me a concrete example or maybe a project, code segment that > demonstrate it? You just return an opaque type something_t, then pass that to a: for_each_symbol(something_t, funcptr_t) john |
From: Yeh, J. <jas...@am...> - 2006-02-28 20:52:07
Attachments:
Oprofile API Proposal.html
|
Please take a look at the html file of the C API proposal. As mentioned before, the C API is just a wrapper of the C++ implementation patch I send a little over a week ago. Please let me know if I missed any functionality or profile configurations. Jason |
From: Ralf W. <Ral...@gm...> - 2006-02-28 22:02:53
|
Hi Jason, * Yeh, Jason wrote on Tue, Feb 28, 2006 at 09:51:47PM CET: > Please take a look at the html file of the C API proposal. As mentioned > before, the C API is just a wrapper of the C++ implementation patch I > send a little over a week ago. I hate to be a bugger again, but: | +----------------------------------------------------------------------+ | |Data Types | | |----------------------------------------------------------------------| | |op_t|An opaque data type which represents an instance of Oprofile API.| | +----------------------------------------------------------------------+ I think this name is too generic. Searching the web for it shows that several software packages use op_t for a generic operator. It would be nice to use a more descriptive name in a global namespace. What about op_instance, given that names ending in _t are reserved by ISO C anyway? Cheers, Ralf |
From: Yeh, J. <jas...@am...> - 2006-02-28 22:36:40
|
Thanks for the suggestion. =20 Looks like there are quiet a few of reserved prefix and suffix: http://www.gnu.org/software/libc/manual/html_node/Reserved-Names.html Jason -----Original Message----- From: Ralf Wildenhues [mailto:Ral...@gm...]=20 Sent: Tuesday, February 28, 2006 4:03 PM To: Yeh, Jason Cc: opr...@li... Subject: Re: OProfile API discussion -- CodeAnalyst Hi Jason, * Yeh, Jason wrote on Tue, Feb 28, 2006 at 09:51:47PM CET: > Please take a look at the html file of the C API proposal. As mentioned > before, the C API is just a wrapper of the C++ implementation patch I > send a little over a week ago. I hate to be a bugger again, but: | +----------------------------------------------------------------------+ | |Data Types | | |----------------------------------------------------------------------| | |op_t|An opaque data type which represents an instance of Oprofile API.| | +----------------------------------------------------------------------+ I think this name is too generic. Searching the web for it shows that several software packages use op_t for a generic operator. It would be nice to use a more descriptive name in a global namespace. What about op_instance, given that names ending in _t are reserved by ISO C anyway? Cheers, Ralf |
From: Yeh, J. <jas...@am...> - 2006-03-29 23:49:26
Attachments:
oprofile-0.9.1-api.patch
|
Here is the latest Oprofile API patch. The patch is created against 0.9.1 release. In addition to what was in the last patch, this patch contains the C API. The API differs slightly from the html document I sent. Most of the doc still applies though. If there is a need to update the html doc, I will send it out shortly after. Jason |
From: John L. <le...@mo...> - 2006-03-29 23:59:35
|
On Wed, Mar 29, 2006 at 05:43:37PM -0600, Yeh, Jason wrote: > the doc still applies though. If there is a need to update the html > doc, I will send it out shortly after. Please do. Sorry I'm being so useless on this... regards john |
From: Yeh, J. <jas...@am...> - 2006-03-30 20:26:22
Attachments:
Oprofile API Proposal.html
|
Here is the updated doc of the API. Let me know if there is any addition functionality needed. Jason -----Original Message----- From: John Levon [mailto:le...@mo...]=20 Sent: Wednesday, March 29, 2006 6:00 PM To: Yeh, Jason Cc: opr...@li... Subject: Re: OProfile API discussion -- CodeAnalyst On Wed, Mar 29, 2006 at 05:43:37PM -0600, Yeh, Jason wrote: > the doc still applies though. If there is a need to update the html > doc, I will send it out shortly after. Please do. Sorry I'm being so useless on this... regards john |
From: John L. <le...@mo...> - 2006-03-30 20:52:20
|
On Thu, Mar 30, 2006 at 02:26:08PM -0600, Yeh, Jason wrote: > Here is the updated doc of the API. Let me know if there is any > addition functionality needed. General: all functions should indicate what errors if any can happen. > void op_delete_instance(struct op_instance ** op_pt) Why does this take a ** ? > op_set_sep Confusion over what it returns if anything. > op_add_image_spec Command line supports globbing. Does this? How do I say "show me libc samples used by /bin/bash" etc. ? Need way to specify dep images like command line. > op_remove_session_spec Clarify that this returns back to the default. > op_add_cpu_spec Define behaviour when CPU isn't present in samples. Define behaviour if merged (as with all cases of these options). > op_add_event_spec What about unit mask, count? > op_iterate_sessions Does this apply to all available, or just those specified? > op_iterate_profile_class Expand unitmask type to 'long' for generality. How does user lookup from unit mask value to unit mask name? > name - Human readable name of profile class. Give examples/definition. > cpu - CPU number. > tgid - Thread group ID. > tid - Thread ID Define values if merged. > op_iterate_summary > only when the profile is separated by lib (see SEP_TID). Typo. > count_array - Sample count array. Do we need a simple way to get the number of profile classes for iterating across this? > op_iterate_symbol Do we need a way to iterate just for particular matching values? How can this be extended later to give detailed values (--details) ? Need way to ask for file/line number of just symbol - this implies we need a way to ask that we want this information in the first place. > op_iterate_symbol_sample Not clear on the difference here? > op_demangle Clarify return type if it won't fit. Size it should have been (see strlcpy) ? Others: how to specify --exclude-dependent (can be faster). How to specify -p. How to specify an oparchive to work on. Sort status of iterators, if any. How to manage versioning (op_create_instance should take an OP_VERSION value, and verify it can deal with it). Need to think about anonymous region code we have now and how it could be worked in. I'm probably missing some things; please, everyone, take a close look at this. All in all, though, a great start I think, Jason. thanks! john |
From: Yeh, J. <jas...@am...> - 2006-03-30 22:01:43
|
>> Here is the updated doc of the API. Let me know if there is any=20 >> addition functionality needed. > >General: all functions should indicate what errors if any can happen. > >> void op_delete_instance(struct op_instance ** op_pt) > >Why does this take a ** ? I reset the op_pt to NULL in the code. >> op_set_sep >Confusion over what it returns if anything. It does not return anything. It can not fail. >> op_add_image_spec >Command line supports globbing. Does this? The API simply passes the string into the Oprofile internal. I have not test globbing, but I am inclined to say yes. >How do I say "show me libc samples used by /bin/bash" etc. ? Need way to specify >dep images like command line. The API does not support it directly. To do that with current API the user need to add "/bin/bash" into the image list using "op_add_image_spec()".=20 The user then needs to find "libc" in the subsequent call to "op_iterate_summary_deps". >> op_remove_session_spec >Clarify that this returns back to the default. > >> op_add_cpu_spec > >Define behaviour when CPU isn't present in samples. Define behaviour if merged >(as with all cases of these options). When any of the options isn't present in samples "op_add_cpu_spec" has not effect. If the particular option is merged, it has not effect. >> op_add_event_spec >What about unit mask, count? The API currently has not way to specify unit mask and count now. I will add them in the next iteration of patch.=20 > >> op_iterate_sessions >Does this apply to all available, or just those specified? All available. It provide to let the user know what sessions are available. >> op_iterate_profile_class >Expand unitmask type to 'long' for generality. How does user lookup from unit mask >value to unit mask name? >> name - Human readable name of profile class. I will provide a function similar to the demangle fucntion to translate unit mask name to unit mask value or vice versa. >Give examples/definition. > >> cpu - CPU number. >> tgid - Thread group ID. >> tid - Thread ID >Define values if merged. If merged, the value will be NULL. =20 > >> op_iterate_summary >> only when the profile is separated by lib (see SEP_TID). Summary is always available, but summary depends is only available when profile is separated by lib. >Do we need a simple way to get the number of profile classes for iterating across >this? > >> op_iterate_symbol > >Do we need a way to iterate just for particular matching values? How can this be >extended later to give detailed values (--details) ? Need way to ask for file/line >number of just symbol - this implies we need a way to ask that we want this >nformation in the first place. Matching the particular values can be easily done by allowing the user to specify a comma separted list. The list will be used to create a string filter for matching correct value. I am a bit concerned on which name we should match, the mangled or demangled names? Details are giving in "op_iterate_symbol_sample". =20 >> op_iterate_symbol_sample >Not clear on the difference here? "op_iterate_symbol" contains profile results for a symbol as whole. "op_iterate_symbol_samples" contains profile results for individual vma. Perhaps a less confusing name is needed. >> op_demangle > >Clarify return type if it won't fit. Size it should have been (see >strlcpy) ? I will look into this. >Others: how to specify --exclude-dependent (can be faster). How to specify -p. How >to specify an oparchive to work on. Sort status of iterators, if any. How to manage >versioning (op_create_instance should take an OP_VERSION value, and verify it >can deal with it). > >Need to think about anonymous region code we have now and how it could be >worked in. --exclude-dependent and -p is not supported by API right now. I would be happy to expand the API if we need to. =20 As for the anonymous regions, I would like to hear opinions from Java and Mono developers especially on how they would use anonymous region code data. This is a great feedback. It gives me more sense on what the API needs to be and a broader view on the uses of API. Jason |
From: John L. <le...@mo...> - 2006-03-30 22:40:51
|
On Thu, Mar 30, 2006 at 04:01:19PM -0600, Yeh, Jason wrote: > >> void op_delete_instance(struct op_instance ** op_pt) > > > >Why does this take a ** ? > > I reset the op_pt to NULL in the code. This is a bit unusual and not necessary, I think. > >> op_add_image_spec > >Command line supports globbing. Does this? > The API simply passes the string into the Oprofile internal. I have not > test > globbing, but I am inclined to say yes. Should document if so. > >How do I say "show me libc samples used by /bin/bash" etc. ? Need way > to specify >dep images like command line. > The API does not support it directly. To do that with current API the > user need to add "/bin/bash" into the image list using > "op_add_image_spec()". > The user then needs to find "libc" in the subsequent call to > "op_iterate_summary_deps". So, should we do this or not? > >> op_add_cpu_spec > > > >Define behaviour when CPU isn't present in samples. Define behaviour if > merged >(as with all cases of these options). > When any of the options isn't present in samples "op_add_cpu_spec" has > not effect. If the particular option is merged, it has not effect. Could we return an error? > >> op_add_event_spec > >What about unit mask, count? > The API currently has not way to specify unit mask and count now. I will > add them in the next iteration of patch. OK. > >> cpu - CPU number. > >> tgid - Thread group ID. > >> tid - Thread ID > >Define values if merged. > If merged, the value will be NULL. ??? are these strings? I thought they were integers. > >> op_iterate_summary > >> only when the profile is separated by lib (see SEP_TID). > Summary is always available, but summary depends is only available when > profile is separated by lib. Yes, but you say "SEP_TID" which is wrong. > Matching the particular values can be easily done by allowing the user > to specify a comma separted list. The list will be used to create a > string filter for matching correct value. I am a bit concerned on which > name we should match, the mangled or demangled names? Good question. > "op_iterate_symbol" contains profile results for a symbol as whole. > "op_iterate_symbol_samples" contains profile results for individual vma. > Perhaps a less confusing name is needed. Name probably OK, but need to clarify the descriptions. > --exclude-dependent and -p is not supported by API right now. I would > be > happy to expand the API if we need to. It's needed. > As for the anonymous regions, I would like to hear opinions from Java > and Mono > developers especially on how they would use anonymous region code data. It's not really useful yet except as an image summary. thanks, john P.S. your email client isn't very good at quoting!! |
From: Yeh, J. <jas...@am...> - 2006-03-30 23:10:36
|
>>>How do I say "show me libc samples used by /bin/bash" etc. ? Need way >> to specify >dep images like command line. >> The API does not support it directly. To do that with current API the=20 >> user need to add "/bin/bash" into the image list using=20 >> "op_add_image_spec()". >> The user then needs to find "libc" in the subsequent call to=20 >> "op_iterate_summary_deps". > >So, should we do this or not? I would prefer to leave this out of API. I did not expect to the API to be a query engine that the user can use like a database. >>>> op_add_cpu_spec >>> >>>Define behaviour when CPU isn't present in samples. Define behaviour=20 >>>if >>merged >(as with all cases of these options). >>When any of the options isn't present in samples "op_add_cpu_spec" has >>not effect. If the particular option is merged, it has not effect. > >Could we return an error? Yes. > >>>> cpu - CPU number. >>>> tgid - Thread group ID. >>>> tid - Thread ID >>>Define values if merged. >> If merged, the value will be NULL. =20 > >??? are these strings? I thought they were integers. It is human readable name. The values are copied from "profile_class" in arrange_profiles.h. Would a numerical value be prefered?=20 > >> Matching the particular values can be easily done by allowing the user=20 >> to specify a comma separted list. The list will be used to create a=20 >> string filter for matching correct value. I am a bit concerned on=20 >> which name we should match, the mangled or demangled names? > >Good question. >Name probably OK, but need to clarify the descriptions. > >> --exclude-dependent and -p is not supported by API right now. I would=20 >> be happy to expand the API if we need to. > >It's needed. > >> As for the anonymous regions, I would like to hear opinions from Java >> and Mono developers especially on how they would use anonymous region >> code data. > >It's not really useful yet except as an image summary. I digressed a bit while I was working on the API and started working on a new proposal for JITted code. It is nearly complete, but I decided to hold onto it until the API is settled. If anyone's interested, I can clean up the document and send it to the list. This is why I am interested in how anonymous region can be used. >P.S. your email client isn't very good at quoting!! Outlook is the only email client available here. I still haven't find a good plug-in for a quoting. =20 Jason |
From: John L. <le...@mo...> - 2006-03-31 01:12:58
|
On Thu, Mar 30, 2006 at 05:10:21PM -0600, Yeh, Jason wrote: > >>>> cpu - CPU number. > >>>> tgid - Thread group ID. > >>>> tid - Thread ID > It is human readable name. The values are copied from "profile_class" > in > arrange_profiles.h. Would a numerical value be prefered? Numerical seems a /lot/ better for numerical values. > >P.S. your email client isn't very good at quoting!! > Outlook is the only email client available here. I still haven't find a > good plug-in for a quoting. How unfortunate :) john |
From: John L. <le...@mo...> - 2006-01-16 13:40:54
|
On Wed, Dec 21, 2005 at 03:28:40PM -0600, Yeh, Jason wrote: > I. Use Oprofile's post processing code to generates summary in the > following > manner: > 1. Create profile_spec. > 2. Generate a list of sample files. > 3. Arrange sample files into profile classes. > 4. Create summary_container from profile classes. > 5. CodeAnalyst saves the summary. > > > II. Use Oprofile's post processing code to retrieve the results for each > binary file in terms of symbols: > 1. Create profile_spec. > 2. Generate a list of sample files. > 3. Arrange sample files into profile classes. > 4. Invert profile classes. > 5. For each inverted profile class add symbols into one > profile_container. > 6. Select all symbols from profile_container > 7. Sort the symbols by file name and by reverse sort. > 8. For each symbol, CodeAnalyst saves the data. > > > If this level of API is believed to be practical and viable, I can go > through headers in libpp in the same fashion Maynard Johnson has done > for libdb/odb.h. Most of these steps are essentially internal. We should take a step back from the internal headers and ask what is a sensible, simple API that: o) provides a way to specify what data we want o) provides a way to iterate through that data. In particular, this should be decoupled as much as possible from internal things like inverting the profiles. Needless to say, I would encourage discussion of what exactly such an API looks like. regards john |