From: Maynard J. <may...@us...> - 2009-03-10 15:56:04
|
Suravee Suthikulpanit wrote: > See my comment below. > > Suravee > > Maynard Johnson wrote: >> Suravee Suthikulpanit wrote: >>> Hi, >>> >>> Please see my comment below. >>> >>> Thank you, >>> >>> Suravee >>> >>> Maynard Johnson wrote: >>>> Suthikulpanit, Suravee wrote: >>>>> Oprofile Extended Feature Interface Proposal >>>> Thanks for putting this together. >>>>> Suravee Suthikulpanit <sur...@am...> >>>>> Date: 3/3/2009 >>>>> >>>>> Goals: >>>>> ===== >>>>> - Implement an interface to allow non-generic functionalities (i.e. >>>>> architecture specific features) to seamlessly extend Oprofile daemon. >>>>> >>>>> - The extended feature can be enable/disable from the command line to >>>>> start Oprofile daemon. >>>>> >>>>> - Flexible and easily extended. >>>>> >>>>> >>>>> Introduction: >>>>> ======== >>>>> As part of the attempt to include the Instruction Based Sampling (IBS) >>>>> feature for AMD Processor family10h into the Oprofile, Maynard has >>>>> recommended me to design an interface to support not only IBS, but >>>>> also >>>>> future extension of architecture-specific features. (Please see >>>>> http://marc.info/?l=oprofile-list&m=123266349409087&w=2 ) >>>>> >>>>> IBS uses a specialized set of registers different from the traditional >>>>> performance counters available on many systems. The IBS performance >>>>> events are derived from bits combinations of these registers, and >>>>> therefore should not be limited by the number of available performance >>>>> counters. Existing Oprofile infrastructure could be extended to >>>>> support >>>>> this design by extending the opd_sfile to allow extension of the data >>>>> structure holding the performance event information. However, the >>>>> extended infrastructure should not affect the normal operation of >>>>> Oprofile when feature is not available. >>>>> >>>>> IBS is only an example of why we need the Extended Interface. Other >>>>> new >>>>> features could also easily extend different areas of Oprofile daemon >>>>> functionalities (i.e. data translation). Design: >>>>> ====== >>>>> >>>>> - Command line Interface >>>>> * -x|--ext-opt=<extended feature type> >>>>> This option enable a specific type of extended feature. >>>>> >>>>> * -o|--ext-val=<extended feature value> >>>>> This option is to be used with the "--ext-opt" to specify >>>>> the type specific parameters >>>> I know there's been some discussion between you and John about >>>> re-working the above options for oprofiled. But what about options >>>> for opcontrol and ophelp to show event info for the extended >>>> events. Is that what you had in mind for the two options below? >>> [SURAVEE] >>> opcontrol -l|--list-event should be listing all the events including >>> the extended events. Extended events should be listed in the event file >> >> Which event file? I would argue that a particular type of extended >> events should be placed in their own event file rather than trying to >> shoehorn them into an existing processor-specific event file. I don't >> know what AMD's plans might be for IBS support in future processors, >> but just for the sake of argument, let's say that IBS support is >> built-into the next generation AMD processor (let's call it >> "family13h"). We don't really want to duplicate the IBS events in >> both family10h and family13h events files. Maybe we need some way to >> link "normal" events to optional "extended" events so that ophelp will >> display ALL relevant information for the given cpu type. > > [SURAVEE] > The future generation of processor will most likely have a slightly > different set of IBS events. The events will be specific to the > processor generation. That's why I feel like it could be included in > the existing processor-specific event file. This should only require > small changes to existing code. OK, maybe we can stick with the simple single event file for now and extend that in the future if necessary. Just need to be sure these changes don't break other architectures . . . to state the obvious :-). > > Let's say we have a separate file to store extended events based on the > feature name (i.e. events/extended/ibs/events,unit_masks), Maybe 'events/x86_64/extended/ibs-1.0/events' instead . . . > we would > still need to have some sort of tagging to indicate which cpu_type will > a particular extended event be available. We could add a line in the "real" events file that points to an extended events file. The line could start with a special symbol so we recognize it as such when parsing it. > > Also, from the users point of view, the IBS events are generally the > same as the PMC events. The only difference is they don't have to worry > about the number of actual PMC hardware available on the system. > >> >>> same as normal events. However, we might need a new tags (instead of >>> using "counters"0,1,2,3") to differentiate the types of events since >>> it is not using the actual hardware counters. This would require >>> changes in the event files, libop/op_events.c, and utils/ophelp.c. >>> >>> The two options below (--feature-help and --list-feature) are meant >>> for different purposes. They are internal to oprofiled and should not >>> be concerned by users who should be using opcontrol. >>> >>>>> * -p|--ext-help=<extended feature type> >>>>> This option shows the help for the speicfied extended feature >>>>> type. >>>>> >>>>> * -l|--list-ext-opt >>>>> This option shows the list of available extended features >>>>> >>>>> >>>>> - "opd_ext_options" Structure >>>>> This structure contains information for each extended feature >>>>> including >>>>> type name, and a structure containing handlers function. >>>>> struct opd_ext_options { >>>>> const char* opt; // Option type >>>>> struct opd_ext_handlers *handlers; // Handlers >>>>> }; >>>>> >>>>> >>>>> - "opd_ext_handlers" Structure >>>>> This structure will contain handlers for each extended feature. >>>>> struct opd_ext_handlers { >>>>> int (*ext_init)(char const * ); // Extended init >>>>> int (*ext_help)(); // Extended option >>>>> help >>>>> struct opd_ext_sfile_handlers *ext_sfile; // Extended >>>>> sfile handlers (optional) >>>>> struct opd_ext_trans_handlers *ext_trans; // Extended >>>>> trans handlers (optional) >>>>> }; >>>>> >>>>> The two main handlers are: * ext_init: handle the >>>>> initialization of the feature at the beginning when oprofiled >>>>> parsing the command line options. >>>>> >>>>> * ext_help: will print out the help message for the feature. >>>>> >>>>> * ext_sfle: a pointer to structure containing handlers >>>>> for the opetration necessary to extend opd_sfile.c : >>>>> struct opd_ext_sfile_handlers { >>>>> int (*create)(struct sfile *, struct transient >>>>> const *); >>>>> int (*dup)(struct sfile *, struct sfile * ); >>>>> int (*close)(struct sfile *); >>>>> int (*sync)(struct sfile *); >>>>> odb_t * (*get)(struct transient const * , int ); >>>>> struct opd_event * >>>>> (*find_counter_event)(unsigned long); >>>>> }; >>>>> >>>>> * ext_trans: a pointer to structure containing handlers for >>>>> the operation necessary to extend opd_trans.c >>>>> struct opd_ext_trans_handlers { >>>>> // TBD >>>>> }; >>>>> - Extended Feature Table >>>>> This is a global array of "struct opd_ext_options" described above. >>>>> Each >>>>> extended feature must be listed here. >>>>> struct opd_ext_options ext_opt_table[] = { >>>>> {"ibs", &opd_ibs_handlers}, >>>> I think both a short name and long name (i.e., description) would be >>>> helpful. >>> [SURAVEE] >>> I plan to put description in the "oprofiled --feature-help=<feature >>> name>". For example: >>> >>> # oprofiled --feature-help=ibs >>> >>> ************************************ >>> * Instruction-based Sampling (IBS) * >>> ************************************ >>> Usage: >>> oprofiled \ >>> --feature-handler=ibs:fetch:<comma-separated IBS fetch >>> event>:<count>:<unitmask> >>> --feature-handler=ibs:op:<comma-separated IBS op >>> events>:<count>:<unitmask> >>> >>> Description: >>> Instruction-Based Sampling (IBS) is a new profiling technique that >>> provides rich, precise program performance information. IBS is >>> introduced by AMD Family10h processors (AMD Opteron Quad-Core >>> processor “Barcelona.”) >>> >>> Please see the "BIOS and Kernel Developer's Guide (BKDG) For AMD >>> Family10h Processor for more information on IBS. >>> >>> (http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/31116.pdf) >> >> >> So you need to start oprofile with some IBS-specific events before you >> can get the help text for IBS-specific events? I think we need a way >> to show this info from ophelp, the same way that's done for other >> arch's when the events are listed to stdout. > > [SURAVEE] > To further clarify my thought, here are some use cases that I have in mind: > > 1. On familiy10h system, users run "opcontrol -l|--list-event" to get > the list of IBS-specific events. Here, ophelp accesses the > events/x86_64/family10/events and unit_masks files which contain both > PMC and IBS events, and print them to stdout. > > 2. Users can run "opcontrol --feature-list" which in turns run > "oprofiled --feature-list" in order to get a list of supported features. ummmm . . . I don't think that would be a good idea. oprofiled should be invoked *only* when we really want to start the daemon, not for the purpose of printing out info to the user. The opcontrol script is designed around the idea that only one daemon should be running at a time. I'd prefer not to start adding exceptions to this rule if we don't have to. Also, I thought there was some agreement that, for now, there would be only one possible extended feature per cpu type. If that is true, then this --feature-list option seems pointless. > > 3. We can run "opcontrol --feature-help=ibs" which in turns run > "oprofiled --feature-help=ibs" in order to get a help/description about > the feature. This option doesn't seem all that useful to me -- at least not the '=<value>' part of it. When a user wants help/description text for an extended feature available for their processor, they shouldn't have to tell opcontrol what the feature is -- we should be smart enough to figure it out, since there can be only one (right?). Besides, as you indicated above, you'd like to have 'opcontrol --list-events' show both the standard and the extended events. Since the standard listing already includes a documentation blurb, you should work out a way to add a documentation blurb for the extended features, too. If the extended features were in a separate file, the output could have some sort of separator to indicate the beginning of extended events, and that's where you print out the extended events documentation blurb, too. By the way, keep in mind that, currently, one can use ophelp directly to get a list of standard events, along with the documentation blurb. Also, the user can pass in the --cpu-type option to get the event list for a different processor than the one they're running on. And lastly, ophelp can output events in xml format. You want to make sure that the extended events can be output in the same xml format. -Maynard > > The reason I interface opcontrol with oprofiled instead of ophelp is > because I would like to keep the extended-feature related implementation > contained within the scope of oprofiled (i.e. the code inside daemon > directory). ophelp is a separate utility which depends on the > events/unitmasks files, cpu_type, and mainly uses the libop and libutil. I'm coming back to the idea that a separate extended events file might be a better option. If the user does "opcontrol --feature-list", this request could be passed down to ophelp and it would open the main events file for the given cpu_type and search for the special line that points to the extended events file. There wouldn't need to be too much knowledge of the extended feature built into ophelp to support this concept. > >> >> -Maynard >>> >>>> Thanks. >>>> -Maynard >>>>> {.....}, >>>>> {....}, >>>>> {NULL, NULL} >>>>> } >>>>> >>>>> - Extension to the "struct sfile" in opd_sfile.h >>>>> To extend sfile, I am adding "odb_t * ext_file" to this structure >>>>> (opd_sfile.h), which is an extension of "opd_t file[OP_MAX_COUNTERS]". >>>>> >>>>> - Extension to the "struct transient" >>>>> To extend translation operation, I am adding "*void * ext" to this >>>>> structure (opd_trans.h). >>>>> >>>>> >>>>> NOTE: >>>>> ===== >>>>> I can provide a prototype of this interface next week if the design >>>>> looks good. One of the limitations with this design is only one >>>>> type of >>>>> extended feature is allowed at a time. I don't see the need to allow >>>>> multiple extended features to run concurrently yet, and do not want to >>>>> introduce unnecessary overhead. However, it should be easily extended >>>>> in the future. >>>>> >>>>> >>>>> ------------------------------------------------------------------------------ >>>>> >>>>> Open Source Business Conference (OSBC), March 24-25, 2009, San >>>>> Francisco, CA >>>>> -OSBC tackles the biggest issue in open source: Open Sourcing the >>>>> Enterprise >>>>> -Strategies to boost innovation and cut costs with open source >>>>> participation >>>>> -Receive a $600 discount off the registration fee with the source >>>>> code: SFAD >>>>> http://p.sf.net/sfu/XcvMzF8H >>>>> _______________________________________________ >>>>> oprofile-list mailing list >>>>> opr...@li... >>>>> https://lists.sourceforge.net/lists/listinfo/oprofile-list >>>> >>> >> >> >> ------------------------------------------------------------------------------ >> >> Open Source Business Conference (OSBC), March 24-25, 2009, San >> Francisco, CA >> -OSBC tackles the biggest issue in open source: Open Sourcing the >> Enterprise >> -Strategies to boost innovation and cut costs with open source >> participation >> -Receive a $600 discount off the registration fee with the source >> code: SFAD >> http://p.sf.net/sfu/XcvMzF8H >> _______________________________________________ >> oprofile-list mailing list >> opr...@li... >> https://lists.sourceforge.net/lists/listinfo/oprofile-list >> > > |