From: Andrew M. <ak...@os...> - 2004-08-26 05:20:44
|
Jay Lan <jl...@en...> wrote: > > I have broken up one big CSA kernel patch into four smaller ones > as attached: > > csa_io - collects io accounting data > csa_mm - collects mm accounting data > csa_eop - provides a hook to perform end-of-process accounting > csa_module - builds csa loadable module Please don't send patches as attachments, and please don't send more than one patch per email. If you stick to the one-patch-per-email rule, you also get an opportunity to nicely describe each patch within its email. Coding style nit: if (foo) bar(); is preferable to if (foo) { bar(); } More broadly: Help! I am 100% not in a position to judge whether Linux needs Comprehensive System Accounting, nor am I able to define what the requirements for such a thing should be. All I can tell from your patch is the quality of its implementation, and that's leaping far, far ahead of where we should be. We're going to need help from you, and from all the other stakeholders in judging how useful this feature is to Linux implementors and how well this implementation meets the (unknown) requirements. See my problem? I've cc'ed lse-tech, where enterprise folks hang out. I would request that the people who are stakeholders in this feature a) stick their hands up b) let us know how important this kind of feature is for their users c) review the offered feature set against their requirements d) let us know how well the implementation fits that requirement and e) inform us of any competing implementations. Compare and contrast. Thanks. > There are no functional changes in this set of csa patches compared > to the 2.6.7 patch linux-2.6.7.csa.patch. > > Patches csa_io, csa_mm, and csa_eop are independent of each other. You may > apply any one, any two or all three and you will be able to build a > functional kernel. However, data collected needs an agent to use it. The > csa_module is one agent that takes advangtage of the feature and it works > with csa-2.0.0 (or later) to report system accounting data of the host > system. The csa-2.0.0 rpm can be downloaded from > ftp://oss.sgi.com/projects/csa/download > > The csa_module patch requires all three accounting data patches to be fully > functional. > > This set of csa patches has been tested with the pagg and job kernel > patches to linux 2.6.8 kernel. The information of pagg and job project can > be found at http://oss.sgi.com/projects/pagg/ > > The csa_module requires the pagg and job kernel patches. > > Feedback, bug reports, and comments are very welcome! > |
From: Guillaume T. <gui...@bu...> - 2004-08-26 11:45:58
|
> Jay Lan <jl...@en...> wrote: > > > > I have broken up one big CSA kernel patch into four smaller ones > > as attached: > > > > csa_io - collects io accounting data > > csa_mm - collects mm accounting data > > csa_eop - provides a hook to perform end-of-process accounting > > csa_module - builds csa loadable module > > I am 100% not in a position to judge whether Linux needs Comprehensive > System Accounting, nor am I able to define what the requirements for such a > thing should be. All I can tell from your patch is the quality of its > implementation, and that's leaping far, far ahead of where we should be. > > We're going to need help from you, and from all the other stakeholders in > judging how useful this feature is to Linux implementors and how well this > implementation meets the (unknown) requirements. See my problem? Here is my point of view about the different projects: - To do group accounting we need a mechanism that allows to manage groups of processes and attach some hook to perform accounting operations. Today there are PAGG and "banks" from ELSA. I think PAGG is more mature than ELSA and as it is already in the -mm tree, we can use this mechanism. - There are already accounting features in the linux kernel with a BSD process accounting. Thus, maybe the good solution for doing per group of process accounting is PAGG + BSD-accounting. Now with ELSA I try to integrate BSD pre-process accounting independently from the mechanism used to manage group of process. Guillaume |
From: Tim S. <ti...@ph...> - 2004-08-26 17:17:29
|
On Wed, 25 Aug 2004, Andrew Morton wrote: > More broadly: Help! > > I am 100% not in a position to judge whether Linux needs Comprehensive > System Accounting, nor am I able to define what the requirements for such a > thing should be. All I can tell from your patch is the quality of its > implementation, and that's leaping far, far ahead of where we should be. > > We're going to need help from you, and from all the other stakeholders in > judging how useful this feature is to Linux implementors and how well this > implementation meets the (unknown) requirements. See my problem? > > I've cc'ed lse-tech, where enterprise folks hang out. I would request that > the people who are stakeholders in this feature > > a) stick their hands up > > b) let us know how important this kind of feature is for their users > > c) review the offered feature set against their requirements > > d) let us know how well the implementation fits that requirement and > > e) inform us of any competing implementations. Compare and contrast. Judging from the feedback during it's stay in -mm (none at all!), general interest in BSD accounting seems quite limited. The rate of downloads of the updated userspace tools is hardly distinguishable from background noise. (This might change with the correct URL in the help text now, but even that was broken for months and nobody cared). Also general interest in the user space tools is low, the latest release of the GNU acct package is from 1998 (and yes, there _are_ problems warranting updates). Funnily enough, with three competing implementation even interest from developers seems larger than that from users (This statement includes me, I did a patch but am not a user of it).) But communication between developers is poor. I for myself only recently learned about ELSA and CSA. Therefore I've Cc:ed some people from whom I got valuable feedback on the BSD accounting format patch. IMHO CSA, ELSA and BSD accounting are too similar to have more than one of them in the kernel. We should either improve BSD accounting to do the job, or kill it in favor of a different implementation. Tim |
From: Arthur C. <co...@di...> - 2004-08-26 19:25:28
|
On Thu, 26 Aug 2004, Tim Schmielau wrote: <snip> > Therefore I've Cc:ed some people from whom I got valuable feedback on the > BSD accounting format patch. > > IMHO CSA, ELSA and BSD accounting are too similar to have more than one of > them in the kernel. We should either improve BSD accounting to do the job, > or kill it in favor of a different implementation. I would be very interested in a CSA implementation similar to what I have on IRIX. I will also plead guilty to not having downloaded the updated patches for either the kernel or the tools. I'm continuing to use my poor hack until a permanent solution gets accepted into the kernel, at which point I'll adopt that. And if it counters the impression at all, I'm not a kernel developer, I proposed my hack out of need as a user of the tools. I also try to stay away from modified kernels, so I'm running Marcelos' 2.4 stable branch with only the 32bit u/gid_t hack applied. That's why I haven't had any feedback on the -mm branch. In short, for my use BSD accounting is sufficient, but I'd love to see CSA in Linux as well. Linux hasn't moved too far into roles where it's a necessity (for what I'm doing, anyway), but I see CSA as something that would certainly help it assume those roles. --Arthur Corliss Bolverk's Lair -- http://arthur.corlissfamily.org/ Digital Mages -- http://www.digitalmages.com/ "Live Free or Die, the Only Way to Live" -- NH State Motto |
From: Tim S. <ti...@ph...> - 2004-08-26 20:05:43
|
On Thu, 26 Aug 2004, Arthur Corliss wrote: > I would be very interested in a CSA implementation similar to what I have on > IRIX. I will also plead guilty to not having downloaded the updated patches > for either the kernel or the tools. I'm continuing to use my poor hack until > a permanent solution gets accepted into the kernel, at which point I'll > adopt that. That's ok, we carefully discussed the changes to make sure no new tools are required ;-) > And if it counters the impression at all, I'm not a kernel developer, I > proposed my hack out of need as a user of the tools. I also try to stay away > from modified kernels, so I'm running Marcelos' 2.4 stable branch with only > the 32bit u/gid_t hack applied. That's why I haven't had any feedback on the > -mm branch. I haven't even tried to get a patch into 2.4, since Marcelo is (rightly) quite resilent to new features. > > In short, for my use BSD accounting is sufficient, but I'd love to see CSA in > Linux as well. Linux hasn't moved too far into roles where it's a necessity > (for what I'm doing, anyway), but I see CSA as something that would certainly > help it assume those roles. Does this mean you would want to have both in the same kernel, potentially turning on both at the same time? Ok, let me summarize what I learned until now: It should be easy to combine the data collection enhancements from CSA and ELSA to provide a common superset of information. Output file formats vary, but might be unified if projects don't insist too much. Main difference between CSA and ELSA on the one hand and BSD acct on the other is that the latter writes one record per process, while the former write one per job. With the new BSD acct v3 format, it should be possible to do per job accounting entirely from userspace, using pid and ppid information to reconstruct the process tree and some userland database for the pid -> job mapping. It would, however, be greatly simplified if the accounting records provided some kind of job id, and some indicator whether or not this process was the last of a job (group). CSA and ELSA might even be more lightweight since fewer accounting records are actually written. Sounds like it should be possible to fulfill the different needs by having loadable modules for the different output formats, or by a /proc entry that controls some aspects like whether records are written per job or per process. Comments? |
From: Jay L. <jl...@en...> - 2004-08-26 20:48:57
|
I do like to see a common data collection method in the kernl. Kernel does not need to decide how the data to be presented to the user space. An accounting loadable module such as CSA or ELSA will take care of how the data to be presented to meet the needs of different users. Sounds reasonable? Regards, - jay Tim Schmielau wrote: > On Thu, 26 Aug 2004, Arthur Corliss wrote: > > >>I would be very interested in a CSA implementation similar to what I have on >>IRIX. I will also plead guilty to not having downloaded the updated patches >>for either the kernel or the tools. I'm continuing to use my poor hack until >>a permanent solution gets accepted into the kernel, at which point I'll >>adopt that. > > > That's ok, we carefully discussed the changes to make sure no new tools > are required ;-) > > >>And if it counters the impression at all, I'm not a kernel developer, I >>proposed my hack out of need as a user of the tools. I also try to stay away >>from modified kernels, so I'm running Marcelos' 2.4 stable branch with only >>the 32bit u/gid_t hack applied. That's why I haven't had any feedback on the >>-mm branch. > > > I haven't even tried to get a patch into 2.4, since Marcelo is (rightly) > quite resilent to new features. > >>In short, for my use BSD accounting is sufficient, but I'd love to see CSA in >>Linux as well. Linux hasn't moved too far into roles where it's a necessity >>(for what I'm doing, anyway), but I see CSA as something that would certainly >>help it assume those roles. > > > Does this mean you would want to have both in the same kernel, potentially > turning on both at the same time? > > > Ok, let me summarize what I learned until now: > > It should be easy to combine the data collection enhancements from > CSA and ELSA to provide a common superset of information. > > Output file formats vary, but might be unified if projects don't insist > too much. > Main difference between CSA and ELSA on the one hand and BSD acct on the > other is that the latter writes one record per process, while the former > write one per job. > With the new BSD acct v3 format, it should be possible to do per job > accounting entirely from userspace, using pid and ppid information to > reconstruct the process tree and some userland database for the > pid -> job mapping. It would, however, be greatly simplified if the > accounting records provided some kind of job id, and some indicator > whether or not this process was the last of a job (group). > > CSA and ELSA might even be more lightweight since fewer accounting records > are actually written. > > Sounds like it should be possible to fulfill the different needs by > having loadable modules for the different output formats, or by a /proc > entry that controls some aspects like whether records are written per > job or per process. > > Comments? |
From: Arthur C. <co...@di...> - 2004-08-28 01:28:03
|
On Thu, 26 Aug 2004, Jay Lan wrote: > I do like to see a common data collection method in the kernl. Kernel > does not need to decide how the data to be presented to the > user space. An accounting loadable module such as CSA or ELSA will > take care of how the data to be presented to meet the needs of > different users. > > Sounds reasonable? Seconded. --Arthur Corliss Bolverk's Lair -- http://arthur.corlissfamily.org/ Digital Mages -- http://www.digitalmages.com/ "Live Free or Die, the Only Way to Live" -- NH State Motto |
From: Guillaume T. <gui...@bu...> - 2004-08-30 12:30:01
|
On Fri, Aug 27, 2004 at 05:27:19PM -0800, Arthur Corliss wrote: > On Thu, 26 Aug 2004, Jay Lan wrote: > > > I do like to see a common data collection method in the kernl. Kernel > > does not need to decide how the data to be presented to the > > user space. An accounting loadable module such as CSA or ELSA will > > take care of how the data to be presented to meet the needs of > > different users. > > > > Sounds reasonable? > > Seconded. Thus, to be clear, the enhanced accounting can be divided into three parts: 1) A common data collection method in the kernel. We could start from BSD-accounting and add CSA information. Could it be something like BSD version4? 2) A module that will manage a job history. I mean, it will manage a structure in which we will keep the relationship between processes and "containers" and also between process and its children. The property needed here is that a child belongs to the same "job" as its parent. This allows to do per-job accounting. I will have a look to PAGG and JOB. Can it be done in userspace? Is the lost of data (as observed by Arthur Corliss with SAR) can be avoided? 3) Finally we need module that will be in charge of datas presentation. This will allow to be easily compatible with other applications. I will have a look to the per-job accounting patch. If I understand well, this patch falls into the second requirement (manage groups of processes). Regards, Guillaume |
From: Tim S. <ti...@ph...> - 2004-08-31 14:20:56
|
On Mon, 30 Aug 2004, Guillaume Thouvenin wrote: > Thus, to be clear, the enhanced accounting can be divided into > three parts: > > 1) A common data collection method in the kernel. > We could start from BSD-accounting and add CSA information. Could > it be something like BSD version4? I've had a quick look at the CSA data collection patches. To get the discussion started, here are my comments: > --- linux.orig/drivers/block/ll_rw_blk.c 2004-08-13 22:36:16.000000000 -0700 > +++ linux/drivers/block/ll_rw_blk.c 2004-08-18 12:07:10.000000000 -0700 > @@ -1948,10 +1950,12 @@ > > if (rw == READ) { > disk_stat_add(rq->rq_disk, read_sectors, nr_sectors); > + current->rblk += nr_sectors; > if (!new_io) > disk_stat_inc(rq->rq_disk, read_merges); > } else if (rw == WRITE) { > disk_stat_add(rq->rq_disk, write_sectors, nr_sectors); > + current->wblk += nr_sectors; > if (!new_io) > disk_stat_inc(rq->rq_disk, write_merges); > } Andi Kleen's comment on the ELSA patch also applies here - most writes will get accounted to pdflushd. See http://www.lib.uaa.alaska.edu/linux-kernel/archive/2004-Week-31/0047.html for his comment. > --- /dev/null 1970-01-01 00:00:00.000000000 +0000 > +++ linux/include/linux/csa_internal.h 2004-08-19 15:19:05.000000000 -0700 [...] > +#else /* CONFIG_CSA || CONFIG_CSA_MODULE */ > + > +#define csa_update_integrals() do { } while (0); > +#define csa_clear_integrals(task) do { } while (0); > +#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */ I suppose the semicolons are unintentional. > --- linux.orig/include/linux/sched.h 2004-08-19 15:17:52.000000000 -0700 > +++ linux/include/linux/sched.h 2004-08-19 15:19:05.000000000 -0700 [...] > @@ -525,6 +527,10 @@ > > /* i/o counters(bytes read/written, blocks read/written, #syscalls, waittime */ > unsigned long rchar, wchar, rblk, wblk, syscr, syscw, bwtime; > +#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE) > + unsigned long csa_rss_mem1, csa_vm_mem1; > + clock_t csa_stimexpd; > +#endif These probably need to be u64, otherwise they might easily overflow within a view seconds on 32 bit platforms. > --- /dev/null 1970-01-01 00:00:00.000000000 +0000 > +++ linux/include/linux/acct_eop.h 2004-08-19 18:48:44.000000000 -0700 This should probably be unified with BSD accounting to a general accounting hook. Tim |
From: Jay L. <jl...@en...> - 2004-08-31 23:02:42
|
Adding cs...@os..., the CSA user group mailing list, to Cc. Tim Schmielau wrote: > On Mon, 30 Aug 2004, Guillaume Thouvenin wrote: > > >> Thus, to be clear, the enhanced accounting can be divided into >>three parts: >> >> 1) A common data collection method in the kernel. >> We could start from BSD-accounting and add CSA information. Could >> it be something like BSD version4? > > > I've had a quick look at the CSA data collection patches. To get the > discussion started, here are my comments: > > >>--- linux.orig/drivers/block/ll_rw_blk.c 2004-08-13 22:36:16.000000000 -0700 >>+++ linux/drivers/block/ll_rw_blk.c 2004-08-18 12:07:10.000000000 -0700 >>@@ -1948,10 +1950,12 @@ >> >> if (rw == READ) { >> disk_stat_add(rq->rq_disk, read_sectors, nr_sectors); >>+ current->rblk += nr_sectors; >> if (!new_io) >> disk_stat_inc(rq->rq_disk, read_merges); >> } else if (rw == WRITE) { >> disk_stat_add(rq->rq_disk, write_sectors, nr_sectors); >>+ current->wblk += nr_sectors; >> if (!new_io) >> disk_stat_inc(rq->rq_disk, write_merges); >> } > > > Andi Kleen's comment on the ELSA patch also applies here - most writes > will get accounted to pdflushd. See > > http://www.lib.uaa.alaska.edu/linux-kernel/archive/2004-Week-31/0047.html > > for his comment. I need more time on this. :) > > >>--- /dev/null 1970-01-01 00:00:00.000000000 +0000 >>+++ linux/include/linux/csa_internal.h 2004-08-19 15:19:05.000000000 -0700 > > [...] > >>+#else /* CONFIG_CSA || CONFIG_CSA_MODULE */ >>+ >>+#define csa_update_integrals() do { } while (0); >>+#define csa_clear_integrals(task) do { } while (0); >>+#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */ > > > I suppose the semicolons are unintentional. Good catch! I fixed this in our internal tree. > > >>--- linux.orig/include/linux/sched.h 2004-08-19 15:17:52.000000000 -0700 >>+++ linux/include/linux/sched.h 2004-08-19 15:19:05.000000000 -0700 > > [...] > >>@@ -525,6 +527,10 @@ >> >> /* i/o counters(bytes read/written, blocks read/written, #syscalls, waittime */ >> unsigned long rchar, wchar, rblk, wblk, syscr, syscw, bwtime; >>+#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE) >>+ unsigned long csa_rss_mem1, csa_vm_mem1; >>+ clock_t csa_stimexpd; >>+#endif > > > These probably need to be u64, otherwise they might easily overflow within > a view seconds on 32 bit platforms. Will fix it. > > >>--- /dev/null 1970-01-01 00:00:00.000000000 +0000 >>+++ linux/include/linux/acct_eop.h 2004-08-19 18:48:44.000000000 -0700 > > > This should probably be unified with BSD accounting to a general accounting > hook. Do you suggest to merge acct_eop.h into acct.h? It sounds good to me! Thanks! - jay > > > Tim > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech |
From: Jay L. <jl...@en...> - 2004-09-08 18:35:01
|
The block IO data was not used in billing for CSA customers. Nobody has ever charged for this data. It is more for accountability and resource consumption tracking. Accuracy changes as the program runs for hours or days in our market. Repeatable billing is critical and block IO is one that is not repeatable. The write blocks was useful and with the type of programs running for hours and days (which are the ones of interest in block IO data). The delayed write after the process terminates is ignored because that is insignificant for a process doing io for hours or days. We use the bytes transferred vs blocks transferred to see programs dominating and polluting the cache. Processes big in one and not the other are of interest to profile more closely. However, if nobody else wants to have this feature, we can pull it out until we can find a way of doing this that makes people happy. I will submit a new set of patch based on 2.6.8.1 later. Thanks! - jay Jay Lan wrote: > Adding cs...@os..., the CSA user group mailing list, to Cc. > > Tim Schmielau wrote: > >> On Mon, 30 Aug 2004, Guillaume Thouvenin wrote: >> >> >>> Thus, to be clear, the enhanced accounting can be divided into >>> three parts: >>> >>> 1) A common data collection method in the kernel. >>> We could start from BSD-accounting and add CSA information. Could >>> it be something like BSD version4? >> >> >> >> I've had a quick look at the CSA data collection patches. To get the >> discussion started, here are my comments: >> >> >>> --- linux.orig/drivers/block/ll_rw_blk.c 2004-08-13 >>> 22:36:16.000000000 -0700 >>> +++ linux/drivers/block/ll_rw_blk.c 2004-08-18 12:07:10.000000000 >>> -0700 >>> @@ -1948,10 +1950,12 @@ >>> >>> if (rw == READ) { >>> disk_stat_add(rq->rq_disk, read_sectors, nr_sectors); >>> + current->rblk += nr_sectors; >>> if (!new_io) >>> disk_stat_inc(rq->rq_disk, read_merges); >>> } else if (rw == WRITE) { >>> disk_stat_add(rq->rq_disk, write_sectors, nr_sectors); >>> + current->wblk += nr_sectors; >>> if (!new_io) >>> disk_stat_inc(rq->rq_disk, write_merges); >>> } >> >> >> >> Andi Kleen's comment on the ELSA patch also applies here - most writes >> will get accounted to pdflushd. See >> >> http://www.lib.uaa.alaska.edu/linux-kernel/archive/2004-Week-31/0047.html >> >> for his comment. > > > I need more time on this. :) > >> >> >>> --- /dev/null 1970-01-01 00:00:00.000000000 +0000 >>> +++ linux/include/linux/csa_internal.h 2004-08-19 15:19:05.000000000 >>> -0700 >> >> >> [...] >> >>> +#else /* CONFIG_CSA || CONFIG_CSA_MODULE */ >>> + >>> +#define csa_update_integrals() do { } while (0); >>> +#define csa_clear_integrals(task) do { } while (0); >>> +#endif /* CONFIG_CSA || CONFIG_CSA_MODULE */ >> >> >> >> I suppose the semicolons are unintentional. > > > Good catch! I fixed this in our internal tree. > >> >> >>> --- linux.orig/include/linux/sched.h 2004-08-19 15:17:52.000000000 >>> -0700 >>> +++ linux/include/linux/sched.h 2004-08-19 15:19:05.000000000 -0700 >> >> >> [...] >> >>> @@ -525,6 +527,10 @@ >>> >>> /* i/o counters(bytes read/written, blocks read/written, #syscalls, >>> waittime */ >>> unsigned long rchar, wchar, rblk, wblk, syscr, syscw, bwtime; >>> +#if defined(CONFIG_CSA) || defined(CONFIG_CSA_MODULE) >>> + unsigned long csa_rss_mem1, csa_vm_mem1; >>> + clock_t csa_stimexpd; >>> +#endif >> >> >> >> These probably need to be u64, otherwise they might easily overflow >> within >> a view seconds on 32 bit platforms. > > > Will fix it. > >> >> >>> --- /dev/null 1970-01-01 00:00:00.000000000 +0000 >>> +++ linux/include/linux/acct_eop.h 2004-08-19 18:48:44.000000000 >>> -0700 >> >> >> >> This should probably be unified with BSD accounting to a general >> accounting >> hook. > > > Do you suggest to merge acct_eop.h into acct.h? It sounds good to me! > > Thanks! > - jay > >> >> >> Tim >> >> >> ------------------------------------------------------- >> This SF.Net email is sponsored by BEA Weblogic Workshop >> FREE Java Enterprise J2EE developer tools! >> Get your free copy of BEA WebLogic Workshop 8.1 today. >> http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click >> _______________________________________________ >> Lse-tech mailing list >> Lse...@li... >> https://lists.sourceforge.net/lists/listinfo/lse-tech > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by BEA Weblogic Workshop > FREE Java Enterprise J2EE developer tools! > Get your free copy of BEA WebLogic Workshop 8.1 today. > http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech |
From: Guillaume T. <gui...@bu...> - 2004-08-27 05:44:41
|
On Thu, Aug 26, 2004 at 10:05:37PM +0200, Tim Schmielau wrote: > > It should be easy to combine the data collection enhancements from > CSA and ELSA to provide a common superset of information. ELSA uses current BSD accounting. The only difference with BSD is that accounting is done for a group of processes. I didn't use PAGG and rewrite something because I thought (I was wrong) that PAGG project wasn't maintained. I continue to maintain ELSA just because there is, until today, no solution for doing job accounting. So, the data collection enhancements from ELSA is not very useful. > With the new BSD acct v3 format, it should be possible to do per job > accounting entirely from userspace, using pid and ppid information to > reconstruct the process tree and some userland database for the > pid -> job mapping. It would, however, be greatly simplified if the > accounting records provided some kind of job id, and some indicator > whether or not this process was the last of a job (group). I like this solution. In fact what I proposed was to have PAGG and a modified BSD accounting that can be used with PAGG as both are already in the -mm tree. But manage group of processes from userspace is, IMHO, a better solution as modifications in the kernel will be minimal. Therefore the solution could be to enhance BSD accounting with data collection from CSA and provide per job accounting with a userspace mechanism. Sounds great to me... Best, Guillaume |
From: Jay L. <jl...@sg...> - 2004-08-27 19:59:24
|
Hi Guillaume, Please visit http://oss.sgi.com/projects/pagg/ The page has been updated to provide information on a per job accounting project called 'job' based on PAGG. There is one userspace rpm and one kernel module for job. This may provide what you are looking for. It is a mature product as well. I am sure Limin(job) and Erik(pagg) would appreciate any input you can provide to make 'job' more useful. Regards, - jay Guillaume Thouvenin wrote: > On Thu, Aug 26, 2004 at 10:05:37PM +0200, Tim Schmielau wrote: > >>It should be easy to combine the data collection enhancements from >>CSA and ELSA to provide a common superset of information. > > > ELSA uses current BSD accounting. The only difference with BSD is that > accounting is done for a group of processes. I didn't use PAGG and > rewrite something because I thought (I was wrong) that PAGG project > wasn't maintained. I continue to maintain ELSA just because there is, > until today, no solution for doing job accounting. > So, the data collection enhancements from ELSA is not very useful. > > >>With the new BSD acct v3 format, it should be possible to do per job >>accounting entirely from userspace, using pid and ppid information to >>reconstruct the process tree and some userland database for the >>pid -> job mapping. It would, however, be greatly simplified if the >>accounting records provided some kind of job id, and some indicator >>whether or not this process was the last of a job (group). > > > I like this solution. > In fact what I proposed was to have PAGG and a modified BSD accounting > that can be used with PAGG as both are already in the -mm tree. But > manage group of processes from userspace is, IMHO, a better solution as > modifications in the kernel will be minimal. > > Therefore the solution could be to enhance BSD accounting with data > collection from CSA and provide per job accounting with a userspace > mechanism. Sounds great to me... > > Best, > Guillaume > > > ------------------------------------------------------- > SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media > 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 > Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. > http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech |
From: Guillaume T. <gui...@bu...> - 2004-08-31 09:18:19
|
On Fri, Aug 27, 2004 at 12:55:03PM -0700, Jay Lan wrote: > Please visit http://oss.sgi.com/projects/pagg/ > The page has been updated to provide information on a per job > accounting project called 'job' based on PAGG. > > There is one userspace rpm and one kernel module for job. > This may provide what you are looking for. It is a mature product > as well. I am sure Limin(job) and Erik(pagg) would appreciate any > input you can provide to make 'job' more useful. I have a question about job. If I understand how it works, you can not add a process in a job. I mean when you start a session, a container is created and it's the only way to create it. If I'm right, I think that it could be interesting to add a process using ioctl and /proc interface. For example, if I want to know how resources are used by a compilation, I need to add the process gcc in a container. Any comments? Best, Guillaume |
From: Guillaume T. <gui...@bu...> - 2004-08-31 10:31:54
|
On Tue, Aug 31, 2004 at 11:06:47AM +0200, Guillaume Thouvenin wrote: > On Fri, Aug 27, 2004 at 12:55:03PM -0700, Jay Lan wrote: > > Please visit http://oss.sgi.com/projects/pagg/ > > The page has been updated to provide information on a per job > > accounting project called 'job' based on PAGG. > > > > There is one userspace rpm and one kernel module for job. > > This may provide what you are looking for. It is a mature product > > as well. I am sure Limin(job) and Erik(pagg) would appreciate any > > input you can provide to make 'job' more useful. > > I have a question about job. If I understand how it works, you can not > add a process in a job. I mean when you start a session, a container is > created and it's the only way to create it. If I'm right, I think that it > could be interesting to add a process using ioctl and /proc interface. I think that I'm not very clear. You can add a process to a container using the /proc/csa interface but it seems that currently this feature is not available with job-1.2.1 package. Therefore, maybe we can add a command called jattach that will attach a process to a given jid... Best, Guillaume |
From: Limin Gu <li...@db...> - 2004-08-31 16:05:13
|
> > On Tue, Aug 31, 2004 at 11:06:47AM +0200, Guillaume Thouvenin wrote: > > On Fri, Aug 27, 2004 at 12:55:03PM -0700, Jay Lan wrote: > > > Please visit http://oss.sgi.com/projects/pagg/ > > > The page has been updated to provide information on a per job > > > accounting project called 'job' based on PAGG. > > > > > > There is one userspace rpm and one kernel module for job. > > > This may provide what you are looking for. It is a mature product > > > as well. I am sure Limin(job) and Erik(pagg) would appreciate any > > > input you can provide to make 'job' more useful. > > > > I have a question about job. If I understand how it works, you can not > > add a process in a job. I mean when you start a session, a container is > > created and it's the only way to create it. If I'm right, I think that it > > could be interesting to add a process using ioctl and /proc interface. > > I think that I'm not very clear. You can add a process to a container > using the /proc/csa interface but it seems that currently this feature > is not available with job-1.2.1 package. Therefore, maybe we can add a > command called jattach that will attach a process to a given jid... The current job package is job-1.4.0-1, you can find it at ftp://oss.sgi.com/projects/pagg/download And it is correct that JOB_ATTACH ioctl is not implemented right now. We could implement that ioctl and also add a user command like jattach. We are trying to push the job kernel module to the community, but the ioctl and /proc binary interface seems not an appropriate kernel and user space communication interface. When job gets more users other CSA and there are more interest about job, maybe we could request a new syscall number in linux. --Limin > > Best, > Guillaume > |
From: John H. <jh...@sg...> - 2004-09-01 21:47:47
|
On Tue, Aug 31, 2004 at 11:06:47AM +0200, Guillaume Thouvenin wrote: > On Fri, Aug 27, 2004 at 12:55:03PM -0700, Jay Lan wrote: > > Please visit http://oss.sgi.com/projects/pagg/ > > The page has been updated to provide information on a per job > > accounting project called 'job' based on PAGG. > > > > There is one userspace rpm and one kernel module for job. > > This may provide what you are looking for. It is a mature product > > as well. I am sure Limin(job) and Erik(pagg) would appreciate any > > input you can provide to make 'job' more useful. > > I have a question about job. If I understand how it works, you can not > add a process in a job. I mean when you start a session, a container is > created and it's the only way to create it. Right, that's the current implementation. Any privileged process can create a job, though, it doesn't *have* to be at the start of a session. I believe job is currently hardwired that the initial member process is the creator, and the only other way in is via inheritance, and there's no way out of the job other than exiting or creating your own job. > If I'm right, I think that it could be interesting to add a process > using ioctl and /proc interface. We're planning on changing that interface, but I think your question applies regardless of what interface is used. > For example, if I want to know how resources are used by a > compilation, I need to add the process gcc in a container. Any > comments? > > Best, > Guillaume It sounds like a slightly different kind of job. The gcc process should already be in a job via it's parent. If it's already in a job, we don't let it out, as jobs are designed to be inescapable so that users can't sneak processes outside their job. If the only client of job is accounting, that might not be required. Maybe as long as they become a member of another job, and the usage is tracked, that would be OK. I'm not sure what that would do to the current CSA tools, though. On IRIX, I think jobs are also used to do resource limits, and that's probably where the hard requirement for jobs being inescapable came from. The ISP/ASP non-acct uses of job would probably want it to be inescapable. Different inheritance and creation policies could be implemented in job. Or, since it's just a loadable module, different job modules could be written to implement different styles of job as are required for different uses. John |
From: Arthur C. <co...@di...> - 2004-08-28 01:34:43
|
On Fri, 27 Aug 2004, Guillaume Thouvenin wrote: > I like this solution. > In fact what I proposed was to have PAGG and a modified BSD accounting > that can be used with PAGG as both are already in the -mm tree. But > manage group of processes from userspace is, IMHO, a better solution as > modifications in the kernel will be minimal. > > Therefore the solution could be to enhance BSD accounting with data > collection from CSA and provide per job accounting with a userspace > mechanism. Sounds great to me... The only concern I have with a userspace solution is that you run the risk of losing that data. What happens if a process on the box drives drives it out of memory and paging space? The box would still be working, it just wouldn't be able to fork new processes, and those already running that aren't purposely made high priority may not get much of a chance to execute as well. I've lost SAR data that way. --Arthur Corliss Bolverk's Lair -- http://arthur.corlissfamily.org/ Digital Mages -- http://www.digitalmages.com/ "Live Free or Die, the Only Way to Live" -- NH State Motto |
From: Arthur C. <co...@di...> - 2004-08-28 01:27:14
|
On Thu, 26 Aug 2004, Tim Schmielau wrote: > That's ok, we carefully discussed the changes to make sure no new tools > are required ;-) Understood, and appreciated. > Does this mean you would want to have both in the same kernel, potentially > turning on both at the same time? I think that's definitely the route to go. Not everyone needs the level of visibility that CSA provides, so I think that with a unified data collection methodology the users can decide for themselves what they need, and the kernel stays in good shape with no implementation-specific bloat. > Ok, let me summarize what I learned until now: > > It should be easy to combine the data collection enhancements from > CSA and ELSA to provide a common superset of information. > > Output file formats vary, but might be unified if projects don't insist > too much. I don't think we'd necessarly want a unified accounting *file*. BSD account files grow pretty damned quick as it is, and that's with a lot less visibility than if you included the extract counters that CSA provides. Make the data available the logging module(s) or what have you via a common struct, but let each module decide what to actually commit to disk, and when (on job exit, etc.). > Main difference between CSA and ELSA on the one hand and BSD acct on the > other is that the latter writes one record per process, while the former > write one per job. > With the new BSD acct v3 format, it should be possible to do per job > accounting entirely from userspace, using pid and ppid information to > reconstruct the process tree and some userland database for the > pid -> job mapping. It would, however, be greatly simplified if the > accounting records provided some kind of job id, and some indicator > whether or not this process was the last of a job (group). > > CSA and ELSA might even be more lightweight since fewer accounting records > are actually written. > > Sounds like it should be possible to fulfill the different needs by > having loadable modules for the different output formats, or by a /proc > entry that controls some aspects like whether records are written per > job or per process. On one hand, loadable modules would be more helpful for people doing side-by-side comparisons of the accounting systems, but the /proc method would be better for dynamic adjustments of a single system. I don't think I have a horse in this race, either way. --Arthur Corliss Bolverk's Lair -- http://arthur.corlissfamily.org/ Digital Mages -- http://www.digitalmages.com/ "Live Free or Die, the Only Way to Live" -- NH State Motto |
From: Jay L. <jl...@en...> - 2004-08-26 18:30:12
|
The reason for breaking up one CSA patch into four patches was so that the only CSA (http://oss.sgi.com/projects/csa/) specific thing is the csa_module. My intention is to improve the system accounting data collection and make the data available to any clients that can use the data. The three areas of accounting data we try to improve are io, mm, and per-process area. As Tim said the problem of BSD accounting was that it has been inactive for a long time. I do not mind incoporating the three accounting data collection patches i submitted into BSD or others as long as the data made available to modules that plan to make use of the data. :) Thanks, - jay Tim Schmielau wrote: > On Wed, 25 Aug 2004, Andrew Morton wrote: > > >>More broadly: Help! >> >>I am 100% not in a position to judge whether Linux needs Comprehensive >>System Accounting, nor am I able to define what the requirements for such a >>thing should be. All I can tell from your patch is the quality of its >>implementation, and that's leaping far, far ahead of where we should be. >> >>We're going to need help from you, and from all the other stakeholders in >>judging how useful this feature is to Linux implementors and how well this >>implementation meets the (unknown) requirements. See my problem? >> >>I've cc'ed lse-tech, where enterprise folks hang out. I would request that >>the people who are stakeholders in this feature >> >>a) stick their hands up >> >>b) let us know how important this kind of feature is for their users >> >>c) review the offered feature set against their requirements >> >>d) let us know how well the implementation fits that requirement and >> >>e) inform us of any competing implementations. Compare and contrast. > > > Judging from the feedback during it's stay in -mm (none at all!), general > interest in BSD accounting seems quite limited. The rate of downloads of > the updated userspace tools is hardly distinguishable from background > noise. (This might change with the correct URL in the help text now, but > even that was broken for months and nobody cared). > Also general interest in the user space tools is low, the latest release > of the GNU acct package is from 1998 (and yes, there _are_ problems > warranting updates). > > Funnily enough, with three competing implementation even interest from > developers seems larger than that from users (This statement includes me, > I did a patch but am not a user of it).) But communication between > developers is poor. I for myself only recently learned about ELSA and CSA. > > Therefore I've Cc:ed some people from whom I got valuable feedback on the > BSD accounting format patch. > > IMHO CSA, ELSA and BSD accounting are too similar to have more than one of > them in the kernel. We should either improve BSD accounting to do the job, > or kill it in favor of a different implementation. > > Tim > > > ------------------------------------------------------- > SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media > 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 > Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. > http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech |
From: Arthur C. <co...@di...> - 2004-08-26 19:45:13
|
On Thu, 26 Aug 2004, Jay Lan wrote: > The reason for breaking up one CSA patch into four patches was > so that the only CSA (http://oss.sgi.com/projects/csa/) specific > thing is the csa_module. My intention is to improve the system > accounting data collection and make the data available to any > clients that can use the data. The three areas of accounting > data we try to improve are io, mm, and per-process area. > > As Tim said the problem of BSD accounting was that it has been > inactive for a long time. I do not mind incoporating the > three accounting data collection patches i submitted into BSD or > others as long as the data made available to modules that plan > to make use of the data. :) All right, I'm going to shut up now. I had no idea that SGI had gone behind my back and started porting CSA to Linux. :-P Before I shut up, though, as a user of these tools I'd vote for a unified data collection method, as suggested above. There has to be at least five of us who have volunteered at one point or another to help make the GNU utilities conform. That should make everyone happy. --Arthur Corliss Bolverk's Lair -- http://arthur.corlissfamily.org/ Digital Mages -- http://www.digitalmages.com/ "Live Free or Die, the Only Way to Live" -- NH State Motto |
From: John H. <jh...@sg...> - 2004-08-26 18:41:33
|
On Wed, Aug 25, 2004 at 10:18:42PM -0700, Andrew Morton wrote: > Jay Lan <jl...@en...> wrote: > > > > I have broken up one big CSA kernel patch into four smaller ones > > as attached: > > > > csa_io - collects io accounting data > > csa_mm - collects mm accounting data > > csa_eop - provides a hook to perform end-of-process accounting > > csa_module - builds csa loadable module > > More broadly: Help! > > I am 100% not in a position to judge whether Linux needs Comprehensive > System Accounting, nor am I able to define what the requirements for such a > thing should be. All I can tell from your patch is the quality of its > implementation, and that's leaping far, far ahead of where we should be. Linux needs something beyond what it has today, at least for the HPC market SGI is familiar with. We believe it will more generally benefit Linux HPC and enterprise markets, which is one reason we've released the whole CSA stack as open source. > > We're going to need help from you, and from all the other stakeholders in > judging how useful this feature is to Linux implementors and how well this > implementation meets the (unknown) requirements. See my problem? > > I've cc'ed lse-tech, where enterprise folks hang out. I would request that > the people who are stakeholders in this feature > > a) stick their hands up We're running CSA in production on Altix (our Itanium/Linux platform) for several years now. > > b) let us know how important this kind of feature is for their users A substantial number of our customers require it. CSA has been developed over the years on SGI's HPC systems in response to our customers needs. It's been reimplemented and opensourced for Linux, originally as an SGI/LANL collaboration. > c) review the offered feature set against their requirements > > d) let us know how well the implementation fits that requirement and It fits. :-) Actually, one secondary feature on our wishlist is 'projects'. Our customers are tied into the current CSA user interface. However, there is lots of room for cooperation under that, particularly in the kernel. We can always consider a migration project as well. John |
From: Peter W. <pwi...@bi...> - 2004-08-27 03:43:06
|
John Hesterberg wrote: > On Wed, Aug 25, 2004 at 10:18:42PM -0700, Andrew Morton wrote: > >>Jay Lan <jl...@en...> wrote: >> >>>I have broken up one big CSA kernel patch into four smaller ones >>> as attached: >>> >>> csa_io - collects io accounting data >>> csa_mm - collects mm accounting data >>> csa_eop - provides a hook to perform end-of-process accounting >>> csa_module - builds csa loadable module >> >>More broadly: Help! >> >>I am 100% not in a position to judge whether Linux needs Comprehensive >>System Accounting, nor am I able to define what the requirements for such a >>thing should be. All I can tell from your patch is the quality of its >>implementation, and that's leaping far, far ahead of where we should be. > > > Linux needs something beyond what it has today, at least for the > HPC market SGI is familiar with. We believe it will more generally > benefit Linux HPC and enterprise markets, which is one reason we've > released the whole CSA stack as open source. > > >>We're going to need help from you, and from all the other stakeholders in >>judging how useful this feature is to Linux implementors and how well this >>implementation meets the (unknown) requirements. See my problem? >> >>I've cc'ed lse-tech, where enterprise folks hang out. I would request that >>the people who are stakeholders in this feature >> >>a) stick their hands up > > > We're running CSA in production on Altix (our Itanium/Linux platform) > for several years now. > > >>b) let us know how important this kind of feature is for their users > > > A substantial number of our customers require it. CSA has been > developed over the years on SGI's HPC systems in response to our > customers needs. It's been reimplemented and opensourced for Linux, > originally as an SGI/LANL collaboration. > > >>c) review the offered feature set against their requirements >> >>d) let us know how well the implementation fits that requirement and > > > It fits. :-) > > Actually, one secondary feature on our wishlist is 'projects'. Should be easy to implement with PAGG :-) > > Our customers are tied into the current CSA user interface. > However, there is lots of room for cooperation under that, particularly > in the kernel. We can always consider a migration project as well. > > John > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to maj...@vg... > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ > -- Peter Williams pwi...@bi... "Learning, n. The kind of ignorance distinguishing the studious." -- Ambrose Bierce |
From: Andrew M. <ak...@os...> - 2004-08-27 03:55:52
|
Thanks, guys. So we now know that there are three potential implementations which do much the same thing, yes? I didn't get a sense of a preferred direction, but at least nobody is flaming anybody else yet ;) It strikes me that CSA is the most actively developed and is the furthest along. But that enhancing BSD accounting might be the least intrusive and most back-compatible approach. Is that a fair summary? If not, what should I have said? |
From: Jay L. <jl...@sg...> - 2004-08-27 20:28:25
|
My proposed patchset includes three accounting data collection patches and one CSA loadable module that make use of the collected data. The three data collection patches (in the area of io, mm and end-of-process) do not conflict with BSD accounting or ELSA. They can be viewed as "enhancement" to both BSD and ELSA as Guillaume, the maintainer of ELSA put it: Therefore the solution could be to enhance BSD accounting with data collection from CSA and provide per job accounting with a userspace mechanism. Sounds great to me... It would be then up to BSD, ELSA or CSA to decide how to process the collected data and present it to users. All response so far seems to favor this unified data collection method proposal. The ELSA approach appears to make changes to kernel/acct.c while CSA will provide its own loadable module. Of the three data collection patches, csa_io and csa_eop are very much CSA-independent. I certainly can make csa_mm more so as well. I think it is a everyone-win situation for all three projects. :) Regards, - jay Andrew Morton wrote: > Thanks, guys. So we now know that there are three potential > implementations which do much the same thing, yes? > > I didn't get a sense of a preferred direction, but at least nobody is > flaming anybody else yet ;) > > It strikes me that CSA is the most actively developed and is the furthest > along. But that enhancing BSD accounting might be the least intrusive and > most back-compatible approach. > > Is that a fair summary? If not, what should I have said? > > > ------------------------------------------------------- > SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media > 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 > Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. > http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech |