Re: oprofile 2.5.42 patch not running on athlon machine

[John Levon <levon@mo...>]

On Wed, Oct 16, 2002 at 05:48:50PM -0400, William Cohen wrote:

> I don't think the system isn't allowing it to mount multiple times, but 
> it definitely did not work when it tried to mount oprofilefs and 
> oprofilefs was already mounted. unmount /dev/oprofile, try again and 
> things worked.


Weird.

> I will work on improvements for the scripts. That is going to be needed 
> if the plan is to move oprofile to just using the Linux 2.5 patch.

That's not the plan. I don't intend to remove the 2.4 stuff any time
soon.

However, obviously we need to clean up the 2.5 scripts stuff. I think
that "op_ctrl" is really the best option here, so we can clean up the
interface too. So I'd like to see a new op_ctrl roughly as described in
the previous mail, so we can support quick start/stop etc. properly

regards
john
-- 
"It's a cardboard universe ... and if you lean too hard against it, you fall
 through." 
	- Philip K. Dick 

Re: oprofile 2.5.42 patch not running on athlon machine

[William Cohen <wcohen@nc...>]

John Levon wrote:
> On Wed, Oct 16, 2002 at 05:13:50PM -0400, William Cohen wrote:
> 
> 
>>I have tracked the problem I have on the athlon machine to the 
>>op_start_25 script. The script blindly tries to mount the oprofilefs 
>>file system. This appeared to cause problems on later attempts when the 
>>oprofilefs file system is already mounted on /dev/oprofile. I have a 
> 
> 
> I'm perplexed as to how this could possibly cause problems. Is your
> "mount" letting you mount multiple times ?

I don't think the system isn't allowing it to mount multiple times, but 
it definitely did not work when it tried to mount oprofilefs and 
oprofilefs was already mounted. unmount /dev/oprofile, try again and 
things worked.

>>The daemon is running and samples are being written to the disk on the 
>>RH Linux 8 athlon machine.
> 
> 
> Great.

I will let it run with a gcc 3.1 "make bootstrap >& problems" providing 
some machine load. I will check it tomorrow to see whether everything 
worked.

>>The op_start_25, op_stop_25, and op_dump scripts still need work. The 
>>scripts are making assumptions about the machine state (no oprofilefs 
>>mounted and no daemon running). op_dump doesn't work at all for 2.5.
> 
> 
> Yup.
> 
> 
>>Could the 30 second timeouts in op_start and op_stop be reduce to 10 or 
>>15 seconds?
> 
> 
> Probably...

I will work on improvements for the scripts. That is going to be needed 
if the plan is to move oprofile to just using the Linux 2.5 patch.

-Will

>>--- ../oprofile/utils/op_start_25	2002-10-14 19:40:33.000000000 -0400
>>+++ ./utils/op_start_25	2002-10-16 17:36:50.000000000 -0400
>>@@ -90,5 +90,7 @@
>> 		fi
>> 	fi
>>-	mount -t oprofilefs nodev /dev/oprofile >/dev/null
>>+	if test ! `grep -o oprofilefs /etc/mtab`; then
>>+		mount -t oprofilefs nodev /dev/oprofile >/dev/null
>>+	fi
>> }
> 
> 
> We want this patch anyway regardless of the above
> 
> thanks
> john
> 

Re: oprofile 2.5.42 patch not running on athlon machine

[John Levon <levon@mo...>]

On Wed, Oct 16, 2002 at 05:13:50PM -0400, William Cohen wrote:

> I have tracked the problem I have on the athlon machine to the 
> op_start_25 script. The script blindly tries to mount the oprofilefs 
> file system. This appeared to cause problems on later attempts when the 
> oprofilefs file system is already mounted on /dev/oprofile. I have a 

I'm perplexed as to how this could possibly cause problems. Is your
"mount" letting you mount multiple times ?

> The daemon is running and samples are being written to the disk on the 
> RH Linux 8 athlon machine.

Great.

> The op_start_25, op_stop_25, and op_dump scripts still need work. The 
> scripts are making assumptions about the machine state (no oprofilefs 
> mounted and no daemon running). op_dump doesn't work at all for 2.5.

Yup.

> Could the 30 second timeouts in op_start and op_stop be reduce to 10 or 
> 15 seconds?

Probably...

> --- ../oprofile/utils/op_start_25	2002-10-14 19:40:33.000000000 -0400
> +++ ./utils/op_start_25	2002-10-16 17:36:50.000000000 -0400
> @@ -90,5 +90,7 @@
>  		fi
>  	fi
> -	mount -t oprofilefs nodev /dev/oprofile >/dev/null
> +	if test ! `grep -o oprofilefs /etc/mtab`; then
> +		mount -t oprofilefs nodev /dev/oprofile >/dev/null
> +	fi
>  }

We want this patch anyway regardless of the above

thanks
john

-- 
"It's a cardboard universe ... and if you lean too hard against it, you fall
 through." 
	- Philip K. Dick 

Re: oprofile 2.5.42 patch not running on athlon machine

[William Cohen <wcohen@nc...>]

I have tracked the problem I have on the athlon machine to the 
op_start_25 script. The script blindly tries to mount the oprofilefs 
file system. This appeared to cause problems on later attempts when the 
oprofilefs file system is already mounted on /dev/oprofile. I have a 
patch for op_start_25 which check and only mounts it if oprofilefs isn't 
mounted. That seems to make things work better. I see sample counts in 
/dev/oprofile/stats/cpu0/sample_received

The daemon is running and samples are being written to the disk on the 
RH Linux 8 athlon machine.

The op_start_25, op_stop_25, and op_dump scripts still need work. The 
scripts are making assumptions about the machine state (no oprofilefs 
mounted and no daemon running). op_dump doesn't work at all for 2.5.

Could the 30 second timeouts in op_start and op_stop be reduce to 10 or 
15 seconds?

-Will

John Levon wrote:
> On Wed, Oct 16, 2002 at 09:36:22AM -0400, William Cohen wrote:
> 
> 
>>The syslog has the "Using NMI interrupt."
>>
>>All entries in /dev/oprofile/stats are 0, doesn't seem to be any 
>>measurements at all.

Re: OProfile problem on kernel 2.4.20-pre10-ac1 and AthlonXP

[Philippe Elie <phil.el@wa...>]

Andy Tai wrote:
> Hi, I have the two kernel configuration option enabled
> in .config as you suggest.  
> CONFIG_X86_LOCAL_APIC=y
> CONFIG_X86_IO_APIC=y
> 
> I did remove /var/lib/oprofile and restart.  Same
> error message.
> 
> dmesg shows nothing unusual:
> 
> oprofile: APIC was already enabled
> oprofile 0.3 loaded, major 254

ok, that's right.

module/oprofiled.c:oprof_open() start with a

  if (!capable(CAP_SYS_PTRACE))
      return -EPERM;

that's the only path in oprof_open() where we return EPERM
but I dunno why your "root" isn't capable of that.

perhaps strace -f will give some clue. You must look for the
open on device name "ophashmapdev"

Add a printk in oprof_open() to check the returned
EPERM come from this code path can also give a clue.

regards,
Phil

Re: OProfile problem on kernel 2.4.20-pre10-ac1 and AthlonXP

[John Levon <levon@mo...>]

On Tue, Oct 15, 2002 at 08:24:42PM -0700, Andy Tai wrote:

> Failed to open hash map device: Operation not permitted

This is very odd. Are the permissions right ? You must be running as
root otherwise op_start would fail earlier.

oprofile itself can't return -EPERM from the opening of the hash map

regards
john

-- 
"It's a cardboard universe ... and if you lean too hard against it, you fall
 through." 
	- Philip K. Dick 

Re: oprofile 2.5.42 patch not running on athlon machine

[John Levon <levon@mo...>]

On Wed, Oct 16, 2002 at 09:36:22AM -0400, William Cohen wrote:

> The syslog has the "Using NMI interrupt."
> 
> All entries in /dev/oprofile/stats are 0, doesn't seem to be any 
> measurements at all.

OK, this means that for some reason the counters aren't being abled
properly. Here is the diff between oprofile 2.4 and 2.5 for athlon
model.

I suggest experimenting with backing parts of this out in the kernel and
seeing where it went wrong.

regards
john

--- src/oprofile/module/x86/op_model_athlon.c	Sun Sep 22 20:20:37 2002
+++ /old/linux-2.5/linux/arch/i386/oprofile/op_model_athlon.c	Wed Oct 16 17:53:18 2002
@@ -10,14 +10,18 @@
  * @author Graydon Hoare
  */
 
+#include <linux/oprofile.h>
+#include <asm/ptrace.h>
+#include <asm/msr.h>
+ 
 #include "op_x86_model.h"
-#include "op_msr.h"
+#include "op_counter.h"
 
 #define NUM_COUNTERS 4
 #define NUM_CONTROLS 4
 
 #define CTR_READ(l,h,msrs,c) do {rdmsr(msrs->counters.addrs[(c)], (l), (h));} while (0)
-#define CTR_WRITE(l,msrs,c) do {wrmsr(msrs->counters.addrs[(c)], -(u32)(l), -1);} while (0)
+#define CTR_WRITE(l,msrs,c) do {wrmsr(msrs->counters.addrs[(c)], -(unsigned int)(l), -1);} while (0)
 #define CTR_OVERFLOWED(n) (!((n) & (1U<<31)))
 
 #define CTRL_READ(l,h,msrs,c) do {rdmsr(msrs->controls.addrs[(c)], (l), (h));} while (0)
@@ -31,6 +35,7 @@
 #define CTRL_SET_UM(val, m) (val |= (m << 8))
 #define CTRL_SET_EVENT(val, e) (val |= e)
 
+static unsigned long reset_value[NUM_COUNTERS];
  
 static void athlon_fill_in_addresses(struct op_msrs * const msrs)
 {
@@ -48,9 +53,9 @@
  
 static void athlon_setup_ctrs(struct op_msrs const * const msrs)
 {
-	uint low, high;
+	unsigned int low, high;
 	int i;
-
+ 
 	/* clear all counters */
 	for (i = 0 ; i < NUM_CONTROLS; ++i) {
 		CTRL_READ(low, high, msrs, i);
@@ -65,46 +70,51 @@
 
 	/* enable active counters */
 	for (i = 0; i < NUM_COUNTERS; ++i) {
-		if (sysctl.ctr[i].event) {
+		if (counter_config[i].event) {
+			reset_value[i] = counter_config[i].count;
 
-			CTR_WRITE(sysctl.ctr[i].count, msrs, i);
+			CTR_WRITE(counter_config[i].count, msrs, i);
 
 			CTRL_READ(low, high, msrs, i);
 			CTRL_CLEAR(low);
 			CTRL_SET_ENABLE(low);
-			CTRL_SET_USR(low, sysctl.ctr[i].user);
-			CTRL_SET_KERN(low, sysctl.ctr[i].kernel);
-			CTRL_SET_UM(low, sysctl.ctr[i].unit_mask);
-			CTRL_SET_EVENT(low, sysctl.ctr[i].event);
+			CTRL_SET_USR(low, counter_config[i].user);
+			CTRL_SET_KERN(low, counter_config[i].kernel);
+			CTRL_SET_UM(low, counter_config[i].unit_mask);
+			CTRL_SET_EVENT(low, counter_config[i].event);
 			CTRL_WRITE(low, high, msrs, i);
+		} else {
+			reset_value[i] = 0;
 		}
 	}
 }
 
  
-static void athlon_check_ctrs(uint const cpu, 
+static int athlon_check_ctrs(unsigned int const cpu, 
 			      struct op_msrs const * const msrs, 
 			      struct pt_regs * const regs)
 {
-	ulong low, high;
+	unsigned int low, high;
+	int handled = 0;
 	int i;
 	for (i = 0 ; i < NUM_COUNTERS; ++i) {
 		CTR_READ(low, high, msrs, i);
 		if (CTR_OVERFLOWED(low)) {
-			op_do_profile(cpu, regs, i);
-			CTR_WRITE(oprof_data[cpu].ctr_count[i], msrs, i);
+			oprofile_add_sample(regs->eip, i, cpu);
+			CTR_WRITE(reset_value[i], msrs, i);
+			handled = 1;
 		}
 	}
-	
+	return handled;
 }
 
  
 static void athlon_start(struct op_msrs const * const msrs)
 {
-	uint low, high;
+	unsigned int low, high;
 	int i;
 	for (i = 0 ; i < NUM_COUNTERS ; ++i) {
-		if (sysctl.ctr[i].count) {
+		if (reset_value[i]) {
 			CTRL_READ(low, high, msrs, i);
 			CTRL_SET_ACTIVE(low);
 			CTRL_WRITE(low, high, msrs, i);
@@ -115,14 +125,15 @@
 
 static void athlon_stop(struct op_msrs const * const msrs)
 {
-	uint low,high;
+	unsigned int low,high;
 	int i;
+
+	/* Subtle: stop on all counters to avoid race with
+	 * setting our pm callback */
 	for (i = 0 ; i < NUM_COUNTERS ; ++i) {
-		if (sysctl.ctr[i].count) {
-			CTRL_READ(low, high, msrs, i);
-			CTRL_SET_INACTIVE(low);
-			CTRL_WRITE(low, high, msrs, i);
-		}
+		CTRL_READ(low, high, msrs, i);
+		CTRL_SET_INACTIVE(low);
+		CTRL_WRITE(low, high, msrs, i);
 	}
 }
 
-- 
"It's a cardboard universe ... and if you lean too hard against it, you fall
 through." 
	- Philip K. Dick 

Re: OProfile problem on kernel 2.4.20-pre10-ac1 and AthlonXP

[Andy Tai <lichengtai@ya...>]

Hi, I have the two kernel configuration option enabled
in .config as you suggest.  
CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y

I did remove /var/lib/oprofile and restart.  Same
error message.

dmesg shows nothing unusual:

oprofile: APIC was already enabled
oprofile 0.3 loaded, major 254

Thanks for any help.

Andy


--- Philippe Elie <phil.el@...> wrote:
> Andy Tai wrote:
> > Hi, I am having problem with oprofile 0.3 on a AMD
> > Athlon XP 1800+ machine running Linux kernel
> > 2.4.20-pre10-ac1.
> > I installed according to the instructions.  When I
> > tried to start it, it shows error message as
> below:
> > 
> > op_start
> >
>
--vmlinux=/share/software/kernel-2.4.20/linux-2.4.19/vmlinux
> > --ctr0-event=HARDWARE_INTERRUPTS --ctr0-count=3000
> 
> > Failed to open hash map device: Operation not
> > permitted
> > Couldn't start oprofiled.
> 
> I've tried 2.4.20-pre10-ac1 with cvs oprofile on a
> PIII
> and it work fine. Kernel configured with
> 
> CONFIG_X86_LOCAL_APIC=y
> CONFIG_X86_IO_APIC=y
> 
> can you try to rm -rf /var/lib/oprofile and restart,
> if this fail do you get any messages with dmesg ?
> 
> 
> $ dmesg | tail
> ...
> oprofile: APIC was already enabled
> oprofile 0.4cvs loaded, major 254
> 
> 
> > I tried other event types for AMD Athlon XP and
> got
> > the same results. 
> > 
> > I have oprofile working on a Pentium III machine
> > before on kernel 2.4.18.
> > 
> > Any help on solving this problem is greatly
> > appropriated.
> 
> regards,
> Phil
> 
> 
> 
>
-------------------------------------------------------
> This sf.net email is sponsored by: viaVerio will pay
> you up to
> $1,000 for every account that you consolidate with
> us.
> http://ad.doubleclick.net/clk;4749864;7604308;v?
> http://www.viaverio.com/consolidator/osdn.cfm
> _______________________________________________
> oprofile-list mailing list
> oprofile-list@...
>
https://lists.sourceforge.net/lists/listinfo/oprofile-list


__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: oprofile 2.5.42 patch not running on athlon machine

[William Cohen <wcohen@nc...>]

John Levon wrote:
> On Tue, Oct 15, 2002 at 05:13:17PM -0400, William Cohen wrote:
> 
> 
>>versions.  I still haven't been successful in getting any data on the 
>>athlon machine. I am thinking about the difference between my PIII and 
> 
> 
> Ugh, I forgot about this.

I am installing RH Linux 8.0 on a unused PII machine. If that works, 
then the problem is the related to the Athlon driver. I will let you 
know what the results are as soon as I install RH 8.0, get the 2.5.42 
kernel and oprofile code running on it.

>>The data collection does not work on my athlon machine. There is an 
>>oprofiled in the process table, but it doesn't appear that any samples 
>>are being taken. Suggestions on things I should look for?
> 
> 
> o check syslog has "Using NMI interrupt"
> o check /dev/oprofile/stats
> o printk/printfs

The syslog has the "Using NMI interrupt."

All entries in /dev/oprofile/stats are 0, doesn't seem to be any 
measurements at all.

Any suggested places to check with printk/printfs?

>>The op_start_25 script it doesn't kill the old process. Have to kill 
>>things by hand. John, why was that stripped out?
> 
> 
> User request - we should split up "starting the daemon" from "starting
> profiling". I'm thinking of an op_ctrl :
> 
> op_ctrl --setup --vmlinux=...
> 
> 	- starts/restarts the daemon
> 
> op_ctrl --start
> 
> 	- echo 1 >/dev/oprofile/enable
> 
> op_ctrl --stop
> 
> 	- echo 0 >/dev/oprofile/enable
> 
> op_ctrl --reset/--save
> 
> 	- rm -rf /var/lib/oprofile/samples; kill -HUP `cat
> 	  /var/lib/oprofile/lock`
> 
> op_ctrl --shutdown
> 
> 	kill -HUP `cat /var/lib/oprofile/lock`
> 
> op_ctrl --dump
> 
> 	See below
> 
> 
>>oprof_start is going to need to be massaged for 2.5 oprofile because it 
>>expects to get the cpu_type information and that isn't available until 
>>the "mount /dev/oprofile" occurs in op_start_25.
> 
> 
> Yes it needs fixing
> 
> 
>>How does one force a dump of the profiling data in the 2.5 system? The 
>>files that op_dump touches are no longer around?
> 
> 
> I'm not sure how we shuold do this yet. A simple echo 0 then echo 1
> won't work unless we can be sure the daemon has gone back to sleep
> before the "echo 1"
> 
> I'd like a way that's less fugly than what happens in 2.4
> 
> regards
> john

correction on thread profiling

[Ka Fai Lau <kafai0928@ya...>]

I miss some file patches in last email. new patch
attached.

remarking -Wunused is needed because opd_go_daemon and
opd_get_ascii_procs are remarked temporary.

KaFai




__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

thread profiling

[Ka Fai Lau <kafai0928@ya...>]

It is my second try already. In my first try, I did
modify the opd_get_image_by_hash,
opd_handle_hashmap...etc.  In my second try, I realize
that it is not necessary because each pid has its own
sample file and also image.

I remarked the opd_go_daemon for my own debugging and
the opd_get_ascii_procs because I didn't parse the
/proc for thread_profiling.

I am asking comments for this approach.

KaFai



__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: OProfile problem on kernel 2.4.20-pre10-ac1 and AthlonXP

[Philippe Elie <phil.el@wa...>]

Andy Tai wrote:
> Hi, I am having problem with oprofile 0.3 on a AMD
> Athlon XP 1800+ machine running Linux kernel
> 2.4.20-pre10-ac1.
> I installed according to the instructions.  When I
> tried to start it, it shows error message as below:
> 
> op_start
> --vmlinux=/share/software/kernel-2.4.20/linux-2.4.19/vmlinux
> --ctr0-event=HARDWARE_INTERRUPTS --ctr0-count=3000 
> Failed to open hash map device: Operation not
> permitted
> Couldn't start oprofiled.

I've tried 2.4.20-pre10-ac1 with cvs oprofile on a PIII
and it work fine. Kernel configured with

CONFIG_X86_LOCAL_APIC=y
CONFIG_X86_IO_APIC=y

can you try to rm -rf /var/lib/oprofile and restart,
if this fail do you get any messages with dmesg ?


$ dmesg | tail
...
oprofile: APIC was already enabled
oprofile 0.4cvs loaded, major 254


> I tried other event types for AMD Athlon XP and got
> the same results. 
> 
> I have oprofile working on a Pentium III machine
> before on kernel 2.4.18.
> 
> Any help on solving this problem is greatly
> appropriated.

regards,
Phil

OProfile problem on kernel 2.4.20-pre10-ac1 and AthlonXP

[Andy Tai <lichengtai@ya...>]

Hi, I am having problem with oprofile 0.3 on a AMD
Athlon XP 1800+ machine running Linux kernel
2.4.20-pre10-ac1.
I installed according to the instructions.  When I
tried to start it, it shows error message as below:

op_start
--vmlinux=/share/software/kernel-2.4.20/linux-2.4.19/vmlinux
--ctr0-event=HARDWARE_INTERRUPTS --ctr0-count=3000 
Failed to open hash map device: Operation not
permitted
Couldn't start oprofiled.


I tried other event types for AMD Athlon XP and got
the same results. 

I have oprofile working on a Pentium III machine
before on kernel 2.4.18.

Any help on solving this problem is greatly
appropriated.

Andy





__________________________________________________
Do you Yahoo!?
Faith Hill - Exclusive Performances, Videos & More
http://faith.yahoo.com

Re: oprofile 2.5.42 patch not running on athlon machine

[John Levon <levon@mo...>]

On Tue, Oct 15, 2002 at 05:13:17PM -0400, William Cohen wrote:

> versions.  I still haven't been successful in getting any data on the 
> athlon machine. I am thinking about the difference between my PIII and 

Ugh, I forgot about this.

> The data collection does not work on my athlon machine. There is an 
> oprofiled in the process table, but it doesn't appear that any samples 
> are being taken. Suggestions on things I should look for?

o check syslog has "Using NMI interrupt"
o check /dev/oprofile/stats
o printk/printfs

> The op_start_25 script it doesn't kill the old process. Have to kill 
> things by hand. John, why was that stripped out?

User request - we should split up "starting the daemon" from "starting
profiling". I'm thinking of an op_ctrl :

op_ctrl --setup --vmlinux=...

	- starts/restarts the daemon

op_ctrl --start

	- echo 1 >/dev/oprofile/enable

op_ctrl --stop

	- echo 0 >/dev/oprofile/enable

op_ctrl --reset/--save

	- rm -rf /var/lib/oprofile/samples; kill -HUP `cat
	  /var/lib/oprofile/lock`

op_ctrl --shutdown

	kill -HUP `cat /var/lib/oprofile/lock`

op_ctrl --dump

	See below

> oprof_start is going to need to be massaged for 2.5 oprofile because it 
> expects to get the cpu_type information and that isn't available until 
> the "mount /dev/oprofile" occurs in op_start_25.

Yes it needs fixing

> How does one force a dump of the profiling data in the 2.5 system? The 
> files that op_dump touches are no longer around?

I'm not sure how we shuold do this yet. A simple echo 0 then echo 1
won't work unless we can be sure the daemon has gone back to sleep
before the "echo 1"

I'd like a way that's less fugly than what happens in 2.4

regards
john
-- 
"It's a cardboard universe ... and if you lean too hard against it, you fall
 through." 
	- Philip K. Dick