You can subscribe to this list here.
2008 |
Jan
(1) |
Feb
(2) |
Mar
|
Apr
|
May
|
Jun
|
Jul
|
Aug
|
Sep
|
Oct
|
Nov
|
Dec
|
---|
From: Satoshi F. <sa...@sd...> - 2008-02-26 08:45:24
|
Hi Vaidy-san, Thank you for your feedback. I'm very glad that you use btrax and think it's cool. > The kernel symbols seems to be matched correct, but what about user > space symbols? > > I would like to see the symbol names for 0x080493ba and related > addressed are are from my application. Can I get glibc symbols also > decoded? > If you want to analyze application's behavior, you need to execute 'bt_split' at first. This command converts each cpu's log file to each process's log file. Then, specify the process log file to the bt_execpath. For example, # bt_collect_log -d /tmp/1 -p 11610 # bt_split -d /tmp/1 ... this command creates $(PID) file(s). # bt_execpath --all -f /tmp/1/$(PID) > Can I ask for time stamps at each trace entry as well? I think the > hardware will record only the branch address... so we will have to > switch to other software only methods to collect time stamps. > Time stamp records are added by btrax's kernel module. Btrax uses 'gettimeofday' for the time stamp. So, if you change the machine time by 'time' command, you'll get the wrong sorted process log. Btrax will use 'TSC' (time stamp counter) for time stamp in next release. It will fix the problem shown above. Btrax-1.0.4 will be released in a few days. Best regards, -- Satoshi Fujiwara Vaidyanathan Srinivasan wrote: > * Satoshi Fujiwara <sa...@sd...> [2008-02-04 17:37:15]: > >> Hello Vaidy, >> >> I'm sorry for my late reply. >> >> I checked our machines and found one which has the same cpu >> that you use (cpuid=6f6). >> I tried btrax-1.0.3 on it, and got same panic message >> (general protection fault). >> > Hi Satoshi-san, > > Thanks for the info regarding the processor. I tried on different CPU > and got the tool working. It was a P4 with HT desktop processor. > > cpu family : 15 > model : 4 > model name : Intel(R) Pentium(R) 4 CPU 3.20GHz > stepping : 1 > cpu MHz : 3192.390 > cache size : 1024 KB > > The tool worked great and gives lots of trace information. I have > a sample trace below: > > vmlinux 0xc0103a0b <math_state_restore+0x1b> nop > vmlinux 0xc0103a18 <math_state_restore+0x28> ret > vmlinux 0xc010383c <device_not_available+0x24> jmp 0x00000000c0102b80 <ret_from_exception> > vmlinux 0xc0102b80 <ret_from_exception> mov $0xffffe000,%ebp > vmlinux 0xc0102bac <resume_userspace+0x10> jmp 0x00000000c0102c8e <restore_all> > vmlinux 0xc0102c8e <restore_all> mov 0x30(%esp),%eax > vmlinux 0xc0102cb2 <restore_nocheck+0xc> iret > -------- 0xb7e3dd53 > -------- 0xb7e3dd5c > -------- 0x080493ba > -------- 0x080493ce > -------- 0x080486d8 > -------- 0xb7e831f0 > -------- 0xb7e831ff > -------- 0xb7f10400 > -------- 0xb7f10405 > vmlinux 0xc0102bb4 <sysenter_entry> mov 0xffffde04(%esp),%esp > vmlinux 0xc0102c0a <sysenter_past_esp+0x4f> call *0xc02854a0(,%eax,4) > ctr_mod.ko 0x000000a8 <sys_call_stub> pop %ebp > ctr_mod.ko 0x000000a9 <sys_call_stub+0x1> call 0x00000000000000aa <sys_call_stub+0x2> > ctr_mod.ko 0x00000000 <pre_sys_call> push %ebx > > I used > bt_collect_log -d /tmp/1 -p 11610 > and then > bt_execpath --all -f /tmp/1/cpu0 > > The kernel symbols seems to be matched correct, but what about user > space symbols? > > I would like to see the symbol names for 0x080493ba and related > addressed are are from my application. Can I get glibc symbols also > decoded? > > Please let me know if there are existing solutions to decode user > space symbols as well. > > The ability to trace across system calls, interrupts and exceptions is > very cool. This helps in understanding the application/library and > kernel behavior. > > Can I ask for time stamps at each trace entry as well? I think the > hardware will record only the branch address... so we will have to > switch to other software only methods to collect time stamps. > > Thanks for all the help. I will continue to use the tool and provide > feedback. > > --Vaidy > > |
From: Satoshi F. <sa...@sd...> - 2008-02-04 08:39:27
|
Hello Vaidy, I'm sorry for my late reply. I checked our machines and found one which has the same cpu that you use (cpuid=6f6). I tried btrax-1.0.3 on it, and got same panic message (general protection fault). As a result of analyzing, I found out that panic occurs when btrax writes the MSR_DEBUGCTL. This panic does not occur on other processors, so I doubt that is the processor's errata. I'd like to ask the processor developers of Intel about it, but I don't know such developers... If you know such kind of developers, please introduce me. --- Satoshi Fujiwara Vaidyanathan Srinivasan wrote: > Dear btrax developer, > > I am interested in running branch tracer to analyse multi threaded > application behavior. > > I downloaded, compiled and ran btrax 1.0.3 but I am not successful in > collecting any useful trace. All the modes of operation crash with > the following OOPS: > > bt_mod loaded(5) > general protection fault: 0000 [#1] > SMP > Modules linked in: bt_mod ctr_mod > CPU: 0 > EIP: 0060:[<d0839081>] Not tainted VLI > EFLAGS: 00010002 (2.6.18 #1) > EIP is at bts_facility_on+0x39/0x3e [bt_mod] > eax: 0000001c ebx: 0000001c ecx: 000001d9 edx: 00000000 > esi: cf7be070 edi: 00000001 ebp: c633da1c esp: c633da18 > ds: 007b es: 007b ss: 0068 > Process top (pid: 3741, ti=c633c000 task=cf7be070 task.ti=c633c000) > Stack: 00000000 c633da24 d083916e c633da44 d083a916 00000000 d0853000 00000000 > d083e9a0 c633dab8 c13456c0 c633da6c c042f05b c05fb920 c0102e44 c0136484 > cf7be5d8 00000001 00000000 c04e3f5c c633da94 c633da84 c042fbe0 00000002 > Call Trace: > [<c0104eeb>] show_stack_log_lvl+0x8a/0x95 > [<c010505f>] show_registers+0x12d/0x19a > [<c010524d>] die+0x181/0x26b > [<c042ed54>] do_general_protection+0x16b/0x173 > [<c0104a89>] error_code+0x39/0x40 > [<d083916e>] bts_on_and_set_pid_info+0x36/0x38 [bt_mod] > [<d083a916>] context_switch_kp_wrapper+0x131/0x147 [bt_mod] > [<c042f05b>] kprobe_exceptions_notify+0x192/0x3f5 > [<c042fbe0>] notifier_call_chain+0x20/0x31 > [<c012cc86>] atomic_notifier_call_chain+0xb/0xd > [<c042ed95>] do_int3+0x39/0x70 > [<c042ea7a>] int3+0x1e/0x24 > Code: e0 ff ff 8b 52 10 8b 14 95 00 8b 5b c0 83 3c 10 00 74 1a b9 d9 01 00 00 0f > 32 83 3d b8 e8 83 d0 02 8b 1d 10 e8 83 d0 74 04 09 d8 <0f> 30 5b 5d c3 55 89 e5 > 56 53 89 c3 83 ec 04 b8 80 de 5f c0 89 > EIP: [<d0839081>] bts_facility_on+0x39/0x3e [bt_mod] SS:ESP 0068:c633da18 > > I did some forward porting to 2.6.24-rc5 but this problem still exist. > The above crash dump is from 2.6.18 kernel on which btrax compiles > without any changes. > > I used 2.6.18 upstream kernel on RHEL 5.1 distro. The kernel config is > attached. The test machine is a dual socket dual core xeon with the > following CPU > > /proc/cpuinfo: > > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 15 > model name : Intel(R) Xeon(R) CPU 5160 @ 3.00GHz > stepping : 6 > cpu MHz : 2992.568 > cache size : 4096 KB > physical id : 0 > siblings : 1 > core id : 0 > cpu cores : 1 > fdiv_bug : no > hlt_bug : no > f00f_bug : no > coma_bug : no > fpu : yes > fpu_exception : yes > cpuid level : 10 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat > pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe nx lm > constant_tsc up pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm > bogomips : 5991.13 > > The features of btrax looks very good and I intend to use it to trace > multi-threaded application and library in user space. Please help me > to get btrax working on 2.6.18 and latest kernel. Most probably I am > missing some config options on my kernel. > > Thanks for the help, > Vaidy > > |
From: Satoshi F. <sa...@sd...> - 2008-01-10 06:26:37
|
This is a text mail. Please ignore. |