Re: [pyprof-devel] saw your post on python-dev

[Christian S. <ma...@ap...>]

Dear Geremy,

On 2009-05-04 22:02:06 you wrote:
> I just saw your post on python-dev and set up your module, but I'm not sure
> how to go about
> actually using it. Mind posting a walkthrough someplace?
> 
> thanks for your time,
> Geremy Condra
> 

Currently there's no documentation except for the initial posting. Some more notes:

- the sampling profiler can only handle threads in which Profiler.thread_init() has been invoked (on linux we need the thread ID (tid) of the thread to query per-thread metrics, but we can only query the tid from within the thread); if the application you're trying to profile only creates threads via the threading.Thread interface, then this can be easily accomplished by hooking the Thread.__bootstrap_inner method, e.g.:

	from Profiler import sampling_profiler, thread_init
	import threading
	old_bootstrap = threading.Thread._Thread__bootstrap_inner
	def new_bootstrap(self):
		thread_init()
		return old_bootstrap(self)
	threading.Thread._Thread__bootstrap_inner = new_bootstrap

- the deterministic profiler only "sees" thread that are created while the profiled function is being executed

For the moment I'd recommend patching the application you're trying to profile. Most of the time, you'll have some main() function, e.g.:

	def main():
		...
	
	if __name__ == '__main__':
		main()


to use the profiler, the call to main() needs to be delegated to the profiler function, e.g. sampling_profiler (and for this profiler you also need the thread_init() hook if you're trying to profile a multi-threaded program):

	from Profiler import sampling_profiler, thread_init
	import threading
	old_bootstrap = threading.Thread._Thread__bootstrap_inner
	def new_bootstrap(self):
		thread_init()
		return old_bootstrap(self)
	threading.Thread._Thread__bootstrap_inner = new_bootstrap

	def main():
		...
	
	if __name__ == '__main__':
		sampling_profiler(main)


In a single-threaded program the hook is unnecessary but thread_init() must still be invoked for the main thread:

	from Profiler import sampling_profiler, thread_init

	def main():
		...
	
	if __name__ == '__main__':
		thread_init()
		sampling_profiler(main)


Building the module itself should not be too difficult, depending on the python version in use you might have to alter PYTHON_VERSION in Makefile, additionally you will need at least python-dev, libboost-dev, g++, linux-headers, make (debian package names). If everything is in place simply typing "make" should do the job of building the native module.

Best regards,
Christian