52

I'd like to profile a method of a function in Python, using cProfile. I tried the following:

import cProfile as profile # Inside the class method... profile.run("self.myMethod()", "output_file")

But it does not work. How can I call a self.method with "run"?

Improve this question
1

5 Answers 5

Reset to default
76

run just tries to exec the string you pass it. If self isn't bound to anything in the scope of the profiler you are using, you can't use it in run! Use the runctx method to pass in the local and global variables in the scope of the call to the profiler:

>>> import time >>> import cProfile as profile >>> class Foo(object): ... def bar(self): ... profile.runctx('self.baz()', globals(), locals()) ... ... def baz(self): ... time.sleep(1) ... print 'slept' ... time.sleep(2) ... >>> foo = Foo() >>> foo.bar() slept 5 function calls in 2.999 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 1 0.000 0.000 2.999 2.999 <stdin>:5(baz) 1 0.000 0.000 2.999 2.999 <string>:1(<module>) 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects} 2 2.999 1.499 2.999 1.499 {time.sleep}

Notice the last line: time.sleep is what's taking up the time.

Improve this answer
Sign up to request clarification or add additional context in comments.

7 Comments

This just doesn't work on Python 2.6.4 on Mac OS X. I get the error NameError: name 'foo' is not defined
@user: sorry, my bad. Edited.
How can I make the profiler "dig deeper", i.e. not just say that all the time was spent in some function of a module but recursively delve into functions called in that module?
What do you mean? The profiling times code starting from the call of the method until the return value. It should give results for every method called in that module. You should check out my answer, it will give you a nice graph of what the result is
Brilliantness! Passing cProfile.runctx() the current globals() and locals() permits arbitrary methods to be profiled – exactly as advertised. Thanks for this, katrielalex.
|
32

Use the profilehooks decorator

http://pypi.python.org/pypi/profilehooks

Improve this answer

4 Comments

Handy, but doesn't work on Python 3, though 2to3 fixes that. :)
Profilehooks now supports Python 3. See here: pypi.python.org/pypi/profilehooks
No. Don't install heavyweight dependencies. Just call cProfile.runctx('self.myMethod()', globals(), locals(), output_file), as katrielalex suggests in what should have been the accepted answer.
profilehooks works perfectly to profile Tkinter applications. With the cProfile approach, you're having it directly execute a function, which if the function is supposed to be a callback to a Tkinter.Button, may not be so easy w/o restructuring code. With profilehooks, importing the profile decorator and applying it to a specific callback lets you test just that callback only when the button is clicked.
5 
 import cProfile p = cProfile.Profile() p.runcall(self.myMethod) p.print_stats()

The Profile class is documented here.

Improve this answer

Comments

4

If your function under profile returns value(s), you need to change the excellent answer from @katrielalex slightly:

... profile.runctx('val = self.baz()', globals(), locals()) ... print locals()['val']
Improve this answer

Comments

-1

I wouldn't recommend profiling a single routine, because that implies knowing in advance there's a problem there.

A fundamental aspect of performance problems is they're sneaky. They're not where you think they are, because if they were you would have solved them already.

It's better to run the whole program with a realistic workload and let the profiling technique tell you where the problems are.

Here's an example where profiling finds the problem, and it is not where expected.

Improve this answer

8 Comments

All I want to do is run a profiler on this top level routine and have the profiler actually delve in to the functions instead of just saying all the top was spend in some top level routine, since that is completely useless.
@user248237: Right. Just do what I said in the link. No profiler will give you better information.
@Mike Dunlavey: Not necessarily. While profiling the entire code, you may then have determined that the bottleneck appears in one function. In which case, it would make sense to then profile that one function to see what's going on there..
@Gino: What's a bottleneck? Lots of people talk about bottlenecks as if "everybody" knows what they are. If a mega-line C# application spends 50% of its time reading dlls to extract string attributes so that they can be translated for the current geography, when you see by looking at the strings that it doesn't need to, is that a bottleneck? Would a profiler find it? (No) Would random-pausing find it? (Yes) Random-pausing will locate any bottleneck that any profiler will, and more that it won't.
@Mike Dunlavey: Frankly, I'm not sure what your point is. The basic assumption is that the programmer understands the code well enough to discern what is relevant and what is not. When I say a bottleneck appears a function, one obvious way that this can happen is if I run a profiler and the profiler tells me that 90% of the time is spent in function A. So function A is a bottleneck that I want to drill down into. And I want to know which lines of function A are taking up most of the time..
|