Watchpoints in Python

This post is going to be a bit more technical. I’m going to discuss an issue I ran into when trying to track down a bug in Allura, the Open Source project that powers SourceForge, and the solution I was able to use, namely watchpoints.


The issue I was trying to debug was that a variable I was setting was getting overwritten at some point but I could not figure out where or why. I’m a pretty dab hand at using pdb (or, more accurately, ipdb), but while trying to step through the code, I got lost in the maze that is the Pylons / TurboGears controller dispatch. All was bleak, and I despaired.

If only, I thought to myself, pdb supported watchpoints! If you’re not familiar with them, watchpoints are similar to breakpoints, except that instead of stopping on a specific line, they stop when a particular variable changes. Perfect for my situation. gdb, for C, supports them, but pdb, alas, does not. There are some Python IDEs that support them, such as Eric and PyScripter (neither of which I have used), but my development environment isn’t suited to using an IDE.

After some digging, I did manage to come across a stackoverflow answer in which Michael Hoffman managed to implement watchpoints in pdb using the alias command and some internal knowledge of pdb. Perfect, I thought! However, as with many things in life, it was not as easy as it seemed.

The commands work by repeatedly adding themselves to the internal command queue in pdb until the condition is met. The first hurdle I ran into is that, as I mentioned earlier, I prefer ipdb. However, the implementation of the command queue in ipdb is essentially the same as in pdb, such that it only took a few minor modifications to account for those differences. The second issue I ran into was that the original snippet was set up to watch variables, while I needed to watch an attribute. This, too, was not terribly difficult to work around. The part that caused me the most difficulty was in getting the stepwatch command working.

Because the nextwatch command stays in the current frame, it worked fine. However, the stepwatch implementation was relying on some global variables and imports to always be available, and unfortunately global only applies to code parsed at the same time as the global statement. What this meant for me was that all of the internal variables used to implement the watchpoint commands disappeared as soon as I left the current source file. Since my issue required jumping in and out of the Pylons and TurboGears dispatch code, that happened almost immediately.

Well, I thought, if they can’t be global, I’ll just have to stash them somewhere where I know I can get them again later. So, I stashed them in pdb and just had the command reimport pdb before using it to ensure it was available in the current scope. This, in turn, ran me into the issue that statement lists can only include simple statements, and not compound statements like if / else.

After jumping through some hoops to get the syntax correct, I was finally able to get it working, and found the spot that was overwriting my variable in no time flat.

Here is my modified version, which I have saved away in my .pdbrc file:

It’s pretty dense, unfortunately, because of pdb’s limitations on what python code it will accept within the .pdbrc file or in aliases. However, it’s very straightforward to use. To give a brief example of the stepwatch command in action, consider this contrived example python file:

Running this will stop on line 25 and drop you into ipdb, from which you can use stepwatch to find where a.value gets changed:

Note that once the stepwatch command was issued, no further input was required on my part to reach the exact spot where the value was modified. This worked across multiple stack frames and would have worked across multiple files.

Aside from the IDEs with watch commands available, I found out later that PyConquer, while not having watchpoints per se, does have the ability to watch a variable and notify you of the exact location where it is changed, and to what. However, I prefer not having to switch to a different tool to find the location, then go set a breakpoint there.

I also subsequently found Pdb++, written by Antonio Cuni to aid working on PyPy, which has a few additional commands, watch being one of them. I have not had a chance, yet, to test this out, but it seems like it could be a cleaner solution.

As it turns out, actually fixing my variable being overwritten was a much more difficult proposition, but you can’t win them all, I guess.

Watch image from Shutterstock


One Response to “Watchpoints in Python”

  1. Kapil Apr 18, 2014 at 1:43 pm #

    Nice Post! I am testing a tool for helping automate discovery of such bugs. Do you have a link to the Allure bug that you were tracking. I will try to test our tool on that bug to see if it’s helpful. Thanks!