Jim Baker wrote:Any help you could provide on digging information out of the $py.class file would be great. We don't need to interpret it any more than to just get the line numbers out. For example, we don't need to know which lines correspond to which functions or classes. Simply getting the set of lines that are executable in any given file would be enough.You have two options:
1. Get the $py.class file corresponding to the module in question, then use a tool like ASM to visit the Java bytecode and obtain the desired info. Note that there is no way to get the bytecode directly - we don't support co_code, this is yet another JVM restriction, so it requires a step similar to obtaining the source with inspect. This option also requires knowing the layout of Python modules in generated Java bytecode, but to quickly summarize, we use a switch statement to dispatch to the right code body (see org.python.core.PyTableCode). You can directly drive this analysis from Jython itself, and I have written some code you potentially could use a starting point. Let me find that code...
Is there any possibility of Jython providing the information in a more accessible form? For example, while compiling to Java, the list of line numbers could be maintained and then output in some other form, perhaps a global constant in the $py.class file? I'm making this up as I go along, and know nothing about .class files, so maybe it's crazy. My point is that getting that line number information in a more digestible form would be wonderful, and it doesn't have to be literally like co_lnotab in order to be useful. I don't mind writing code in coverage.py just to support Jython, but I don't want to get mired in complex reverse engineering of .class files.
On Mon, May 11, 2009 at 9:16 AM, nedbatchelder <email@example.com> wrote:
Just to clarify: to measure code coverage, we need to get two types of
information: 1) what lines were executed in the code, and 2) what lines
*could* have been executed. #1 is working fine because Jython supports
sys.settrace, as you mentioned. But we're looking for advice on how to get
#2. In CPython, we can examine the co_lnotab table to get the list of all
executable lines. In Jython, the set of potentially executable lines is
represented implicitly in the .class file in the form of calls to setline(),
but that information is not readily accessible, unless you can tell me a
relatively simple way to access it from the .class file.
So Chris and I are looking for advice from the Jython experts: how can we
tell which lines in a Jython program are candidates for execution?
View this message in context: http://www.nabble.com/Identifying-executed-lines-in-jython-tp23482970p23484891.html
Jim Baker wrote:
> Jython fully supports the standard tracing/profiling API (sys.settrace,
> sys.setprofile). In addition, a subset of debugging (pdb) is supported,
> namely we don't allow for setting jumps. However, we don't support
> exactly the same sequence of events. You can see this in the 2.5 tests,
> test_trace and test_profile, as well as run standard profiling. (Note we
> only support profile, not cProfile.)
> Standard attributes are available on the frame, including f_lineno. You
> access directly or use inspect, the latter works in 2.5, and perhaps
> earlier. Generated Java bytecode sets this attribute in the PyFrame
> as well as emitting the corresponding JVM metadata. This allows for
> debugging either by pdb type debuggers or by standard Java source
> We need both because we cannot introspect running Java bytecode in process
> without using an agent, that's a (security) restriction of the JVM.
> co_lnotab is therefore an implementation detail that supports this
> functionality for CPython bytecode.
> Jython 2.5.0 supports the execution of CPython bytecode, although this
> feature must be considered experimental at this point. A future release
> (likely 2.5.1) will enable the compilation of CPython bytecode, until then
> you need a separate compiler. (CPython is a good choice, although it
> support quite the same grammar, eg no direct usage of class, in, and other
> attributes that are normally keywords in Python - Jython of course relaxes
> this restriction.) For these code objects, the stdlib dis seems to work
> fine. You can see an example of how to work with this in test_pbcvm.py.
> Certainly feel free to contact me for more details or better yet discuss
> I don't know the specifics of Coverage or figleaf, and how much they
> on the CPython bytecode format, but hopefully they don't rely too much on
> the CPython bytecode format. Alternatively, you could try our CPython
> bytecode support, most of the regrtest passes it, with failures only
> occurring for code object introspection and slightly different float
> (for NaN, that sort of thing) when marshaled.
> - Jim
> On Mon, May 11, 2009 at 7:05 AM, Chris Burroughs
>> Most python coverage tools depend on the CPython co_lnotab
>> implementation. For example, both Coverage and figleaf use code like
>> this modeled on the stdlib dis and trace modules:
>> byte_increments = [ord(c) for c in code.co_lnotab[0::2]]
>> line_increments = [ord(c) for c in code.co_lnotab[1::2]]
>> The jython differences page
>> <http://jython.sourceforge.net/docs/differences.html> says that this
>> co_lnotab will probably never be supported. What is the recommended
>> way for developers to determine which lines of jython code were
>> executed? Have the co_lnotab support plans changed, or should the
>> .class files be examined, or is there an alternative approach?
>> Thank you.
>> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
>> production scanning environment may not be a perfect world - but thanks
>> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
>> Series Scanner you'll get full speed at 300 dpi even with all image
>> processing features enabled. http://p.sf.net/sfu/kodak-com
>> Jython-dev mailing list
> Jim Baker
> The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
> production scanning environment may not be a perfect world - but thanks to
> Kodak, there's a perfect scanner to get the job done! With the NEW KODAK
> Series Scanner you'll get full speed at 300 dpi even with all image
> processing features enabled. http://p.sf.net/sfu/kodak-com
> Jython-dev mailing list
Sent from the jython-dev mailing list archive at Nabble.com.
The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your
production scanning environment may not be a perfect world - but thanks to
Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700
Series Scanner you'll get full speed at 300 dpi even with all image
processing features enabled. http://p.sf.net/sfu/kodak-com
Jython-dev mailing list
-- Ned Batchelder, http://nedbatchelder.com