While trying to debug an application, I left the instrumented jar on the classpath ('cause I was too lazy to remove it). This led to weird random JVM crashes as I stepped through the code. I am using Eclipse 3.1.
I recognize that the sourcecode and the instrumented bytecode don't align precisely. Is this just a side effect that I'll have to deal with?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
If this is the case, then you probably shouldn't debug with instrumented code. I'd automate this by modifying your build process to have EMMA compile to a different location just for the purpose of reporting.
PATIENT: It hurts when I press here.
DOCTOR: Don't press there.
-- Chad
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
EMMA-instrumented bytecode has source line numbers adjusted so that the exception stack traces are preserved perfectly (and they are).
However, it does not have variable table offsets adjusted, because that info is only used by debuggers and decompilers. Neither of these use cases needs to be supported, right?
So, if you try to debug instrumented code you will see issues. An easy fix is not to do it.
Vlad.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
This is causing a problem for the JBoss Transactions project. Our unit, integration and system use a Java agent called Byteman (see http://www.org.jboss.com/byteman) to rewrite bytecode at load time. Byteman injects highly specific faults into the product code in order to ensure that the code exercises the test scenario. It also injects trace code in order to allow verification of correct execution. We would like also to be able to instrument our tests for code coverage. However, the bad entries in the local variable table are interfering with the operation of the ASM bytecode manipulation library on which Byteman is based. It is throwing exceptions when it processes instrumented classes because it (quite legitimately) expects the extents for local var table entries to begin and end on an instruction boundary. Fixing ASM is not an option - it cannot make up values for the local var table entries.
Could you please include the patch supplied by mdlavin into an upcoming release?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Another thing: The supplied patch is not quite correct. Method new_Attribute_info of class Attribute_info needs to create a LocalVariableTableAttribute_info instance in two cases, for both a LVT attribute and an LVTT attribute. The latter are teh hack introduced in JDK 1.5 to introduce signatures as well as types for local variables. Without this code written using generics wil not transform correctly.
This requires adding the following two cases to the if else cascade in new_Attribute_info()
else if (ATTRIBUTE_LOCAL_VARIABLE_TABLE.equals (name))
{
return new LocalVariableTableAttribute_info (attribute_name_index, attribute_length, bytes);
}
else if (ATTRIBUTE_LOCAL_VARIABLE_TYPE_TABLE.equals (name))
{
return new LocalVariableTableAttribute_info (attribute_name_index, attribute_length, bytes);
}
ATTRIBUTE_LOCAL_VARIABLE_TYPE_TABLE needs to be defined in class Attribute_info as the string "LocalVariableTypeTable".
Note that LVT and LVTT entries both use the same format so one size fits all.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I have found another error in the patch supplied by mdlavin. The code which updates the extents for local variables currently increments the start and end positions by the relevant number of bytes when instrumentation code is placed before the start of a variable. This is normally correct. However, there is a special case. When the start location of the variable is zero then it should remain zero even when instrumentation code is placed at the beginning of the bytecode sequence. Start position zero indicates a method argument and method arguments remain in scope even if instrumentation code is added at the head of the method. This requires changing method visit (LocalVariableTableAttribute_info_info, Object) of class InstrVisitor which was added in the patch as follows
original code in patch
final int lineCount = attribute.size ();
for (int l = 0; l < lineCount; ++ l)
{
final LocalVariable_info var = attribute.get (l);
// TODO: make this faster using either table assist or the sorted array in 'sortedLines'
// adjust bytecode offset for line number mapping:
int end_pc = var.m_start_pc+var.m_length; // use m_start_pc before adjustment to get original end
int adjSegment = lowbound (m_methodJumpAdjOffsets, var.m_start_pc);
var.m_start_pc += m_methodJumpAdjValues ;
I have found one more bug which affects the patch supplied by mdlavin although the problem lies in the emma code, not in his patch code. The loop which adjusts local variable table entry start and size fields is not correctly updating the size of local variables whose extent reaches the final instruction of the original bytecode. It leaves these variables going out of scope at a point part way into the instrumentation code inserted before the final return.
When a local variable extends right to the end of the method body the local variable table entry start and size fields add together to produce an offset which lies 1 byte beyond the final return instruction at the end of the last block. The instrumentation process adds code before this return instruction which means that the original end point now precedes the return statement. However, the code in InstrVisitor does not add an adjustment table entry for this last block. Looking at the allocation statement for this array the size of the table is allocated using blockcount:
final int jumpAdjOffsets = new int ; // room for initial 0 + (blockCount - 1)
final int jumpAdjMap = new int ; // room for initial 0 + (blockCount - 1)
Clearly the code assumes that the last insertion cannot affect positions/sizes for attributes. This is an invalid assumption for local variable table entries which extend to the end of the method. They should still do so after the final instrumentation code has been added. This causes a problem when trying to feed the emma transformed code through an agent based on ASM because the instruction inserted where the original return was placed is an aload. This does not employ the same number of bytes as a return. This means that the end position in the transformed bytecode is no longer aligned on an instruction boundary.
This can be fixed by including the offset for the final inserted bytecode into the adjustment table ie.. allocate room for one more block and change the < comparison to a <= in the branches where a decision is made to update the adjustment table.
This fix should not affect any other offset computations since it only affects cases where a position off the the end of a block is used to mark a region end. The only other case (I am aware of) which might be subject to the same miscalculation is exception table end points which also calculate the end of the excepting block using start + size. The end point arrived at is one byte beyond the end of the bytecode segment in which the exception handler is valid. I think these end points will never computed incorrectly though because the excepting block will terminate the bytecode - it is always followed by a handler block. Even if the handler code is only a simple return statement the end point of the excepting block will be at the return instruction rather than beyond it.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
One more correction. The fix required to enable local variable table offset to work is to include an extra entry in the table which maps any position just beyond the end of the old bytecode array to a comparable position just beyond the end of the new array. So, this requires increasing the size of jumpAdjOffsets and jumpAdjMap by one and then adding the following lines at the end of the loop which initialises them
int oldBytecodeEnd = attribute.getCodeSize();
int newBytecodeEnd = emitctx.m_out.size ();
This ensures that local variables whose extent reaches to the end of the original bytecode will maintain the smae property with respect to the new bytecode array.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Ok, this patch has now been incorporated into a fix for JBoss Transactions issue JBTM-682 (https://jira.jboss.org/jira/browse/JBTM-682). The JIRA includes a diffs file detailing all changes made by me or mdlavin relative to the 2.0.5312 source release.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I came across another problem when feeding emma instrumented bytecode through ASM. The ASM code does not update StackMapTable attributes which detail the layout of local var and stack frames at various points in the code (these are new to JDK6). Passing unadjusted versions of these attributes through to ASM for further bytecode transformation causes muchas problemas.
I have patched the emma code a second time so it correctly remaps offsets contained in StackMapTable entries. A diffs file of both patches against the base 2.0.5312 release and a link to the patched code in the JBossTS repo wrkspace are available via JBoss Transactions issue JBTM-682 (https://jira.jboss.org/jira/browse/JBTM-682).
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
I am new to Emma, Can you plz tell me how to instrument jar files with emma and run test cases against the instrumented jar file to calculate the coverage?
Thanks
Cheers, ALi
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
While trying to debug an application, I left the instrumented jar on the classpath ('cause I was too lazy to remove it). This led to weird random JVM crashes as I stepped through the code. I am using Eclipse 3.1.
I recognize that the sourcecode and the instrumented bytecode don't align precisely. Is this just a side effect that I'll have to deal with?
If this is the case, then you probably shouldn't debug with instrumented code. I'd automate this by modifying your build process to have EMMA compile to a different location just for the purpose of reporting.
PATIENT: It hurts when I press here.
DOCTOR: Don't press there.
-- Chad
EMMA-instrumented bytecode has source line numbers adjusted so that the exception stack traces are preserved perfectly (and they are).
However, it does not have variable table offsets adjusted, because that info is only used by debuggers and decompilers. Neither of these use cases needs to be supported, right?
So, if you try to debug instrumented code you will see issues. An easy fix is not to do it.
Vlad.
This is causing a problem for the JBoss Transactions project. Our unit, integration and system use a Java agent called Byteman (see http://www.org.jboss.com/byteman) to rewrite bytecode at load time. Byteman injects highly specific faults into the product code in order to ensure that the code exercises the test scenario. It also injects trace code in order to allow verification of correct execution. We would like also to be able to instrument our tests for code coverage. However, the bad entries in the local variable table are interfering with the operation of the ASM bytecode manipulation library on which Byteman is based. It is throwing exceptions when it processes instrumented classes because it (quite legitimately) expects the extents for local var table entries to begin and end on an instruction boundary. Fixing ASM is not an option - it cannot make up values for the local var table entries.
Could you please include the patch supplied by mdlavin into an upcoming release?
Another thing: The supplied patch is not quite correct. Method new_Attribute_info of class Attribute_info needs to create a LocalVariableTableAttribute_info instance in two cases, for both a LVT attribute and an LVTT attribute. The latter are teh hack introduced in JDK 1.5 to introduce signatures as well as types for local variables. Without this code written using generics wil not transform correctly.
This requires adding the following two cases to the if else cascade in new_Attribute_info()
else if (ATTRIBUTE_LOCAL_VARIABLE_TABLE.equals (name))
{
return new LocalVariableTableAttribute_info (attribute_name_index, attribute_length, bytes);
}
else if (ATTRIBUTE_LOCAL_VARIABLE_TYPE_TABLE.equals (name))
{
return new LocalVariableTableAttribute_info (attribute_name_index, attribute_length, bytes);
}
ATTRIBUTE_LOCAL_VARIABLE_TYPE_TABLE needs to be defined in class Attribute_info as the string "LocalVariableTypeTable".
Note that LVT and LVTT entries both use the same format so one size fits all.
I have found another error in the patch supplied by mdlavin. The code which updates the extents for local variables currently increments the start and end positions by the relevant number of bytes when instrumentation code is placed before the start of a variable. This is normally correct. However, there is a special case. When the start location of the variable is zero then it should remain zero even when instrumentation code is placed at the beginning of the bytecode sequence. Start position zero indicates a method argument and method arguments remain in scope even if instrumentation code is added at the head of the method. This requires changing method visit (LocalVariableTableAttribute_info_info, Object) of class InstrVisitor which was added in the patch as follows
original code in patch
final int lineCount = attribute.size ();
for (int l = 0; l < lineCount; ++ l)
{
final LocalVariable_info var = attribute.get (l);
// TODO: make this faster using either table assist or the sorted array in 'sortedLines'
// adjust bytecode offset for line number mapping:
int end_pc = var.m_start_pc+var.m_length; // use m_start_pc before adjustment to get original end
int adjSegment = lowbound (m_methodJumpAdjOffsets, var.m_start_pc);
var.m_start_pc += m_methodJumpAdjValues ;
adjSegment = lowbound (m_methodJumpAdjOffsets, end_pc);
end_pc += m_methodJumpAdjValues ;
var.m_length = end_pc - var.m_start_pc;
}
return null;
replacement code
final int lineCount = attribute.size ();
for (int l = 0; l < lineCount; ++ l)
{
final LocalVariable_info var = attribute.get (l);
// TODO: make this faster using either table assist or the sorted array in 'sortedLines'
// adjust bytecode offset for line number mapping:
int end_pc = var.m_start_pc+var.m_length; // use m_start_pc before adjustment to get original end
int adjSegment;
if (var.m_start_pc != 0) {
adjSegment = lowbound (m_methodJumpAdjOffsets, var.m_start_pc);
var.m_start_pc += m_methodJumpAdjValues ;
}
adjSegment = lowbound (m_methodJumpAdjOffsets, end_pc);
end_pc += m_methodJumpAdjValues ;
var.m_length = end_pc - var.m_start_pc;
}
return null;
I have found one more bug which affects the patch supplied by mdlavin although the problem lies in the emma code, not in his patch code. The loop which adjusts local variable table entry start and size fields is not correctly updating the size of local variables whose extent reaches the final instruction of the original bytecode. It leaves these variables going out of scope at a point part way into the instrumentation code inserted before the final return.
When a local variable extends right to the end of the method body the local variable table entry start and size fields add together to produce an offset which lies 1 byte beyond the final return instruction at the end of the last block. The instrumentation process adds code before this return instruction which means that the original end point now precedes the return statement. However, the code in InstrVisitor does not add an adjustment table entry for this last block. Looking at the allocation statement for this array the size of the table is allocated using blockcount:
final int jumpAdjOffsets = new int ; // room for initial 0 + (blockCount - 1)
final int jumpAdjMap = new int ; // room for initial 0 + (blockCount - 1)
Clearly the code assumes that the last insertion cannot affect positions/sizes for attributes. This is an invalid assumption for local variable table entries which extend to the end of the method. They should still do so after the final instrumentation code has been added. This causes a problem when trying to feed the emma transformed code through an agent based on ASM because the instruction inserted where the original return was placed is an aload. This does not employ the same number of bytes as a return. This means that the end position in the transformed bytecode is no longer aligned on an instruction boundary.
This can be fixed by including the offset for the final inserted bytecode into the adjustment table ie.. allocate room for one more block and change the < comparison to a <= in the branches where a decision is made to update the adjustment table.
This fix should not affect any other offset computations since it only affects cases where a position off the the end of a block is used to mark a region end. The only other case (I am aware of) which might be subject to the same miscalculation is exception table end points which also calculate the end of the excepting block using start + size. The end point arrived at is one byte beyond the end of the bytecode segment in which the exception handler is valid. I think these end points will never computed incorrectly though because the excepting block will terminate the bytecode - it is always followed by a handler block. Even if the handler code is only a simple return statement the end point of the excepting block will be at the return instruction rather than beyond it.
One more correction. The fix required to enable local variable table offset to work is to include an extra entry in the table which maps any position just beyond the end of the old bytecode array to a comparable position just beyond the end of the new array. So, this requires increasing the size of jumpAdjOffsets and jumpAdjMap by one and then adding the following lines at the end of the loop which initialises them
int oldBytecodeEnd = attribute.getCodeSize();
int newBytecodeEnd = emitctx.m_out.size ();
jumpAdjOffsets = oldBytecodeEnd;
jumpAdjMap = newBytecodeEnd - oldBytecodeEnd;
This ensures that local variables whose extent reaches to the end of the original bytecode will maintain the smae property with respect to the new bytecode array.
Ok, this patch has now been incorporated into a fix for JBoss Transactions issue JBTM-682 (https://jira.jboss.org/jira/browse/JBTM-682). The JIRA includes a diffs file detailing all changes made by me or mdlavin relative to the 2.0.5312 source release.
I came across another problem when feeding emma instrumented bytecode through ASM. The ASM code does not update StackMapTable attributes which detail the layout of local var and stack frames at various points in the code (these are new to JDK6). Passing unadjusted versions of these attributes through to ASM for further bytecode transformation causes muchas problemas.
I have patched the emma code a second time so it correctly remaps offsets contained in StackMapTable entries. A diffs file of both patches against the base 2.0.5312 release and a link to the patched code in the JBossTS repo wrkspace are available via JBoss Transactions issue JBTM-682 (https://jira.jboss.org/jira/browse/JBTM-682).
Hi,
I am new to Emma, Can you plz tell me how to instrument jar files with emma and run test cases against the instrumented jar file to calculate the coverage?
Thanks
Cheers, ALi