Re: [Jython-dev] some microbenchmarks results

[Charlie G. <cha...@gm...>]

On Jan 1, 2008 9:45 AM, Frank Wierzbicki <fwi...@gm...> wrote:
> Hi Samuele -- thanks for these results!  They really are not too bad
> at all :) -- I bet someone around here will try it on the trunk soon,
> there have been some thoughts about performance lately.

Indeed, I ran pybench for the first time a few weeks ago and fixed
some of the blatant problems on trunk.  This looks like this is
uncovering another set of rough patches.

Even as things stand, if I stack the deck completely in Jython's favor
and run trunk with java 6 -server and add 10 warmup runs to microbench
to let hotspot kick in, things aren't awful at all:

exe: python2.5
exe: jythontrunk_java6_server
 0.23x slower on test_exception.test_try_except_finally()
 0.32x slower on test_dict.test_dict_instance_setattr_instance_dict()
 0.33x slower on test_exception.test_try_except()
 0.34x slower on test_dict.test_dict_instance_setnewattr_instance_dict()
 0.34x slower on test_exception.test_try_except_else()
 0.36x slower on test_list.test_list_setitem()
 0.36x slower on test_count1.test_loop_other_count()
 0.40x slower on test_count1.test_count_in_dict()
 0.40x slower on test_dict.test_dict_instance_getattr_instance_dict()
 0.48x slower on test_dict.test_dict_raw_range()
 0.51x slower on test_count1.test_loop()
 0.58x slower on test_count1.test_call_nested_function_other_count()
 0.63x slower on test_count1.test_loop_unrolled()
 0.67x slower on test_exception.test_reraise()
 0.67x slower on test_exception.test_except_multiple_user_exception()
 0.67x slower on test_exception.test_except_multiple_builtin_exception()
 0.69x slower on test_count1.test_call_nested_function()
 0.70x slower on test_count1.test_call_method_of_old_style_class()
 0.70x slower on test_count1.test_call_nested_function_many_args()
 0.72x slower on test_bltn.test_call_sin()
 0.73x slower on test_bltn.test_call_fabs()
 0.75x slower on test_unicode.test_bad_case_python2_5()
 0.77x slower on test_unicode.test_find_pattern2()
 0.83x slower on test_formatting.test_repr_formatting()
 0.86x slower on test_count1.test_count_in_global()
 0.86x slower on test_count1.test_count_in_global2()
 0.86x slower on test_unicode.test_find_pattern1()
 0.86x slower on test_list.test_list_getitem()
 0.87x slower on test_count1.test_call_function_with_arguments_in_cellvars()
 0.88x slower on test_count1.test_call_function_without_arguments_in_cellvars()
 0.93x slower on test_exception.test_except_specific_builtin_exception()
 1.01x slower on test_count1.test_call_function()
 1.03x slower on test_create1.test_simple_loop()
 1.03x slower on test_exception.test_except_specific_user_exception()
 1.04x slower on test_dict.test_dict_getitem()
 1.07x slower on test_formatting.test_dict_formatting()
 1.07x slower on test_list.test_list_append()
 1.10x slower on test_create1.test_simple_loop_with_new_style_class_new()
 1.12x slower on test_bltn.test_isinstance1()
 1.16x slower on test_unicode.test_find_pattern4()
 1.18x slower on test_bltn.test_isinstance2()
 1.21x slower on test_create1.test_simple_loop_with_old_style_class_creation()
 1.33x slower on test_dict.test_dict_class_dict_getmethod()
 1.36x slower on test_exception.test_raise_builtin_exception()
 1.40x slower on test_exception.test_raise_user_exception()
 1.45x slower on test_formatting.test_simple_formatting()
 1.49x slower on test_dict.test_dict_creation_mode1()
 1.50x slower on test_list.test_list_extend()
 1.50x slower on test_formatting.test_format_long()
 1.54x slower on test_bltn.test_isinstance3()
 1.74x slower on test_count1.test_call_method_of_new_style_class()
 1.77x slower on test_dict.test_dict_creation_mode2()
 1.77x slower on test_unicode.test_find_pattern8()
 1.78x slower on test_dict.test_dict_setitem1()
 1.87x slower on test_count1.test_count_in_slot()
 1.99x slower on test_count1.test_count_in_attr()
 2.08x slower on test_dict.test_dict_setitem2()
 2.16x slower on test_create1.test_simple_loop_with_new_style_class_creation()
 2.17x slower on test_formatting.test_format_unicode()
 2.42x slower on test_exception.test_instantiate_user_exception()
 2.50x slower on test_unicode.test_find_pattern16()
 2.53x slower on test_formatting.test_number_formatting()
 2.83x slower on test_dict.test_dict_creation_mode3()
 4.33x slower on test_exception.test_instantiate_builtin_exception()
 5.29x slower on test_unicode.test_count_worstcase()
 5.49x slower on test_unicode.test_find_worstcase()
 6.16x slower on test_list.test_list_slice()

I don't know if this is an unfair comparison or not.  I think Java 7
will essentially have -server on by default, and any long running
application will give hotspot enough time to do its magic, so I don't
think it's too far off the mark.

Charlie