#25 Include resource usage in regression tests

closed
Maarten Brock
None
5
2005-12-07
2003-03-09
No

It would be fine if the regression tests would
also report resource usage like:

Summary for 'mcs51': 0 failures, 2479 tests, 480 test cases,
xxx cpu cycles, yyy byte code

This would help to see advances / avoid regressions
in code generation.

One could f.e. notice if a peephole does not
apply any more and would have to be adapted.
(Stuff like mov a,#0x00 versus clr a, ordering
of registers, label not there anymore and the like.)

Furthermore there finally is an objective argument in
the MyCpuIsBiggerThanYours debate;)

Frieder

Discussion

  • Maarten Brock
    Maarten Brock
    2005-12-07

    Logged In: YES
    user_id=888171

    Implemented in SDCC 2.5.4 #1182

     
  • Maarten Brock
    Maarten Brock
    2005-12-07

    • assigned_to: nobody --> maartenbrock
    • status: open --> closed
     
  • diagram

     
    Attachments
  • Logged In: YES
    user_id=589052

    Hi Maarten,

    thanks adding this!

    To give a little feedback I append a diagram which shows the
    code size and the number of tics over the SDCC version:)

    Data was generated for the mcs51 small model for SDCC
    version 2.4.0, 2.5.0, 2.5.4 when running over the test files
    of 2.4.0.

    The diagram was generated with gnuplot and is appended to
    this feature request.

    Greetings
    Frieder

     
  • diagram, data and gnuplot file

     
    Attachments
  • Bernhard Held
    Bernhard Held
    2005-12-10

    Logged In: YES
    user_id=203539

    This is a very, very valuable extension to the regression
    tests!

    The improvement of sdcc over time is very satisfying,
    thanks for pointing this out, Frieder!

    Thanks Maarten!

     
  • Borut Ražem
    Borut Ražem
    2005-12-11

    Logged In: YES
    user_id=568035

    Nice tool, Frieder! What do you think about to put the graph
    on the SDCC Release WIKI?

    Borut

     
  • Logged In: YES
    user_id=589052

    > What do you think about to put the graph on the SDCC
    Release WIKI?

    Good idea! I want to include the data for the minor versions
    .1, .2, .3 and .4 during next week. Also I'd want to put
    some more comments into the bench.gnuplot file. Meanwhile do
    whatever you want to do with them!)
    (Optimum from my point of view would probably be to include
    a script into the build system - unfortunately this is
    beyond my scope)