#14 Coverage and Meck

-ANY-
closed-fixed
None
5
2012-03-16
2012-03-11
No

Using Meck in one of the eunit tests make the coverage goal fails.

Plugin version 2.2.0-SNAPSHOT

Updating the cover2 module with the latest OTP version fix the problem.

Discussion

  • Hi Nicolas,

    thanks for the report, it's always nice to hear from people that are interested in the plugin. I just took a look at the diff between our (patched) cover2.erl and the one from the master, and I can imagine that we are a bit outdated.

    I will address this and make sure to get up-to-date with the latest changes. It would be great if you could help me out and provide a small test with Meck mocks, that fails, so I can put it in our integration test suite for the plugin.

    We are currently close to our 2.2.0 release (within this week hopefully) but I see no problem with getting this sorted out before that.

    With kind regards
    /Olle

     
    • assigned_to: nobody --> olle-t
     

  • Anonymous
    2012-03-14

    Hi Olle,

    How can I send you the test project ?

    And which patched had you make on cover2.erl ?
    To make it work, I just used the version from OTP and changed coveragereportscript.erl to call the function cover2:compile_beam_directory/1 and not cover2:compile_beam_directory/2.
    ( I also re-ordered the case clauses in order to generate the report even if eunit failed )

    Thanks,
    Nicolas

     
  • The patch we made was for more control and finer granularity when doing beam-file cover compilation, adding `Options' to those clauses as well. I'm not sure it will ever end up in the main Erlang/OTP branch (currently the test setup for Erlang/OTP is simply too brittle, seems it only runs on some developer boxes/Linux distros or with a whole lot of hackery, so I'm not in the mood to commit changes I can't test).

    There are no problems in merging the changes from the master into our patched module though, and that's at least a good solution for our next 2.2.0 release.

    Anyway, you could post me your test-case either over my profile here at sourceforge, or you can find me at GitHub, LinkedIn or though my personal web site.

    http://sourceforge.net/users/olle-t
    https://github.com/olle
    http://www.linkedin.com/in/olletornstrom
    http://studiomediatech.com/

    Cheers
    /O

     
    • status: open --> open-fixed
     
  • I've managed to add a very simple integration test, that verifies coverage with tests that use the Meck mocking library. Adding this to the current snapshot, planned for release 2.2.0.

    Please try it out using the plugin dependency version 2.2.0-SNAPSHOT, I just released it to the Sonatype OSS snapshot repository. I'm hoping for a positive response so I can close this bug.

    Regards
    /Olle

     
  • Hi Nicolas, just a note, I saw your comment to Tobias on the other issue - If you want to build the plugin and run the integration tests, you need to have access to the Sonatype OSS repository (https://oss.sonatype.org/).

    I've built the Meck library as a `erlang-std' dependency, and the integration test of course requires it to work.

    With kind regards
    /Olle

     
  • Hello again. Had some time to back-track the problem with coverage and Meck and I'm quite shocked about my findings. I don't know if you know Adam Lindberg personally (was thinking since you guys both work for/with/close to Erlang Solutions) but what he's doing in meck_cover.erl and meck_mod.erl surely makes it tricky.

    He basically want's to get hold of some none-exported functions in cover.erl, and goes about it by modifying the running BEAM, replacing it with one with some extra exports. This way of hacking the running VM seems to mix up the coverage state-process with it's surrounding BEAM module; mine is supposed to be a separate interface called cover2.erl.

    The inner state loop in cover2.erl has an extended clause for `compile_beam' where `Options' are expected, and after the meck-magic reload trick, it seems to receive messages from a module API that no longer sends the extra parameter - crash.

    I think what would be best for all parties is to see if a public patch for cover.erl could be accepted, that allows for BEAM directory compilation with user options (my patch) and then modify the meck_cover.erl accordingly.

    Of course, in my opinion, it would be preferrable if Meck would also be able to work with cover.erl over a public API.

    I'm patching our plugin again, to use a cover2.erl that also contains the necessary state-loop-clause, so back to basics. Oh, yes, the reason for my patch from the beginning was that BEAM cover compilation with user options is a way better method to ensure that i can use `export_all' and also get coverage for non exported functions, without having to compile from source once again.

    Regards
    /O

     
  • ...ah, found it - there's of course a registered state server process in cover.erl, it is called `cover_server' - so that's why the Meck BEAM hijack/reload attempts to post it. Oh well.

     

  • Anonymous
    2012-03-16

    Hi,

    I try with a fresh checkout of your repository (without running integration-tests) and it seems to work.

    I don't know what my ex-colleague Adam wants to achieve playing with the cover data file.
    I guess it's when the pass through option is used to mock some of the functions. In this case "we wants" to capture the fact that the other functions can be covered.

     
  • Ok, but sounds great then - my problem solved, your problem solved. Perhaps I'll find a way to run the OTP test-suite for cover, and post a patch that suggests a homogenous API for cover_*, to allow for user options.

    Marking this bug as fixed/closed. Please let us know if you run in to any other issues, and feel free to contact us if you have any other feedback, thoughts or ideas about the Maven Erlang Plugin.

    With kind regards
    /Olle

     
    • status: open-fixed --> closed-fixed