On Mar 27, 2014, at 5:49 AM, Peter Oberparleiter <oberpar@linux.vnet.ibm.com> wrote:

On 26.03.2014 18:18, Jonah Petri wrote:
Xcode 5.1 is using an LLVM based compiler which apparently
generates coverage data files which are similar but not fully
compatible with the GCC ones. If maintaining coverage file format
compatibility with GCC would be a goal of the Xcode project, this
could be filed as a bug report against Xcode. Since lcov was
designed to work with GCC/GCOV, I don't see much that can be done
in lcov to support Xcode.


Regards, Peter Oberparleiter

To those who mind find this thread in the future, it should be noted
that you can fall back on using gcc’s gcov, which is available as
"gcov-4.2”.  So, you can run lcov as “lcov --gcov-tool `xcrun -f
gcov-4.2` …"

llvm-cov is intended to be a reimplementation of gcov’s major
features, and at first glance it appears to be compatible with lcov.

lcov relies on the following aspects of gcov/gcc to function properly:
- Command line options
- Command output
- .gcno file format

I don't have llvm-cov installed anywhere but would like to get a better
understanding on how similar the tools are. Could you provide the following:
- Output of llvm-cov —help

Thanks very much for your assistance!

One thing I should clarify: where I mentioned “llvm-cov” above, I should have written “llvm’s gcov”.  Xcode 5.1 includes llvm’s reimplementation of gcov (and calls it “gcov”), moving the gcc version to “gcov-4.2”.   There is also a tool called “llvm-cov” which appears to want to generate reports from gcda/gcno files, but I can’t get it to do anything but fail due to "invalid gcno files” with Xcode 5.1’s version of clang.  The llvm-cov tool isn’t relevant to this conversation, and I apologize for the confusion.

The improvement which I’m hoping we can make for lcov is compatibility with llvm’s gcov implementation.

So, to that end:

$ gcov —help
OVERVIEW: LLVM code coverage tool

USAGE: gcov [options] SOURCEFILE

OPTIONS:

General options:

  -a                         - Display all basic blocks
  -b                         - Display branch probabilities
  -c                         - Display branch counts instead of percentages (requires -b)
  -f                         - Show coverage for each function
  -help                      - Display available options (-help-hidden for more)
  -help-list                 - Display list of available options (-help-list-hidden for more)
  -o=<DIR|FILE>              - Find objects in DIR or based on FILE's path
  -object-directory=<string> - 
  -object-file=<string>      - 
  -p                         - Preserve path components
  -u                         - Display unconditional branch info (requires -b)
  -version                   - Display the version of this program

Internal and debugging options:

  -dump                      - Dump the gcov file to stderr
  -gcda=<string>             - Override inferred gcda file
  -gcno=<string>             - Override inferred gcno file

This seems compatible enough with gcc’s gcov.  One exception is that llvm’s gcov does not recognize the -v switch.  It does output version information for the “—version” and “-version” switches.  I see that lcov (specifically geninfo) uses "gcov -v” to probe its version information, and I suggest switching that to “gcov —version”.  Alternatively, for maximum backwards compatibility safety, the “gcov -v” test could remain in place, and a supplemental test for llvm’s gcov could be added to get_gcov_version() in the case where the regex match fails.  This would only be required if some older (yet supported) versions of gcov did not support the —version long option.

I note that get_gcov_capabilities() relies on the long options being listed in gcov --help.  I’ve worked around that limitation in my patch by explicitly enumerating the capabilities of LLVM gcov.  This is the most brittle part of the patch, but I don’t see a better way around it.

- .gcov file for a small sample program (2 functions, one if-statement,
 ideally with branch and function coverage switches enabled)

$ cat test.c                
#include <stdio.h>

void test( int value ) {
    if (value) {
        printf("true\n");
    } else {
        printf("false\n");
    }
}

int main(void) {
    test(1);
    test(0);
}

$ clang -ftest-coverage -fprofile-arcs test.c
$ ./a.out
true
false
$ gcov -b -c -a -p test.c  # llvm’s gcov
File 'test.c'
Lines executed:100.00% of 8
Branches executed:100.00% of 2
Taken at least once:100.00% of 2
No calls
test.c:creating 'test.c.gcov'

$ cat test.c.gcov
        -:    0:Source:test.c
        -:    0:Graph:test.gcno
        -:    0:Data:test.gcda
        -:    0:Runs:2
        -:    0:Programs:1
        -:    1:#include <stdio.h>
        -:    2:
function test called 4 returned 100% blocks executed 100%
        -:    3:void test( int value ) {
        4:    4:    if (value) {
        4:    4-block  0
branch  0 taken 2
branch  1 taken 2
        2:    5:        printf("true\n");
        2:    6:    } else {
        2:    6-block  0
        2:    7:        printf("false\n");
        2:    7-block  0
        -:    8:    }
        4:    9:}
        4:    9-block  0
        -:   10:
function main called 2 returned 100% blocks executed 100%
        -:   11:int main(void) {
        2:   12:    test(1);
        2:   13:    test(0);
        2:   14:}
        2:   14-block  0


- Output of running lcov --capture --initial on a project compiled with
 llvm (--initial will try to read and parse the .gcno files) - lcov
 should report errors when it cannot parse .gcno files

I tested this on a medium-sized project, and stdout is the same for running lcov against llvm’s gcov and gcc’s gcov 4.2.1 (except for the detected version of gcov).  Encouragingly, the info files generated are also byte for byte identical.


The error comes from lcov’s inability to detect llvm-cov’s version
number, thus the change recommended in the previously noted
stackoverflow answer is much of what’s needed (but not quite all).
If it’s desired, I would happily submit a signed patch which allows
llvm-cov to be used by lcov without modifications.

Such a patch would generate a certain level of expectation regarding
lcov's compatibility with llvm-cov. Assuming that the information
requested earlier confirms a sufficient level of compatibility, I'm open
for such a patch.


Great.  I have a patch prepared.  If you’re interested in taking it after reviewing the above, I’ll gladly submit it for your comments.

Jonah