Re: [avr-llvm-devel] Regression tests

[Borja F. <bor...@gm...>]

This is what I'm getting:

  Expected Passes    : 9914
  Expected Failures  : 66
  Unsupported Tests  : 569
  Unexpected Failures: 11

I've seen in that file you attached that you're getting errors because of
the target triple, errors like:

error: unable to create target: 'No available targets are compatible
with this triple, see -version for the available targets.'

or

error auto-selecting target for module 'No available targets are
compatible with this triple, see -version for the available targets

so I guess that's the reason, so what is your target triple? do clang -v to
get it, mine is i386-pc-linux-gnu


2012/5/10 Borja Ferrer <bor...@gm...>

> Interesting, I'll check it out tomorrow to see if I can find some reason
> behind this, because we're getting very different results here.
>
>
> 2012/5/9 Nicklas Bo Jensen <nbj...@gm...>
>
>> Sorry for my slow response.Got i working again now, thanks.
>>
>> Building with optimization disabled and assertions turned on i get:
>>
>> Avr llvm+clang:
>>   Expected Passes    : 6978
>>   Expected Failures  : 39
>>   Unsupported Tests  : 3281
>>   Unexpected Failures: 279
>>
>> In the 279 unexpected failures for example 143 of them are
>> test/CodeGen/Generic. I'm not really sure how these tests are supposed to
>> work? None of them seems to be using FileCheck.
>>
>> However building for msp430 seems to be giving me similar results,
>> indicating that it is not a avr-llvm specific problem. Please see the
>> attached output "make check-all" from avr-llvm.
>>
>>
>> On Fri, May 4, 2012 at 11:30 PM, Borja Ferrer <bor...@gm...>wrote:
>>
>>> I'm asking because last week you said that you were getting +200 fails,
>>> and since I'm getting 11 I wanted to know what's going on.
>>>
>>> faluco is me btw, I updated a patch today because I got a conflict when
>>> updating my clang repo. These variables are defined in
>>> DiagnosticSemaKinds.td so check that you have them, they come from the
>>> flash.diff patch.
>>>
>>>
>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>
>>>> The new tests should pass. I cannot check other tests now, I'm having
>>>> some issues after applying the newest patches that came in today by faluco:
>>>>
>>>> /home/nicklas/install/avr-llvm/llvm/tools/clang/lib/Sema/SemaDecl.cpp:4374:38:
>>>> error:
>>>>       no member named 'err_flash_variable_requires_const' in namespace
>>>>       'clang::diag'
>>>>     Diag(NewVD->getLocation(), diag::err_flash_variable_requires_const);
>>>>                                ~~~~~~^
>>>> /home/nicklas/install/avr-llvm/llvm/tools/clang/lib/Sema/SemaDecl.cpp:4383:38:
>>>> error:
>>>>       no member named 'err_flash_pointer_requires_const' in namespace
>>>>       'clang::diag'
>>>>     Diag(NewVD->getLocation(), diag::err_flash_pointer_requires_const);
>>>>                                ~~~~~~^
>>>> /home/nicklas/install/avr-llvm/llvm/tools/clang/lib/Sema/SemaDecl.cpp:7208:25:
>>>> error:
>>>>       no member named 'err_flash_pointer_requires_const' in namespace
>>>>       'clang::diag'
>>>>     Diag(NameLoc, diag::err_flash_pointer_requires_const);
>>>>                   ~~~~~~^
>>>> Have i done something wrong?
>>>>
>>>> Thanks,
>>>> Nicklas
>>>>
>>>> On Fri, May 4, 2012 at 8:50 PM, Borja Ferrer <bor...@gm...>wrote:
>>>>
>>>>> How many tests failures are you getting now?
>>>>> I'm getting 11, 10 from clang due to address space stuff (you'll need
>>>>> to patch clang to get those) and 1 from llvm.
>>>>>
>>>>>
>>>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>>>
>>>>>> Ah, perfect.
>>>>>>
>>>>>> I would actually guess that you need to run "make check-all" before
>>>>>> being able to test new tests individually. Sorry :)
>>>>>>
>>>>>> On Fri, May 4, 2012 at 8:17 PM, Borja Ferrer <bor...@gm...>wrote:
>>>>>>
>>>>>>> Nevermind, got it.
>>>>>>>
>>>>>>>
>>>>>>> 2012/5/4 Borja Ferrer <bor...@gm...>
>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> One thing, i dont have llvm-lit in that dir, am I supposed to build
>>>>>>>> llvm with an addtional param or something in order to get it?
>>>>>>>>
>>>>>>>>
>>>>>>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>>>>>>
>>>>>>>>> Ok, I've committed now. To only run the AVR specific tests run
>>>>>>>>> something like:
>>>>>>>>>
>>>>>>>>> ./build/Debug+Asserts/bin/llvm-lit llvm/test/CodeGen/AVR/
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Fri, May 4, 2012 at 1:47 PM, Borja Ferrer <
>>>>>>>>> bor...@gm...> wrote:
>>>>>>>>>
>>>>>>>>>> Oh a small one I've just noticed, leave only 1 newline between
>>>>>>>>>> tests not 2.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2012/5/4 Borja Ferrer <bor...@gm...>
>>>>>>>>>>
>>>>>>>>>>> Ok they look great now, I don't have any other objections.
>>>>>>>>>>> Please commit this test to SVN, post your SF username here so that Eric or
>>>>>>>>>>> John can give you commit permissions. Once it gets commited I'll check if I
>>>>>>>>>>> need to add any pattern specific tests to it, these tests are corner cases
>>>>>>>>>>> of some instructions.
>>>>>>>>>>> Before commiting it, remove the testcases dir with all the files
>>>>>>>>>>> there, and create a new one called test/CodeGen/AVR, and place your file
>>>>>>>>>>> there, just to keep the same structure like in llvm's repo.
>>>>>>>>>>>
>>>>>>>>>>> About the rest of tests to do, from that list remove mul and
>>>>>>>>>>> division for now, mul isn't yet implemented and division can come later.
>>>>>>>>>>> Sub tests should be very similar to add, then add binary ops (or, and,
>>>>>>>>>>> xor). Add calling convention tests (argument passing through regs and
>>>>>>>>>>> stack, return values for each value type, calls). Memory operations, etc...
>>>>>>>>>>>
>>>>>>>>>>> There are quite a few tests to add, but for now do the sub and
>>>>>>>>>>> binary op tests and we'll discuss the others by then.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> 1) Sure
>>>>>>>>>>>> 2) Yes, I've followed the msp430 backend that does include it.
>>>>>>>>>>>> I've removed it now.
>>>>>>>>>>>> 3) Again, I've moved the CHECK lines, but it differs from
>>>>>>>>>>>> backend to backend.
>>>>>>>>>>>> 4) Done
>>>>>>>>>>>> 5) Done
>>>>>>>>>>>> 6) Done
>>>>>>>>>>>>
>>>>>>>>>>>> I will continue working other tests, which do we need?
>>>>>>>>>>>> Something like(and please add/comment):
>>>>>>>>>>>>
>>>>>>>>>>>> -Sub
>>>>>>>>>>>> -division(Only working for multiples of 2?)
>>>>>>>>>>>> -mult
>>>>>>>>>>>> -Return argument(In a few variants)
>>>>>>>>>>>> -and
>>>>>>>>>>>> -or
>>>>>>>>>>>> -xor
>>>>>>>>>>>> -increment
>>>>>>>>>>>> -branching?
>>>>>>>>>>>>
>>>>>>>>>>>> Any ideas?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks,
>>>>>>>>>>>> Nicklas
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, May 2, 2012 at 12:29 AM, Borja Ferrer <
>>>>>>>>>>>> bor...@gm...> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Obviously in the last line I meant adiw or subi/sbci pair, add
>>>>>>>>>>>>> for immediates doesn't exist for imms greater than 63
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2012/5/2 Borja Ferrer <bor...@gm...>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> They look good, very nice :)
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Some small details and nitpicks:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 1) I think it's not necessary to add the reg_reg_imm variant
>>>>>>>>>>>>>> because you covered it with the other two tests.
>>>>>>>>>>>>>> 2) Do the tests work if you remove the target triple line?
>>>>>>>>>>>>>> I've seen other backends don't include it.
>>>>>>>>>>>>>> 3) Move the CHECK lines after the function prototype like
>>>>>>>>>>>>>> other backends do.
>>>>>>>>>>>>>> 4) In every CHECK line, can you remove the tabs between the
>>>>>>>>>>>>>> instr mnemonic and the first operand and add a single space? (I'm unsure
>>>>>>>>>>>>>> about this because i dont know if CHECK eats spaces). Also for the future,
>>>>>>>>>>>>>> the llvm coding standards says to config your editor to replace tabs with
>>>>>>>>>>>>>> spaces, so it's a good moment time to do it.
>>>>>>>>>>>>>> 5) Oh and the most important one, please add this in only one
>>>>>>>>>>>>>> file, add.ll or something like that, so we can keep this convention in the
>>>>>>>>>>>>>> future, otherwise we'll end having too many test files.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Something that should be covered is, when doing 16 bit
>>>>>>>>>>>>>> additions we can use adiw or add depending on the imm value, can you cover
>>>>>>>>>>>>>> this aswell?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2012/5/1 John Myers <ato...@gm...>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sun, Apr 29, 2012 at 10:46 AM, Nicklas Bo Jensen <
>>>>>>>>>>>>>>> nbj...@gm...> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> I have successfully been able to compile your testcases
>>>>>>>>>>>>>>>> (/avr-llvm/testcases/*.ll) to something looking like valid avr assembler.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> How should I test/simulate the assembler? I get errors when
>>>>>>>>>>>>>>>> trying to simulate the generated assembler in AVRStudio. Perhaps they use a
>>>>>>>>>>>>>>>> different assembler?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> avr-llvm produces GNU assembler syntax, which is different
>>>>>>>>>>>>>>> then the Atmel assembler syntax.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Eventually we could support multiple asm syntax's like the
>>>>>>>>>>>>>>> X86 target does.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>