Re: [avr-llvm-devel] Regression tests

[Borja F. <bor...@gm...>]

Interesting, I'll check it out tomorrow to see if I can find some reason
behind this, because we're getting very different results here.

2012/5/9 Nicklas Bo Jensen <nbj...@gm...>

> Sorry for my slow response.Got i working again now, thanks.
>
> Building with optimization disabled and assertions turned on i get:
>
> Avr llvm+clang:
>   Expected Passes    : 6978
>   Expected Failures  : 39
>   Unsupported Tests  : 3281
>   Unexpected Failures: 279
>
> In the 279 unexpected failures for example 143 of them are
> test/CodeGen/Generic. I'm not really sure how these tests are supposed to
> work? None of them seems to be using FileCheck.
>
> However building for msp430 seems to be giving me similar results,
> indicating that it is not a avr-llvm specific problem. Please see the
> attached output "make check-all" from avr-llvm.
>
>
> On Fri, May 4, 2012 at 11:30 PM, Borja Ferrer <bor...@gm...>wrote:
>
>> I'm asking because last week you said that you were getting +200 fails,
>> and since I'm getting 11 I wanted to know what's going on.
>>
>> faluco is me btw, I updated a patch today because I got a conflict when
>> updating my clang repo. These variables are defined in
>> DiagnosticSemaKinds.td so check that you have them, they come from the
>> flash.diff patch.
>>
>>
>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>
>>> The new tests should pass. I cannot check other tests now, I'm having
>>> some issues after applying the newest patches that came in today by faluco:
>>>
>>> /home/nicklas/install/avr-llvm/llvm/tools/clang/lib/Sema/SemaDecl.cpp:4374:38:
>>> error:
>>>       no member named 'err_flash_variable_requires_const' in namespace
>>>       'clang::diag'
>>>     Diag(NewVD->getLocation(), diag::err_flash_variable_requires_const);
>>>                                ~~~~~~^
>>> /home/nicklas/install/avr-llvm/llvm/tools/clang/lib/Sema/SemaDecl.cpp:4383:38:
>>> error:
>>>       no member named 'err_flash_pointer_requires_const' in namespace
>>>       'clang::diag'
>>>     Diag(NewVD->getLocation(), diag::err_flash_pointer_requires_const);
>>>                                ~~~~~~^
>>> /home/nicklas/install/avr-llvm/llvm/tools/clang/lib/Sema/SemaDecl.cpp:7208:25:
>>> error:
>>>       no member named 'err_flash_pointer_requires_const' in namespace
>>>       'clang::diag'
>>>     Diag(NameLoc, diag::err_flash_pointer_requires_const);
>>>                   ~~~~~~^
>>> Have i done something wrong?
>>>
>>> Thanks,
>>> Nicklas
>>>
>>> On Fri, May 4, 2012 at 8:50 PM, Borja Ferrer <bor...@gm...>wrote:
>>>
>>>> How many tests failures are you getting now?
>>>> I'm getting 11, 10 from clang due to address space stuff (you'll need
>>>> to patch clang to get those) and 1 from llvm.
>>>>
>>>>
>>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>>
>>>>> Ah, perfect.
>>>>>
>>>>> I would actually guess that you need to run "make check-all" before
>>>>> being able to test new tests individually. Sorry :)
>>>>>
>>>>> On Fri, May 4, 2012 at 8:17 PM, Borja Ferrer <bor...@gm...>wrote:
>>>>>
>>>>>> Nevermind, got it.
>>>>>>
>>>>>>
>>>>>> 2012/5/4 Borja Ferrer <bor...@gm...>
>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> One thing, i dont have llvm-lit in that dir, am I supposed to build
>>>>>>> llvm with an addtional param or something in order to get it?
>>>>>>>
>>>>>>>
>>>>>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>>>>>
>>>>>>>> Ok, I've committed now. To only run the AVR specific tests run
>>>>>>>> something like:
>>>>>>>>
>>>>>>>> ./build/Debug+Asserts/bin/llvm-lit llvm/test/CodeGen/AVR/
>>>>>>>>
>>>>>>>>
>>>>>>>> On Fri, May 4, 2012 at 1:47 PM, Borja Ferrer <bor...@gm...
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>> Oh a small one I've just noticed, leave only 1 newline between
>>>>>>>>> tests not 2.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2012/5/4 Borja Ferrer <bor...@gm...>
>>>>>>>>>
>>>>>>>>>> Ok they look great now, I don't have any other objections. Please
>>>>>>>>>> commit this test to SVN, post your SF username here so that Eric or John
>>>>>>>>>> can give you commit permissions. Once it gets commited I'll check if I need
>>>>>>>>>> to add any pattern specific tests to it, these tests are corner cases of
>>>>>>>>>> some instructions.
>>>>>>>>>> Before commiting it, remove the testcases dir with all the files
>>>>>>>>>> there, and create a new one called test/CodeGen/AVR, and place your file
>>>>>>>>>> there, just to keep the same structure like in llvm's repo.
>>>>>>>>>>
>>>>>>>>>> About the rest of tests to do, from that list remove mul and
>>>>>>>>>> division for now, mul isn't yet implemented and division can come later.
>>>>>>>>>> Sub tests should be very similar to add, then add binary ops (or, and,
>>>>>>>>>> xor). Add calling convention tests (argument passing through regs and
>>>>>>>>>> stack, return values for each value type, calls). Memory operations, etc...
>>>>>>>>>>
>>>>>>>>>> There are quite a few tests to add, but for now do the sub and
>>>>>>>>>> binary op tests and we'll discuss the others by then.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2012/5/4 Nicklas Bo Jensen <nbj...@gm...>
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> 1) Sure
>>>>>>>>>>> 2) Yes, I've followed the msp430 backend that does include it.
>>>>>>>>>>> I've removed it now.
>>>>>>>>>>> 3) Again, I've moved the CHECK lines, but it differs from
>>>>>>>>>>> backend to backend.
>>>>>>>>>>> 4) Done
>>>>>>>>>>> 5) Done
>>>>>>>>>>> 6) Done
>>>>>>>>>>>
>>>>>>>>>>> I will continue working other tests, which do we need? Something
>>>>>>>>>>> like(and please add/comment):
>>>>>>>>>>>
>>>>>>>>>>> -Sub
>>>>>>>>>>> -division(Only working for multiples of 2?)
>>>>>>>>>>> -mult
>>>>>>>>>>> -Return argument(In a few variants)
>>>>>>>>>>> -and
>>>>>>>>>>> -or
>>>>>>>>>>> -xor
>>>>>>>>>>> -increment
>>>>>>>>>>> -branching?
>>>>>>>>>>>
>>>>>>>>>>> Any ideas?
>>>>>>>>>>>
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Nicklas
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, May 2, 2012 at 12:29 AM, Borja Ferrer <
>>>>>>>>>>> bor...@gm...> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Obviously in the last line I meant adiw or subi/sbci pair, add
>>>>>>>>>>>> for immediates doesn't exist for imms greater than 63
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> 2012/5/2 Borja Ferrer <bor...@gm...>
>>>>>>>>>>>>
>>>>>>>>>>>>> They look good, very nice :)
>>>>>>>>>>>>>
>>>>>>>>>>>>> Some small details and nitpicks:
>>>>>>>>>>>>>
>>>>>>>>>>>>> 1) I think it's not necessary to add the reg_reg_imm variant
>>>>>>>>>>>>> because you covered it with the other two tests.
>>>>>>>>>>>>> 2) Do the tests work if you remove the target triple line?
>>>>>>>>>>>>> I've seen other backends don't include it.
>>>>>>>>>>>>> 3) Move the CHECK lines after the function prototype like
>>>>>>>>>>>>> other backends do.
>>>>>>>>>>>>> 4) In every CHECK line, can you remove the tabs between the
>>>>>>>>>>>>> instr mnemonic and the first operand and add a single space? (I'm unsure
>>>>>>>>>>>>> about this because i dont know if CHECK eats spaces). Also for the future,
>>>>>>>>>>>>> the llvm coding standards says to config your editor to replace tabs with
>>>>>>>>>>>>> spaces, so it's a good moment time to do it.
>>>>>>>>>>>>> 5) Oh and the most important one, please add this in only one
>>>>>>>>>>>>> file, add.ll or something like that, so we can keep this convention in the
>>>>>>>>>>>>> future, otherwise we'll end having too many test files.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Something that should be covered is, when doing 16 bit
>>>>>>>>>>>>> additions we can use adiw or add depending on the imm value, can you cover
>>>>>>>>>>>>> this aswell?
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2012/5/1 John Myers <ato...@gm...>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Apr 29, 2012 at 10:46 AM, Nicklas Bo Jensen <
>>>>>>>>>>>>>> nbj...@gm...> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I have successfully been able to compile your testcases
>>>>>>>>>>>>>>> (/avr-llvm/testcases/*.ll) to something looking like valid avr assembler.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> How should I test/simulate the assembler? I get errors when
>>>>>>>>>>>>>>> trying to simulate the generated assembler in AVRStudio. Perhaps they use a
>>>>>>>>>>>>>>> different assembler?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> avr-llvm produces GNU assembler syntax, which is different
>>>>>>>>>>>>>> then the Atmel assembler syntax.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Eventually we could support multiple asm syntax's like the
>>>>>>>>>>>>>> X86 target does.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>