avr-llvm-devel

[avr-llvm-devel] Tests

[Sherard D. <sc...@gm...>]

Hey everyone,

So I started to work on writing some tests, and... not as easy as I
thought it would be. Writing the tests are easy, the hard part is
*defining* what is correct. I've mostly been studying the code GCC
produces, however I don't want to end up writing tests that verify that
the output is just like what GCC would produce, or otherwise limit the
possible output.

I have looked at the tests written for most of the other backends, and
what I see usually seems either really strict (restricting possible code
generation), really specific (targeted at a specific bug report), or
simply tests that don't check their output.

Pasted below is one of the test files I've written. Any comments would
be greatly appreciated, as I seem to be having a rather difficult time
grasping the idea of verifying correct code, for which multiple pieces
of code can all be considered correct.

%struct.Large = type { i32, i32, i32, i32 }

declare i8 @targetCallRegisters8(i8, i8, i8, i8)
declare i32 @targetCallRegisters32(i32, i32, i32, i32)
declare i32 @targetCallStack(i32, %struct.Large)

define i8 @callerStack(i32 %a, %struct.Large %b) {
  ; CHECK: callerStack
  ; CHECK: in [[SSUBRL:r[0-9]+]], 0x3D
  ; CHECK: in [[SSUBRH:r[0-9]+]], 0x3E
  ; CHECK: sbiw [[STACKRL]], 0x10
  ; CHECK: out 0x3E, [[SSUBRH]]
  ; CHECK: out 0x3D, [[SSUBRL]]
  ; CHECK: ldi r22, 0xE0
  ; CHECK: ldi r23, 0x00
  ; CHECK: ldi r24, 0x00
  ; CHECK: ldi r25, 0x00
  ; CHECK: {{[eir]*call}}
  ; CHECK: in [[SADDRL:r[0-9]+]], 0x3D
  ; CHECK: in [[SADDRH:r[0-9]+]], 0x3E
  ; CHECK: adiw [[STACKRL]], 0x10
  ; CHECK: out 0x3E, [[SADDRH]]
  ; CHECK: out 0x3D, [[SADDRL]]
  
  %1 = call i32 @targetCallStack(i32 224, %struct.Large %b)
  ret i8 0  
}

define i8 @callerRegisters8() {
  ; CHECK: callerRegisters8
  ; CHECK: ldi r24, 0xA0
  ; CHECK: ldi r22, 0xB0
  ; CHECK: ldi r20, 0xC0
  ; CHECK: ldi r18, 0xD0
  ; CHECK: {{[eir]*call}}
  
  %1 = call i8 @targetCallRegisters8(i8 160, i8 176, i8 192, i8 208)
  ret i8 %1
}

define i8 @callerRegisters32() {
  ; CHECK: callerRegisters32
  ; CHECK: ldi r22, 0xA0
  ; CHECK: ldi r18, 0xB0
  ; CHECK: ldi [[TMPREG1:r[0-9]+]], 0xC0
  ; CHECK: mov r14, [[TMPREG1]]
  ; CHECK: ldi [[TMPREG2:r[0-9]+]], 0xD0
  ; CHECK: mov r10, [[TMPREG2]]
  ; CHECK: {{[eir]*call}}
  
  %1 = call i32 @targetCallRegisters32(i32 160, i32 176, i32 192, i32
208)
  ret i8 0
}

[avr-llvm-devel] Tests

[Borja F. <bor...@gm...>]

Hello, first thank both of you for working into this. As Joshua suggested I
would start by working with simple very tests, I haven't checked other
backends but I'm sure they have to check simple things like arith
instructions and calling conventions as Sherard did. However I would start
with arith instructions because they are easier and shorter to write, and
very important, testing all datatypes (from i8 to i64). Also testing both
regs, and imm instruction operands. Ideally we should test that each
instruction for every datatype produces the correct asm output.
Note on this, for now don't write tests on multiplications and shifts, the
first is not implemented yet, and for the second they require special
treatment that we will discuss later.
Then, move on to testing calling conventions, so check if the right
registers are being read as input arguments of a function, check if the
right registers are being passed in function calls etc. Some info about
calling conventions, I haven't tested passing big structs byval or
returning them so I wouldn't add tests for this for now, maybe make a note
on this to check in it the future when it's done. Ah, and don't test
varargs, they are a bit tricky to implement, and they arent high priority
for now.

One note about Joshua's test, i dont know if CHECK is case sensitive, but
all instructions are lowercase in our backend, so please careful with this.
Another thing, i have a patched version of clang that doesn't emit the
"signext" argument attribute in functions, we dont want to signext an i8 to
i16 beause the upper reg is not used, so we may be able to use it for other
purposes. I can't remember now if the patch in svn addresses this or not,
but it was something easy to do in clang, so you may want to add it to your
code.

For these simple tests you can use GCC to check the produced asm because we
should produce exactly the same code, and check the generated IR that LLVM
produces like Joshua showed, that way you can write directly tests in C
instead of IR.

Re: [avr-llvm-devel] Tests

[Joshua N. <ne...@em...>]

Hey Sherard,
	I don't know if that is the first test you have written, but I would start with the easy tests first. (see attached, if that works on this list serve) But as far as the test you would like to write based on calling conditions, you are right, the tests written are quite strict, specific, or simply there just to check compilation, and in order to write a test for something like a call, that you would need to know the underlying code, so until Borja commits his code so far you will not be able to write code for this part.
	For a digression, as far as the philosophy behind having very strict tests, from what I can tell is that once we have decided on a way to compile a piece of LLVM IR it should compile the same way ever time until we change code that affects it. This is a great way to catch code that directly or indirectly affects the testing IR compilation, if we somehow tweak the compilation to work better, it is ok that we fail, for at that time we check the failed code, see that it works correctly (hopefully even better) and tweak the test. Knowing this the smaller and more specific the test is, the better, for instance, write a couple test for a calling convention, each one testing a specific part of the call. I am sorry if I have written too much here, my formal training is in Biology, so I do not expect much prior knowledge from people, as I am often left out in the dark when people do not explain lots of things to the T.
	Anyways, about how to go about writing the tests,  I would first patch clang (I don't know if patch in the repository is up to date, so I attached mine) and execute the following `clang -emit-llvm -ccc-host-triple avr -Os -S -o - scratch.c`, this will emit LLVM IR based on scratch.c so you know what valid IR to use, then use llc to compile this IR into assembly `llc -march=avr arithmetic_instruction_selection.ll -o -` (this you will need the actual backend code, I am currently using my own avr backend code) If it outputs correct assembly, write the test to conform to this output, if not. figure out why it does not and let us know or help fix it. Anyways, I hope this helps, that you get my attached files and do not take offense.
	Joshua

Joshua Nedrud
Master of Science, Biomedical Engineering

Example, if the attached files do not send:

; RUN: llc < %s -march=avr | FileCheck %s

define signext i8 @add8rr(i8 signext %a, i8 signext %b) nounwind readnone optsize {
  %1 = add i8 %a, %b
  ret i8 %1
}
; CHECK: add8rr:
; CHECK: ADD

This test checks for the function definition add8rr, and then checks if the assembly instruction ADD follows this, which it should for this is about the simplest LLVM IR function you can get when run with -Os optimizations.