Re: [LTP] device-driver testsuites in LTP

[Németh M. <nm...@fr...>]

Hello Subrata and David,

David, as you can read below I had a look on your nls testsuite. Maybe
you can help me to answer my questions.

Subrata Modak wrote:
> Hi Németh,
> 
> On Mon, 2009-01-05 at 09:01 +0100, Németh Márton wrote:
>> Hi,
>> Subrata Modak wrote:
>>> On Sun, 2009-01-04 at 13:57 +0100, Németh Márton wrote:
>>>
>>> [...]
>>> I had also suggested during our initial conversation for reviving the
>>> existing device-drivers tests in LTP. I wanted somebody to help me fix
>>> them and bring them up and compile/run. Would you be able to help us
>>> here ?
>> I am interested in the topic, but I think it is better that I don't stop
>> with the v4l-test project, because then it will be never reach a good state.
>> (We can never finish testing completely...)
>>
>> If I understand correctly the task is to get the all the sources under
>> ltp/testcases/kernel/device-drivers. The files in question are:
>>
>> [...]
> 
> Yes, absolutely.
> 
>> What kernel version should it work with? The most recent patch is for
>> 2.6.25 in the latest gcov-kernel-2.tar.gz .
> 
> I would always want them to be compiled against the latest kernel,
> provided the updates are available. As you said that the latest patches
> are available for 2.6.25, it would be great if we can see them compile
> against that. And when the recent patches get available, we can merge
> them as well.
> 
>> Is there any need for special hardware for these tests?
> 
> May not be necessary. As your tests can also run against "Virtual Video
> Driver" and generate results in absence of the real hardware, i would
> like these tests to run if any Virtual driver is available for them.
> 
> Since these device-drivers are not built/run in any default/integrated
> run, and mostly left on the tester to execute them individually, we will
> not have much problem. So, we can have the following strategy:
> 1) We will initially fix them, so that they start compiling on the
> latest kernels,

I had a short look on the smallest testsuite in bytes to see how this could
work:

testcases/kernel/device-drivers/nls/Makefile
testcases/kernel/device-drivers/nls/nlsTest.c
testcases/kernel/device-drivers/nls/nlsTest.h
testcases/kernel/device-drivers/nls/userBlockNLS.c

Although I could modify the source code so that it is compilable the
most difficult part is still not solved. I could not find any note or
documentation what this testsiute is about:

 - What do we test? Where are the boundaries of the module under test?
   I guess that nls stands for Native language support.

 - How does the architecture of the testsuite look like? I see that there
   is a kernelspace and a userspace part.

I found two little notes about the nls testsuite:

in Chagelog: LTP-20030710
- Added testcases to for the NLS filesystem.  ( Marty Ridgeway )

and that the module author is David Cruz <cr...@us...>. Maybe
he could also help.

There is an additional step here we have to do:
1.5) Understand the testsuite and make it semantically correct so it
     won't crash immediately and produce relayable test results.

> 2) Individual testers can then go to the sub-directories of their driver
> tests, and then do individual make and run. If a virtual driver is
> present, he/she can run the tests against it, else, they would require
> to attach a real hardware before testing. We will leave it on their
> choice. Our responsibility will be to provide them clean tests which
> they can at least build/install in the initial stage.
> 
> And all info like:
> 1) Options to enable during kernel build/run,
> 2) Special Library, etc requirements,
> 
> should be put in the ltp/README, just like the "Video For Linux" info
> has been added.
> 
> Now, since we have decided to work something to make the device-drivers
> tests in LTP to look a little better, i have another request:
> 
> Can you also point out similar device-drivers tests for the Linux
> kernel, those who exist as separate project(s) either on sf.net, or,
> somewhere else. We can then talk to the maintainer and host them in LTP
> in similar arrangement as "Video For Linux" tests. Let me know whenever
> you come across any such info.
> 
> Regards--
> Subrata
> 
>> Regards,
>>
>> 	Márton Németh
>>
> 
> 
>