[Crackerjack-users 00068] Re: [LTP] [RFC] [OLS 2008 FALLOUT] Issue # 2

[Garrett C. <yan...@gm...>]

On Mon, Aug 18, 2008 at 7:43 PM, Masatake YAMATO <ya...@re...> wrote:
>> 2.5) LTP should start focusing on IN-KERNEL-API testing, basically
>> testing the API´s which are available only inside the kernel.
>
> Comparing with system calls, I think IN-KERNEL-API has much chances to
> be tested well tested by the other kernel code. So the bug newly
> injected to IN-KERNEL-API-IMPLEMENTATION will be found quickly.  In
> addition such APIs will be changed quickly. It means test cases for
> usch API becomes garbage quickly.
>
> I think it will be more fruity if LTP people look at
> OUTSIDE-KERNEL-BEHAVIOR with using kvm(or other vitalization
> technologies) ; e.g. we can conduct a test kernel behavior when
> plugging a usb device to a computer.
>
> Masatake YAMATO

Whitebox testing is a must because chasing down issues with blackbox
tests becomes annoying sometimes, and more importantly code coverage
can be lower than desired with blackbox tests. "Graybox" testing (if I
understand my manager's use of the term correctly) is the majority of
what LTP does nowadays though (only selectively works API's which are
unmasked to end users)...

How deep you want to dig into the rabbit hole is up to you as a test
writer with your unit tests, but the level of functional testing done
and the coverage attempted without being tied too much to a given arch
or device or non-common feature-anchored set of API's is fairly
substantial compared to other opensource test suites.

-Garrett