Re: [LTP] Modularized gzip'd tarballs created....

[William J. H. <wj...@us...>]

I can say from our perspective right now we only run the kernel section.
        Although it's not what I would prefer it's just the way things are 
currently.

        I hope the kernel development community becomes more consistent at 
checking their work with LTP (regression tests being a big part). That 
reduces our superficial efforts at running LTP as a glibc/kernel 
validation tool and we can start exploring the other testcases (network) 
which can get more interesting. Because as we all know just cause the 
network can send data doesn't mean it works if it consumes 80% of the CPU.

        Things like the network test tools, and other tools which have a 
dual purpose, like growfiles for example, are interesting. It not only 
tests parts of the syscall layers, but it also tests hardware and io and 
so forth. If combined with say a network IO tool you can really stress 
test that particular subsystem i a "real world" type of test. And maybe 
come up with a magic simulation # for pretenting to be a webserver.

        Again, once the kernel work is pretty much in effect I think 
there's a cause for branching out into more of a "system test tool". You 
can get into interesting things too like maybe being able to plug in the 
"contest" kernel benchmarking tool along with other tools say "bonnie" 
(which may be part of contest already now that I think of it) and whatever 
else and then LTP could be run as an overall system benchmark / stress 
tool.

        I don't know if I see the command testing section as a big 
priority. Simply because I feel that most cmds have really stagnated and 
the "does ar support the -v flag" test doesn't really tell you much about 
whether or not it's a real bug for a lot of effort just getting there. But 
that's just my reasoning. 
        I really liked the idea that someone posted about trying to use 
automake's testing for ./configure scripts. There are also tools that just 
randomly pick options and feed in garbage as a means of checking for 
buffer overflows. More generic tools rather then a test case for each 
cmdline tool. Just picking say emacs or vi (way too many options / 
combinations), even say ping and traceroute (potentially too much setup), 
ls (how do you know the #'s it's returning are valid). 
        Each of the examples illustrates one reason why it can be 
problematic to create a testcase for that cmdline tool. Now combine the 
possibilities of multiple issues for the same tool, along with an 
unfavorable multiplication rate (1 tool * ~10 options) seems to indicate 
to me the low yield for a given amount of effort.

        My vision at least.
        j


Jay Huie 
  wj...@us...
  zSeries Linux System Test
  Phone: 845-435-8164


Sent by:        ltp...@li...
To:     Robert Williamson/Austin/IBM@IBMUS
cc:     ltp <ltp...@li...> 
Subject:        Re: [LTP] Modularized gzip'd tarballs created....




Robbie,

I think we should understand what percent of LTP network or Command
section are used by those who use LTP regularly. It will be nice to get
some feedback on this from those using LTP. If the testing community
doesint care about the large tgz file or RPM they have to download, we
dont have to take the trouble of modularizing LTP. Personally I think
there are benifits in the long run by modularizing LTP.

My thought is, the runall script should have options to run specific
sections of LTP, also, change the name of that script to run-ltp.sh
something like that. run-ltp.sh should be able to handle a scenario file
based test execution, this will provide flexibility.

If there is support/agreement on such issues in the LTP-community I can
work on a script.

Cheers
Manjo

*******************************************************************************
cognito ergo sum
*******************************************************************************

On Mon, 13 Jan 2003, Robert Williamson wrote:

> As promised, I created the gzip'd tarball versions of the RPMs created
> Friday.  The network and commands modules are just simply the 
directories,
> that would need to be placed under /testcases when opened.  This is all
> being done in anticipation of the commands section getting bigger.  I 
may
> even toy around with modularizing the kernel section.....but that, if 
ever,
> will come later.
>
> - Robbie
>
> Robert V. Williamson <ro...@us...>
> Linux Test Project
> IBM Linux Technology Center
> Phone: (512) 838-9295   T/L: 678-9295
> Fax: (512) 838-4603
> http://ltp.sourceforge.net
> ====================
> "Only two things are infinite, the universe and human stupidity, and I'm
> not sure about the former." -Albert Einstein
>
>
>
>
> -------------------------------------------------------
> This SF.NET email is sponsored by: FREE  SSL Guide from Thawte
> are you planning your Web Server Security? Click here to get a FREE
> Thawte SSL guide and find the answers to all your  SSL security issues.
> http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
> _______________________________________________
> Ltp-list mailing list
> Ltp...@li...
> https://lists.sourceforge.net/lists/listinfo/ltp-list
>



-------------------------------------------------------
This SF.NET email is sponsored by: FREE  SSL Guide from Thawte
are you planning your Web Server Security? Click here to get a FREE
Thawte SSL guide and find the answers to all your  SSL security issues.
http://ads.sourceforge.net/cgi-bin/redirect.pl?thaw0026en
_______________________________________________
Ltp-list mailing list
Ltp...@li...
https://lists.sourceforge.net/lists/listinfo/ltp-list