From: Subrata M. <su...@li...> - 2008-08-13 08:34:03
|
Hi, Recent OLS 2008 was a critical point in LTP´s evolution, as i got the opportunity to meet several people across the Linux ecosystem, and listened to their opinion about LTP. Here i would start a mail chain with the above Subject line, discuss each and every issue in this mailing list, collate everybody´s opinion on those issue(s) and take action accordingly. These are the people i encountered: 1) People, who uses LTP heavily. And they suggested lots of improvement to it. We will discuss those issues in mails from now, 2) People, who have heard about LTP and not used it till now. They promised that they will give a try, 3) People, who has never heard about it. So, it was an opportunity to convey them what LTP is all about. I hope people in Category 2 & 3 will start using LTP soon, and we will get an enlarged user base and hence bringing more contribution in future. ================= ISSUE # 1 ================= The heavy users made a point of LTP having the capability to automate testing completely. What they meant was LTP to have capability to do: 1) Kernel Build, 2) Kernel Install/Distro install, 3) Then do specific/all tests, They said that this feature will simplify the way they work. I would like to know what you all think about this. What i feel is, every project should evolve and should be flexible enough to meet their users requirement dynamically, and should not be tied down with the limitations of it´s initial design constraints. If automating kernel build, install and tests is a requirement coming from the user community, then we need to give a hard look at it. I would like to know what you think about this. Regards-- Subrata |
From: Serge E. H. <se...@us...> - 2008-08-13 13:59:46
|
Quoting Subrata Modak (su...@li...): > Hi, > > Recent OLS 2008 was a critical point in LTP´s evolution, as i got the > opportunity to meet several people across the Linux ecosystem, and > listened to their opinion about LTP. Here i would start a mail chain > with the above Subject line, discuss each and every issue in this > mailing list, collate everybody´s opinion on those issue(s) and take > action accordingly. These are the people i encountered: > > 1) People, who uses LTP heavily. And they suggested lots of improvement > to it. We will discuss those issues in mails from now, > > 2) People, who have heard about LTP and not used it till now. They > promised that they will give a try, > > 3) People, who has never heard about it. So, it was an opportunity to > convey them what LTP is all about. I hope people in Category 2 & 3 will > start using LTP soon, and we will get an enlarged user base and hence > bringing more contribution in future. > > ================= > ISSUE # 1 > ================= > The heavy users made a point of LTP having the capability to automate > testing completely. What they meant was LTP to have capability to do: > 1) Kernel Build, > 2) Kernel Install/Distro install, > 3) Then do specific/all tests, > > They said that this feature will simplify the way they work. I would > like to know what you all think about this. > > What i feel is, every project should evolve and should be flexible (my 2c) if every project evolves, then every program will end up being emacs+firefox+eclipse all in one. If people want what you describe above, then a new project should be created. It could actually be pretty spiffy, and quite simple. It could grab distro images to autoinstall a kvm image, install some software and/or patches that I specify, grab a kernel I specify, build it, grab the most recent ltp release and compile/install it, run the tests, and give me the results. I know there are suites out there that do that type of things on physical grids now. A smaller version of that which just creates a kvm partition on my own machine would be like a personal version of one of those. Ideally it would be accompanied by an online store of very targeted distro install .isos that auto-install themselves if I just do kvm -hda newimage.img -cdrom distro.iso -boot d. And through judicious saving of installed images and use of -snapshot, this project could lead to truly repeatable ltp results. "Use this kernel with this config on this qemu-img, and you'll see that chown is failing." Cool. But putting this in ltp seems wrong to me. Let's keep ltp's focus on testing. So really the hardest part of starting something like this might be the creation of some auto-install distro images. > enough to meet their users requirement dynamically, and should not be > tied down with the limitations of it´s initial design constraints. If > automating kernel build, install and tests is a requirement coming from > the user community, then we need to give a hard look at it. I would like > to know what you think about this. > > Regards-- > Subrata > |
From: Garrett C. <yan...@gm...> - 2008-08-13 16:14:19
|
On Wed, Aug 13, 2008 at 6:58 AM, Serge E. Hallyn <se...@us...> wrote: > Quoting Subrata Modak (su...@li...): >> Hi, >> >> Recent OLS 2008 was a critical point in LTP´s evolution, as i got the >> opportunity to meet several people across the Linux ecosystem, and >> listened to their opinion about LTP. Here i would start a mail chain >> with the above Subject line, discuss each and every issue in this >> mailing list, collate everybody´s opinion on those issue(s) and take >> action accordingly. These are the people i encountered: >> >> 1) People, who uses LTP heavily. And they suggested lots of improvement >> to it. We will discuss those issues in mails from now, >> >> 2) People, who have heard about LTP and not used it till now. They >> promised that they will give a try, >> >> 3) People, who has never heard about it. So, it was an opportunity to >> convey them what LTP is all about. I hope people in Category 2 & 3 will >> start using LTP soon, and we will get an enlarged user base and hence >> bringing more contribution in future. >> >> ================= >> ISSUE # 1 >> ================= >> The heavy users made a point of LTP having the capability to automate >> testing completely. What they meant was LTP to have capability to do: >> 1) Kernel Build, >> 2) Kernel Install/Distro install, >> 3) Then do specific/all tests, >> >> They said that this feature will simplify the way they work. I would >> like to know what you all think about this. >> >> What i feel is, every project should evolve and should be flexible > > (my 2c) if every project evolves, then every program will end up being > emacs+firefox+eclipse all in one. > > If people want what you describe above, then a new project should be > created. It could actually be pretty spiffy, and quite simple. It > could grab distro images to autoinstall a kvm image, install some > software and/or patches that I specify, grab a kernel I specify, build > it, grab the most recent ltp release and compile/install it, run the > tests, and give me the results. > > I know there are suites out there that do that type of things on > physical grids now. A smaller version of that which just creates > a kvm partition on my own machine would be like a personal version of > one of those. Ideally it would be accompanied by an online store of > very targeted distro install .isos that auto-install themselves if > I just do kvm -hda newimage.img -cdrom distro.iso -boot d. > > And through judicious saving of installed images and use of -snapshot, > this project could lead to truly repeatable ltp results. "Use this > kernel with this config on this qemu-img, and you'll see that chown > is failing." Cool. > > But putting this in ltp seems wrong to me. Let's keep ltp's focus on > testing. > > So really the hardest part of starting something like this might be > the creation of some auto-install distro images. > >> enough to meet their users requirement dynamically, and should not be >> tied down with the limitations of it´s initial design constraints. If >> automating kernel build, install and tests is a requirement coming from >> the user community, then we need to give a hard look at it. I would like >> to know what you think about this. >> >> Regards-- >> Subrata I agree with Serge completely. There should be a series of wrappers around LTP which do the "right thing" when invoking runltp, et all on a target system or kvm. I think the issue is that LTP lacks the internal knobs to do the job appropriately within the makefiles, for detecting that the system can't build librt stuff because of mismatch in ABI, or other tests because of missing dependencies (like Perl + Ballista for instance). I tried filling that in with some stubs for detecting headers, etc which may not be a part of a standard distro, but that can all vary depending on what distribution is put into play, and again I want to avoid autotools as much as possible, even though things are sort of evolving in that kind of direction... -Garrett |
From: Randy D. <rd...@xe...> - 2008-08-13 16:42:53
|
On Wed, 13 Aug 2008 14:02:55 +0530 Subrata Modak wrote: > Hi, > > Recent OLS 2008 was a critical point in LTP´s evolution, as i got the > opportunity to meet several people across the Linux ecosystem, and > listened to their opinion about LTP. Here i would start a mail chain > with the above Subject line, discuss each and every issue in this > mailing list, collate everybody´s opinion on those issue(s) and take > action accordingly. These are the people i encountered: > > 1) People, who uses LTP heavily. And they suggested lots of improvement > to it. We will discuss those issues in mails from now, > > 2) People, who have heard about LTP and not used it till now. They > promised that they will give a try, > > 3) People, who has never heard about it. So, it was an opportunity to > convey them what LTP is all about. I hope people in Category 2 & 3 will > start using LTP soon, and we will get an enlarged user base and hence > bringing more contribution in future. > > ================= > ISSUE # 1 > ================= > The heavy users made a point of LTP having the capability to automate > testing completely. What they meant was LTP to have capability to do: > 1) Kernel Build, With what/how many configs? > 2) Kernel Install/Distro install, With which/how many distros? This is an unending mess... and $someone will say "you don't support my $distro". > 3) Then do specific/all tests, (Like Serge said) Just focus here. > They said that this feature will simplify the way they work. I would > like to know what you all think about this. > > What i feel is, every project should evolve and should be flexible > enough to meet their users requirement dynamically, and should not be > tied down with the limitations of it´s initial design constraints. If > automating kernel build, install and tests is a requirement coming from > the user community, then we need to give a hard look at it. I would like > to know what you think about this. Kernel builds are relatively easy to do & automating them isn't difficult. OTOH, installing a distro is a much larger task in my experience. --- ~Randy Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA http://linuxplumbersconf.org/ |
From: Paul L. <pl...@us...> - 2008-08-13 20:36:43
|
Agreeing with what pretty much everyone else has said, I'd also like to add that this isn't the first time someone has requested something like this. Iirc, there was a small attempt at automating at least some part of it, you may still find artifacts of it lingering around in the scripts directory. Other alternatives include things like autotest, STP at OSDL, or even rolling your own. As Randy suggested, it isn't all that hard to automate kernel build and boot. But end to end automation normally requires some level of integration with your hardware/network infrastructure. It can be done in a generic enough way, sure, but normally ends up spiraling out of control into something difficult to setup and manage. People tend to look at something like that and decide they could come up with something simpler (but not generic enough to be used anywhere) themselves. -Paul Larson On Wed, 2008-08-13 at 09:43 -0700, Randy Dunlap wrote: > On Wed, 13 Aug 2008 14:02:55 +0530 Subrata Modak wrote: > > > Hi, > > > > Recent OLS 2008 was a critical point in LTP´s evolution, as i got the > > opportunity to meet several people across the Linux ecosystem, and > > listened to their opinion about LTP. Here i would start a mail chain > > with the above Subject line, discuss each and every issue in this > > mailing list, collate everybody´s opinion on those issue(s) and take > > action accordingly. These are the people i encountered: > > > > 1) People, who uses LTP heavily. And they suggested lots of improvement > > to it. We will discuss those issues in mails from now, > > > > 2) People, who have heard about LTP and not used it till now. They > > promised that they will give a try, > > > > 3) People, who has never heard about it. So, it was an opportunity to > > convey them what LTP is all about. I hope people in Category 2 & 3 will > > start using LTP soon, and we will get an enlarged user base and hence > > bringing more contribution in future. > > > > ================= > > ISSUE # 1 > > ================= > > The heavy users made a point of LTP having the capability to automate > > testing completely. What they meant was LTP to have capability to do: > > 1) Kernel Build, > > With what/how many configs? > > > 2) Kernel Install/Distro install, > > With which/how many distros? This is an unending mess... > and $someone will say "you don't support my $distro". > > > > 3) Then do specific/all tests, > > (Like Serge said) Just focus here. > > > > They said that this feature will simplify the way they work. I would > > like to know what you all think about this. > > > > What i feel is, every project should evolve and should be flexible > > enough to meet their users requirement dynamically, and should not be > > tied down with the limitations of it´s initial design constraints. If > > automating kernel build, install and tests is a requirement coming from > > the user community, then we need to give a hard look at it. I would like > > to know what you think about this. > > Kernel builds are relatively easy to do & automating them isn't difficult. > OTOH, installing a distro is a much larger task in my experience. > > > --- > ~Randy > Linux Plumbers Conference, 17-19 September 2008, Portland, Oregon USA > http://linuxplumbersconf.org/ |
From: Louis R. <Lou...@ke...> - 2008-08-18 13:07:59
|
On Wed, Aug 13, 2008 at 02:02:55PM +0530, Subrata Modak wrote: > Hi, > > Recent OLS 2008 was a critical point in LTP´s evolution, as i got the > opportunity to meet several people across the Linux ecosystem, and > listened to their opinion about LTP. Here i would start a mail chain > with the above Subject line, discuss each and every issue in this > mailing list, collate everybody´s opinion on those issue(s) and take > action accordingly. These are the people i encountered: > > 1) People, who uses LTP heavily. And they suggested lots of improvement > to it. We will discuss those issues in mails from now, > > 2) People, who have heard about LTP and not used it till now. They > promised that they will give a try, > > 3) People, who has never heard about it. So, it was an opportunity to > convey them what LTP is all about. I hope people in Category 2 & 3 will > start using LTP soon, and we will get an enlarged user base and hence > bringing more contribution in future. > > ================= > ISSUE # 1 > ================= > The heavy users made a point of LTP having the capability to automate > testing completely. What they meant was LTP to have capability to do: > 1) Kernel Build, > 2) Kernel Install/Distro install, > 3) Then do specific/all tests, > > They said that this feature will simplify the way they work. I would > like to know what you all think about this. > > What i feel is, every project should evolve and should be flexible > enough to meet their users requirement dynamically, and should not be > tied down with the limitations of it´s initial design constraints. If > automating kernel build, install and tests is a requirement coming from > the user community, then we need to give a hard look at it. I would like > to know what you think about this. [ Sorry for arriving late in the discussion (hollydays) ] I definitely agree with others that 1) and 2) should be done in separate projects. However, 3) (doing selected tests) has to be done in LTP, IMHO. AFAICS, what LTP currently provides for this is (please correct me where I am wrong): - run all tests, - run a (whole) suite of tests (eg "syscalls"), -? run a single test. What I would like: i) run a custom subset of tests, out of existing suites (eg not all "syscalls"), ii) disable some tests in a suite (eg disable fork07 in "syscalls"), but still report that the test was disabled. For i), why not writing my own file under runtest/? Because some tests need a few setup before being run, and I don't want to copy these setup in all custom runlists. To address this, I suggest to move those setup commands in dedicated executables located together with the test sources (say abort01.setup under testcases/kernel/syscalls/abort/). Those *.setup could be simple shell scripts (abort01.setup would contain "ulimit -c 1024") or binaries as well. For ii) a usecase is the following: when implementing a new feature, often the feature is buggy and breaks several tests for different reasons. "Break" means the test result is "FAIL", or the machine was crashed as well. As long as fixing those bugs is fast, one can simply comment out the failing tests in the related file under runtest/. However, when it takes a long time to fix those bugs, having a report giving the list of successful tests as well as the ones that were (intentionally) disabled gives some bugfix progress bar. To address this, I suggest to, first, do as I suggested for i), and second, replace the setup code in the runlists by wrapping code when desired Example to disable fork07: fork07 fork07 becomes fork07 disabled fork07 Where "disabled" is a script living in the same directory as test binaries, that contains: <begin script> #!/bin/sh echo $1 disabled <end script> If what I suggest looks ok, I can write patches. Thanks, Louis -- Dr Louis Rilling Kerlabs Skype: louis.rilling Batiment Germanium Phone: (+33|0) 6 80 89 08 23 80 avenue des Buttes de Coesmes http://www.kerlabs.com/ 35700 Rennes |
From: Subrata M. <su...@li...> - 2008-10-20 20:30:21
|
I would apolozise that i did not bring these set of discussions in their logical conclusion. I would hence do this in some couple of days to come. ======================================================================== Issue # 1 ======================================================================== As i can infer the logic majority of you have proposed, to keep LTP away from the realm of Kernel Build and Test, and concentrate more on real testing of the Kernel and other improvement areas; i would agree with you. It is not that i did not understand what and where LTP should really focus into, but, i wanted to bring to everybody´s notice some aspirations from some quarters. Now, having strongly convinced the actual need for this, we would not think of LTP needing to do Auto Kernel Build/install, Distro fetch/install. We would rather let some other project do that and keep LTP focussed on the real need to test the kernel better. > On Wed, Aug 13, 2008 at 02:02:55PM +0530, Subrata Modak wrote: > > Hi, > > > > Recent OLS 2008 was a critical point in LTP´s evolution, as i got the > > opportunity to meet several people across the Linux ecosystem, and > > listened to their opinion about LTP. Here i would start a mail chain > > with the above Subject line, discuss each and every issue in this > > mailing list, collate everybody´s opinion on those issue(s) and take > > action accordingly. These are the people i encountered: > > > > 1) People, who uses LTP heavily. And they suggested lots of improvement > > to it. We will discuss those issues in mails from now, > > > > 2) People, who have heard about LTP and not used it till now. They > > promised that they will give a try, > > > > 3) People, who has never heard about it. So, it was an opportunity to > > convey them what LTP is all about. I hope people in Category 2 & 3 will > > start using LTP soon, and we will get an enlarged user base and hence > > bringing more contribution in future. > > > > ================= > > ISSUE # 1 > > ================= > > The heavy users made a point of LTP having the capability to automate > > testing completely. What they meant was LTP to have capability to do: > > 1) Kernel Build, > > 2) Kernel Install/Distro install, > > 3) Then do specific/all tests, > > > > They said that this feature will simplify the way they work. I would > > like to know what you all think about this. > > > > What i feel is, every project should evolve and should be flexible > > enough to meet their users requirement dynamically, and should not be > > tied down with the limitations of it´s initial design constraints. If > > automating kernel build, install and tests is a requirement coming from > > the user community, then we need to give a hard look at it. I would like > > to know what you think about this. > > [ Sorry for arriving late in the discussion (hollydays) ] > > I definitely agree with others that 1) and 2) should be done in separate > projects. > > However, 3) (doing selected tests) has to be done in LTP, IMHO. > > AFAICS, what LTP currently provides for this is (please correct me where I am > wrong): > - run all tests, > - run a (whole) suite of tests (eg "syscalls"), > -? run a single test. > > What I would like: > i) run a custom subset of tests, out of existing suites (eg not all "syscalls"), > ii) disable some tests in a suite (eg disable fork07 in "syscalls"), but still > report that the test was disabled. > Yes, this can be done. What i can think of presently, the best way to do so is to: 1) Provide a custom command file under ltp/runtest/*, which contains a set of entries (similar way you enter in ltp/runtest/* files) you would not like to run. 2) Let runltp do an internal grep from it´s -f <files> and exclude those tests you mentioned. We may need to introduce another option, say -F <file2>. Where file2 basically contains tests that you want to exclude from running. > For i), why not writing my own file under runtest/? Because some tests need a > few setup before being run, and I don't want to copy these setup in all custom > runlists. > To address this, I suggest to move those setup commands in > dedicated executables located together with the test sources (say abort01.setup > under testcases/kernel/syscalls/abort/). Those *.setup could be simple shell > scripts (abort01.setup would contain "ulimit -c 1024") or binaries as well. Yes, this is absolutely possible. You can install your setup script during make install, and invoke the script during test run. Let the script execute the original test case binary if it feels that all system requirements are made, and report results. We have already started doing it for lots of scripts, the latest i remember is this: http://ltp.cvs.sourceforge.net/viewvc/ltp/ltp/testcases/kernel/syscalls/ioctl/test_ioctl?revision=1.1&view=markup > For ii) a usecase is the following: when implementing a new feature, often the > feature is buggy and breaks several tests for different reasons. "Break" means > the test result is "FAIL", or the machine was crashed as well. As long as > fixing those bugs is fast, one can simply comment out the failing tests in the > related file under runtest/. However, when it takes a long time to fix those > bugs, having a report giving the list of successful tests as well as the ones > that were (intentionally) disabled gives some bugfix progress bar. > To address this, I suggest to, first, do as I suggested for i), and > second, replace the setup code in the runlists by wrapping code when desired > Example to disable fork07: > fork07 fork07 > becomes > fork07 disabled fork07 > > Where "disabled" is a script living in the same directory as test binaries, that > contains: > > <begin script> > #!/bin/sh > > echo $1 disabled > <end script> > > If what I suggest looks ok, I can write patches. Sure. Please let us know the way you would like to do this. Regards-- Subrata > > Thanks, > > Louis > |
From: Louis R. <Lou...@ke...> - 2008-10-20 14:15:55
|
On Mon, Oct 20, 2008 at 05:41:21PM +0530, Subrata Modak wrote: > I would apolozise that i did not bring these set of discussions in their > logical conclusion. I would hence do this in some couple of days to > come. > > ======================================================================== > Issue # 1 > ======================================================================== > As i can infer the logic majority of you have proposed, to keep LTP away > from the realm of Kernel Build and Test, and concentrate more on real > testing of the Kernel and other improvement areas; i would agree with > you. It is not that i did not understand what and where LTP should > really focus into, but, i wanted to bring to everybody´s notice some > aspirations from some quarters. Now, having strongly convinced the > actual need for this, we would not think of LTP needing to do Auto > Kernel Build/install, Distro fetch/install. We would rather let some > other project do that and keep LTP focussed on the real need to test the > kernel better. > > > On Wed, Aug 13, 2008 at 02:02:55PM +0530, Subrata Modak wrote: > > > Hi, > > > > > > Recent OLS 2008 was a critical point in LTP´s evolution, as i got the > > > opportunity to meet several people across the Linux ecosystem, and > > > listened to their opinion about LTP. Here i would start a mail chain > > > with the above Subject line, discuss each and every issue in this > > > mailing list, collate everybody´s opinion on those issue(s) and take > > > action accordingly. These are the people i encountered: > > > > > > 1) People, who uses LTP heavily. And they suggested lots of improvement > > > to it. We will discuss those issues in mails from now, > > > > > > 2) People, who have heard about LTP and not used it till now. They > > > promised that they will give a try, > > > > > > 3) People, who has never heard about it. So, it was an opportunity to > > > convey them what LTP is all about. I hope people in Category 2 & 3 will > > > start using LTP soon, and we will get an enlarged user base and hence > > > bringing more contribution in future. > > > > > > ================= > > > ISSUE # 1 > > > ================= > > > The heavy users made a point of LTP having the capability to automate > > > testing completely. What they meant was LTP to have capability to do: > > > 1) Kernel Build, > > > 2) Kernel Install/Distro install, > > > 3) Then do specific/all tests, > > > > > > They said that this feature will simplify the way they work. I would > > > like to know what you all think about this. > > > > > > What i feel is, every project should evolve and should be flexible > > > enough to meet their users requirement dynamically, and should not be > > > tied down with the limitations of it´s initial design constraints. If > > > automating kernel build, install and tests is a requirement coming from > > > the user community, then we need to give a hard look at it. I would like > > > to know what you think about this. > > > > [ Sorry for arriving late in the discussion (hollydays) ] > > > > I definitely agree with others that 1) and 2) should be done in separate > > projects. > > > > However, 3) (doing selected tests) has to be done in LTP, IMHO. > > > > AFAICS, what LTP currently provides for this is (please correct me where I am > > wrong): > > - run all tests, > > - run a (whole) suite of tests (eg "syscalls"), > > -? run a single test. > > > > What I would like: > > i) run a custom subset of tests, out of existing suites (eg not all "syscalls"), > > ii) disable some tests in a suite (eg disable fork07 in "syscalls"), but still > > report that the test was disabled. > > > > Yes, this can be done. What i can think of presently, the best way to do > so is to: > 1) Provide a custom command file under ltp/runtest/*, which contains a > set of entries (similar way you enter in ltp/runtest/* files) you would > not like to run. > 2) Let runltp do an internal grep from it´s -f <files> and exclude those > tests you mentioned. We may need to introduce another option, say -F > <file2>. Where file2 basically contains tests that you want to exclude > from running. I like this idea. This is less intrusive than what I suggested. > > > For i), why not writing my own file under runtest/? Because some tests need a > > few setup before being run, and I don't want to copy these setup in all custom > > runlists. > > To address this, I suggest to move those setup commands in > > dedicated executables located together with the test sources (say abort01.setup > > under testcases/kernel/syscalls/abort/). Those *.setup could be simple shell > > scripts (abort01.setup would contain "ulimit -c 1024") or binaries as well. > > Yes, this is absolutely possible. You can install your setup script > during make install, and invoke the script during test run. Let the > script execute the original test case binary if it feels that all system > requirements are made, and report results. We have already started doing > it for lots of scripts, the latest i remember is this: > > http://ltp.cvs.sourceforge.net/viewvc/ltp/ltp/testcases/kernel/syscalls/ioctl/test_ioctl?revision=1.1&view=markup Yes something like that. This is however not absolutely required if we implement selective test disabling the way you suggested it. > > > For ii) a usecase is the following: when implementing a new feature, often the > > feature is buggy and breaks several tests for different reasons. "Break" means > > the test result is "FAIL", or the machine was crashed as well. As long as > > fixing those bugs is fast, one can simply comment out the failing tests in the > > related file under runtest/. However, when it takes a long time to fix those > > bugs, having a report giving the list of successful tests as well as the ones > > that were (intentionally) disabled gives some bugfix progress bar. > > To address this, I suggest to, first, do as I suggested for i), and > > second, replace the setup code in the runlists by wrapping code when desired > > Example to disable fork07: > > fork07 fork07 > > becomes > > fork07 disabled fork07 > > > > Where "disabled" is a script living in the same directory as test binaries, that > > contains: > > > > <begin script> > > #!/bin/sh > > > > echo $1 disabled > > <end script> > > > > If what I suggest looks ok, I can write patches. > > Sure. Please let us know the way you would like to do this. Following your suggestion, using a -F <disabled_tests> option for runltp: 1) put a list of test names, one per line, in <disabled_tests>; 2) in runltp, build a sed script from the list of test names in <disabled_tests>, which purpose is to replace every line of its input beginning with a disabled test name with 'disabled <test_name>'; 3) in runltp, filter the generated 'alltests' file with the generated sed script. That's it. I have no time to do this right now, unfortunately. However, your feedback is welcome :) Thanks, Louis -- Dr Louis Rilling Kerlabs Skype: louis.rilling Batiment Germanium Phone: (+33|0) 6 80 89 08 23 80 avenue des Buttes de Coesmes http://www.kerlabs.com/ 35700 Rennes |