From: Dan K. <dk...@ix...> - 2003-05-05 21:53:09
|
I do this, on systems with as little as 32MB RAM. I originally tried to do it without NFS, but it was too much of a pain. I only run the basic tests, and even then, I exclude a few, e.g. mm01 mm02 page01 page02 stack_space swapoff01 swapoff02 swapon01 swapon02 because they make little sense on diskless nodes with no swap areas. - Dan -----Original Message----- From: David A. Marlin To: ltp...@li... Sent: 05.05.2003 13:27 Subject: [LTP] Running LTP on resource constrained systems Has anyone run the LTP Testsuite (or parts of the testsuite) on a resource constrained system? I am working with devices that may have up to 64MB RAM and only a flash file system (no hard drive). They may, or may not, include an Ethernet connection. Because of these limitations, the systems will not include a full Linux installation (no native toolchain, only a small subset of packages). The kernel and all packages will be built using a cross compiler. If anyone has run LTP on a similar system, I would appreciate your comments or suggestions based on your experience. Thank you, d.marlin ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Ltp-list mailing list Ltp...@li... https://lists.sourceforge.net/lists/listinfo/ltp-list |
From: Dan K. <dk...@ix...> - 2003-05-06 00:17:59
|
We run without those mods, I think. The mods we needed to get it to run with busybox were included in cvs in mid-april. I rather thought busybox's ash was happy with runalltests now. FWIW, we're using busybox-0.60.5 (but we wish we were still using 0.60.2, which was less buggy). We do configure a lot of stuff into busybox, though. I'd be happy to post a busybox config file if needed. Also, one option can't be turned on via the usual config file, or at least couldn't last time I checked: --- busybox-0.60.5/ash.c 2002-10-22 15:14:29.000000000 -0700 +++ busybox-0.60.5/ash.c.new 2003-03-12 16:44:41.000000000 -0800 @@ -47,7 +47,7 @@ * applet if you want to do this sort of thing. There are some scripts * out there that use it, so if you need it, enable it. Most people will * leave this disabled. This adds 1k on an x86 system. */ -#undef CONFIG_ASH_GETOPTS +#define CONFIG_ASH_GETOPTS - Dan -----Original Message----- From: Manoj Iyer To: David A. Marlin; ltp...@li... Sent: 05.05.2003 15:57 Subject: Re: [LTP] Running LTP on resource constrained systems ... do you have BASH? if NOT then runalltests.sh script will have to be modified to work with buybox (if you have busybox instead), coz there are certain things in runnalltests.sh that busy box ash does not like. I would hack it this way>>> Under ltp-mmddyy create results directory, In runalltests.sh remove the -l option and hardcode the logfile name in the line that executes pan (this is towards the end of that file). Remove $quite_mode and replace it with -q Remove $pretty_print and replace it with -p, Create your own custom command file with the list of tests you want to execute and replace -f ${TMP}/alltests with -f {PWD}/runtests/my_command_file.txt |
From: Manoj I. <ma...@au...> - 2003-05-06 00:53:54
|
Ah! that is cool dan! you are correct most of the stuff wrt runalltests.sh that I posed in my prev email apply if you have busybox. Thanks for the patch. Manjo On Mon, 2003-05-05 at 19:17, Dan Kegel wrote: > We run without those mods, I think. The mods we needed > to get it to run with busybox were included in cvs in mid-april. > > I rather thought busybox's ash was happy with runalltests now. > FWIW, we're using busybox-0.60.5 (but we wish we were still using > 0.60.2, which was less buggy). > > We do configure a lot of stuff into busybox, though. I'd > be happy to post a busybox config file if needed. Also, one > option can't be turned on via the usual config file, or > at least couldn't last time I checked: > > --- busybox-0.60.5/ash.c 2002-10-22 15:14:29.000000000 -0700 > +++ busybox-0.60.5/ash.c.new 2003-03-12 16:44:41.000000000 -0800 > @@ -47,7 +47,7 @@ > * applet if you want to do this sort of thing. There are some scripts > * out there that use it, so if you need it, enable it. Most people will > * leave this disabled. This adds 1k on an x86 system. */ > -#undef CONFIG_ASH_GETOPTS > +#define CONFIG_ASH_GETOPTS > > - Dan > > -----Original Message----- > From: Manoj Iyer > To: David A. Marlin; ltp...@li... > Sent: 05.05.2003 15:57 > Subject: Re: [LTP] Running LTP on resource constrained systems > ... > do you have BASH? if NOT then runalltests.sh script will have to be > modified to work with buybox (if you have busybox instead), coz there > are certain things in runnalltests.sh that busy box ash does not like. I > would hack it this way>>> > > Under ltp-mmddyy create results directory, > In runalltests.sh remove the -l option and hardcode the logfile name in > the line that executes pan (this is towards the end of that file). > Remove $quite_mode and replace it with -q > Remove $pretty_print and replace it with -p, > Create your own custom command file with the list of tests you > want to execute and replace -f ${TMP}/alltests with -f > {PWD}/runtests/my_command_file.txt > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Ltp-list mailing list > Ltp...@li... > https://lists.sourceforge.net/lists/listinfo/ltp-list |
From: David A. M. <dm...@re...> - 2003-05-07 01:37:26
|
Thanks to all for responding. <some snipping> > I recently fixed some of them, sendmsg01 and all the syslog tests. You > will have these changes in the May 8th release of LTP. I have been using an older version of LTP. I will look for this release. > do you have BASH? if NOT then runalltests.sh script will have to be > modified to work with buybox (if you have busybox instead), coz there > are certain things in runnalltests.sh that busy box ash does not like. We do have bash on the systems I'm using now, so this should not be an issue. > Like others responded select the testcase you want to execute and create > a custom command file and use the -f option to execute your custom list > of testcases. This looks like the best option for us to use, including only the testcases which are applicable to the system being tested, which brings me to my next question: Are the system requirements for these test cases documented? I have not been able to find these (although I may have missed it). I was looking for things like how much memory is required, how much storage is required (for file system and file transfer tests), is a network connection required, is swap space required, etc., on a per test case basis. For example: 1) I ran 'sched' tests, which create 1000 threads, but received an error. I think I just ran out of resources on this system. If I change it to create 500 threads (or up to 890 threads) there is no error. 2) I ran 'syscalls' tests, and several tests failed (recvmsg01, sendfile02, etc.) because I had no network connection. I would like to run all tests possible on a given target system, but need a way to identify which are possible. This leads to my third question: Has there been any effort to create a "tiered" test configuration? We were considering a way to select specific tests based on the system's capabilities, i.e., amount of memory, disk capacity (if present), network connection present, etc. In other words, a way to create a custom command file based on the system configuration. Does this sound like something that would be useful to others? > These might not be all that you have to do... you will have to hack > further to fit your requirements. It will be nice if you can share ur > hacks with us. </some snipping> We would certainly like to contribute any changes that we make back to the project. Thank you, d.marlin |
From: Manoj I. <ma...@au...> - 2003-05-07 02:11:09
|
> > > Like others responded select the testcase you want to execute and create > > a custom command file and use the -f option to execute your custom list > > of testcases. > > This looks like the best option for us to use, including only the > testcases which are applicable to the system being tested, which brings > me to my next question: Are the system requirements for these test > cases documented? The best documentation available is 'code'. There is a small descriptions file that gives you some idea as to what these tests are supposed to do. Well it will be a great thing to have if someone came up with something like that.! :-) > I would like to run all tests possible on a given target system, but > need a way to identify which are possible. This leads to my third > question: Has there been any effort to create a "tiered" test > configuration? We were considering a way to select specific tests based > on the system's capabilities, i.e., amount of memory, disk capacity (if > present), network connection present, etc. In other words, a way to > create a custom command file based on the system configuration. Does > this sound like something that would be useful to others? In the next release of LTP there will be a menu driven scheme to selectively execute testcase, this menu is based on ncurses etc, so might not work well in your case. One solution to this could be to autoconf LTP. I am looking into it... But if you can come up with a scheme that will be cool! -- Manjo |
From: David A. M. <dm...@re...> - 2003-05-07 17:17:38
|
Manoj Iyer wrote: > > > > > > Like others responded select the testcase you want to execute and create > > > a custom command file and use the -f option to execute your custom list > > > of testcases. > > > > This looks like the best option for us to use, including only the > > testcases which are applicable to the system being tested, which brings > > me to my next question: Are the system requirements for these test > > cases documented? > > The best documentation available is 'code'. . . But there's a lot of code. ;-) Seriously, the code can tell you what each test does, but does not necessarily reveal resource requirements. Such things often require empirical data based on previous test runs. > . . . There is a small > descriptions file that gives you some idea as to what these tests are > supposed to do. Well it will be a great thing to have if someone came up > with something like that.! :-) As we run the tests and determine resource requirements and limits, we can document these and add them back to the descriptions file, if that would be appropriate. If others do the same, we can eventually develop a rather comprehensive list. > > > I would like to run all tests possible on a given target system, but > > need a way to identify which are possible. This leads to my third > > question: Has there been any effort to create a "tiered" test > > configuration? We were considering a way to select specific tests based > > on the system's capabilities, i.e., amount of memory, disk capacity (if > > present), network connection present, etc. In other words, a way to > > create a custom command file based on the system configuration. Does > > this sound like something that would be useful to others? > > In the next release of LTP there will be a menu driven scheme to > selectively execute testcase, this menu is based on ncurses etc, so > might not work well in your case. > > One solution to this could be to autoconf LTP. I am looking into it... > But if you can come up with a scheme that will be cool! autoconf can gather a lot of information about the 'build' system, but we are using a cross compiler for these systems, which limits the information available about the 'target' system (cannot determine amount of memory, storage, if network is present, etc.). The menu driven system would probably be appropriate since it could be run on the build system to configure the tests to be run on the target. I'll get the next release of LTP and see how the menu system works. If possible, we'll look into expanding this to configure tests based on the target system configuration (some set of criteria). Thanks again, d.marlin |
From: Dan K. <dk...@ix...> - 2003-05-07 18:11:46
|
David A. Marlin wrote: > autoconf can gather a lot of information about the 'build' system, but > we are using a cross compiler for these systems, which limits the > information available about the 'target' system (cannot determine amount > of memory, storage, if network is present, etc.). The menu driven > system would probably be appropriate since it could be run on the build > system to configure the tests to be run on the target. I face this problem as well. How about this: when starting, the test should declare how much RAM it needs. Then the framework could skip the test if that wasn't available. Extending the idea, tests could declare that they needed various other resources. I'd prefer it be a declarative statement embedded in the code -- that way it could be used both by source-scanning tools and by the runtime. No sense segregating it off in a text file to rot... - Dan |
From: E. <jo...@wo...> - 2003-05-07 18:20:48
|
On Wed, 7 May 2003 10:55:52 -0700, Dan Kegel wrote: > How about this: when starting, the test should declare how > much RAM it needs. Then the framework could skip the test > if that wasn't available. Extending the idea, tests could > declare that they needed various other resources. > > I'd prefer it be a declarative statement embedded in the > code -- that way it could be used both by source-scanning > tools and by the runtime. No sense segregating it off in > a text file to rot... If you want to do this, how about the following code? char *buf; buf = malloc(AS_MUCH_AS_I_NEED); if (!buf) exit(what_do_you_prefer ? 0 : 1); memset(buf, 0x88, AS_MUCH_AS_I_NEED); free(buf); This should also catch the case of overcommitted memory, the only problem is to figure out a sane value for AS_MUCH_AS_I_NEED. Jörn -- Everything should be made as simple as possible, but not simpler. -- Albert Einstein |
From: Dan K. <dk...@ix...> - 2003-05-07 18:29:22
|
Jörn Engel wrote: > On Wed, 7 May 2003 10:55:52 -0700, Dan Kegel wrote: > >>How about this: when starting, the test should declare how >>much RAM it needs. Then the framework could skip the test >>if that wasn't available. Extending the idea, tests could >>declare that they needed various other resources. >> >>I'd prefer it be a declarative statement embedded in the >>code -- that way it could be used both by source-scanning >>tools and by the runtime. No sense segregating it off in >>a text file to rot... > > > If you want to do this, how about the following code? > > char *buf; > buf = malloc(AS_MUCH_AS_I_NEED); > if (!buf) > exit(what_do_you_prefer ? 0 : 1); > memset(buf, 0x88, AS_MUCH_AS_I_NEED); > free(buf); > > This should also catch the case of overcommitted memory, the only > problem is to figure out a sane value for AS_MUCH_AS_I_NEED. Well... that's what should happen under the hood, but I'd prefer to put that in an ltp library, and make individual tests only do ltp_requires_ram(AS_MUCH_AS_I_NEED); That's a lot easier to grep for. - Dan |
From: Paul L. <pl...@li...> - 2003-05-07 19:23:15
|
On Wed, 2003-05-07 at 13:13, Dan Kegel wrote: > Well... that's what should happen under the hood, > but I'd prefer to put that in an ltp library, and > make individual tests only do > ltp_requires_ram(AS_MUCH_AS_I_NEED); > That's a lot easier to grep for. Your talking about doing this for every test program? (something like 1500 or 1800 last I heard of someone counting) What about the ones with variable memory requirements? |
From: Dan K. <dk...@ix...> - 2003-05-07 19:34:37
|
Paul Larson wrote: > On Wed, 2003-05-07 at 13:13, Dan Kegel wrote: > >>Well... that's what should happen under the hood, >>but I'd prefer to put that in an ltp library, and >>make individual tests only do >> ltp_requires_ram(AS_MUCH_AS_I_NEED); >>That's a lot easier to grep for. > > Your talking about doing this for every test program? Nope, only on an as-needed basis. So test programs that turn out not to run on a 32MB platform would get that little line added at the top of main. Should only be ten or twenty files affected. > What about the ones with variable memory requirements? Depends on why the memory requirements vary, but basically, only tests that can't run on 32MB machines would get tagged. Hmm, maybe what ltp_requires_ram(int x) should expand to is just a comparison with an environment variable LTP_HAS_RAM_MEGABYTES rather than actually trying to allocate the memory. Then tests that run properly only with 24MB of RAM would have the line ltp_requires_ram_megabytes(24); which would expand to something like if ($LTP_HAS_RAM_MEGABYTES < 24) don't run test, just report "not enough ram to run" - Dan |
From: Paul L. <pl...@li...> - 2003-05-07 21:25:16
|
On Wed, 2003-05-07 at 14:18, Dan Kegel wrote: > Nope, only on an as-needed basis. So test programs that > turn out not to run on a 32MB platform would get that > little line added at the top of main. Should only > be ten or twenty files affected. There's already a much easier way to handle all of this, as well as any problem related to having a unique arch/machine/setup. If a test is giving you problems because of your setup then just comment it out of the runtest/* file. -Paul Larson |
From: Dan K. <dk...@ix...> - 2003-05-07 22:05:36
|
Paul Larson wrote: > On Wed, 2003-05-07 at 14:18, Dan Kegel wrote: > >>Nope, only on an as-needed basis. So test programs that >>turn out not to run on a 32MB platform would get that >>little line added at the top of main. Should only >>be ten or twenty files affected. > > There's already a much easier way to handle all of this, as well as any > problem related to having a unique arch/machine/setup. If a test is > giving you problems because of your setup then just comment it out of > the runtest/* file. And that's what we currently do. What was being proposed was documenting the resource requirements of the various tests so people using constrained systems could know which tests to comment out. And I had the bright idea of doing the documentation in the code, so the commenting out could be done at runtime. - Dan |