From: Zhouping L. <zl...@re...> - 2011-08-24 11:57:18
|
There are several vm tunable files under /proc/sys/vm, I will compose some cases for testing the basic functions of them. This patch is adding the read/write/check functions to mem lib so that I can include them in my testcases. set_sys_tune(): set a long int value to a vm tunable file. get_sys_tune(): get a value from a vm tunable file. check_sys_tune(): check to confirm the value in tunable file meets our expectation. read_meminfo(): read the special value from /proc/meminfo. Since most of the values in vm tunable files contain only single integer, the functions I added only read/write long int values for now. For those files not storing single interger (e.g. lowmem_reserve_ratio), These functions will not be used in the case. Signed-off-by: Zhouping Liu <zl...@re...> --- testcases/kernel/mem/include/mem.h | 7 +++ testcases/kernel/mem/lib/mem.c | 78 +++++++++++++++++++++++++++++++++++- 2 files changed, 84 insertions(+), 1 deletions(-) -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-08-24 12:03:56
|
/proc/sys/vm/overcommit_memory contains a flag that control memory overcommitment. - When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory. - When this flag is 1, the kernel pretends there is always enough memory until it actually runs out. - When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory. This feature can be very useful because there are a lot of programs that malloc() huge amounts of memory "just-in-case" and don't use much of it. The default value is 0. memory allocation limit = swap_total + (mem_total * overcommit_ratio / 100) The case is designed to test the feature of this tunable file. Signed-off-by: Zhouping Liu <zl...@re...> --- testcases/kernel/mem/tunable/Makefile | 42 ++++ testcases/kernel/mem/tunable/overcommit_memory.c | 230 ++++++++++++++++++++++ 2 files changed, 272 insertions(+), 0 deletions(-) create mode 100644 testcases/kernel/mem/tunable/Makefile create mode 100644 testcases/kernel/mem/tunable/overcommit_memory.c -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-08-29 05:00:03
|
Hi, v1 -> v2: rewrite the overcommit_memory_test() functions. please review it, any comments are welcome. The case is designed to test overcommit_meory and overcommit_ratio tunable's performance under /proc/sys/vm /proc/sys/vm/overcommit_memory contains a flag that control memory overcommitment. - When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory. - When this flag is 1, the kernel pretends there is always enough memory until it actually runs out. - When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory. When overcommit_memoy is set to 2, the committed address space is not permitted to exceed swap plus overcommit_ratio percentage of physical RAM. the committed address limit can calculate by the following equation: commit_limit = swap_total + overcommit_ratio * physical_total it can read from /proc/meminfo:CommitLimit in the equation, swap_total and physical_total are constant, overcommit_ratio is a val, so we can tune commit_limit by overcommit_ratio. Signed-off-by: Zhouping Liu <zl...@re...> --- runtest/mm | 7 + testcases/kernel/mem/tunable/Makefile | 42 ++++ testcases/kernel/mem/tunable/overcommit_memory.c | 260 ++++++++++++++++++++++ 3 files changed, 309 insertions(+), 0 deletions(-) create mode 100644 testcases/kernel/mem/tunable/Makefile create mode 100644 testcases/kernel/mem/tunable/overcommit_memory.c -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-08-30 12:30:47
|
Hi, v2 -> v3: modified variables' type, updated overcommit_memory_test function, updated the comment lines. please review it again, any comments are welcome. There are two tunables overcommit_memory and overcommit_ratio under /proc/sys/vm/, which can control memory overcommitment. The overcommit_memory tunable contains a flag that enables memory overcommitment, it has three values: - When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory. - When this flag is 1, the kernel pretends there is always enough memory until it actually runs out. - When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory. The overcommit_ratio tunable defines the amount by which the kernel overextends its memory resources in the event that overcommit_memory is set to the value of 2. The value in this file represents a percentage added to the amount of actual RAM in a system when considering whether to grant a particular memory request. The general formula for this tunable is: CommitLimit = SwapTotal + MemTotal * overcommit_ratio CommitLimit, SwapTotal and MemTotal can read from /proc/meminfo. The program is designed to test the two tunables: When overcommit_memory = 0, allocatable memory can't overextends the amount of free memory. I choose the three cases: a. less than free_total: free_total / 2, alloc should pass. b. equal to free_total: free_total, alloc should pass. c. greater than free_total: free_total * 2, alloc should fail. When overcommit_memory = 1, it can alloc enough much memory, I choose the three cases: a. less than sum_total: sum_total / 2, alloc should pass. b. equal to sum_total: sum_total, alloc should pass. c. greater than sum_total: sum_total * 2, alloc should pass. *note: sum_total = SwapTotal + MemTotal When overcommit_memory = 2, allocatable memory = CommitLimit - Committed_As I define it as commit_left, also I choose three cases: a. less than commit_left: commit_left / 2, alloc should pass. b. greater than commit_left: commit_left * 2, alloc should fail. c. overcommit limit: CommitLimit, alloc should fail. *note: CommitLimit is the current overcommit limit. Committed_AS is the amount of memory that system has used. why I din't choose 'equal to commit_left' as a case, because in this case, the result is not fixed after I tested some times. References: - Documentation/sysctl/vm.txt - Documentation/vm/overcommit-accounting Signed-off-by: Zhouping Liu <zl...@re...> --- runtest/mm | 7 + testcases/kernel/mem/tunable/Makefile | 42 ++++ testcases/kernel/mem/tunable/overcommit_memory.c | 282 ++++++++++++++++++++++ 3 files changed, 331 insertions(+), 0 deletions(-) create mode 100644 testcases/kernel/mem/tunable/Makefile create mode 100644 testcases/kernel/mem/tunable/overcommit_memory.c -- Thanks, Zhouping Liu |
From: Cyril H. <ch...@su...> - 2011-09-08 12:50:41
|
Hi! > +static int heavy_malloc(unsigned long size) > +{ > + char *p; > + > + p = (char *)malloc(size); This cast to char is useless, please remove it. -- Cyril Hrubis ch...@su... |
From: Zhouping L. <zl...@re...> - 2011-09-09 07:14:46
|
Hi, > Hi! > > +static int heavy_malloc(unsigned long size) > > +{ > > + char *p; > > + > > + p = (char *)malloc(size); > > This cast to char is useless, please remove it. have updated it, please check the new patch in new mail. > > -- > Cyril Hrubis > ch...@su... -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-09-09 07:19:31
|
Hi, Cyril v3 -> v4: updated set_sys_tune() and get_sys_tune(), please comment. There are several vm tunable files under /proc/sys/vm, I will compose some cases for testing the basic functions of them. This patch is adding the read/write/check functions to mem lib so that I can include them in my testcases. set_sys_tune(): set a long long integer value to a vm tunable file. get_sys_tune(): get a long long integer value from a vm tunable file. check_sys_tune(): check to confirm the value in tunable file meets our expectation. read_meminfo(): read the special value from /proc/meminfo. Since most of the values in vm tunable files contain only single integer, the functions I added only read/write long long values for now. For those files not storing single interger (e.g. lowmem_reserve_ratio), these functions will not be used in the case. Signed-off-by: Zhouping Liu <zl...@re...> --- testcases/kernel/mem/include/mem.h | 7 +++ testcases/kernel/mem/lib/mem.c | 81 +++++++++++++++++++++++++++++++++++- 2 files changed, 87 insertions(+), 1 deletions(-) -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-09-09 07:21:25
|
sorry for forgetting attached the patch. > Hi, Cyril > v3 -> v4: updated set_sys_tune() and get_sys_tune(), please comment. > > There are several vm tunable files under /proc/sys/vm, I will compose > some cases for testing the basic functions of them. This patch is > adding > the read/write/check functions to mem lib so that I can include them > in > my testcases. > > set_sys_tune(): set a long long integer value to a vm tunable file. > get_sys_tune(): get a long long integer value from a vm tunable file. > check_sys_tune(): check to confirm the value in tunable file meets our > expectation. > read_meminfo(): read the special value from /proc/meminfo. > > Since most of the values in vm tunable files contain only single > integer, the functions I added only read/write long long values for > now. > For those files not storing single interger (e.g. > lowmem_reserve_ratio), > these functions will not be used in the case. > > Signed-off-by: Zhouping Liu <zl...@re...> > --- > testcases/kernel/mem/include/mem.h | 7 +++ > testcases/kernel/mem/lib/mem.c | 81 > +++++++++++++++++++++++++++++++++++- > 2 files changed, 87 insertions(+), 1 deletions(-) > -- > Thanks, > Zhouping Liu -- Thanks, Zhouping Liu |
From: Cyril H. <ch...@su...> - 2011-09-09 12:51:44
|
Hi! > diff --git a/testcases/kernel/mem/lib/mem.c b/testcases/kernel/mem/lib/mem.c > index f3983d4..64751fd 100644 > --- a/testcases/kernel/mem/lib/mem.c > +++ b/testcases/kernel/mem/lib/mem.c > @@ -77,7 +77,7 @@ void write_memcg(void) > snprintf(buf, BUFSIZ, "%d", getpid()); > if (write(fd, buf, strlen(buf)) != strlen(buf)) > tst_brkm(TBROK|TERRNO, cleanup, "write %s", buf); > - close(fd); > + close(fd); > } > > void write_cpusets(void) > @@ -712,3 +712,82 @@ void check_ksm_options(int *size, int *num, int *unit) > "process number cannot be less 3."); > } > } > + > +void set_sys_tune(char *sys_file, long long tune) > +{ > + int fd; > + char buf[BUFSIZ]; > + > + tst_resm(TINFO, "set %s to %lld", sys_file, tune); > + > + fd = open(sys_file, O_WRONLY); > + if (fd == -1) > + tst_brkm(TBROK|TERRNO, cleanup, "open %s", sys_file); > + if (snprintf(buf, BUFSIZ, "%lld", tune) < 0) > + tst_brkm(TBROK|TERRNO, cleanup, "snprintf"); > + if (write(fd, buf, strlen(buf)) == -1) > + tst_brkm(TBROK|TERRNO, cleanup, "write"); Please add the filename into the message that gets printed when write has failed too. And possibly enclose the %s with ' eg. tst_brkm(... "open '%s'", sys_file). > + close(fd); > +} > + > +long long get_sys_tune(char *sys_file) > +{ > + int fd; > + long long tune; > + char buf[BUFSIZ], *endptr; > + > + fd = open(sys_file, O_RDONLY); > + if (fd == -1) > + tst_brkm(TBROK|TERRNO, cleanup, "open %s", sys_file); > + if (read(fd, buf, BUFSIZ) < 0) > + tst_brkm(TBROK|TERRNO, cleanup, "read"); And here into the read branch too. > + close(fd); > + > + tune = strtoll(buf, &endptr, 10); > + if (tune == LLONG_MAX || tune == LLONG_MIN) > + tst_brkm(TBROK|TERRNO, cleanup, "strtoll"); > + if (endptr == buf || (*endptr != '\0' && *endptr != '\n')) > + tst_brkm(TBROK, cleanup, "Invalid tunable: %s", buf); > + > + return tune; > +} > + > +/* > + * the function is designed to make sure the value we get from > + * sys_file is equal to what we set last. > + */ > +void check_sys_tune(char *sys_file, long long tune) > +{ > + long long val; > + > + val = get_sys_tune(sys_file); > + if (val == tune) > + tst_resm(TINFO, "confirmed %s = %lld", sys_file, tune); > + else > + tst_brkm(TBROK, cleanup, "%s = %lld, is not %lld", > + sys_file, val, tune); > +} > + > +long long read_meminfo(char *item) > +{ > + FILE *fp; > + char line[BUFSIZ], buf[BUFSIZ]; > + long long val; > + > + fp = fopen(PATH_MEMINFO, "r"); > + if (fp == NULL) > + tst_brkm(TBROK|TERRNO, cleanup, "fopen %s", PATH_MEMINFO); > + while (fgets(line, BUFSIZ, fp) != NULL) { > + if (sscanf(line, "%s %lld ", buf, &val) == 2) > + if (strcmp(buf, item) == 0) { > + fclose(fp); > + return val; > + } > + continue; > + } > + fclose(fp); > + > + tst_brkm(TBROK, cleanup, "cannot find \"%s\" in %s", > + item, PATH_MEMINFO); > +} -- Cyril Hrubis ch...@su... |
From: Zhouping L. <zl...@re...> - 2011-09-09 07:26:54
|
hi, Cyril v3 -> v4: moved useless code. please comment. There are two tunables overcommit_memory and overcommit_ratio under /proc/sys/vm/, which can control memory overcommitment. The overcommit_memory contains a flag that enables memory overcommitment, it has three values: - When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory. - When this flag is 1, the kernel pretends there is always enough memory until it actually runs out. - When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory. The overcommit_ratio tunable defines the amount by which the kernel overextends its memory resources in the event that overcommit_memory is set to the value of 2. The value in this file represents a percentage added to the amount of actual RAM in a system when considering whether to grant a particular memory request. The general formula for this tunable is: CommitLimit = SwapTotal + MemTotal * overcommit_ratio CommitLimit, SwapTotal and MemTotal can read from /proc/meminfo. The program is designed to test the two tunables: When overcommit_memory = 0, allocatable memory can't overextends the amount of free memory. I choose the three cases: a. less than free_total: free_total / 2, alloc should pass. b. equal to free_total: free_total, alloc should pass. c. greater than free_total: free_total * 2, alloc should fail. When overcommit_memory = 1, it can alloc enough much memory, I choose the three cases: a. less than sum_total: sum_total / 2, alloc should pass. b. equal to sum_total: sum_total, alloc should pass. c. greater than sum_total: sum_total * 2, alloc should pass. *note: sum_total = SwapTotal + MemTotal When overcommit_memory = 2, allocatable memory = CommitLimit - Committed_As I define it as commit_left, also I choose three cases: a. less than commit_left: commit_left / 2, alloc should pass. b. greater than commit_left: commit_left * 2, alloc should fail. c. overcommit limit: CommitLimit, alloc should fail. *note: CommitLimit is the current overcommit limit. Committed_AS is the amount of memory that system has used. why I din't choose 'equal to commit_left' as a case, because in this case, the result is not fixed after I tested some times. References: - Documentation/sysctl/vm.txt - Documentation/vm/overcommit-accounting Signed-off-by: Zhouping Liu <zl...@re...> --- runtest/mm | 7 + testcases/kernel/mem/tunable/Makefile | 42 ++++ testcases/kernel/mem/tunable/overcommit_memory.c | 282 ++++++++++++++++++++++ 3 files changed, 331 insertions(+), 0 deletions(-) create mode 100644 testcases/kernel/mem/tunable/Makefile create mode 100644 testcases/kernel/mem/tunable/overcommit_memory.c -- Thanks, Zhouping Liu |
From: Cyril H. <ch...@su...> - 2011-09-09 12:57:59
|
Hi! > @@ -0,0 +1,282 @@ > +/* > + * Description: > + * > + * There are two tunables overcommit_memory and overcommit_ratio under > + * /proc/sys/vm/, which can control memory overcommitment. > + * > + * The overcommit_memory contains a flag that enables memory > + * overcommitment, it has three values: > + * - When this flag is 0, the kernel attempts to estimate the amount > + * of free memory left when userspace requests more memory. > + * - When this flag is 1, the kernel pretends there is always enough > + * memory until it actually runs out. > + * - When this flag is 2, the kernel uses a "never overcommit" policy > + * that attempts to prevent any overcommit of memory. > + * > + * The overcommit_ratio tunable defines the amount by which the kernel > + * overextends its memory resources in the event that overcommit_memory > + * is set to the value of 2. The value in this file represents a > + * percentage added to the amount of actual RAM in a system when > + * considering whether to grant a particular memory request. > + * The general formula for this tunable is: > + * CommitLimit = SwapTotal + MemTotal * overcommit_ratio > + * CommitLimit, SwapTotal and MemTotal can read from /proc/meminfo. > + * > + * The program is designed to test the two tunables: > + * > + * When overcommit_memory = 0, allocatable memory can't overextends > + * the amount of free memory. I choose the three cases: > + * a. less than free_total: free_total / 2, alloc should pass. > + * b. equal to free_total: free_total, alloc should pass. > + * c. greater than free_total: free_total * 2, alloc should fail. > + * > + * When overcommit_memory = 1, it can alloc enough much memory, I > + * choose the three cases: > + * a. less than sum_total: sum_total / 2, alloc should pass > + * b. equal to sum_total: sum_total, alloc should pass > + * c. greater than sum_total: sum_total * 2, alloc should pass > + * *note: sum_total = SwapTotal + MemTotal > + * > + * When overcommit_memory = 2, > + * allocatable memory = CommitLimit - Committed_AS > + * I define it as commit_left, also I choose three cases: > + * a. less than commit_left: commit_left / 2, alloc should pass > + * b. greater than commit_left: commit_left * 2, alloc should fail > + * c. overcommit limit: CommitLimit, alloc should fail > + * *note: CommitLimit is the current overcommit limit. > + * Committed_AS is the amount of memory that system has used. > + * why I din't choose 'equal to commit_left' as a case, because in > + * this case, the result is not fixed after I tested some times. > + * > + * References: > + * - Documentation/sysctl/vm.txt > + * - Documentation/vm/overcommit-accounting > + * > + * ******************************************************************** > + * Copyright (C) 2011 Red Hat, Inc. > + * > + * This program is free software; you can redistribute it and/or > + * modify it under the terms of version 2 of the GNU General Public > + * License as published by the Free Software Foundation. > + * > + * This program is distributed in the hope that it would be useful, > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > + * > + * Further, this software is distributed without any warranty that it > + * is free of the rightful claim of any third person regarding > + * infringement or the like. Any license provided herein, whether > + * implied or otherwise, applies only to this software file. Patent > + * licenses, if any, provided herein do not apply to combinations of > + * this program with other software, or any other product whatsoever. > + * > + * You should have received a copy of the GNU General Public License > + * along with this program; if not, write the Free Software > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > + * 02110-1301, USA. > + * ******************************************************************** > + */ Maybe the licese should be on the top of the file followed with the test description (at least most of the tests are organized this way). > +#include <stdio.h> > +#include <stdlib.h> > +#include <errno.h> > +#include <linux/mman.h> > +#include "test.h" > +#include "usctest.h" > +#include "../include/mem.h" > + > +#define FILE_OVER_MEM PATH_SYSVM "overcommit_memory" > +#define FILE_OVER_RATIO PATH_SYSVM "overcommit_ratio" > +#define DEFAULT_OVER_RATIO 50LL > +#define EXPECT_PASS 0 > +#define EXPECT_FAIL 1 > + > +char *TCID = "overcommit_memory"; > +static long long old_overcommit_memory; > +static long long old_overcommit_ratio; > +static long long overcommit_ratio; > +static long long sum_total; > +static long long free_total; > +static long long commit_limit; > +static long long commit_left; > +static int R_flag; > +static char *R_opt; > +option_t options[] = { > + { "R:", &R_flag, &R_opt }, > + { NULL, NULL, NULL } > +}; > + > + > +static void overcommit_memory_test(void); > +static int heavy_malloc(long long size); > +static void alloc_and_check(long long size, int expect_result); > +static void usage(void); > +static void update_mem(void); > + > +int main(int argc, char *argv[]) > +{ > + char *msg; > + int lc; > + > + overcommit_ratio = DEFAULT_OVER_RATIO; > + > + msg = parse_opts(argc, argv, options, &usage); > + if (msg != NULL) { > + tst_brkm(TBROK, NULL, "OPTION PARSING ERROR - %s", msg); > + tst_exit(); As said before, please drop the tst_exit() here, it's not needed anymore after tst_brkm(). > + } > + > + if (R_flag) { > + overcommit_ratio = atoll(R_opt); > + if (overcommit_ratio < 0) { > + tst_brkm(TBROK, NULL, "-R option is required a" > + "Non-negative"); > + tst_exit(); Here as well. > + } > + } > + > + setup(); > + > + for (lc = 0; TEST_LOOPING(lc); lc++) { > + Tst_count = 0; > + > + overcommit_memory_test(); > + } > + > + cleanup(); > + > + tst_exit(); > +} -- Cyril Hrubis ch...@su... |
From: Zhouping L. <zl...@re...> - 2011-09-09 17:11:40
|
Hi, > Hi! > > @@ -0,0 +1,282 @@ > > +/* > > + * Description: > > + * > > + * There are two tunables overcommit_memory and overcommit_ratio > > under > > + * /proc/sys/vm/, which can control memory overcommitment. > > + * > > + * The overcommit_memory contains a flag that enables memory > > + * overcommitment, it has three values: > > + * - When this flag is 0, the kernel attempts to estimate the > > amount > > + * of free memory left when userspace requests more memory. > > + * - When this flag is 1, the kernel pretends there is always > > enough > > + * memory until it actually runs out. > > + * - When this flag is 2, the kernel uses a "never overcommit" > > policy > > + * that attempts to prevent any overcommit of memory. > > + * > > + * The overcommit_ratio tunable defines the amount by which the > > kernel > > + * overextends its memory resources in the event that > > overcommit_memory > > + * is set to the value of 2. The value in this file represents a > > + * percentage added to the amount of actual RAM in a system when > > + * considering whether to grant a particular memory request. > > + * The general formula for this tunable is: > > + * CommitLimit = SwapTotal + MemTotal * overcommit_ratio > > + * CommitLimit, SwapTotal and MemTotal can read from /proc/meminfo. > > + * > > + * The program is designed to test the two tunables: > > + * > > + * When overcommit_memory = 0, allocatable memory can't overextends > > + * the amount of free memory. I choose the three cases: > > + * a. less than free_total: free_total / 2, alloc should pass. > > + * b. equal to free_total: free_total, alloc should pass. > > + * c. greater than free_total: free_total * 2, alloc should fail. > > + * > > + * When overcommit_memory = 1, it can alloc enough much memory, I > > + * choose the three cases: > > + * a. less than sum_total: sum_total / 2, alloc should pass > > + * b. equal to sum_total: sum_total, alloc should pass > > + * c. greater than sum_total: sum_total * 2, alloc should pass > > + * *note: sum_total = SwapTotal + MemTotal > > + * > > + * When overcommit_memory = 2, > > + * allocatable memory = CommitLimit - Committed_AS > > + * I define it as commit_left, also I choose three cases: > > + * a. less than commit_left: commit_left / 2, alloc should pass > > + * b. greater than commit_left: commit_left * 2, alloc should fail > > + * c. overcommit limit: CommitLimit, alloc should fail > > + * *note: CommitLimit is the current overcommit limit. > > + * Committed_AS is the amount of memory that system has used. > > + * why I din't choose 'equal to commit_left' as a case, because in > > + * this case, the result is not fixed after I tested some times. > > + * > > + * References: > > + * - Documentation/sysctl/vm.txt > > + * - Documentation/vm/overcommit-accounting > > + * > > + * > > ******************************************************************** > > + * Copyright (C) 2011 Red Hat, Inc. > > + * > > + * This program is free software; you can redistribute it and/or > > + * modify it under the terms of version 2 of the GNU General Public > > + * License as published by the Free Software Foundation. > > + * > > + * This program is distributed in the hope that it would be useful, > > + * but WITHOUT ANY WARRANTY; without even the implied warranty of > > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. > > + * > > + * Further, this software is distributed without any warranty that > > it > > + * is free of the rightful claim of any third person regarding > > + * infringement or the like. Any license provided herein, whether > > + * implied or otherwise, applies only to this software file. Patent > > + * licenses, if any, provided herein do not apply to combinations > > of > > + * this program with other software, or any other product > > whatsoever. > > + * > > + * You should have received a copy of the GNU General Public > > License > > + * along with this program; if not, write the Free Software > > + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA > > + * 02110-1301, USA. > > + * > > ******************************************************************** > > + */ > > > Maybe the licese should be on the top of the file followed with the > test > description (at least most of the tests are organized this way). Okay, I will move the license to the top. > > > +#include <stdio.h> > > +#include <stdlib.h> > > +#include <errno.h> > > +#include <linux/mman.h> > > +#include "test.h" > > +#include "usctest.h" > > +#include "../include/mem.h" > > + > > +#define FILE_OVER_MEM PATH_SYSVM "overcommit_memory" > > +#define FILE_OVER_RATIO PATH_SYSVM "overcommit_ratio" > > +#define DEFAULT_OVER_RATIO 50LL > > +#define EXPECT_PASS 0 > > +#define EXPECT_FAIL 1 > > + > > +char *TCID = "overcommit_memory"; > > +static long long old_overcommit_memory; > > +static long long old_overcommit_ratio; > > +static long long overcommit_ratio; > > +static long long sum_total; > > +static long long free_total; > > +static long long commit_limit; > > +static long long commit_left; > > +static int R_flag; > > +static char *R_opt; > > +option_t options[] = { > > + { "R:", &R_flag, &R_opt }, > > + { NULL, NULL, NULL } > > +}; > > + > > + > > +static void overcommit_memory_test(void); > > +static int heavy_malloc(long long size); > > +static void alloc_and_check(long long size, int expect_result); > > +static void usage(void); > > +static void update_mem(void); > > + > > +int main(int argc, char *argv[]) > > +{ > > + char *msg; > > + int lc; > > + > > + overcommit_ratio = DEFAULT_OVER_RATIO; > > + > > + msg = parse_opts(argc, argv, options, &usage); > > + if (msg != NULL) { > > + tst_brkm(TBROK, NULL, "OPTION PARSING ERROR - %s", msg); > > + tst_exit(); > > As said before, please drop the tst_exit() here, it's not needed > anymore after tst_brkm(). Okay, I'll delete tst_exit. > > > + } > > + > > + if (R_flag) { > > + overcommit_ratio = atoll(R_opt); > > + if (overcommit_ratio < 0) { > > + tst_brkm(TBROK, NULL, "-R option is required a" > > + "Non-negative"); > > + tst_exit(); > > Here as well. also as the above. > > > + } > > + } > > + > > + setup(); > > + > > + for (lc = 0; TEST_LOOPING(lc); lc++) { > > + Tst_count = 0; > > + > > + overcommit_memory_test(); > > + } > > + > > + cleanup(); > > + > > + tst_exit(); > > +} > > -- > Cyril Hrubis > ch...@su... -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-09-09 17:14:45
|
Hi, Cyril v4 -> v5: updated message. please comment it. There are several vm tunable files under /proc/sys/vm, I will compose some cases for testing the basic functions of them. This patch is adding the read/write/check functions to mem lib so that I can include them in my testcases. set_sys_tune(): set a long long integer value to a vm tunable file. get_sys_tune(): get a long long integer value from a vm tunable file. check_sys_tune(): check to confirm the value in tunable file meets our expectation. read_meminfo(): read the special value from /proc/meminfo. Since most of the values in vm tunable files contain only single integer, the functions I added only read/write long long values for now. For those files not storing single interger (e.g. lowmem_reserve_ratio), these functions will not be used in the case. Signed-off-by: Zhouping Liu <zl...@re...> --- testcases/kernel/mem/include/mem.h | 7 +++ testcases/kernel/mem/lib/mem.c | 81 +++++++++++++++++++++++++++++++++++- 2 files changed, 87 insertions(+), 1 deletions(-) -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-10-09 07:37:23
|
Hi, Cyril and all can anyone comment for the patch? Thanks, Zhouping ----- Original Message ----- > From: "Zhouping Liu" <zl...@re...> > To: "Cyril Hrubis" <ch...@su...> > Cc: cz...@re..., ltp...@li... > Sent: Saturday, September 10, 2011 1:14:39 AM > Subject: [LTP][PATCH v5 1/2] mem/tunable: new testcase about overcommiting memory > > Hi, Cyril > v4 -> v5: updated message. please comment it. > > There are several vm tunable files under /proc/sys/vm, I will compose > some cases for testing the basic functions of them. This patch is > adding > the read/write/check functions to mem lib so that I can include them > in > my testcases. > > set_sys_tune(): set a long long integer value to a vm tunable file. > get_sys_tune(): get a long long integer value from a vm tunable > file. > check_sys_tune(): check to confirm the value in tunable file meets > our > expectation. > read_meminfo(): read the special value from /proc/meminfo. > > Since most of the values in vm tunable files contain only single > integer, the functions I added only read/write long long values for > now. > For those files not storing single interger (e.g. > lowmem_reserve_ratio), > these functions will not be used in the case. > > Signed-off-by: Zhouping Liu <zl...@re...> > --- > testcases/kernel/mem/include/mem.h | 7 +++ > testcases/kernel/mem/lib/mem.c | 81 > +++++++++++++++++++++++++++++++++++- > 2 files changed, 87 insertions(+), 1 deletions(-) > > -- > Thanks, > Zhouping Liu > |
From: Zhouping L. <zl...@re...> - 2011-09-09 17:19:02
|
hi, Cyril v4 -> v5: delete the useless code: tst_exit(). please comment it. There are two tunables overcommit_memory and overcommit_ratio under /proc/sys/vm/, which can control memory overcommitment. The overcommit_memory contains a flag that enables memory overcommitment, it has three values: - When this flag is 0, the kernel attempts to estimate the amount of free memory left when userspace requests more memory. - When this flag is 1, the kernel pretends there is always enough memory until it actually runs out. - When this flag is 2, the kernel uses a "never overcommit" policy that attempts to prevent any overcommit of memory. The overcommit_ratio tunable defines the amount by which the kernel overextends its memory resources in the event that overcommit_memory is set to the value of 2. The value in this file represents a percentage added to the amount of actual RAM in a system when considering whether to grant a particular memory request. The general formula for this tunable is: CommitLimit = SwapTotal + MemTotal * overcommit_ratio CommitLimit, SwapTotal and MemTotal can read from /proc/meminfo. The program is designed to test the two tunables: When overcommit_memory = 0, allocatable memory can't overextends the amount of free memory. I choose the three cases: a. less than free_total: free_total / 2, alloc should pass. b. equal to free_total: free_total, alloc should pass. c. greater than free_total: free_total * 2, alloc should fail. When overcommit_memory = 1, it can alloc enough much memory, I choose the three cases: a. less than sum_total: sum_total / 2, alloc should pass. b. equal to sum_total: sum_total, alloc should pass. c. greater than sum_total: sum_total * 2, alloc should pass. *note: sum_total = SwapTotal + MemTotal When overcommit_memory = 2, allocatable memory = CommitLimit - Committed_As I define it as commit_left, also I choose three cases: a. less than commit_left: commit_left / 2, alloc should pass. b. greater than commit_left: commit_left * 2, alloc should fail. c. overcommit limit: CommitLimit, alloc should fail. *note: CommitLimit is the current overcommit limit. Committed_AS is the amount of memory that system has used. why I din't choose 'equal to commit_left' as a case, because in this case, the result is not fixed after I tested some times. References: - Documentation/sysctl/vm.txt - Documentation/vm/overcommit-accounting Signed-off-by: Zhouping Liu <zl...@re...> --- runtest/mm | 7 + testcases/kernel/mem/tunable/Makefile | 42 ++++ testcases/kernel/mem/tunable/overcommit_memory.c | 278 ++++++++++++++++++++++ 3 files changed, 327 insertions(+), 0 deletions(-) create mode 100644 testcases/kernel/mem/tunable/Makefile create mode 100644 testcases/kernel/mem/tunable/overcommit_memory.c -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-10-09 07:38:24
|
Hi, Cyril and all can anyone comment for the patch? Thanks, Zhouping ----- Original Message ----- > From: "Zhouping Liu" <zl...@re...> > To: ch...@su... > Cc: cz...@re..., ltp...@li... > Sent: Saturday, September 10, 2011 1:18:54 AM > Subject: [LTP][PATCH v5 2/2] mem/tunable: new testcase about overcommiting memory > > hi, Cyril > v4 -> v5: delete the useless code: tst_exit(). > please comment it. > > There are two tunables overcommit_memory and overcommit_ratio under > /proc/sys/vm/, which can control memory overcommitment. > > The overcommit_memory contains a flag that enables memory > overcommitment, it has three values: > - When this flag is 0, the kernel attempts to estimate the amount > of free memory left when userspace requests more memory. > - When this flag is 1, the kernel pretends there is always enough > memory until it actually runs out. > - When this flag is 2, the kernel uses a "never overcommit" policy > that attempts to prevent any overcommit of memory. > > The overcommit_ratio tunable defines the amount by which the kernel > overextends its memory resources in the event that overcommit_memory > is set to the value of 2. The value in this file represents a > percentage added to the amount of actual RAM in a system when > considering whether to grant a particular memory request. > The general formula for this tunable is: > CommitLimit = SwapTotal + MemTotal * overcommit_ratio > CommitLimit, SwapTotal and MemTotal can read from /proc/meminfo. > > The program is designed to test the two tunables: > > When overcommit_memory = 0, allocatable memory can't overextends > the amount of free memory. I choose the three cases: > a. less than free_total: free_total / 2, alloc should pass. > b. equal to free_total: free_total, alloc should pass. > c. greater than free_total: free_total * 2, alloc should fail. > > When overcommit_memory = 1, it can alloc enough much memory, I > choose the three cases: > a. less than sum_total: sum_total / 2, alloc should pass. > b. equal to sum_total: sum_total, alloc should pass. > c. greater than sum_total: sum_total * 2, alloc should pass. > *note: sum_total = SwapTotal + MemTotal > > When overcommit_memory = 2, > allocatable memory = CommitLimit - Committed_As > I define it as commit_left, also I choose three cases: > a. less than commit_left: commit_left / 2, alloc should pass. > b. greater than commit_left: commit_left * 2, alloc should fail. > c. overcommit limit: CommitLimit, alloc should fail. > *note: CommitLimit is the current overcommit limit. > Committed_AS is the amount of memory that system has used. > why I din't choose 'equal to commit_left' as a case, because in > this case, the result is not fixed after I tested some times. > > References: > - Documentation/sysctl/vm.txt > - Documentation/vm/overcommit-accounting > > Signed-off-by: Zhouping Liu <zl...@re...> > --- > runtest/mm | 7 + > testcases/kernel/mem/tunable/Makefile | 42 ++++ > testcases/kernel/mem/tunable/overcommit_memory.c | 278 > ++++++++++++++++++++++ > 3 files changed, 327 insertions(+), 0 deletions(-) > create mode 100644 testcases/kernel/mem/tunable/Makefile > create mode 100644 testcases/kernel/mem/tunable/overcommit_memory.c > > -- > Thanks, > Zhouping Liu > |
From: Zhouping L. <zl...@re...> - 2011-08-29 04:55:43
|
Hi, v1->v2: update the message info please review it, any comments are welcome. There are several vm tunable files under /proc/sys/vm, I will compose some cases for testing the basic functions of them. This patch is adding the read/write/check functions to mem lib so that I can include them in my testcases. set_sys_tune(): set a long int value to a vm tunable file. get_sys_tune(): get a value from a vm tunable file. check_sys_tune(): check to confirm the value in tunable file meets our expectation. read_meminfo(): read the special value from /proc/meminfo. Since most of the values in vm tunable files contain only single integer, the functions I added only read/write long int values for now. For those files not storing single interger (e.g. lowmem_reserve_ratio), These functions will not be used in the case. Signed-off-by: Zhouping Liu <zl...@re...> --- testcases/kernel/mem/include/mem.h | 7 +++ testcases/kernel/mem/lib/mem.c | 80 +++++++++++++++++++++++++++++++++++- 2 files changed, 86 insertions(+), 1 deletions(-) -- Thanks, Zhouping Liu |
From: Zhouping L. <zl...@re...> - 2011-08-30 12:22:14
|
Hi, v2 -> v3: modified the variable's type. please review it again, any comments are welcome. There are several vm tunable files under /proc/sys/vm, I will compose some cases for testing the basic functions of them. This patch is adding the read/write/check functions to mem lib so that I can include them in my testcases. set_sys_tune(): set a long long value to a vm tunable file. get_sys_tune(): get a value from a vm tunable file. check_sys_tune(): check to confirm the value in tunable file meets our expectation. read_meminfo(): read the special value from /proc/meminfo. Since most of the values in vm tunable files contain only single integer, the functions I added only read/write long long values for now. For those files not storing single interger (e.g. lowmem_reserve_ratio), these functions will not be used in the case. Signed-off-by: Zhouping Liu <zl...@re...> --- testcases/kernel/mem/include/mem.h | 7 +++ testcases/kernel/mem/lib/mem.c | 80 +++++++++++++++++++++++++++++++++++- 2 files changed, 86 insertions(+), 1 deletions(-) -- Thanks, Zhouping Liu |
From: Cyril H. <ch...@su...> - 2011-09-08 12:42:56
|
Hi! > +void set_sys_tune(char *sys_file, long tune) > +{ > + int fd; > + char buf[BUFSIZ]; > + > + fd = open(sys_file, O_WRONLY); > + if (fd == -1) > + tst_brkm(TBROK|TERRNO, cleanup, "open"); > + sprintf(buf, "%ld", tune); > + if (write(fd, buf, strlen(buf)) == -1) > + tst_brkm(TBROK|TERRNO, cleanup, "write"); > + close(fd); > +} I would use snprintf() here, even if you are sure the result fits into the buffer. Also adding name of the file to the tst_brkm() report would be better. So you would get: open 'filename': no such file or directory rather than: open : no such file or directory > +long get_sys_tune(char *sys_file) > +{ > + int fd; > + long tune; > + char buf[BUFSIZ], *endptr; > + > + fd = open(sys_file, O_RDONLY); > + if (fd == -1) > + tst_brkm(TBROK|TERRNO, cleanup, "open"); > + if (read(fd, buf, BUFSIZ) < 0) > + tst_brkm(TBROK|TERRNO, cleanup, "read"); Here as well. > + close(fd); > + > + tune = strtol(buf, &endptr, 10); > + if (((tune == LONG_MAX || tune == LONG_MIN) && errno == ERANGE) || > + (errno != 0 && tune == 0)) > + tst_brkm(TBROK|TERRNO, cleanup, "strtol"); I think you should clear errno before calling the strtol, otherwise you could get invalid conversion for string "0" when errno was set by some previous system call. > + if (endptr == buf || (*endptr != '\0' && *endptr != '\n')) > + tst_brkm(TBROK, cleanup, "Invalid number parameter: %s", buf); > + > + return tune; > +} > > +/* > + * the function is designed to make sure the value we get from > + * sys_file is equal to what we set last. > + */ > +void check_sys_tune(char *sys_file, long tune) > +{ > + long val; > + > + val = get_sys_tune(sys_file); > + if (val == tune) > + tst_resm(TINFO, "val in %s = %ld", sys_file, tune); > + else > + tst_brkm(TBROK, cleanup, "val in %s = %ld, not %ld", > + sys_file, val, tune); > +} > + > +unsigned long read_meminfo(char *item) > +{ > + FILE *fp; > + char line[BUFSIZ], buf[BUFSIZ]; > + unsigned long val; > + > + fp = fopen(PATH_MEMINFO, "r"); > + if (fp == NULL) > + tst_brkm(TBROK|TERRNO, cleanup, "fopen %s", PATH_MEMINFO); > + while (fgets(line, BUFSIZ, fp) != NULL) { > + if (sscanf(line, "%s %lu ", buf, &val) == 2) Using sscanf() to fill string buffer is potentialy dangerous. You could either to use GNU extension to hardcode maximal buffer length into the format string or to go trough the string while you find whitespace (see isspace()). > + if (strcmp(buf, item) == 0) { > + fclose(fp); > + return val; > + } > + continue; > + } > + fclose(fp); > + > + tst_brkm(TBROK, cleanup, "cannot find \"%s\" in %s", > + item, PATH_MEMINFO); > +} -- Cyril Hrubis ch...@su... |
From: Zhouping L. <zl...@re...> - 2011-09-09 07:13:29
|
hi, comment in-line. > Hi! > > +void set_sys_tune(char *sys_file, long tune) > > +{ > > + int fd; > > + char buf[BUFSIZ]; > > + > > + fd = open(sys_file, O_WRONLY); > > + if (fd == -1) > > + tst_brkm(TBROK|TERRNO, cleanup, "open"); > > + sprintf(buf, "%ld", tune); > > + if (write(fd, buf, strlen(buf)) == -1) > > + tst_brkm(TBROK|TERRNO, cleanup, "write"); > > + close(fd); > > +} > > I would use snprintf() here, even if you are sure the result fits into > the buffer. > > Also adding name of the file to the tst_brkm() report would be better. > > So you would get: > > open 'filename': no such file or directory > > rather than: > > open : no such file or directory have updated it, please check the next email. > > > > +long get_sys_tune(char *sys_file) > > +{ > > + int fd; > > + long tune; > > + char buf[BUFSIZ], *endptr; > > + > > + fd = open(sys_file, O_RDONLY); > > + if (fd == -1) > > + tst_brkm(TBROK|TERRNO, cleanup, "open"); > > + if (read(fd, buf, BUFSIZ) < 0) > > + tst_brkm(TBROK|TERRNO, cleanup, "read"); > > Here as well. > > > + close(fd); > > + > > + tune = strtol(buf, &endptr, 10); > > + if (((tune == LONG_MAX || tune == LONG_MIN) && errno == ERANGE) || > > + (errno != 0 && tune == 0)) > > + tst_brkm(TBROK|TERRNO, cleanup, "strtol"); > > I think you should clear errno before calling the strtol, otherwise > you > could get invalid conversion for string "0" when errno was set by some > previous system call. > I removed the errno in updated patch. > > + if (endptr == buf || (*endptr != '\0' && *endptr != '\n')) > > + tst_brkm(TBROK, cleanup, "Invalid number parameter: %s", buf); > > + > > + return tune; > > +} > > > > +/* > > + * the function is designed to make sure the value we get from > > + * sys_file is equal to what we set last. > > + */ > > +void check_sys_tune(char *sys_file, long tune) > > +{ > > + long val; > > + > > + val = get_sys_tune(sys_file); > > + if (val == tune) > > + tst_resm(TINFO, "val in %s = %ld", sys_file, tune); > > + else > > + tst_brkm(TBROK, cleanup, "val in %s = %ld, not %ld", > > + sys_file, val, tune); > > +} > > + > > +unsigned long read_meminfo(char *item) > > +{ > > + FILE *fp; > > + char line[BUFSIZ], buf[BUFSIZ]; > > + unsigned long val; > > + > > + fp = fopen(PATH_MEMINFO, "r"); > > + if (fp == NULL) > > + tst_brkm(TBROK|TERRNO, cleanup, "fopen %s", PATH_MEMINFO); > > + while (fgets(line, BUFSIZ, fp) != NULL) { > > + if (sscanf(line, "%s %lu ", buf, &val) == 2) > > Using sscanf() to fill string buffer is potentialy dangerous. You > could > either to use GNU extension to hardcode maximal buffer length into the > format string or to go trough the string while you find whitespace > (see > isspace()). I'm not sure how to changed it, could you give me more details about how to avoid Using sscanf(). and always I think Using sscanf() can solve this question. > > > + if (strcmp(buf, item) == 0) { > > + fclose(fp); > > + return val; > > + } > > + continue; > > + } > > + fclose(fp); > > + > > + tst_brkm(TBROK, cleanup, "cannot find \"%s\" in %s", > > + item, PATH_MEMINFO); > > +} > > > -- > Cyril Hrubis > ch...@su... -- Thanks, Zhouping Liu |
From: Cyril H. <ch...@su...> - 2011-09-09 12:44:24
|
Hi! > > Using sscanf() to fill string buffer is potentialy dangerous. You > > could > > either to use GNU extension to hardcode maximal buffer length into the > > format string or to go trough the string while you find whitespace > > (see > > isspace()). > I'm not sure how to changed it, could you give me more details about how to avoid > Using sscanf(). and always I think Using sscanf() can solve this question. Well, the problem with sscanf("%s", buf) is that the sscanf() don't know how long is the buffer and that may lead to buffer overflow and strange and unexpected behavior. Generally there are two ways how to fix it. Either use GNU extension to hardcode the maximal length of the buffer into the string, eg. sscanf("%64s, buf); would say sscanf to use at most 64 bytes of the buffer. Or go trough the string with while cycle till you find first occurence of whitespace. -- Cyril Hrubis ch...@su... |
From: Zhouping L. <zl...@re...> - 2011-09-09 17:32:14
|
Hi, Cyril > Hi! > > > Using sscanf() to fill string buffer is potentialy dangerous. You > > > could > > > either to use GNU extension to hardcode maximal buffer length into > > > the > > > format string or to go trough the string while you find whitespace > > > (see > > > isspace()). > > I'm not sure how to changed it, could you give me more details about > > how to avoid > > Using sscanf(). and always I think Using sscanf() can solve this > > question. > > Well, the problem with sscanf("%s", buf) is that the sscanf() don't > know > how long is the buffer and that may lead to buffer overflow and > strange > and unexpected behavior. Hmm, +long long read_meminfo(char *item) +{ + FILE *fp; + char line[BUFSIZ], buf[BUFSIZ]; + long long val; + + fp = fopen(PATH_MEMINFO, "r"); + if (fp == NULL) + tst_brkm(TBROK|TERRNO, cleanup, "fopen %s", PATH_MEMINFO); + while (fgets(line, BUFSIZ, fp) != NULL) { + if (sscanf(line, "%s %lld", buf, &val) == 2) + if (strcmp(buf, item) == 0) { + fclose(fp); + return val; in this case, read_meminfo can only read /proc/meminfo, fgets() read a line from /proc/meminfo one time, and the content of the line would always be like as this: MemTotal: 8080956 kB and sscanf(line, "%s %lld", buf, &val) can match the item. so I think it is enough safe to get it. PTY: it's a little hard for me to use GNU extension to hardcode the maximal length. > > Generally there are two ways how to fix it. Either use GNU extension > to > hardcode the maximal length of the buffer into the string, eg. > sscanf("%64s, buf); would say sscanf to use at most 64 bytes of the > buffer. Or go trough the string with while cycle till you find first > occurence of whitespace. > > -- > Cyril Hrubis > ch...@su... -- Thanks, Zhouping Liu |