From: Cliff W. <cl...@os...> - 2002-08-09 15:23:33
|
> Cliff White wrote: > > > > As promised, i'm running 2.5.30 on the Very Tiny Two-Way. > > (2 CPU x 256MB x 2 cheep IDE drives) > > Caveats all over this one. > > This load can't be compared to the larger 3 tier runs, but > > it was quick to do, and can be compared to itself. > > > > Frankly, results for stock 2.5.30 were horrible. Really bad. > > 2.5 swapped more. A lot more. > > At a guess I'd say that the test is right on the egde of > swapping in 2.4, and over the edge in 2.5. > > If you repeat the test with a little more memory and it gets > better then that would confirm the theory. > Should be do-able. The larger 2-cpu system is having some database issues at the moment, we hope to have it happy today. cliffw |
From: Gerrit H. <gh...@us...> - 2002-08-10 02:18:40
|
In message <200...@wo...>, > : Andi Kleen writes: > On Fri, Aug 09, 2002 at 05:30:47PM -0700, Gerrit Huizenga wrote: > > > > Also, do you know if sapdb uses AIO (asynchronous IO) on any platforms? > > Would it benefit from AIO on Linux? > > There is some posix AIO support in the source code for other Unixes. > Last time I looked it wasn't used on linux though, it uses threaded IO instead. > It could be enabled by fixing some ifdefs and recompiling. > > Another useful change would be probably to change it to use O_DIRECT. Then > it could work with files instead of raw partitions and it would be also > possible to measure differences between rawio and O_DIRECT. I checked > with the sapdb developers some time ago and they told me that all the > IO buffers should be already sufficiently aligned for O_DIRECT, so > changing it should be very simple. This is great to know. We are doing a lot of work like this with DB2 but it would be nice if someone could repeat some of our work externally, especially when using purely open source tools. We can also use that to compare the results to what we are seeing internally to provide comparisons between the stress points on the kernel between the various applications. gerrit |
From: Jenny Z. <je...@os...> - 2002-08-12 22:41:56
|
I will check with SAPDB engineers regarding those questions. It turned out that the database shutdown because the disk is full. I did one run on 2.5.29(with no patch) and SAPDB 7.3.0.25. The results are available at http://www.osdl.org/projects/dbt1prfrns/results/ under linux-2.5.29-2x400. I am using raw devices for my database so it is not affected by IDE driver as the 1tier results. Comparing to the same run on 2.4.18 kernel(under linux-2.4.18-2x400-kernprof), I observed slower database response time in the file 'BT', but the difference is not very big. Other than that, the data collected on both kernels are very close. I will do more runs on the same kernel with different users to see if the scalability curve is the same. Then I will do some tests as you suggested later this week. Jenny On Fri, 2002-08-09 at 19:18, Gerrit Huizenga wrote: > In message <200...@wo...>, > : Andi Kleen writes: > > On Fri, Aug 09, 2002 at 05:30:47PM -0700, Gerrit Huizenga wrote: > > > > > > Also, do you know if sapdb uses AIO (asynchronous IO) on any platforms? > > > Would it benefit from AIO on Linux? > > > > There is some posix AIO support in the source code for other Unixes. > > Last time I looked it wasn't used on linux though, it uses threaded IO instead. > > It could be enabled by fixing some ifdefs and recompiling. > > > > Another useful change would be probably to change it to use O_DIRECT. Then > > it could work with files instead of raw partitions and it would be also > > possible to measure differences between rawio and O_DIRECT. I checked > > with the sapdb developers some time ago and they told me that all the > > IO buffers should be already sufficiently aligned for O_DIRECT, so > > changing it should be very simple. > > This is great to know. We are doing a lot of work like this with > DB2 but it would be nice if someone could repeat some of our > work externally, especially when using purely open source tools. > We can also use that to compare the results to what we are seeing > internally to provide comparisons between the stress points on > the kernel between the various applications. > > gerrit -- Jenny Zhang Open Source Development Lab Inc 15275 SW Koll Parkway - Suite H Beaverton, OR 97006 (503)626-2455 ext 31 |
From: Jenny Z. <je...@os...> - 2002-08-10 00:13:08
|
Found a 'working' combination of kernel and sapdb version. I am now running kernel 2.5.29 and sapdb7.3.0.25. I can not get sapdb to work on 2.5.26. Everything worked fine until I start 80 database connections to do the run. The database got panic and shutdown. I will post the database error log to SAPDB mailing list hoping to get some understanding why it shutdown. So far, no interesting result yet. On Fri, 2002-08-09 at 08:23, Cliff White wrote: > > Cliff White wrote: > > > > > > As promised, i'm running 2.5.30 on the Very Tiny Two-Way. > > > (2 CPU x 256MB x 2 cheep IDE drives) > > > Caveats all over this one. > > > This load can't be compared to the larger 3 tier runs, but > > > it was quick to do, and can be compared to itself. > > > > > > Frankly, results for stock 2.5.30 were horrible. Really bad. > > > > 2.5 swapped more. A lot more. > > > > At a guess I'd say that the test is right on the egde of > > swapping in 2.4, and over the edge in 2.5. > > > > If you repeat the test with a little more memory and it gets > > better then that would confirm the theory. > > > Should be do-able. The larger 2-cpu system is having some > database issues at the moment, we hope to have it happy today. > cliffw > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech -- Jenny Zhang Open Source Development Lab Inc 15275 SW Koll Parkway - Suite H Beaverton, OR 97006 (503)626-2455 ext 31 |
From: Gerrit H. <gh...@us...> - 2002-08-10 00:31:09
|
Thanks, Jenny - even failures are interesting at this point. There is a good chance that the kernel did something mean to the application, so if you can figure out through strace or a debugger or a collection of the exit values what led to the failure that might be useful. Also, do you know if sapdb uses AIO (asynchronous IO) on any platforms? Would it benefit from AIO on Linux? gerrit In message <1028938536.23929.1385.camel@IBM-A>, > : Jenny Zhang writes: > Found a 'working' combination of kernel and sapdb version. I am now > running kernel 2.5.29 and sapdb7.3.0.25. I can not get sapdb to work on > 2.5.26. > > Everything worked fine until I start 80 database connections to do the > run. The database got panic and shutdown. I will post the database error > log to SAPDB mailing list hoping to get some understanding why it > shutdown. > > So far, no interesting result yet. > > On Fri, 2002-08-09 at 08:23, Cliff White wrote: > > > Cliff White wrote: > > > > > > > > As promised, i'm running 2.5.30 on the Very Tiny Two-Way. > > > > (2 CPU x 256MB x 2 cheep IDE drives) > > > > Caveats all over this one. > > > > This load can't be compared to the larger 3 tier runs, but > > > > it was quick to do, and can be compared to itself. > > > > > > > > Frankly, results for stock 2.5.30 were horrible. Really bad. > > > > > > 2.5 swapped more. A lot more. > > > > > > At a guess I'd say that the test is right on the egde of > > > swapping in 2.4, and over the edge in 2.5. > > > > > > If you repeat the test with a little more memory and it gets > > > better then that would confirm the theory. > > > > > Should be do-able. The larger 2-cpu system is having some > > database issues at the moment, we hope to have it happy today. > > cliffw > > > > > > > > ------------------------------------------------------- > > This sf.net email is sponsored by:ThinkGeek > > Welcome to geek heaven. > > http://thinkgeek.com/sf > > _______________________________________________ > > Lse-tech mailing list > > Lse...@li... > > https://lists.sourceforge.net/lists/listinfo/lse-tech > -- > Jenny Zhang > Open Source Development Lab Inc > 15275 SW Koll Parkway - Suite H > Beaverton, OR 97006 > (503)626-2455 ext 31 > > > > ------------------------------------------------------- > This sf.net email is sponsored by:ThinkGeek > Welcome to geek heaven. > http://thinkgeek.com/sf > _______________________________________________ > Lse-tech mailing list > Lse...@li... > https://lists.sourceforge.net/lists/listinfo/lse-tech > > |
From: Andi K. <ak...@su...> - 2002-08-10 02:13:14
|
On Fri, Aug 09, 2002 at 05:30:47PM -0700, Gerrit Huizenga wrote: > Thanks, Jenny - even failures are interesting at this point. There > is a good chance that the kernel did something mean to the application, > so if you can figure out through strace or a debugger or a collection > of the exit values what led to the failure that might be useful. > > Also, do you know if sapdb uses AIO (asynchronous IO) on any platforms? > Would it benefit from AIO on Linux? There is some posix AIO support in the source code for other Unixes. Last time I looked it wasn't used on linux though, it uses threaded IO instead. It could be enabled by fixing some ifdefs and recompiling. Another useful change would be probably to change it to use O_DIRECT. Then it could work with files instead of raw partitions and it would be also possible to measure differences between rawio and O_DIRECT. I checked with the sapdb developers some time ago and they told me that all the IO buffers should be already sufficiently aligned for O_DIRECT, so changing it should be very simple. -Andi |
From: Andrew M. <ak...@zi...> - 2002-08-10 00:33:47
|
Jenny Zhang wrote: > > Found a 'working' combination of kernel and sapdb version. I am now > running kernel 2.5.29 and sapdb7.3.0.25. I can not get sapdb to work on > 2.5.26. > > Everything worked fine until I start 80 database connections to do the > run. The database got panic and shutdown. I will post the database error > log to SAPDB mailing list hoping to get some understanding why it > shutdown. > If the database is using the raw driver then you'll need some bugs fixed. http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.30/direct_io-fixes.patch |
From: Jenny Z. <je...@os...> - 2002-08-14 00:02:29
|
Since not much interesting data found from the runs on 2-proc 2.5.29 kernel, I decided to try 2.5.31. But the problem is when I try to bind raw devices, it complained: 'Cannot open master raw device '/dev/rawctl' (No such device)' I am sure that the device is there, I can see it using ls. The patch Andrew recommended is in 2.5.31. Thanks for any hints, Jenny On Fri, 2002-08-09 at 17:31, Andrew Morton wrote: > Jenny Zhang wrote: > > > > Found a 'working' combination of kernel and sapdb version. I am now > > running kernel 2.5.29 and sapdb7.3.0.25. I can not get sapdb to work on > > 2.5.26. > > > > Everything worked fine until I start 80 database connections to do the > > run. The database got panic and shutdown. I will post the database error > > log to SAPDB mailing list hoping to get some understanding why it > > shutdown. > > > > If the database is using the raw driver then you'll need some bugs > fixed. http://www.zip.com.au/~akpm/linux/patches/2.5/2.5.30/direct_io-fixes.patch -- Jenny Zhang Open Source Development Lab Inc 15275 SW Koll Parkway - Suite H Beaverton, OR 97006 (503)626-2455 ext 31 |
From: Andrew M. <ak...@zi...> - 2002-08-14 00:12:16
|
Jenny Zhang wrote: > > Since not much interesting data found from the runs on 2-proc 2.5.29 > kernel, I decided to try 2.5.31. But the problem is when I try to bind > raw devices, it complained: > 'Cannot open master raw device '/dev/rawctl' (No such device)' > I am sure that the device is there, I can see it using ls. The patch > Andrew recommended is in 2.5.31. I changed the raw driver so that it could be compiled as a module, and I bet it's not in your kernel any more. Check that you have CONFIG_RAW_DRIVER=y. Or CONFIG_RAW_DRIVER=m and some alias which I can never remember in /etc/modules.conf, but I tested it and it works ;) |