From: <jay...@wi...> - 2004-03-05 05:56:06
|
Jp, Do you have vmstat, iostat and the other tools referenced installed in the linux image? Yes Kevin. I am using SuSE Linux 8.2 Thanks -JP -----Original Message----- From: jay...@wi... [mailto:jay...@wi...] Sent: Thursday, March 04, 2004 4:38 AM To: cha...@wi...; je...@os... Cc: Kevin Babst; Mike Black; Sasi Lanka; osd...@li...; kri...@wi... Subject: RE: [osdldbt-general] RE: FW: DBT3 testing Hi Jenny, If I execute the script run_dbt3.sh (under dbdriver/scripts/sapdb) it won't generate q_time.out, vmstat.out,iostat and other files. I have to run the script dbt3_stats.sh (under data_collect/sapdb) get all the information. I have attached q_time.out, power_query.result and output of dbt3_stats.sh Only one file with .result extn is created in the output directory. List of files created under output directory when I run dbt3_stats.sh ------------------------------------------------------------------------ ------------ all_ind_columns.out db_stat - directory iostat.txt ipcs - direcotry meminfo0.out meminfo1.out param.out power_param power_query power_query.result primary_keys.out q_time.out refresh_stream1 refresh_stream2 run.sar.data throughput_param1 throughput_param2 thuput_qs1 thuput_qs2 vmstat.out ------------------------------------------------------------------------ -------- ipcs dir will have the following files: . ipcs0.out ipcs10.out ipcs2.out ipcs4.out ipcs6.out ipcs8.out .. ipcs1.out ipcs11.out ipcs3.out ipcs5.out ipcs7.out ipcs9.out db_stat directory will have many output files for lockstats, m_cache, m_load, m_lock, m_log, m_pages, m_row, m_trans and x_cons. Thanks -JP -----Original Message----- From: Chandrashekhar RS (WT01 - EMBEDDED & PRODUCT ENGINEERING SOLUTIONS) Sent: Wednesday, March 03, 2004 7:50 PM To: 'Jenny Zhang'; Jayaprakash Shetty (WT01 - EMBEDDED & PRODUCT ENGINEERING SOLUTIONS) Cc: ke...@fa...; mb...@fa...; Sasi Lanka; 'dbt-general'; Krishna Mandyamkannan (WT01 - EMBEDDED & PRODUCT ENGINEERING SOLUTIONS) Subject: RE: [osdldbt-general] RE: FW: DBT3 testing Hi Jenny The engineer working on this was not in office today. I have gathered the information from the machine where we ran the dbt3 tests and I am attaching the q_time.out and the power_query.result file. I am not sure whether I am attaching the right files here. Anyway I will ask JP to send you the files if the attachment I am sending is not the correct files you were expecting for analysis. Thanks Chandra -----Original Message----- From: Jenny Zhang [mailto:je...@os...] Sent: Wednesday, March 03, 2004 12:38 AM To: jay...@wi... Cc: ke...@fa...; mb...@fa...; sa...@fa...; dbt-general; kri...@wi...; cha...@wi... Subject: RE: [osdldbt-general] RE: FW: DBT3 testing Hi, I am glad that the database is build successfully. I am not sure that the test went through fine. Since the power test took only 492 seconds and the throughput test took only 233 seconds. I think it errored out during the run. Can you check the q_time.out to see if all the tasks have end time stamp? Also the *.results files should have the query outputs. Jenny On Tue, 2004-03-02 at 04:31, jay...@wi... wrote: > Hi Jenny, > I tried the database build by retaining all the parameter values as > defined in the set_param.sh and create_db.sh except for reducing > DATA_CACHE from 262144 to 51200 and it went through fine. It took > around 35 minutes for database build. I didn't use any raw devices. > > Ran the test using the script run_dbt3.sh. > "run_dbt3.sh 1 2" > The script run_dbt3.sh initially executes build_db.sh which builds the database. > Attached is the output of run_dbt3.sh. > > I used the SAPDB version 7.3.0 and database tables were created using > scale factor 1. Also ran the test using the script dbt3_stats.sh which > generates additional outputs for iostat,vmstat,etc. > > Thanks > -JP > > > > -----Original Message----- > From: osd...@li... > [mailto:osd...@li...]On Behalf Of Jenny > Zhang > Sent: Tuesday, March 02, 2004 2:51 AM > To: Kevin Babst > Cc: Mike Black; Sasi Lanka; dbt-general > Subject: [osdldbt-general] RE: FW: DBT3 testing > > > Is the I/O the bottleneck? If it is, using more drives will help. Did > you see a lot wait for IO or disk busy? > > Another thing I would try is to do build_db.sh step by step and see > why it hangs. Do you have to use SAPDB? > > On Mon, 2004-03-01 at 10:20, Kevin Babst wrote: > > I believe we are trying to run DTB3 on only 1 drive. Would you > > recommend building it on more, if so how many more? > > > > -----Original Message----- > > From: Jenny Zhang [mailto:je...@os...] > > Sent: Monday, March 01, 2004 10:06 AM > > To: Mike Black > > Cc: Kevin Babst; Sasi Lanka; dbt-general > > Subject: RE: FW: DBT3 testing > > > > Hi, > > > > Jayaprakash Shetty contacted us before. As both Mark and I have > > suggested, can you try to reduce the DATA_CACHE to about 400M, since the > > OS need some memory to do other stuff. You can increase it if raw is > > used. The database kernel log knldiag might give some info if the > > configuration is not proper. > > > > Can you tell me the purpose of your test? The reason I am asking is > > that I would like to find out if you can use pgsql instead of SAPDB. > > >From the tests I've done on a 4-way 4G memory system, building a > > database of SF 1 on SAPDB took about 30 minutes, and on PG took > > about 1 hour. But for SAPDB, I layout the database on 11 raw > > devices. I wonder if I/O is the bottleneck for you since the > > database is on one file system. > > > > The old dbt-3 on SAPDB run can be found at: > > http://khack.osdl.org/stp/283794/ The latest dbt-3 on PG run is at: > > http://khack.osdl.org/stp/288997/ > > > > Let me know if I can provide more info. > > > > Jenny > > On Sat, 2004-02-28 at 09:26, Mike Black wrote: > > > Thanks! We could really use your help. > > >=20 > > > Mike B. > > > > > > __________________________________________________________________ > > > ____ > > > From: jenny [mailto:je...@os...] > > > Sent: Fri 2/27/2004 2:42 PM > > > To: Mike Black > > > Cc: Mar...@os...; Ma...@os...; 'Jayaprakash Shetty (WT01 - > > > EMBEDDED & PRODUCT ENGINEERING SOLUTIONS)'; 'Chandrashekhar RS'; Sasi > > > Lanka; Kevin Babst > > > Subject: Re: FW: DBT3 testing > > > > > > > > > > > > Hi Mike, > > > > > > I would like to help on this issue. I will response first thing > > > next Monday if not this weekend. I just got back from China, and > > > I can bearly think after the long flights. > > > > > > thanks, > > > > > > Jenny > > > On Wed, 25 Feb 2004, Mike Black wrote: > > > > > > > Jenny, Mary, and Mark: > > > > > > > > My name is Mike Black and I manage the SQA group for Fabric7 > > > Systems. I > > > > think you know Russ Clapp, Kevin Babst, and a number of other > > > > ex-Sequent/IBMers at F7. We have SW/HWDev/SQA engineers and > > > contractors > > > > in Mt View, CA, Hillsboro, OR, and Bangalore, India. > > > > > > > > Thanks for the help so far. We'd REALLY like to get your DBT-3 > > > > test running in our set up, but are still having issues. JP > > > > below > > > outlined > > > > the exact steps he's taken. Could you review them and see if > > > > we're missing something, or perhaps we don't have enough disk or > > > > ram. JP > > > can > > > > give you the system config. Is there some additional OSDL > > > > debugging flags that might help if turned on? > > > > > > > > We're trying to wrap this up as soon as possible. > > > > > > > > Thanks, > > > > > > > > Mike B. > > > > > > > > -----Original Message----- > > > > From: jay...@wi... > > > [mailto:jay...@wi...] > > > > > > > > Sent: Tuesday, February 24, 2004 6:12 AM > > > > To: mb...@fa... > > > > Cc: ke...@fa...; sas...@wi...; > > > cha...@wi...; > > > > jay...@wi...; kri...@wi... > > > > Subject: DBT3 testing > > > > > > > > > > > > > > > > Hi Mike, > > > > Here are the steps that we have tried to setup and execute DBT3 > > > tests. > > > > With scale factor 0.1 test executed fine. Seeing problems with > > > > scale factor 1. > > > > > > > > 1. Uncompress and untar the dbt3 gz file > > > > tar zxvf dbt3-v1.3.tar.gz > > > > 2. Uncompress and untar 20000511.tar.z(dbgen from > > > > http://www.tpc.org/tpch/). When untarred > > > > a directory called appendix will be created where it puts all > > > the > > > > files. > > > > tar zxvf 20000511.tar.z > > > > 3. Run configure in dbt3-v1.3 directory > > > > autoconf && ./configure --with-sapdb && make && make install > > > > 4. Copy the dbgen files from appendix > > > > cp appendix/dbgen/* dbt3-v1.3/datagen/dbgen > > > > 5. Run the patch command under dbt3-v1.3/datagen > > > > patch -p0 < osdl_dbgen.patch > > > > 6. Run make and make install under dbt3-v1.3 > > > > 7. Modify the file dbt3-v1.3/scripts/sapdb/set_run_env.sh to > > > > look > > > like > > > > > > > > Original ( The path can be edited as per the requirement and > > > these > > > > paths are added when we run configure) > > > > > > > > export > > > > > > > = DSS_QUERY=3D/home/test/tests/performance/dbt/dbt3/datagen/sapdb-quer > > > ies > > > > export DSS_PATH=3D/dbt3_data/sapdb > > > > export > > > DSS_CONFIG=3D/home/test/tests/performance/dbt/dbt3/datagen/dbgen > > > > export SID=3DDBT3 > > > > export > > > > = DBT3_PERL_MODULE=3D/home/test/tests/performance/dbt/dbt3/perlmodul > > > > es > > > > export DBT3_BACKUP=3D/dbt3_backup > > > > export PATH=3D/opt/sapdb/depend/bin/:/opt/sapdb/indep_prog/bin/:$PATH > > > > export SYS_DIR=3D/dbt3/sapdb > > > > export DBUSER=3Ddbt > > > > > > > > New: > > > > export > > > DSS_QUERY=3D/home/test/tests/performance/dbt/dbt3/datagen/queries > > > > export DBT3_INSTALL_PATH=3D/home/test/tests/performance/dbt/dbt3 > > > > export DSS_PATH=3D/tmp/dbt3_data/sapdb > > > > export > > > DSS_CONFIG=3D/home/test/tests/performance/dbt/dbt3/datagen/dbgen > > > > export SID=3DDBT3 > > > > export > > > > = DBT3_PERL_MODULE=3D/home/test/tests/performance/dbt/dbt3/perlmodul > > > > es > > > > export DBT3_BACKUP=3D/tmp/dbt3_backup > > > > export PATH=3D/opt/sapdb/depend/bin/:/opt/sapdb/indep_prog/bin/:$PATH > > > > export SYS_DIR=3D/tmp/dbt3/sapdb > > > > export DBUSER=3Ddbt > > > > > > > > 8. Source the file set_run_env.sh > > > > 9. Create directories for DSS_PATH, DBT3_BACKUP and SYS_DIR 10. > > > > Run the dbgen command to create database tables > > > > cd dbt3-v1.3/datagen/dbgen > > > > ./dbgen -s <SF> > > > > Here is SF stands for scale factor. > > > > Eg: To create database tables with scale factor 0.1 > > > > ./dbgen -s 0.1 > > > > > > > > 11. Build the database > > > > cd dbt3-v1.3/scripts/sapdb > > > > ./build_db.sh > > > > The database build parameters needs to be edited before this > > > step. > > > > More information is given > > > > in the step 13. > > > > > > > > 12. Run the test > > > > cd dbt3-v1.3/dbdriver/scripts/sapdb > > > > ./run_dbt3.sh <scale_factor> <num_stream> [seed] > > > > Eg : ./run_dbt3.sh 1 1 > > > > Seed parameter is optional > > > > > > > > The script dbt3-v1.3/data_collect/sapdb/dbt3_stats.sh can > > > > be executed to run the test as well > > > > as to collect the system statistics. > > > > > > > > 13. The following parameters are used for DATA_CACHE ,LOG and > > > > DATA > > > when > > > > I tried the test > > > > with 0.1(database tables created with SF=3D 0.1) and it > > > > worked > > > fine. > > > > > > > > param_put DATA_CACHE 262144 > > > > in the file dbt3-v1.3/scripts/sapdb/set_param.sh > > > > and > > > > param_adddevspace 1 SYS $SYS_DIR/$SID/SYS_001 F > > > > param_adddevspace 1 DATA $SYS_DIR/$SID/DBT3_DATA_001 F 524228 > > > > param_adddevspace 1 LOG $SYS_DIR/$SID/DBT3_LOG_001 F 8192 > > > > > > > > in the file dbt3-v1.3/scripts/sapdb/create_db.sh > > > > And all these values came with the test suite. > > > > > > > > If I use these parameters with SF 1(database tables created > > > > with > > > SF=3D > > > > 1) it takes more time(or > > > > hangs) during database build. > > > > > > > > I tried the following set of parameters with SF 1 > > > > > > > > (i) When used the following values for DATA_CACHE and LOG I > > > > got > > > the > > > > similar result where it > > > > used take long time for database build. > > > > param_put DATA_CACHE 41943040 > > > > param_adddevspace 1 LOG $SYS_DIR/$SID/DBT3_LOG_001 F 819200 > > > > (ii) With the following values for DATA_CACHE and LOG I see > > > > that loading of system table fails > > > > because of no space. > > > > > > > > param_put DATA_CACHE 41943040 > > > > param_adddevspace 1 LOG $SYS_DIR/$SID/DBT3_LOG_001 F 8192000 > > > > > > > > It created the following files > > > > rw-rw---- 1 sapdb sapsys 4.0G 2004-02-19 22:45 > > > > /tmp/dbt3/sapdb/DBT3/DBT3_DATA_001 > > > > -rw-rw---- 1 sapdb sapsys 15G 2004-02-19 22:54 > > > > /tmp/dbt3/sapdb/DBT3/DBT3_LOG_001 > > > > -rw-rw---- 1 sapdb sapsys 8.9M 2004-02-19 22:43 > > > > /tmp/dbt3/sapdb/DBT3 > > > > > > > > Pre-requisite: > > > > =3D=3D=3D=3D=3D=3D=3D=3D=3D > > > > SAPDB 7.3.0 should be installed and steps from 8 should be done > > > > as > > > sapdb > > > > user. sapdb user should have permissions. > > > > > > > > SAPDB 7.3.0 installation steps [ Steps are given for SAPDB 7.4.0 > > > > but they are similar to 7.3.0 ] > > > > > > > > 1. Unzip and untar the file > > > > 2. tar zxvf sapdb-all-linux-32bit-i386-7_4_3_30.tar.gz > > > > 3. Run the following command in the directory > > > > sapdb-all-linux-32bit-i386-7_4_3_30 > > > > ./SDBINST > > > > > > > > They follow the steps below and enter appropriate values as > > > > given > > > in <> > > > > > > > > > > > > > > > > Installation of SAP DB Software > > > > ******************************** > > > > > > > > > > > > starting installation Fr, Jan 30, 2004 at 14:21:08 operating > > > > system: Linux I386 2.4.19 4GB GLIBC 2.2.5 callers working > > > > directory: /home/guest/sapdb-all-linux-32bit-i386-7_4_3_30 > > > > installer directory: /home/guest/sapdb-all-linux-32bit-i386-7_4_3_30 > > > > archive directory: /home/guest/sapdb-all-linux-32bit-i386-7_4_3_30 > > > > > > > > > > > > existing profiles: > > > > 0: Server > > > > 1: Runtime For SAP AS > > > > 2: DB Analyzer > > > > 3: C Precompiler > > > > 4: Webtools > > > > 5: ODBC > > > > 6: JDBC > > > > 7: Script Interface > > > > 8: XML Indexing Engine > > > > 9: all > > > > 10: none > > > > please enter profile id: <Enter 9. In case of SAPDB 7.3.0 Enter > > > > 7and > > > it > > > > will have totally 8 profiles> > > > > > > > > no updatable installation of package "Base" found please enter > > > > group name for database programs [sapsys]: <Press Enter > > > > Key> > > > > please enter owner name for database programs [sapdb]: <Press > > > > Enter > > > Key> > > > > > > > > unknown user - create "sapdb" on local machine? (y/n) <Enter y> > > > > New password: <Enter sapdb> Bad password: too simple > > > > Re-enter new password:<Enter sapdb> > > > > please enter independent data path > > > [/var/opt/sapdb/indep_data]:<Press > > > > Enter Key> > > > > directory "/var/opt/sapdb/indep_data" does not exist, create? > > > > (y/n) <Enter y> please enter independent program path > > > > [/opt/sapdb/indep_prog]:<Press Enter Key> > > > > directory "/opt/sapdb/indep_prog" does not exist, create? (y/n) > > > <Enter > > > > y> > > > > please enter dependent path [/opt/sapdb/depend]:<Press Enter > > > > Key> directory "/opt/sapdb/depend" does not exist, create? (y/n) > > > > <Enter > > > y> > > > > > > > > I already sent a mail to OSDL asking for the minimum test > > > configuration > > > > to use with SF=3D1 > > > > Let me know if you need more information. > > > > > > > > Thanks and Regards, > > > > -JP > > > > > > > > > > > > Confidentiality Notice > > > > > > > > > > > > > > > > > > > > > > > > The information contained in this electronic message and any > > > attachments > > > > to this message are intended > > > > > > > > for the exclusive use of the addressee(s) and may contain > > > confidential > > > > or privileged information. If > > > > > > > > you are not the intended recipient, please notify the sender at > > > Wipro or > > > > Mai...@wi... immediately > > > > > > > > and destroy all copies of this message and any attachments. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > ------------------------------------------------------- > SF.Net is sponsored by: Speed Start Your Linux Apps Now. Build and > deploy apps & Web services for Linux with a free DVD software kit from > IBM. Click Now! = http://ads.osdn.com/?ad_id=3D1356&alloc_id=3D3438&op=3Dclick > _______________________________________________ > osdldbt-general mailing list > osd...@li... > https://lists.sourceforge.net/lists/listinfo/osdldbt-general Confidentiality Notice The information contained in this electronic message and any attachments to this message are intended for the exclusive use of the addressee(s) and may contain confidential or privileged information. If you are not the intended recipient, please notify the sender at Wipro or Mai...@wi... immediately and destroy all copies of this message and any attachments. |