Menu
▾
▴
[Spades-sim-devel] [ spades-sim-Bugs-1069667 ] Slow on gentoo
|
From: SourceForge.net <no...@so...> - 2005-05-17 20:37:09
|
Bugs item #1069667, was opened at 2004-11-19 15:05 Message generated for change (Settings changed) made by patstg You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498859&aid=1069667&group_id=61929 Category: None Group: None >Status: Closed >Resolution: Works For Me Priority: 6 Submitted By: Patrick Riley (patstg) Assigned to: Patrick Riley (patstg) Summary: Slow on gentoo Initial Comment: From: Daniel Hein dhein AT informatik.hu-berlin.de Hello Patrick, I have a problem with a new Gentoo Linux installation, running spades on it: The Gentoo Linux is fresh installed on my laptop. Everything works fine, but the rcssserver3d-0.3 is quite slowly - compared with Suse Linux 9.2 on the same machine. I tested your sample_agent/sample_world_model example (with a simulation_length of 5.000.000 in ballworld.conf) on both systems - meanwhile under Suse I reach an overall mean simtime/sec of about 102.000 under Gentoo it's only about 53.000. The outputs I have attached (ballworld_output). I installed a benchmark in both systems ("Lmbench 2.0"; http://www.bitmover.com/lmbench/) to test them - but there was no real difference in the results, except the missing local communication-values UDP, RPC/UDP and RPC/TCP. The results I have attached (lmbench2-results). I compiled spades with compiler-option -pg to create a profile-file of the function-calls, but it does not work on both systems. (made: "./configure CPPFLAGS="-p -pg -O2"; make; make install) - there is only a gmon.out in the root-directory of spades, but no idea to which executable it belongs. I expected for ballworld and for commserver extra gmon.out-files, but there are not..(?) I collected some information and outputs of both installations: - cpuinfo: output of cpuinfo - testlauf: the script for running the sample_agent/sample_world_model-example five times - gentoo/ : all outputs/results of the gentoo-system - suse/ : all outputs/results of the suse-system explained: - configure_output: output produced by "./configure" - config.log: copy of config.log-file in the spades-directory after "./configure" - hdparm_output: output produced by "hdparm -tT /dev/hda" - kernel-config: the .config-file of the kernel-configuration - lspci-output: output produced by "lspci" - lmbench2-results: the results of the lmbench2-benchmark The computer is has a ASUS M2400N with a Intel Pentium Centrino 1,7Ghz and 1,24 Gb Ram. I just have no idea what could be the reason for this great difference in the simulation-speed. I already tried different kernel-configurations and another kernel (vanilla-2.6.9), but the results do not change. I am wondering about the UDP, RPC/UDP and RPC/TCP-values in the lmbench2.0-benchmark, but even the author of the benchmark could not help me. I am not sure, but for me it seems, that the local communication is slowlier, also the sample_agent/sample_world_model with the testlauf-script crashes quite often on the Gentoo-System with TCP-Errors (Unable to connect, address already in use) - under Suse it never crashed. The curios is, that all other applications (Screensaver, 3D-Games, Browser) I have seen until now work fine and fast - only Spades is slowlier than with Suse. Maybe you have an idea what could be the problem and/or solution of this problem? ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2005-05-17 11:10 Message: Logged In: YES user_id=189652 More info from Joschka Boedecker Meanwhile, I got some feedback from Takenori Kubo who is responsible for the machine setup in Osaka. He did some tests with SPADES and the rcssserver3D on Gentoo 2005, SUSE 9.2, and Fedora 2 on a Pentium-4, 2.4Ghz, 768MB RAM machine. He didn't find significant variations in the performance, although the SuSE system was a little faster than the two others with SPADES (ballworld) alone: Fedora: Overall mean simtime/sec: 45083.3 Overall mean simtime/sec: 44419.5 Overall mean simtime/sec: 44853.1 Overall mean simtime/sec: 46092.8 Overall mean simtime/sec: 46437.7 Gentoo: Overall mean simtime/sec: 46904.8 Overall mean simtime/sec: 46447 Overall mean simtime/sec: 47147.1 Overall mean simtime/sec: 47107.5 Overall mean simtime/sec: 47036.9 SuSE: Overall mean simtime/sec: 50053.1 Overall mean simtime/sec: 50261.9 Overall mean simtime/sec: 50560.4 Overall mean simtime/sec: 50334.2 Overall mean simtime/sec: 52504.3 and interestingly, SuSE binary on Gentoo kernel: Overall mean simtime/sec: 52693.4 Overall mean simtime/sec: 52936 Overall mean simtime/sec: 52151 Overall mean simtime/sec: 53018.1 For the 3D server without monitor (running agenttest I believe), he got the following values: Gentoo: Overall mean simtime/sec: 68.9036 SuSE: Overall mean simtime/sec: 65.9513 Fedora: Overall mean simtime/sec: 68.0666 ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2005-04-25 15:21 Message: Logged In: YES user_id=189652 I wasn't able to upload the file to sourceforge. Please get it from here: http://zebrafish.coral.cs.cmu.edu/statictest.tar.gz ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2005-04-25 15:17 Message: Logged In: YES user_id=189652 Okay, I have another set of experiments. I've created a statically linked binary and set up an integrated agent (this was broken in the latest releases). Please download statictest.tar.gz, run test.sh on both suse and gentoo and upload the results. There's some chance that this won't work because of shared library weirdness. If that is true, please replace libspadesint.so with the version that you got from compilation (and let me know that you had to do this). The integrated agent test may still not work though... ---------------------------------------------------------------------- Comment By: Daniel Hein (affenschaukel) Date: 2005-04-18 17:39 Message: Logged In: YES user_id=1163050 == Suse 9.2: ==================================== # gcc -v Reading specs from /usr/lib/gcc-lib/i586-suse-linux/3.3.4/specs Configured with: ../configure --enable-threads=posix --prefix=/usr --with-local-prefix=/usr/local --infodir=/usr/share/info --mandir=/usr/share/man --enable-languages=c,c++,f77,objc,java,ada --disable-checking --libdir=/usr/lib --enable-libgcj --with-gxx-include-dir=/usr/include/g++ --with-slibdir=/lib --with-system-zlib --enable-shared --enable-__cxa_atexit i586-suse-linux Thread model: posix gcc version 3.3.4 (pre 3.3.5 20040809) == Gentoo ===================================== # gcc -v Reading specs from /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/specs Configured with: /var/tmp/portage/gcc-3.3.4-r1/work/gcc-3.3.4/configure --prefix=/usr --bindir=/usr/i686-pc-linux-gnu/gcc-bin/3.3 --includedir=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/include --datadir=/usr/share/gcc-data/i686-pc-linux-gnu/3.3 --mandir=/usr/share/gcc-data/i686-pc-linux-gnu/3.3/man --infodir=/usr/share/gcc-data/i686-pc-linux-gnu/3.3/info --enable-shared --host=i686-pc-linux-gnu --target=i686-pc-linux-gnu --with-system-zlib --enable-languages=c,c++ --enable-threads=posix --enable-long-long --disable-checking --disable-libunwind-exceptions --enable-cstdio=stdio --enable-version-specific-runtime-libs --with-gxx-include-dir=/usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/include/g++-v3 --with-local-prefix=/usr/local --enable-shared --enable-nls --without-included-gettext --disable-multilib --enable-__cxa_atexit --enable-clocale=generic Thread model: posix gcc version 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6) ============================================ Greetings :-) ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2005-03-26 15:18 Message: Logged In: YES user_id=189652 (sorry I've been ignoring this) There's one other thing I'd like to know before I consult with a Linux expert. Can you give me the output of gcc -v on both systems? I'm particularly interested in the thread model, e.g.: shamu:~ % gcc -v Reading specs from /usr/lib/gcc-lib/i386-redhat-linux/3.2.2/specs Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --disable-checking --with-system-zlib --enable-__cxa_atexit --host=i386-redhat-linux Thread model: posix gcc version 3.2.2 20030222 (Red Hat Linux 3.2.2-5) Thanks. ---------------------------------------------------------------------- Comment By: Daniel Hein (affenschaukel) Date: 2005-01-19 19:28 Message: Logged In: YES user_id=1163050 This week I installed on another machine (Centrino 1.6Ghz, 512Mb) Gentoo (Kernel 2.6.10-gentoo-r4). This time I used a default install package from gentoo.org (stage 3) with a precomiled basic system to avoid configuring mistakes by myself. Everything works fine but again with the same slowliness of Spades. For comparing the runs again I installed parallel Suse Linux 9.2 (Kernel 2.6.8-28) on the same machine. Here are the outputs of both systems: === Gentoo ====================================== #time ./ballworld --file ballworld.conf --run_integrated_commserver SPADES Simulation Engine, version 1.03 [...] Engine Status: PAUSED simtime=5000000 events=619493 mean simtime/sec=52974.2 Real time used for simulation: 95.9693 Overall mean simtime/sec: 52100 Closing the connections... 84.352u 10.013s 1:38.20 96.0% 0+0k 0+0io 0pf+0w ================================================== === Suse ========================================= #time ./ballworld --file ballworld.conf --run_integrated_commserver SPADES Simulation Engine, version 1.03 [...] Engine Status: PAUSED simtime=5000000 events=618797 mean simtime/sec=109478 Real time used for simulation: 46.2163 Overall mean simtime/sec: 108187 Closing the connections... 35.343u 9.369s 0:48.57 92.0% 0+0k 0+0io 0pf+0w ================================================== Now I wanted to profile the ballworld run - compiling the whole spades with the -pg option: =================================== # ./configure CPPFLAGS="-p -pg -O3" # make # make install =================================== But after the run of the ballworld sample =================================== # cd sample_world_model # time ./ballworld --file ballworld.conf --run_integrated_commserver =================================== only one gmon.out-file is produced, but in the root directory of spades (in my case: /home/xyz/Spades/spades-1.03/gmon.out I tried to run gprof with different options (different executables), but it only produces: =================================== # gprof gmon.out sample_world_model/ballworld gmon.out file is missing call-graph data =================================== So how I can profile the ballworld run to see, where are the differences between the systems? BTW: ldd commserver, gcc --version and /lib/libc-2.3.4.so produce the same output like I posted for the other machine. ---------------------------------------------------------------------- Comment By: Daniel Hein (affenschaukel) Date: 2004-12-15 15:16 Message: Logged In: YES user_id=1163050 Gentoo: # ./configure CPPFLAGS="-p -O2" CXXFLAGS="-O2" # make # cd sample_world_model/ # time ./ballworld --file ballworld.conf --run_integrated_commserver [...] 84.712u 10.296s 1:39.27 95.6% 0+0k 0+0io 0pf+0w So, nothing changed. Actually I have a look on the CFLAGS-Variable in the /etc/make.conf file. It is set to CFLAGS="-march=i686 -O3 -pipe", but maybe there is a better configuration. I don't know how it has an effect on the system, especially the libc-library. I will read about it and maybe when I have time I will rebuild the installation with a different configuration. ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2004-12-03 12:53 Message: Logged In: YES user_id=189652 Actually, there's one other thing I'd like to try. Can you put -O2 in your CXXFLAGS environment variable before compiling on gentoo and let me know if it is still slow? ---------------------------------------------------------------------- Comment By: Daniel Hein (affenschaukel) Date: 2004-11-23 18:40 Message: Logged In: YES user_id=1163050 Well, I have these outputs: ======= ldd commserver ========================= Suse: # ldd commserver linux-gate.so.1 => (0xffffe000) libexpat.so.0 => /usr/lib/libexpat.so.0 (0xb7fcb000) libdl.so.2 => /lib/libdl.so.2 (0xb7fc6000) libstdc++.so.5 => /usr/lib/libstdc++.so.5 (0xb7f09000) libgcc_s.so.1 => /lib/libgcc_s.so.1 (0xb7f01000) libm.so.6 => /lib/tls/libm.so.6 (0xb7ede000) libc.so.6 => /lib/tls/libc.so.6 (0xb7dc8000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0xb7fea000) Gentoo: # ldd commserver linux-gate.so.1 => (0xffffe000) libexpat.so.0 => /usr/lib/libexpat.so.0 (0xb7fc4000) libdl.so.2 => /lib/libdl.so.2 (0xb7fc1000) libstdc++.so.5 => /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libstdc++.so.5 (0xb7ef8000) libgcc_s.so.1 => /usr/lib/gcc-lib/i686-pc-linux-gnu/3.3.4/libgcc_s.so.1 (0xb7eef000) libm.so.6 => /lib/libm.so.6 (0xb7ecd000) libc.so.6 => /lib/libc.so.6 (0xb7db8000) /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0xb7fea000) ======== gcc --version ========================== Suse: # gcc --version gcc (GCC) 3.3.4 (pre 3.3.5 20040809) Copyright (C) 2003 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Gentoo: # gcc --version gcc (GCC) 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6) Copyright (C) 2003 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. ============ libc ========================== Suse: # /lib/tls/libc.so.6 GNU C Library stable release version 2.3.3 (20040917), by Roland McGrath et al. Copyright (C) 2004 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Configured for i686-suse-linux. Compiled by GNU CC version 3.3.4 (pre 3.3.5 20040809). Compiled on a Linux 2.6.8.1 system on 2004-10-05. Available extensions: GNU libio by Per Bothner crypt add-on version 2.1 by Michael Glad and others Native POSIX Threads Library by Ulrich Drepper et al GNU Libidn by Simon Josefsson NoVersion patch for broken glibc 2.0 binaries BIND-8.2.3-T5B NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk Thread-local storage support included. For bug reporting instructions, please see: <http://www.gnu.org/software/libc/bugs.html>. Gentoo: # /lib/libc-2.3.4.so GNU C Library 20040808 release version 2.3.4, by Roland McGrath et al. Copyright (C) 2004 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. Compiled by GNU CC version 3.3.4 20040623 (Gentoo Linux 3.3.4-r1, ssp-3.3.2-2, pie-8.7.6). Compiled on a Linux 2.4.21 system on 2004-11-05. Available extensions: GNU libio by Per Bothner crypt add-on version 2.1 by Michael Glad and others linuxthreads-0.10 by Xavier Leroy BIND-8.2.3-T5B libthread_db work sponsored by Alpha Processor Inc NIS(YP)/NIS+ NSS modules 0.19 by Thorsten Kukuk Thread-local storage support included. For bug reporting instructions, please see: <http://www.gnu.org/software/libc/bugs.html>. ========================================== ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2004-11-23 15:37 Message: Logged In: YES user_id=189652 After some more tests, it appears that kernel is not having any effect. I'm guessing that something is different about libc. Do you know anything about gentoo and suse differences in this respect? ---------------------------------------------------------------------- Comment By: Daniel Hein (affenschaukel) Date: 2004-11-23 12:49 Message: Logged In: YES user_id=1163050 Sorry for the unclear statements: I meant there is no speed difference with the integrated commserver compared to the run *without* integrated commserver *on the same system* (gentoo). But well - I will try to get an 2.6.8-kernel for the gentoo-system and test it again. ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2004-11-23 11:55 Message: Logged In: YES user_id=189652 These two statements from your earlier comment are NOT compatible: The first says there is not diff and the second says there is. Which is it? -- quoted comment below * There is no speed difference using the integrated communication server, even the mean simtime/sec values are similar. * 'time ./ballworld --file ballworld.conf --run_integrated_commserver' produces: Suse: # 32.286u 9.261s 0:46.24 92.7% 0+0k 0+0io 0pf+0w Gentoo: # 84.171u 9.950s 1:37.46 96.5% 0+0k 0+0io 0pf+0w ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2004-11-23 11:54 Message: Logged In: YES user_id=189652 A friend (rajesh as cs) had this to say "2.6.9 has some VM issues I think. I know the OOM killer gets activated much faster in 2.6.9 than in 2.6.8 (I've noticed that myself). that might be causing the problem if you have have 384M or lower RAM" I got another friend to run with a 2.6.9 and a 2.6.8 kernel (but on different speed machines) acm:: [11:49:45] 3 GHz machine results: 21.868u 5.335s 1:20.45 33.7% 0+0k 0+0io 0pf+0w off~/ftp/spades-1.03/sample_world_model> uname -a Linux off.cmcl.cs.cmu.edu 2.6.8-1.521 #1 Mon Aug 16 09:01:18 EDT 2004 i686 i686 i386 GNU/Linux acm:: [11:50:17] 1 GHz machine results: 42.157u 10.162s 2:21.60 36.9% 0+0k 0+0io 0pf+0w squeegee~/ftp/spades-1.03/sample_world_model> uname -a Linux squeegee 2.6.9-1.6_FC2 #1 Thu Nov 18 22:03:19 EST 2004 i686 i686 i386 GNU/Linux This suggests the kernel is not to blame ---------------------------------------------------------------------- Comment By: Daniel Hein (affenschaukel) Date: 2004-11-20 11:26 Message: Logged In: YES user_id=1163050 * There is no speed difference using the integrated communication server, even the mean simtime/sec values are similar. * 'time ./ballworld --file ballworld.conf --run_integrated_commserver' produces: Suse: # 32.286u 9.261s 0:46.24 92.7% 0+0k 0+0io 0pf+0w Gentoo: # 84.171u 9.950s 1:37.46 96.5% 0+0k 0+0io 0pf+0w * 'tcpdump -i -lo' while executing the testlauf-script (without integrated commserver) is producing a lot of output, while running ballworld with integrated commserver is not. ---------------------------------------------------------------------- Comment By: Patrick Riley (patstg) Date: 2004-11-19 15:14 Message: Logged In: YES user_id=189652 A couple things I'd like to know: * Is there a speed difference if you use the integrated communication server? * What's the output of 'time <command>' in those cases? * Can you use tcpdump to verify that the lo interface is being used on the gentoo system? If 'tcpdump -i lo' gives you lots of output while the simulation is running, then it probably is. ---------------------------------------------------------------------- You can respond by visiting: https://sourceforge.net/tracker/?func=detail&atid=498859&aid=1069667&group_id=61929 |