From: ALBERT H. B. <ab...@ti...> - 2011-03-23 12:59:59
|
Hi, I'm trying to set up a benchmark of Yaws with Apache Benchmark (AB) tool. The configuration of Yaws is the default one, only changing the root folder where to get HTML scripts, and also adding where to look for PHP FastCGI. Max_connections and keepalive_maxusers instructions are set to nolimit. My problem is when I am running the test. When I set a concurrency level higher than 200 clients, the ab exits with an error, only doing few requests (by using the -c option). Searching I found some Erlang variable which is ERL_MAX_PORTS which can increase the open ports in the Erlang virtual machine (by default is 1024). But I'm not able to run Yaws with this instruction (inside Yaws executable or when executing it with --erlarg ). Is there a way to increase the concurrency of Yaws? Thanks a lot. Regards, -- Albert Hidalgo ________________________________ Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra pol?tica de env?o y recepci?n de correo electr?nico en el enlace situado m?s abajo. This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. http://www.tid.es/ES/PAGINAS/disclaimer.aspx |
From: Steve V. <vi...@ie...> - 2011-03-23 13:08:48
|
On Wed, Mar 23, 2011 at 8:58 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: > > My problem is when I am running the test. When I set a concurrency level > higher than 200 clients, the ab exits with an error, only doing few requests > (by using the –c option). Searching I found some Erlang variable which is > ERL_MAX_PORTS which can increase the open ports in the Erlang virtual > machine (by default is 1024). But I’m not able to run Yaws with this > instruction (inside Yaws executable or when executing it with --erlarg ). ERL_MAX_PORTS isn't what you're looking for. What you need to do is make sure your operating system allows enough per-process file descriptors. On Linux or other UNIX systems, for example, you can run ulimit -n 65536 in your shell, and then run yaws. This will allow it to have 65536 open file descriptors. --steve |
From: ALBERT H. B. <ab...@ti...> - 2011-03-24 10:16:39
|
Hi, I tested this command and it continues to fail. There is some parameter in Erlang or in Yaws to change/add in order to support a concurrency higher than 300 users? Thanks. Regards, -----Original Message----- From: vi...@gm... [mailto:vi...@gm...] On Behalf Of Steve Vinoski Sent: miércoles, 23 de marzo de 2011 14:09 To: ALBERT HIDALGO BAREA Cc: erl...@li... Subject: Re: [Erlyaws-list] Yaws concurrency problems On Wed, Mar 23, 2011 at 8:58 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: > > My problem is when I am running the test. When I set a concurrency level > higher than 200 clients, the ab exits with an error, only doing few requests > (by using the -c option). Searching I found some Erlang variable which is > ERL_MAX_PORTS which can increase the open ports in the Erlang virtual > machine (by default is 1024). But I'm not able to run Yaws with this > instruction (inside Yaws executable or when executing it with --erlarg ). ERL_MAX_PORTS isn't what you're looking for. What you need to do is make sure your operating system allows enough per-process file descriptors. On Linux or other UNIX systems, for example, you can run ulimit -n 65536 in your shell, and then run yaws. This will allow it to have 65536 open file descriptors. --steve Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. http://www.tid.es/ES/PAGINAS/disclaimer.aspx |
From: Steve V. <vi...@ie...> - 2011-03-24 12:38:03
|
On Thu, Mar 24, 2011 at 6:14 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: > Hi, > > I tested this command and it continues to fail. > There is some parameter in Erlang or in Yaws to change/add in order to support a concurrency higher than 300 users? No, there are no other settings required. Can you post the details of the errors you're seeing? Also, we need to know what version of Erlang you have as well as what version of Yaws you have. Are you running on Linux? Meanwhile I'll run some experiments on my Linux box. --steve > Thanks. > Regards, > > -----Original Message----- > From: vi...@gm... [mailto:vi...@gm...] On Behalf Of Steve Vinoski > Sent: miércoles, 23 de marzo de 2011 14:09 > To: ALBERT HIDALGO BAREA > Cc: erl...@li... > Subject: Re: [Erlyaws-list] Yaws concurrency problems > > On Wed, Mar 23, 2011 at 8:58 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: >> >> My problem is when I am running the test. When I set a concurrency level >> higher than 200 clients, the ab exits with an error, only doing few requests >> (by using the -c option). Searching I found some Erlang variable which is >> ERL_MAX_PORTS which can increase the open ports in the Erlang virtual >> machine (by default is 1024). But I'm not able to run Yaws with this >> instruction (inside Yaws executable or when executing it with --erlarg ). > > ERL_MAX_PORTS isn't what you're looking for. What you need to do is > make sure your operating system allows enough per-process file > descriptors. On Linux or other UNIX systems, for example, you can run > > ulimit -n 65536 > > in your shell, and then run yaws. This will allow it to have 65536 > open file descriptors. > > --steve > > Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. > This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. > http://www.tid.es/ES/PAGINAS/disclaimer.aspx > |
From: Steve V. <vi...@ie...> - 2011-03-24 21:58:18
|
On Thu, Mar 24, 2011 at 8:37 AM, Steve Vinoski <vi...@ie...> wrote: > On Thu, Mar 24, 2011 at 6:14 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: >> Hi, >> >> I tested this command and it continues to fail. >> There is some parameter in Erlang or in Yaws to change/add in order to support a concurrency higher than 300 users? > > No, there are no other settings required. > > Can you post the details of the errors you're seeing? Also, we need to > know what version of Erlang you have as well as what version of Yaws > you have. Are you running on Linux? Meanwhile I'll run some > experiments on my Linux box. I've run some experiments and found some interesting results. On my Ubuntu 10.10 box I have ab version 2.3. No matter what version of Yaws I used on Erlang/OTP R14B02, ab would die as it neared the end of the run. If I said run with 20000 concurrent connections and perform 50000 total requests, it would die saying "connection reset by peer" after ~47000 or so requests. If I dropped to 2500 concurrent connections and 5000 total requests, it would work fine. If I tried 2500 and 15000, it would die after ~13000 requests. I then tried an old version of ab, version 2.0.41-dev running on an old RHEL4 box. No problems there at all: 20000 concurrent connections doing 50000 requests or 100000 requests, it all worked fine. 20000 concurrent connections is as high as ab can go, and Yaws handled it, no problem. So, you might want to try a testing tool other than ab such as tsung or httperf, or try to find yourself an older version of ab. --steve |
From: ALBERT H. B. <ab...@ti...> - 2011-03-25 08:24:34
|
Here are the specs of my server system: I am running Red Hat Linux Enterprise Edition 5.3 (ulimit is set to 1024, the default parameter) Erlang R14B (erts-5.8.1) [source] [64-bit] Yaws 1.89 The client is running apache benchmark "AB". When running the following command it completes the test without problem: ab -n 10000 -c 100 -k http://tas01/webtest.html But when running the next command it produces an error: ab -n 10000 -c 350 -k http://tas01/webtest.html apr_socket_recv: Connection reset by peer (104) Total of 116 requests completed And the report.log or access.log does not report any crash or failure. Also it does not create any erlang crash report. I am testing some other web servers and I cannot reproduce this error with "AB". It seems that there is some problem with concurrency and Yaws. Thanks. Regards, -----Original Message----- From: vi...@gm... [mailto:vi...@gm...] On Behalf Of Steve Vinoski Sent: jueves, 24 de marzo de 2011 13:38 To: ALBERT HIDALGO BAREA Cc: erl...@li... Subject: Re: [Erlyaws-list] Yaws concurrency problems On Thu, Mar 24, 2011 at 6:14 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: > Hi, > > I tested this command and it continues to fail. > There is some parameter in Erlang or in Yaws to change/add in order to support a concurrency higher than 300 users? No, there are no other settings required. Can you post the details of the errors you're seeing? Also, we need to know what version of Erlang you have as well as what version of Yaws you have. Are you running on Linux? Meanwhile I'll run some experiments on my Linux box. --steve > Thanks. > Regards, > > -----Original Message----- > From: vi...@gm... [mailto:vi...@gm...] On Behalf Of Steve Vinoski > Sent: miércoles, 23 de marzo de 2011 14:09 > To: ALBERT HIDALGO BAREA > Cc: erl...@li... > Subject: Re: [Erlyaws-list] Yaws concurrency problems > > On Wed, Mar 23, 2011 at 8:58 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: >> >> My problem is when I am running the test. When I set a concurrency level >> higher than 200 clients, the ab exits with an error, only doing few requests >> (by using the -c option). Searching I found some Erlang variable which is >> ERL_MAX_PORTS which can increase the open ports in the Erlang virtual >> machine (by default is 1024). But I'm not able to run Yaws with this >> instruction (inside Yaws executable or when executing it with --erlarg ). > > ERL_MAX_PORTS isn't what you're looking for. What you need to do is > make sure your operating system allows enough per-process file > descriptors. On Linux or other UNIX systems, for example, you can run > > ulimit -n 65536 > > in your shell, and then run yaws. This will allow it to have 65536 > open file descriptors. > > --steve > > Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. > This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. > http://www.tid.es/ES/PAGINAS/disclaimer.aspx > Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. http://www.tid.es/ES/PAGINAS/disclaimer.aspx |
From: Steve V. <vi...@ie...> - 2011-03-25 15:43:59
|
On Fri, Mar 25, 2011 at 4:22 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: > Here are the specs of my server system: > I am running Red Hat Linux Enterprise Edition 5.3 (ulimit is set to 1024, the default parameter) > Erlang R14B (erts-5.8.1) [source] [64-bit] > Yaws 1.89 > > The client is running apache benchmark "AB". > When running the following command it completes the test without problem: > ab -n 10000 -c 100 -k http://tas01/webtest.html > > But when running the next command it produces an error: > ab -n 10000 -c 350 -k http://tas01/webtest.html > apr_socket_recv: Connection reset by peer (104) > Total of 116 requests completed > > And the report.log or access.log does not report any crash or failure. Also it does not create any erlang crash report. Correct. If there were any odd errors within Yaws, we would see complaints. I ran Yaws in interactive mode with SASL turned on and never saw any errors there or in any of the log files. > I am testing some other web servers and I cannot reproduce this error with "AB". > It seems that there is some problem with concurrency and Yaws. Again, I ran one ab version against Yaws 1.89 that had no problems whatsoever, but a newer ab version running against the same version of Yaws had problems. This points to the problem being in ab, not in Yaws. You keep claiming the connection reset error is due to a concurrency issue in Yaws but so far we have no proof of that. I checked out numerous versions of Yaws via git, built and installed each one, and the two versions of ab behaved the same against every single Yaws version, with the old ab working fine and the new ab failing. I've also run a variety of other load testing tools against Yaws, including commercial ones, without any problems. The QA guys I work with have concurrency tests that beat the crap out of our system that uses Yaws, again without problem. Under the yaws/test/t1 directory is a concurrency test that tries to create load while respecting the number of file descriptors available and not killing the system. It launches a bunch of parallel curl tasks that download files of various sizes, with the goal of creating concurrency and load, and then verifies that each file was received correctly. If you have the latest code from github and set the RUNTEST_CURLS env var before running "make test" in this directory, you can control how many curl client instances are launched. RUNTEST_CURLS=1000 for example creates 999 concurrent curl client processes (because there are 3 file sizes attempted for each and the math makes for 999 instead of 1000). I just ran a test on my Ubuntu 10.10 system with these 999 clients and, while it took a bit to complete due to the number of processes running on the OS concurrently with the web server, it completed successfully. My Ubuntu box is 4+ years old and so isn't all that beefy, so maybe if yours is newer/faster you can try with a higher curl count. My advice remains the same: use a different tool. The ab tool isn't all that great anyway because it doesn't simulate real-world loads. Try httperf or tsung instead. If I get some time I might run wireshark to see why the newer ab has issues. --steve > > Thanks. > Regards, > > -----Original Message----- > From: vi...@gm... [mailto:vi...@gm...] On Behalf Of Steve Vinoski > Sent: jueves, 24 de marzo de 2011 13:38 > To: ALBERT HIDALGO BAREA > Cc: erl...@li... > Subject: Re: [Erlyaws-list] Yaws concurrency problems > > On Thu, Mar 24, 2011 at 6:14 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: >> Hi, >> >> I tested this command and it continues to fail. >> There is some parameter in Erlang or in Yaws to change/add in order to support a concurrency higher than 300 users? > > No, there are no other settings required. > > Can you post the details of the errors you're seeing? Also, we need to > know what version of Erlang you have as well as what version of Yaws > you have. Are you running on Linux? Meanwhile I'll run some > experiments on my Linux box. > > --steve > >> Thanks. >> Regards, >> >> -----Original Message----- >> From: vi...@gm... [mailto:vi...@gm...] On Behalf Of Steve Vinoski >> Sent: miércoles, 23 de marzo de 2011 14:09 >> To: ALBERT HIDALGO BAREA >> Cc: erl...@li... >> Subject: Re: [Erlyaws-list] Yaws concurrency problems >> >> On Wed, Mar 23, 2011 at 8:58 AM, ALBERT HIDALGO BAREA <ab...@ti...> wrote: >>> >>> My problem is when I am running the test. When I set a concurrency level >>> higher than 200 clients, the ab exits with an error, only doing few requests >>> (by using the -c option). Searching I found some Erlang variable which is >>> ERL_MAX_PORTS which can increase the open ports in the Erlang virtual >>> machine (by default is 1024). But I'm not able to run Yaws with this >>> instruction (inside Yaws executable or when executing it with --erlarg ). >> >> ERL_MAX_PORTS isn't what you're looking for. What you need to do is >> make sure your operating system allows enough per-process file >> descriptors. On Linux or other UNIX systems, for example, you can run >> >> ulimit -n 65536 >> >> in your shell, and then run yaws. This will allow it to have 65536 >> open file descriptors. >> >> --steve >> >> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. >> This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. >> http://www.tid.es/ES/PAGINAS/disclaimer.aspx >> > > Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo. > This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at. > http://www.tid.es/ES/PAGINAS/disclaimer.aspx > |