On Fri, Mar 25, 2011 at 4:22 AM, ALBERT HIDALGO BAREA <abarea@...> wrote:
> Here are the specs of my server system:
> I am running Red Hat Linux Enterprise Edition 5.3 (ulimit is set to 1024, the default parameter)
> Erlang R14B (erts-5.8.1) [source] [64-bit]
> Yaws 1.89
> The client is running apache benchmark "AB".
> When running the following command it completes the test without problem:
> ab -n 10000 -c 100 -k http://tas01/webtest.html
> But when running the next command it produces an error:
> ab -n 10000 -c 350 -k http://tas01/webtest.html
> apr_socket_recv: Connection reset by peer (104)
> Total of 116 requests completed
> And the report.log or access.log does not report any crash or failure. Also it does not create any erlang crash report.
Correct. If there were any odd errors within Yaws, we would see
complaints. I ran Yaws in interactive mode with SASL turned on and
never saw any errors there or in any of the log files.
> I am testing some other web servers and I cannot reproduce this error with "AB".
> It seems that there is some problem with concurrency and Yaws.
Again, I ran one ab version against Yaws 1.89 that had no problems
whatsoever, but a newer ab version running against the same version of
Yaws had problems. This points to the problem being in ab, not in
Yaws. You keep claiming the connection reset error is due to a
concurrency issue in Yaws but so far we have no proof of that. I
checked out numerous versions of Yaws via git, built and installed
each one, and the two versions of ab behaved the same against every
single Yaws version, with the old ab working fine and the new ab
failing. I've also run a variety of other load testing tools against
Yaws, including commercial ones, without any problems. The QA guys I
work with have concurrency tests that beat the crap out of our system
that uses Yaws, again without problem.
Under the yaws/test/t1 directory is a concurrency test that tries to
create load while respecting the number of file descriptors available
and not killing the system. It launches a bunch of parallel curl tasks
that download files of various sizes, with the goal of creating
concurrency and load, and then verifies that each file was received
correctly. If you have the latest code from github and set the
RUNTEST_CURLS env var before running "make test" in this directory,
you can control how many curl client instances are launched.
RUNTEST_CURLS=1000 for example creates 999 concurrent curl client
processes (because there are 3 file sizes attempted for each and the
math makes for 999 instead of 1000). I just ran a test on my Ubuntu
10.10 system with these 999 clients and, while it took a bit to
complete due to the number of processes running on the OS concurrently
with the web server, it completed successfully. My Ubuntu box is 4+
years old and so isn't all that beefy, so maybe if yours is
newer/faster you can try with a higher curl count.
My advice remains the same: use a different tool. The ab tool isn't
all that great anyway because it doesn't simulate real-world loads.
Try httperf or tsung instead.
If I get some time I might run wireshark to see why the newer ab has issues.
> -----Original Message-----
> From: vinoski@... [mailto:vinoski@...] On Behalf Of Steve Vinoski
> Sent: jueves, 24 de marzo de 2011 13:38
> To: ALBERT HIDALGO BAREA
> Cc: erlyaws-list@...
> Subject: Re: [Erlyaws-list] Yaws concurrency problems
> On Thu, Mar 24, 2011 at 6:14 AM, ALBERT HIDALGO BAREA <abarea@...> wrote:
>> I tested this command and it continues to fail.
>> There is some parameter in Erlang or in Yaws to change/add in order to support a concurrency higher than 300 users?
> No, there are no other settings required.
> Can you post the details of the errors you're seeing? Also, we need to
> know what version of Erlang you have as well as what version of Yaws
> you have. Are you running on Linux? Meanwhile I'll run some
> experiments on my Linux box.
>> -----Original Message-----
>> From: vinoski@... [mailto:vinoski@...] On Behalf Of Steve Vinoski
>> Sent: miércoles, 23 de marzo de 2011 14:09
>> To: ALBERT HIDALGO BAREA
>> Cc: erlyaws-list@...
>> Subject: Re: [Erlyaws-list] Yaws concurrency problems
>> On Wed, Mar 23, 2011 at 8:58 AM, ALBERT HIDALGO BAREA <abarea@...> wrote:
>>> My problem is when I am running the test. When I set a concurrency level
>>> higher than 200 clients, the ab exits with an error, only doing few requests
>>> (by using the -c option). Searching I found some Erlang variable which is
>>> ERL_MAX_PORTS which can increase the open ports in the Erlang virtual
>>> machine (by default is 1024). But I'm not able to run Yaws with this
>>> instruction (inside Yaws executable or when executing it with --erlarg ).
>> ERL_MAX_PORTS isn't what you're looking for. What you need to do is
>> make sure your operating system allows enough per-process file
>> descriptors. On Linux or other UNIX systems, for example, you can run
>> ulimit -n 65536
>> in your shell, and then run yaws. This will allow it to have 65536
>> open file descriptors.
>> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
>> This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at.
> Este mensaje se dirige exclusivamente a su destinatario. Puede consultar nuestra política de envío y recepción de correo electrónico en el enlace situado más abajo.
> This message is intended exclusively for its addressee. We only send and receive email on the basis of the terms set out at.