Current connection count and concurrent connection counts are wrong

[bob korf]

Both the current connection count and concurrent connection counts for indivdual urls are wrong. The counts increase over time even if website traffic drops. This is a SERIOUS bug that causes mod_qos to throttle back or deny access to legitimate site vistors when the website is running properly.

Any suggestions on how to correct this?

VERSIONS
mod_qos 11.18

Server version: Apache/2.2.31 (Unix)
Server built: Aug 13 2015 23:45:59
Architecture: 64-bit
Server MPM: Worker
threaded: yes (fixed thread count)
forked: yes (variable process count)

[Pascal Buchbinder]

Hi Bob
Do you like to share your configuration with us?
Has your server any child process crashing? (watch your log for child pid .* exit signal messages)

[bob korf]

Thanks for your prompt reply.

There were 3 abnormal exits out of 3.2 million requests since the server last started. After apache uptime of 12 hours, the discrepency between the server-status connections (250) vs mod_qos (350) is significant. If the server is not restarted at least daily, the MaxClients limit is reached and the connections are throttled by mod_qos.

Server Version: Apache/2.2.31 (Unix) DAV/2 mod_fastcgi/2.4.6 mod_fcgid/2.3.9 PHP/5.3.29 mod_qos/11.18

Here are the worker settings:
ServerLimit 16
StartServers 8
MaxClients 640
MinSpareThreads 8
MaxSpareThreads 16
ThreadsPerChild 64
MaxRequestsPerChild 8000

The httpd child processes exit frequently and normally, since MaxRequestsPerChild is relatively low. I can try raising that.

There are 10 virtual hosts on this server and 9 are have very low volume.

Another oddity is the error log has quite a few bursts (about 2 dozen burst each day) of QS_LocRequestLimit entries:

mod_qos(010): access denied, QS_LocRequestLimit* rule: /busy-url(40), concurrent requests=47

These bursts last about 1 second. The number of concurrent requests on "/busy-url" gradually increases over time until the limit is hit.

I'm not convinced there actually are 40 plus concurrent requests on the same url. The server-status shows a steady number of connections, but a spike in connections or newly spawned servers would be expected if there truly were a large number of concurrent requests. None of that happens. netstat also reports a relatively stable number of connections, and no excess SYN_RECV or TIME_WAIT entries. In other words, there is nothing to corraborate the large number of concurrent requests that mod_qos reports.

The configuration files are large. Let me know if there is specific configuration information that would help.

Thanks

[bob korf]

Not sure if this related, but "Max Clients in Memory" limit was reached. What happens when the limit is reached, are the oldest connections no longer tracked? Is it possible that open client connections are not counted when they close because they are no longer tracked in memory?

How does apache keep track of open connection in it's server status?