Activity for mod_qos
-
Jurgen Goelen posted a comment on discussion Open Discussion
The regular expression QS_LocRequestLimitMatch ^http 50 didn't work, but modifying it to QS_LocRequestLimitMatch ^.*$ 50 solved the problem. Thanks!
-
Pascal Buchbinder posted a comment on discussion Open Discussion
QS_LocRequestLimitDefault 50 is basically QS_LocRequestLimit / 50 which does not match your proxy request (which starts with a "h") anymore. I assume that QS_LocRequestLimitMatch ^http 50 will do the trick.
-
Hello *, I've been struggling with an issue where QS_LocRequestLimitDefault isn't working when combined with mod_rewrite. Here's my configuration: <virtualhost *:80=""> ServerName slow.local Header setIfEmpty Strict-Transport-Security "max-age=31536000; includeSubDomains"</virtualhost> ProxyPreserveHost On RewriteEngine On RequestHeader unset Origin QS_LocRequestLimitDefault 50 RewriteRule ^/(.*)$ http://slow_backend:8000/$1 [L,P] ProxyPassReverse / http://slow_backend:8000/ The interesting part...
-
've been struggling with an issue where QS_LocRequestLimitDefault isn't working when combined with mod_rewrite. Here's my configuration: <VirtualHost *:80> ServerName slow.local Header setIfEmpty Strict-Transport-Security "max-age=31536000; includeSubDomains" ProxyPreserveHost On RewriteEngine On RequestHeader unset Origin QS_LocRequestLimitDefault 50 RewriteRule ^/(.*)$ http://slow_backend:8000/$1 [L,P] ProxyPassReverse / http://slow_backend:8000/ </VirtualHost> The interesting part is that when...
-
Craig posted a comment on discussion Open Discussion
It appears this issue was fixed in version 11.76.
-
mod_qos released /mod_qos-11.76.tar.gz
-
11.76 cleanup utils
-
tools cleanup part7
-
-
11.76 (util cleanup)
-
tools cleanup part7
-
tools cleanup part6
-
tools cleanup part5
-
tools cleanup part4
-
tools cleanup part3
-
tools cleanup part2
-
tools cleanup part1
-
You can log the events and counts as part of you access log entries (adding the to the format definition), e.g. %{SlowRequest}e and %{SlowRequest_Counter}e
-
A.J. A.J. posted a comment on discussion Open Discussion
Thanks for the reply. My aggressive solution is based on the behavior of some overly-aggressive spiders. I've excluded static resources like images, so the idea is to enforce a reasonable rate on all other requests to my application (30 requests within 2 seconds). I indeed made a typo in my rules: the 20 should be 30 instead. So that explains why I saw log messages related to the "RepeatedlySlow" rule without first seeing "SlowRequest" entries. However, it's still a mystery why that rule was being...
-
I don't know why you decided on an aggressive two-stage solution. I suggest to start with a simple rule. Know the critical resources (e.g. slow application) which you want to protect and define a rule for that. About the two counters: RepeatedlySlow comes first as it blocks already at 20 requests (though the second time only) while SlowRequest waits until 30.
-
don't worry, apr_time_t is a 64bit (long or long long depending on your OS) you probably think about the year 2038 bug where 32bit integers will overflow
-
Ahmed Ahmed posted a comment on discussion Open Discussion
Yes, the problem is that the calculation for requests per time interval uses timestamps, and they are seconds that have passed from the default year of 1970, this number increases everyday, and now it overflows the datatypes used in the project, specially the apr_time_t. This happens to me with the official docker images.
-
First, thank you very much for creating this very useful module! My issue is that legitimate spiders like Googlebot are being blocked by mod_qos, even if the number of requests they send is well below the limit I've set. Here are my rules, which are based on your suggestions for some basic DOS protection: # Allows max 100 connections from a single ip address: QS_SrvMaxConnPerIP 100 # Don't allow a client IP to access a "handler" (not a static resource like # a jpg, gif, ..) more than 30 times within...
-
QS_EventPerSecLimit measures the current "requests per time interval" (default is 5 seconds https://mod-qos.sourceforge.net/#QS_SrvSampleRate) and calculates and applies a delay if necessary, https://mod-qos.sourceforge.net/glossary.html#requestPerSecond You should see "mod_qos(050): request rate limit" messages if a delay gets added. You can also watch the the calculated request rate using the status viewer (mod_status) or measure it by counting the log lines matching your rule (per five second...
-
I have this simple configuration here:- SetEnvIfExpr "%{HTTP:userId} =~ /([0-9]*)/" USER_ID=$1 CustomLog "logs/custom.log" "USER_ID=%{USER_ID}e" QS_EventPerSecLimit USER_ID 10 And the rate rate limiting is not there at all, I checked the source code and All I could find is that sconf->location_t is getting assigned in the qos_event_rs_cmd function but it never got used for actual rate limiting anywhere. Also many other rate limiting directives there that just modify the sconf variable for the qs_hp_ccfunction...
-
I have this simple configuration here:- SetEnvIfExpr "%{HTTP:msisdn} =~ /([0-9]*)/" MSISDN=$1 CustomLog "logs/custom.log" "MSISDN=%{MSISDN}e" QS_EventPerSecLimit MSISDN 10 And the rate rate limiting is not there at all, I checked the source code and All I could find is that sconf->location_t is getting assigned in the qos_event_rs_cmd function but it never got used for actual rate limiting anywhere. Also many other rate limiting directives there that just modify the sconf variable for the qs_hp_ccfunction...
-
I have this simple configuration here:- SetEnvIfExpr "%{HTTP:msisdn} =~ /([0-9]*)/" MSISDN=$1 CustomLog "logs/custom.log" "MSISDN=%{MSISDN}e" QS_EventPerSecLimit MSISDN 10 And the rate rate limiting is not there at all, I checked the source code and All I could find is that sconf->location_t is getting assigned in the qos_event_rs_cmd function but it never got used for actual rate limiting anywhere. Also many other rate limiting directives there that just modify the sconf variable for the qs_hp_ccfunction...
-
I have this simple configuration here:- SetEnvIfExpr "%{HTTP:msisdn} =~ /([0-9]*)/" MSISDN=$1 CustomLog "logs/custom.log" "MSISDN=%{MSISDN}e" QS_EventPerSecLimit MSISDN 10 And the rate rate limiting is not there at all, I checked the source code and All I could find is that sconf->location_t is getting assigned in the qos_event_rs_cmd function but it never got used for actual rate limiting anywhere. Also other rate limiting directives that just modify the sconf variable for the qoqs_hp_ccfunction...
-
psre mig util snapshot
-
Thank you for reporting the issue.
-
The current latest version of mod_qos, version 11.75, does not build with GCC 14. gcc -DHAVE_CONFIG_H -I. -I.. -DLINUX -D_REENTRANT -D_GNU_SOURCE -Wdate-time -D_FORTIFY_SOURCE=2 -I/usr/include/libpng16 -g -O2 -Werror=implicit-function-declaration -ffile-prefix-map=/<<PKGBUILDDIR>>=. -fstack-protector-strong -fstack-clash-protection -Wformat -Werror=format-security -fcf-protection -I/usr/include/apr-1.0 -I/usr/include/apr-1.0 -I/usr/include -I/usr/lib/include -c -o qsfilter2.o qsfilter2.c qsfilter2.c:...
-
You could use a client event counter to permanently mark an IP address (just ensure you never reach the counter's limit). Example for illustration (I've not tested this myself): # counter to remember VIP QS_ClientEventLimitCount 100 31536000 VIP_IP # keep the counter low (never reach the limit) QS_SetEnv VIP_IP_REFRESH 10 QS_SetEnvIfCmp VIP_IP_Counter gt VIP_IP_REFRESH VIP_IP_Decrement=2 # detect returning VIP user by IP QS_SetEnv VIP_IP_IS_SET 0 QS_SetEnvIfCmp VIP_IP_Counter gt VIP_IP_IS_SET QS_VipRequest=1...
-
Simon Studer modified a comment on discussion Open Discussion
Directives like QS_VipIPUser declare an IP as VIP based on request information. Is there a way to arbitrarily mark IPs as VIP, based on request information, similar to how requests can arbitrarily be marked as VIP by setting an environment variable? For example, if the username matches a regular expression, set environment variable, which will then not only mark the request as VIP but all requests from this IP. The directives QS_VipRequest and QS_IsVipRequest appear to only mark the request itself...
-
Directives like QS_VipIPUser declare an IP as VIP based on request information. Is there a way to arbitrarily mark IPs as VIP, based on request information, similar to how requests can arbitrarily be marked as VIP by setting an environment variable? For example, if the username matches a regular expression, set environment variable, which will then not only mark the request as VIP but all requests from this IP. The directives QS_VipRequest and QS_IsVipRequest appear to only mark the request itself...
-
Rahul Sharma posted a comment on discussion Open Discussion
Thanks. I was able to solve it.
-
I have been trying the exact same thing but its not working for me. Can you please check? Here's my config - # get client IP from HTTP request header instead of client IP QS_ClientIpFromHeader X-Forwarded-For #set this counter on every access ( "/" below is the URL ) SetEnvIf Request_URI / RepeatClientRequest SetEnvIf Remote_Addr 122.160.81.155 !RepeatClientRequest # no more than 2000 requests for any url in 5 minutes QS_ClientEventLimitCount 2000 300 RepeatClientRequest # deny a client IP for 30...
-
RemoteAddr is the IP address of the client (peer X-Forwarded-For is a HTTP request header field which contains an IP address (often set by proxy servers).
-
I have been trying the exact same thing but its not working for me. Can you please check? Here's my config - # get client IP from HTTP request header instead of client IP QS_ClientIpFromHeader X-Forwarded-For #set this counter on every access ( "/" below is the URL ) SetEnvIf Request_URI / RepeatClientRequest SetEnvIf Remote_Addr 122.160.81.155 !RepeatClientRequest # no more than 2000 requests for any url in 5 minutes QS_ClientEventLimitCount 2000 300 RepeatClientRequest # deny a client IP for 30...
-
@pbuchbinder Do we need to do this SetEnvIf X-Forwarded-For xxx.xxx.xxx.xxx !RepeatClientRequest if we are using X-Forwarded-For? QS_ClientIpFromHeader X-Forwarded-F
-
-
I solved it using the X-Forwarded-For instead of Remote_Addr The problem was that Remote_Addr did not have the client IP so I had to use the X-Forwarded-For - SetEnvIf X-Forwarded-For xxx.xxx.xxx.xxx !RepeatClientRequest But I wonder how it worked for you when you are also using the X-Forwarded-For QS_ClientIpFromHeader X-Forwarded-F
-
I solved it using the X-Forwarded-For instead of Remote_Addr The problem was that Remote_Addr did not have the client IP so I had to use the X-Forwarded-For - SetEnvIf X-Forwarded-For xxx.xxx.xxx.xxx !RepeatClientRequest But I wonder how it worked for 'rafa' when he was also using the X-Forwarded-For QS_ClientIpFromHeader X-Forwarded-F
-
The problem was that Remote_Addr did not have the client IP so I had to use the X-Forwarded-For - SetEnvIf X-Forwarded-For xxx.xxx.xxx.xxx !RepeatClientRequest But I wonder how it worked for @rafa when he was also using the X-Forwarded-For QS_ClientIpFromHeader X-Forwarded-F
-
Of course, we greatly appreciate the work you do for this project!
-
setting connection note short-lingering-close when aborting connection
-
Sali Simon Indeed, MPM event uses a shorter timeout when closing the connection if a module passes this note. I'm going to add this to mod_qos as well. Thank you for sharing this information. Regards, Pascal
-
Looking through the source code of mod_reqtimeout, I saw that it uses the short-lingering-close request note to considerably shorten the potential duration of a lingering close: https://github.com/apache/httpd/blob/trunk/modules/filters/mod_reqtimeout.c#L329-L335 I did not see this note being used in mod_qos. Would this be useful for mod_qos as well to shorten the lingering close duration in case of QoS intervention?
-
Looking through the source code of mod_reqtimeout, I saw that it uses the short-lingering-close request note to considerably shorten the potential duration of a lingering close: https://github.com/apache/httpd/blob/trunk/modules/filters/mod_reqtimeout.c#L329-L335 I did not see this note being used in mod_qos. Would this be useful for mod_qos as well to shorten the lingering close duration in case of QoS intervention?
-
mod_qos version 11.75 is now available. It uses the IP address determined by mod_remoteip by configuring QS_ClientIpFromHeader #USERAGENT_IP
-
-
11.75
-
test env
-
USERAGENT_IP
-
USERAGENT_IP
-
new build srv
-
I agree: mod_remoteip got fully integrated into Apache as of version 2.4.19... it's time for mod_qos to adapt and also support Apache's "useragent_ip" feature.
-
William Edwards posted a comment on discussion Open Discussion
Hi @pbuchbinder, I missed that nuance in the documentation. Sorry about that, and thanks for finding it. If mod_remoteip removes X-Forwarded-For, there are no headers containing the IP address (I could let the proxy add a header, but I'd rather not). And without QS_ClientIpFromHeader, mod_qos (rightly) believes the connection comes from the proxy. Are you familiar with successful mod_qos + mod_remoteip approaches? mod_remoteip is not a very exotic module. I found https://sourceforge.net/p/mod-qos/discussion/697421/thread/96040e61/,...
-
I can confirm that X-Forwarded-For is present. In fact, mod_remoteip uses it successfully I assume mod_remoteip is causing the problem. https://httpd.apache.org/docs/2.4/mod/mod_remoteip.html this header is removed from the request altogether mod_remoteip.c: else { apr_table_unset(r->headers_in, config->header_name); }
-
Hi, When using the QS_ClientIpFromHeader directive, mod_qos returns: mod_qos(069): no valid IP header found (@hp): header 'X-Forwarded-For' not available, fallback to connection's IP 10.40.23.9 Why? I can confirm that X-Forwarded-For is present. In fact, mod_remoteip uses it successfully (and I would expect mod_qos to honour it, but that's a different matter). The full config can be found below, although I can reproduce the issue when using the directive standalone. <IfModule mod_qos.c> QS_LogOnly...
-
you asked about QS_ClientIpFromHeader ....
-
Marc posted a comment on discussion Open Discussion
no, it does not. At least QS_SrvMaxConnPerIP uses the IP of the loadbalancer. Since all my trouble I switched to my own loadbalancer and now remove the bad traffic in front - but not all users will have that possibility.
-
Looking at the mod_remoteip code, I assume that using HAProxy's proxy protocol is transparent to other modules loaded into your Apache server. I assume you don't need to configure QS_ClientIpFromHeader.
-
So there is currently no way to use the parsed client IP that is available in log as %a ? Since this is a transparent proxy, the http header is not modified (hetzner).
-
mod_qos doesn't support HAProxy's proxy protocol yet but QS_ClientIpFromHeader can be used to read the client's address from a HTTP request header field such as X-Forwarded-For if another module adds it.
-
I have generic loadbalancer in front of my webservers. So the RemoteIPProxyProtocol is enabled and the remote ip adresse can be accessed via %a in the log file. I've seen the "QS_ClientIpFromHeader" directive, but I don't know how to configure it, in order to use the client address set by the proxy protocol. Any help would be great.
-
Thank you for the clarification!
-
-
Georg von Zezschwitz posted a comment on discussion Open Discussion
Well, this request is quite a while ago :-) I would love this feature, and woud be even willing to implement it. Would you accept (or let's say: check) a patch? Suggestion: Currently, mod_qos seems to "accept" something like "1.2.3.4/27", but it "ignores" it. As people might enter it like this, because they don't read the manual: I guess that reporting an error when somebody specifies a netmask might help.
-
Hi Simon I don't see the possibility to access the hello package to calculate the fingerprint. We would also need to persist the fingerprint as part of the SSL session (in order to have it for connections resuming the session). And: I would not suggest to add the fingerprint calculation to mod_qos but rather to another module fingerprinting module. Regards, Pascal
-
Anonymous modified a comment on discussion Open Discussion
I have loaded the module qos using Loadmodule in httpd.conf file and but getting access denied error cannot parse uri. Please let me know if i am missing something. I am trying to connect in secure mode 443.
-
Hi mod_qos changed how it calculates the maximum number of connections. As you don't use any connection based limitations, you can override the default by using the QS_MaxClients directive (to the same value as you have set for MaxRequestWorkers). Hope this helps... Regards, Pascal
-
Gabriel Pineda modified a comment on discussion Open Discussion
I understand that support with event is limited and noticed that there has been work to make it work better with it. We are currently using Apache 2.4.57 + event with mod_qos 11.70. When we test with 11.74 we are noticing increased memory usage by the parent httpd proc. We are not seeing same behavior if we use worker with 11.74 224246 root 20 0 8170m 7.6g 6.9g S 0.0 17.8 0:45.07 /usr/sbin/httpd vs 339370 root 20 0 3471m 3.0g 2.4g S 0.0 7.1 0:09.88 /usr/sbin/httpd it more than doubles the amount...
-
I understand that support with event is limited and noticed that there has been work to make it work better with it. We are currently using Apache 2.4.57 + event with mod_qos 11,70. When we test with 11.74 we are noticing increased memory usage by the parent httpd proc. We are not seeing same behavior if we use worker with 11.74 224246 root 20 0 8170m 7.6g 6.9g S 0.0 17.8 0:45.07 /usr/sbin/httpd vs 339370 root 20 0 3471m 3.0g 2.4g S 0.0 7.1 0:09.88 /usr/sbin/httpd it more than doubles the amount...
-
With other QoS solutions, we have seen the JA3 Fingerprint [1] being used to rate limit (often in conjunction with different connections attributes like IP address). Is there any chance JA3 fingerprints could be implemented in mod_qos? Is this even something that could be implemented in an Apache module or would it have to be implemented in Apache httpd directly or mod_ssl? [1] https://engineering.salesforce.com/tls-fingerprinting-with-ja3-and-ja3s-247362855967/
-
Hi Graig https://898860.bugs.gentoo.org/attachment.cgi?id=855734 shows mod_qos.c:1915:12: error: incompatible pointer to integer conversion assigning to 'unsigned int' from 'apr_os_thread_t' (aka 'struct __pthread *') [-Wint-conversion] id.tid = apr_os_thread_current(); ^ ~~~~~~~~~~~~~~~~~~~~~~~ which was soled in mod_qos 11.74. I recommend to use the latest version of mod_qos. Regards, Pascal
-
mod_qos 11.73 fails to compile with clang 16 due to an incompatible pointer to integer conversion. You can find the full error information at https://bugs.gentoo.org/898860 I've attached a patch which fixes this issue - could you please apply it and include it in future mod_qos releases? Thank you!
-
This is a sanity check. mod_qos verifies the request uses a request URI with at least a path portion. If you intend to tunnel other traffic through you web server, mod_qos is probably not suitable.
-
I have loaded the module qos using Loadmodule in httpd.conf file and but getting access denied error cannot parse uri. [Fri Jun 23 06:28:06.132536 2023] [ssl:debug] [pid 3745132:tid 4096785216] ssl_engine_kernel.c(391): [client ..*.27:52753] AH02034: Initial (No.1) HTTPS request received for child 0 (server localhost.localdomain:443) [Fri Jun 23 06:28:06.132961 2023] [authz_core:debug] [pid 3745132:tid 4096785216] mod_authz_core.c(844): [client ..*.27:52753] AH01628: authorization result: granted...
-
I am also facing the same issue after enable the mod qos module. Please help.
-
I am also facing the same issue after enable the mod qos module. Please help.
-
Ravi GH posted a comment on discussion Open Discussion
Hi Pascal, Thank for looking into it. Regarding SSL variables, I tried some more and found below: 1 <IfModule qos_module> 2 RequestHeaderPlus unset X-Hash 3 <Location /rest/> 4 SetEnvIfPlus SSL_CLIENT_S_DN_CN (.*) cn=$1 late 5 </Location> 6 SetHashHeaderPlus X-Hash cn 7 QS_ClientIpFromHeader X-Hash 8 QS_ClientEventLimitCount 3 60 9 QS_ErrorResponseCode 429 10 </IfModule> For above qos.config, I confirm from the log that Line-4 works as expected and value of variable ‘cn’ is correctly getting set...
-
Hi Pascal, Thank for looking into it. Regarding SSL variables, I tried some more and found below: 1 <IfModule qos_module> 2 RequestHeaderPlus unset X-Hash 3 <Location /rest/> 4 SetEnvIfPlus SSL_CLIENT_S_DN_CN (.*) cn=$1 late 5 </Location> 6 SetHashHeaderPlus X-Hash cn 7 QS_ClientIpFromHeader X-Hash 8 QS_ClientEventLimitCount 3 60 9 QS_ErrorResponseCode 429 10 </IfModule> For above qos.config, I confirm from the log that L-4 works as expected and value of variable ‘cn’ is correctly getting set from...
-
apr and apache update
-
-
11.74
-
fix potential counter overflow for early event detection
-
fix potential counter overflow for early event detection
-
fix potential counter overflow for early event detection
1 >
