QS_LocKBytesPerSecLimitMatch not working?

Anonymous
2011-11-17
2013-05-23

  • Anonymous
    2011-11-17

    Hi,

    I've just installed mod_qos and it seems to be set up and mostly working - I've got the /qos handler there and confirmed that QS_LocRequestLimitMatch is working as I would expect.

    However, when I try to use QS_LocKBytesPerSecLimitMatch it doesn't seem to limit the transfer speed for the URL. The download shows up as getting caught by the rule in /qos, I can see the limit is set (as 50), but "wait rate" and "current" values never change and my download speed is consistently above 50kbytes/sec.

    In the log file I see this:

       mod_qos(052): byte rate limit, rule: ^.*\\.exe$(50), kbytes/sec=537, delay=1705ms

    So it looks like it is catching the rule properly, but the download speed isn't limited.

    I've also tried using QS_LocKBytesPerSecLimit  but it seems to have the same problem.

    My apache config just looks like the below (this is in the VirtualHost section):

    QS_LocRequestLimitMatch         "^.*\.exe$"     2
    QS_LocKBytesPerSecLimitMatch    "^.*\.exe$"     50

    Any help would be appreciated!

     
  • Hi trog.
    Thanks for sharing your experiences.

    Yes, it can be possible that the bandwidth limitation can't be enforced accurately, e.g., the limitation is defined too low, clients do vary their behavior very fast, or the downloaded files are too small (especially if mod_qos is not used in a reverse proxy setup).

    But what you mentioned does not mean the the bandwidth is not correctly limited.

    1) The delay time (1705ms) is very hight. This may have two reasons: the limitation (50kbytes) is quite low and/or the ramp up time of the client requests was probably very short (fast changed from low to high traffic).
    2) The viewer does not show the bandwidth if the measured value has recently been updated. Note: update happens only at the end of a request processing and this might take some time if your clients download large files (I should probably improve this behavior).

    So my question: Do you have measured the bandwidth? If yes, for how long did you measure (since mod_qos uses a closed loop control algorithm to throttle the bandwidth, the transient oscillation may take some time)?

    An easy way to measure the bandwidth looking at the Apache's transfer log (per request log, you may use the http://opensource.adnovum.ch/mod_qos/qslog.html tool).

     

  • Anonymous
    2011-11-21

    Hi,

    Thanks for the reply.

    I measured the bandwidth (I assume you mean download speed?) by using wget - I ran a wget for maybe four or five minutes without seeing any effect before stopping. I tried from a few different networks just to rule out local network effects but I saw the same thing - full-speed downloads.

    For my purposes I really need the bandwidth limiting to take effect immediately after the download has started - and I also need to support fairly low limits (like 50kbytes/sec and below).

    I have since switched over to mod_bw  ( http://bwmod.sourceforge.net/ ) which does what I want - the download is immediately limited to 50 kbytes/sec. I would rather use mod_qos if possible though just because it seems to be getting actively developed :)

    Thanks!

     
  • Eric Janz
    Eric Janz
    2012-05-23

    Hi,

    I have also a similar issue.

    I setup mod_qos as follows:

    SetEnvIfNoCase Referer "http://www.somedomain.com" SLOWDOWN_REFERER=1

    QS_EventRequestLimit SLOWDOWN_REFERER 100
    QS_EventKBytesPerSecLimit SLOWDOWN_REFERER 640

    (I tried also with other values, lower ones, without QS_EventRequestLimit, but the result still is the same)

    Apache restarts correctly:
      Apache/2.2.8 (Ubuntu) DAV/2 SVN/1.4.6 PHP/5.2.4-2ubuntu5.23 with Suhosin-Patch mod_qos/10.5 configured - resuming normal operations

    But retrieving file with wget still gets not limited:

    me@myhost:~/t$ wget www.mydomain.com/tools.tar -referer="http://www.somedomain.com" -output-document=/dev/null -no-proxy -no-cache
    -2012-05-23 19:44:56-  http://www.mydomain.com/tools.tar
    Resolving Hostname www.mydomain.com… XX.YY.ZZ.WW
    Connecting to www.mydomain.com|XX.YY.ZZ.WW|:80… connected.
    HTTP-Request sent, waiting for response… 200 OK
    Size: 107458560 (102M)
    Save in »/dev/null«.

    100% 107.458.560 2,03M/s   in 42s    

    2012-05-23 19:45:37 (2,46 MB/s) - »/dev/null« saved

    me@myhost:~/t$

    In the Apache  error log I can see:

       mod_qos(052): byte rate limit, rule: var={SLOWDOWN_REFERER}(640), kbytes/sec=1520, delay=68ms, referer: http://www.somedomain.com

    Any ideas why this seems not to work for me?
    Best regards,
    Eric Janz

     
  • As mentioned above, bandwidth throttling works best when using mod_qos within a revers proxy setup (using mod_proxy) because the module adds the delay on a per bucket brigade basis (best when sending the data in 8kb blocks). It does not work very well when serving local files, especially when having sendfile enabled on plaintext (non HTTPS) connections.

     
  • Eric Janz
    Eric Janz
    2012-05-25

    Hi,

    thanks for your fast reply!

    OK, I now do understand. The problem is using mod_qos to throttle bandwidth accessing files which were served by Apache from local. I should use Apache + mod_qos to throttle bandwidth accessing remote apps with mod_proxy or similar instead of using it directly on local files.

    I have now tried succesfully  a setup with tc + iptables using the string match capabilities to mark and classify packets into qos queues.

    Thanks and best regards!
    Eric Janz