tag/classify fix in branches 'tentative'?

nnn
2011-11-28
2013-03-05
  • nnn
    nnn
    2011-11-28

    struggling mightily to get fwb to 'do' adequate VoIP traffic shaping, I've created a rule,

    test/QOSrules
    Any:Any:Any:Any:Both:Continue:Any:(options)

    (options)
    General:  Stateless rule
    Tag: Tag140
          Tag connections … (adds … CONNMARK)
    Classify: 1:40

    which single-rule compiles to

    test / QOSrules / rule 0
    $IPTABLES -N QOSrules -t mangle
    $IPTABLES -t mangle -A QOSrules  -j MARK -set-mark qos140
    $IPTABLES -t mangle -A QOSrules  -j CONNMARK -save-mark
    $IPTABLES -t mangle -A QOSrules  -j CLASSIFY -set-class 1:40

    and in full compile,

        # ================ Table 'mangle', rule set QOSrules
        #
        # Rule QOSrules 0 (global)
        #
        echo "Rule QOSrules 0 (global)"
        #
        $IPTABLES -N QOSrules -t mangle
        $IPTABLES -t mangle -A QOSrules  -j MARK -set-mark qos140
        $IPTABLES -t mangle -A QOSrules  -j CONNMARK -save-mark
        $IPTABLES -t mangle -A QOSrules  -j CLASSIFY -set-class 1:40
        #
        # Rule QOSrules 1 (global)


        # ================ Table 'mangle', rule set _Policy
        #
        # Rule _Policy 9 (global)
        #
        echo "Rule _Policy 9 (global)"
        #
        $IPTABLES -t mangle -A PREROUTING  -j QOSrules
        $IPTABLES -t mangle -A POSTROUTING  -j QOSrules
        $IPTABLES -t mangle -A FORWARD  -j QOSrules

        # Rule _Policy 9 (global)
        #
        echo "Rule _Policy 9 (global)"
        #
        $IPTABLES -A OUTPUT  -j QOSrules
        $IPTABLES -A INPUT  -j QOSrules
        $IPTABLES -A FORWARD  -j QOSrules

    none of which looks exactly like what i read further @ http://www.fwbuilder.org/4.0/docs/users_guide/tag-rules.html

    # ================ Table 'mangle', automatic rules
    $IPTABLES -t mangle -A PREROUTING -j CONNMARK -restore-mark

    reading further @ http://www.fwbuilder.org/4.0/docs/firewall_builder_release_notes.html

    Changes in support for iptables

    "Tag and classify actions dont work properly with branches". When branching rule points to a rule set that has rules with Tag and Classify options, branching should occur in mangle table even when checkbox "create branch in mangle table" is not checked. The fix in this change is tentative as it creates branch in chains PREROUTING, POSTROUTING and OUTPUT. Since target CLASSIFY is only allowed in POSTROUTING, this may create conflict. Need to test more."

    I can't manage to follow through FWB's output to figure out if FWB is working the way it's supposed to or not.

    Is the "fix" still "tentative"?  Is there still a need to "test more"?

     
  • Vadim Kurland
    Vadim Kurland
    2011-11-28

    its tentative in the sense that we might be able to find better algorithm in the future. We haven't found it yet.

    the page in the Firewall Builder Users Guide you linked to above is quite accurate. May be you see an older version ? Try to reload the page (Shift-reload to bypass cached version)

     
  • nnn
    nnn
    2011-11-30

    @ kkomsalov

    > I see from your rules that you are attempting to do boot.

    'boot'?  do you mean 'both'?  i not, then i'm a bit confused.

    > Simple construction like next one should do the job
    > …

    thanks for the references.

    my current goal is to craft QoS policy for:

    one linux/iptables router/firewall
    multiple servers and VoIP client devices on the lan (no asterisk inside … yet)
    Hensema's fixes for openssh TOS (http://www.docum.org/docum.org/faq/cache/49.html)
    QoS for 'cleanest' VoIP traffic - both inbound and outbound.
    as much as possible "in" FWB gui …

    so far, i've been basing the script on:
    http://jaredwatkins.com/wordpress/wp-content/uploads/2010/04/TrafficShapingUnderLinuxForVoiceOverIP.pdf

    i'm not yet sure if that's similar, or somehow different than your approach …

    trying to get all the pieces working has been challenging …

     
  • >>I see from your rules that you are attempting to do boot.

    >>'boot'?  do you mean 'both'?  i not, then i'm a bit confused.

    Yes “both”, sorry.

    >my current goal is to craft QoS policy for:
    >
    >one linux/iptables router/firewall
    >multiple servers and VoIP client devices on the lan (no asterisk inside …
    >yet)
    >Hensema's fixes for openssh TOS
    >(http://www.docum.org/docum.org/faq/cache/49.html)
    >QoS for 'cleanest' VoIP traffic - both inbound and outbound.
    >as much as possible "in" FWB gui …

    Now it is really confusing as you are introducing third thing; actually even forth. “TOS -set-tos” and “-m limit” are two different things and are not queuing disciplines.
    Booth are very limited. -set-tos is reliable only if you have control on all routers on the route. “-m limit” it is good only to drive crazy some user you don’t like.

    >so far, i've been basing the script on:
    > http://jaredwatkins.com/wordpress/wp->content/uploads/2010/04/TrafficShapingUnde
    >rLinuxForVoiceOverIP.pdf

    >i'm not yet sure if that's similar, or somehow different than your approach …

    >trying to get all the pieces working has been challenging …

    Yes this is the same approach like main for the incoming traffic.
    This will give you warranted bandwidth for the VoIP and also ability to borrow and tec.
    It is not the same approach for the incoming traffic.
    If one of your users starts watching some streaming video, he will kill your phone.
    The construction in the document you follow:
    tc qdisc add dev $DEV handle ffff: ingress
    # filter *everything* to it (0.0.0.0/0), drop everything that's
    # coming in too fast:
    tc filter add dev $DEV parent ffff: protocol ip prio 100 u32 match ip src \
    0.0.0.0/0 police rate ${DOWNLINK}kbit burst 10k drop flowid :1
    Will only “if you make ${DOWNLINK}kbit slightly less than your real downlink speed” move the queue from the router of your provider to your box. Then it will rely that the box will balance the links fairly.
    It will not, if your clients are using most of the new streaming tricks, they have more than 1 to 10 relation of uplink to downlink. Limiting the client to 15k will still have around 600k udp steaming video from Akamai.
    You mentioned that you have only servers, then this is good enough.
    But if you have such troublesome users then you have to look on the second part of my script and do some control over the incoming traffic.
    /sbin/modprobe ifb
    ifconfig ifb0 up
    Is the device you need – there is no much documentation about it. It is really new.
    And the mirror income traffic to it.
    tc filter add dev $DEV parent ffff: protocol ip prio 10 u32 \
    match ip dst 0.0.0.0/0 flowid 1: \
    action mirred egress redirect dev ifb0

    As result your income traffic will be outgoing for ifb0, and you may apply some queuing disciplines similarly to the first part of either my script or the document you follow “it is same approach”.
    Then you may have “and have to have” rules for boot bad clients and warranted traffic for VoIP.
    Like this for bad client for example:
    tc class add dev ifb0 parent 1:3 classid 1:33 htb rate 100kbit burst 6k ceil 150kbit prio 3
    tc qdisc add dev ifb0 parent 1:33 handle 70: sfq perturb 10
    tc filter add dev ifb0 parent 1: protocol ip prio 2 u32 \
    match ip dst 172.17.128.178 flowid 1:33

    172.17.128.178 – loves streaming udp video

    P.S.
    Probably there is some more misspellings and missgramars, sorry in advance

    This is how it looks like:
    root@acer:/etc/rc.d# tc class ls dev ifb0
    class htb 1:33 parent 1:3 leaf 70: prio 3 rate 100000bit ceil 150000bit burst 6Kb cburst 1599b
    class htb 1:32 parent 1:3 leaf 60: prio 2 rate 1024Kbit ceil 5120Kbit burst 6Kb cburst 1599b
    class htb 1:31 parent 1:3 leaf 50: prio 1 rate 4096Kbit ceil 5120Kbit burst 6Kb cburst 1599b
    class htb 1:3 root rate 5120Kbit ceil 5120Kbit burst 6Kb cburst 1599b
    class htb 1:4 root leaf 22: prio 0 rate 100000Kbit ceil 100000Kbit burst 6125b cburst 1600b
    root@acer:/etc/rc.d#
    root@acer:/etc/rc.d#
    root@acer:/etc/rc.d# tc -s -d qdisc show dev ifb0
    qdisc htb 1: root refcnt 2 r2q 10 default 4 direct_packets_stat 0 ver 3.17
    Sent 1183547238 bytes 3181349 pkt (dropped 396, overlimits 1308551 requeues 0)
    backlog 0b 0p requeues 0
    qdisc sfq 22: parent 1:4 limit 127p quantum 1514b flows 127/1024 perturb 10sec
    Sent 5578432 bytes 15017 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0
    qdisc sfq 50: parent 1:31 limit 127p quantum 1514b flows 127/1024 perturb 10sec
    Sent 809780398 bytes 2686947 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0
    qdisc sfq 60: parent 1:32 limit 127p quantum 1514b flows 127/1024 perturb 10sec
    Sent 344915522 bytes 258162 pkt (dropped 396, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0
    qdisc sfq 70: parent 1:33 limit 127p quantum 1514b flows 127/1024 perturb 10sec
    Sent 23272886 bytes 221223 pkt (dropped 0, overlimits 0 requeues 0)
    backlog 0b 0p requeues 0
    root@acer:/etc/rc.d#
    root@acer:/etc/rc.d# tc -s class show dev eth0
    class htb 1:11 parent 1:1 leaf 10: prio 1 rate 460000bit ceil 512000bit burst 6Kb cburst 1600b
    Sent 97479190 bytes 1053576 pkt (dropped 0, overlimits 0 requeues 0)
    rate 5840bit 12pps backlog 0b 0p requeues 0
    lended: 1053547 borrowed: 29 giants: 0
    tokens: 1654359 ctokens: 376954

    class htb 1:1 root rate 512000bit ceil 512000bit burst 6Kb cburst 1600b
    Sent 429711076 bytes 1654207 pkt (dropped 0, overlimits 0 requeues 0)
    rate 11936bit 17pps backlog 0b 0p requeues 0
    lended: 374934 borrowed: 0 giants: 0
    tokens: 1486329 ctokens: 376954

    class htb 1:13 parent 1:1 leaf 30: prio 3 rate 10000bit ceil 15000bit burst 6Kb cburst 1599b
    Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
    rate 0bit 0pps backlog 0b 0p requeues 0
    lended: 0 borrowed: 0 giants: 0
    tokens: 76800000 ctokens: 13333328

    class htb 1:2 root leaf 12: prio 0 rate 100000Kbit ceil 100000Kbit burst 6125b cburst 1600b
    Sent 181812626 bytes 933903 pkt (dropped 0, overlimits 0 requeues 0)
    rate 488bit 0pps backlog 0b 0p requeues 0
    lended: 933237 borrowed: 0 giants: 0
    tokens: 7546 ctokens: 1875

    class htb 1:12 parent 1:1 leaf 20: prio 2 rate 51000bit ceil 512000bit burst 6Kb cburst 1600b
    Sent 332200887 bytes 600631 pkt (dropped 807, overlimits 0 requeues 0)
    rate 6096bit 6pps backlog 0b 0p requeues 0
    lended: 225726 borrowed: 374905 giants: 0
    tokens: 14534691 ctokens: 376954

     
  • One mistake:
    >Yes this is the same approach like main for the incoming traffic.
    >This will give you warranted bandwidth for the VoIP and also ability to borrow and tec.
    >It is not the same approach for the incoming traffic.

    Should be:
    >Yes this is the same approach like main for the outgoing traffic.

    This web interface is killing me.

     
  • nnn
    nnn
    2011-11-30

    thanks again for the insights.

    fyi, i found a very helpful, and seemingly thorough, document:

    "Complex Traffic Shaping/Control"
      http://www.shorewall.net/traffic_shaping.htm

    that addresses both inbound and outbound traffic shaping, as well as VoIP optimization AND IPv6 shaping.

    I need to evaluate whether I'm better off trying to create a piecemeal, undocumented solution wrapped around FWB, or switch to use of shorewall, losing the UI, but gaining the documented solution, and, apparently, a community of users that also do the same.

    I know there are a few folks doing pieces of it 'here' …  I'd really prefer to have a well-documented full solution, that can be handed off to a commercial support contract, but it's just not there yet, afaict.