Implementing Quality of Service with iptables CLASSIFY rules

ImageStream's version 4.2 releases include support for the iptables CLASSIFY directive, making quality of service configuration simpler and more flexible. Instead of using complex matches in the advanced traffic control (tc) utility, you can use ImageStream's bwinit and bwadd utilities and the powerful iptables utilities to sort traffic into individual queues. In a default network, all traffic shares a single queue, and the router does not provide any service differentiation. In this type of configuration, any one traffic flow could use all of the available bandwidth and cause service issues with critical traffic flows. In the example below, we will use a network configuration that has a single local Ethernet segment and a single Serial device. All Quality of Service rules are applied in the transmit direction. Once traffic has arrived at the router, bandwidth usage has already happened and quality of service in the receive direction would have little effect.

====ETHERNET===================== 192.168.1.0/24 ===
|
| LAN IP: 192.168.1.1 (100 Mbps)
+-------+--------+
| Ethernet0 |
| Router A |
| Serial0 |
+-------+--------+
| WAN IP: 192.168.54.1/30 (255.255.255.252) (1.5 Mbps)
|
====INTERNET/WAN=================

Before You Start

Map out the service classes that you want to provide on your router. Be careful not to overallocate bandwidth, or lower priority services may not receive bandwidth allocations. The IP addresses used in this Technical note are examples only. You will need to use an IP network given to you by your Internet Service Provider.

Determining Service Classes

In this example, we are going to assume the following service classes:

1. Voice over IP phone with highest priority at 192.168.1.5
2. SSH/telnet (interactive character) traffic with high priority
3. "Default" service class for all non-classified traffic
4. Non-realtime e-mail traffic at low priority
5. Security camera (192.168.1.6) FTP traffic at low priority

For detailed information on blocking or limiting Peer-to-Peer networking traffic, please see our Limiting P2P Traffic Technical Note.

The configuration will define five service classes under a common leaf class:
+---------+
| root 1: |
+---------+
|
+---------------------------------------+
| class 1:1 |
+---------------------------------------+
| | | | |
+----+ +----+ +----+ +----+ +----+
|1:10| |1:15| |1:20| |1:30| |1:35|
+----+ +----+ +----+ +----+ +----+

It is possible to configure classes under the root class, but this limits your ability to create additional leaf classes with different service classes and priorities. Using multiple classes can be important in networks that have differeing requirements for business and residential users, for example. The service classes will be identical on both Serial0 and Ethernet0.

Configuring Quality of Service Queues Using bwinit and bwadd

ImageStream's bwinit and bwadd utilities allow users to define traffic control classes with a simpler utility than the advanced 'tc' utility provided with ImageStream routers. In this example, we will start with the rules for bandwidth allocation (please note the use of the shorthand "eth0" in the configurations below):

Bandwidth allocations:

Total bandwidth available: 1.5 Mbps (limit of Serial0 speed)

1. VoIP -- Minimum guarantee: 256 Kbps, Maximum allowed: 1 Mbps, Priority: 1
2. SSH/telnet -- Minimum guarantee: 32 Kbps, Maximum allowed: 128 Kbps, Priority: 2
3. "Default" -- Minimum guarantee: 128 Kbps, Maximum allowed: 1 Mbps, Priority: 3
4. E-Mail -- Minimum guarantee: 128 Kbps, Maximum allowed: 512 Kbps, Priority: 4
5. Security camera -- Minimum guarantee: 128 Kbps, Maximum allowed: 1500Kbps, Priority: 5

We have chosen to allocate a small amount of bandwidth to SSH/telnet, since these applications are character-oriented and depend on user typing speeds (typically only a few bits per second). Please note that if the VoIP requires 1 Mbps, the remaining classes will have very little bandwidth available. It is possible with this configuration for e-mail traffic and security camera traffic to receive no bandwidth during high traffic conditions in the VoIP or default classes. You may need to adjust the maximum guaranteed bandwidths to ensure available bandwidth for all classes. In this example, e-mail and security camera traffic are low priority and will not be guaranteed bandwidth if the higher priority classes require 1.5 Mbps of bandwidth.

As noted above, the configurations for Ethernet0 and Serial0 are identical. First, we must initialize the router's QoS layers and specify both the maximum bandwidth available to the device as well as the default traffic class. Use of the default traffic class is not required, but strongly recommended to avoid the possibility that traffic is not classified into a QoS queue. In the router's Quality of Service configuration (Option 1, Option 4, Option 1, Option 1 from the router's main menu), we will add the following statements:
bwinit --dev eth0 --bandwidth 1536Kbit --default 20
bwinit --dev Serial0 --bandwidth 1536Kbit --default 20

In the above configuration, we have specified a device (eth0 and Serial0, respectively), the total bandwidth available (1.5 Mbps, or T1 speeds) to both devices and a default traffic class (20, in both cases). The default class instructs the router to sort all unclassified traffic into class 20 automatically.

Next, we must configure the bandwidth allocations for each of the five classes specified above:
#Rules for Ethernet0
bwadd --dev eth0 --classid 10 --minimum 256Kbit --maximum 1Mbit --priority 1
bwadd --dev eth0 --classid 15 --minimum 32Kbit --maximum 128Kbit --priority 2
bwadd --dev eth0 --classid 20 --minimum 128Kbit --maximum 1Mbit --priority 3 --fair-queue
bwadd --dev eth0 --classid 30 --minimum 128Kbit --maximum 512Kbit --priority 4 --fair-queue
bwadd --dev eth0 --classid 35 --minimum 128Kbit --maximum 1500Kbit --priority 5 --fair-queue

#Rules for Serial0
bwadd --dev Serial0 --classid 10 --minimum 256Kbit --maximum 1Mbit --priority 1
bwadd --dev Serial0 --classid 15 --minimum 32Kbit --maximum 128Kbit --priority 2
bwadd --dev Serial0 --classid 20 --minimum 128Kbit --maximum 1Mbit --priority 3 --fair-queue
bwadd --dev Serial0 --classid 30 --minimum 128Kbit --maximum 512Kbit --priority 4 --fair-queue
bwadd --dev Serial0 --classid 35 --minimum 128Kbit --maximum 1500Kbit --priority 5 --fair-queue

Again, please note that the rules for Ethernet0 and Serial0 are identical. Each set of queues will apply only in the transmit direction ("upstream" traffic leaves on Serial0, "downstream" traffic leaves the router on Ethernet0). In most networks where upload and download speeds are symmetric, the rules for the inward-facing and outward-facing interfaces will be identical. The class identification numbers we have chosen for this example are arbitrary. You may select a range of valid integers, and the class identifiers do not have to match from interface to interface.

The rules follow a common format:

bwadd:
Specifies that the router is adding a rule to an initialized interface.

--dev DEVICE:
Specifies that the router is adding a rule to the device named DEVICE.

--classid XX:
Configures a QoS class with the identifier "XX". This class value is used later by iptables.

--minimum XX:
Configures the minimum guaranteed bandwidth ("XX") for this class. The router will attempt to always guarantee this amount of traffic to the class.

Kbit, Mbit:
Notation used to specify Kilobits and Megabits. Only whole numbers are valid, so 1.5 Megabit becomes 1500Kbit.

--maximum XX:
Configures the maximum allowed bandwidth ("XX") for this class. The router will not allow traffic in this class to exceed the specified maximum.

--priority XX:
Specifies an optional priority value ("XX") from 0 to 8 used to rank classes in order of bandwidth allocations.

--fair-queue:
Enables Stochastic Fair Queuing (SFQ) for the class. This option accepts no configuration variables. The router will allocate bandwidth fairly to all class members. This is especially useful in busy classes where a combination of users with different bandwidth usage profiles share a common bandwidth limit. SFQs help to avoid a single user or small group of users from using all of the bandwidth available to a class to the exclusion of other class members.

Both the --minimum and --maximum values are not required. You may specify only a minimum or maximum, and the router will automatically set the two values equal to each other. The router will not attempt to check for oversubscriptions or overallocations. Please carefully check your configurations for proper bandwidth allocation for your network requirements. Save your Quality of Service configuration. It is not necessary to instate the rules immediately, but you will need to instate the rules before the configuration will take effect.

Classifying Traffic Using iptables CLASSIFY

Now that we have configured the Quality of Service queues, we must sort traffic into the proper queues. While it is possible to do this from within the 'tc' QoS utility, using the iptables CLASSIFY directive provides a simpler, more flexible and more powerful method. In the router's firewall configuration (Option 1, Option 4, Option 2, Option 1 from the main menu), we will add the following statements below. For our VoIP phone, we can match traffic based on the source or destination IP address. We will map traffic coming from (Serial0) or going to (Ethernet0) the address 192.168.69.5 and add that traffic to class 1:10, which we mapped out originally and configured in the previous step:

#Phone traffic
iptables -A POSTROUTING -t mangle -o eth0 -d 192.168.69.5 -j CLASSIFY --set-class 1:10
iptables -A POSTROUTING -t mangle -o Serial0 -s 192.168.69.5 -j CLASSIFY --set-class 1:10

iptables CLASSIFY directives are always added to the POSTROUTING chain's mangle table. For more information about the path a packet follows through iptables, please see ImageStream's iptables tutorial or the official iptables HOWTO. Please note the significant difference between the two rules: the location of the "192.168.69.5" address. For traffic leaving the network on Serial0, 192.168.69.5 will be the source address. For reply traffic returning to the network (Ethernet0), the 192.168.69.5 will appear as the destination address.

The rules use several different elements, explained below:

iptables:
Specifies that the router is adding an iptables rule

-A POSTROUTING:
Appends (-A) a rule to the router's POSTROUTING chain

-t mangle:
Appends the rule to the mangle (-t mangle) table inside the specified chain

-o eth0, -o Serial0:
Specifies that only packets that match the specified outbound (-o) interface will match the rule. CLASSIFY rules will always be applied to an outbound interface.

-d XX:
Specifies that only packets with a destination address of "XX" will match the rule.

-s XX:
Specifies that only packets with a source address of "XX" will match the rule.

-j CLASSIFY:
Instructs iptables to take an action (-j) on packets matching this rule, in this case to CLASSIFY them into a QoS queue

--set-class 1:XX:
Instructs iptables to add matching packets to class ID 1:XX.

Next, we will add rules for our interactive traffic class. First, add the rules for telnet traffic, which uses port 23:

#telnet traffic
iptables -A POSTROUTING -t mangle -o eth0 -p tcp --sport 23 -j CLASSIFY --set-class 1:15
iptables -A POSTROUTING -t mangle -o Serial0 -p tcp --dport 23 -j CLASSIFY --set-class 1:15

Next, we will add rules for interactive SSH traffic. This traffic uses port 22, but we must also match the ToS bit for Minimize-Delay (0x10). Secure copy (SCP) traffic also uses port 22, but does not set the ToS bit. Please note that some SSH applications, such "putty" and "SecureCRT", do not set the ToS bit on interactive SSH traffic and will not match this rule. There is no workaround for programs that do not properly set the ToS bit.

#ssh traffic
iptables -A POSTROUTING -t mangle -o eth0 -p tcp -m tos --tos 0x10 --sport 22 -j CLASSIFY --set-class 1:15
iptables -A POSTROUTING -t mangle -o Serial0 -p tcp -m tos --tos 0x10 --dport 22 -j CLASSIFY --set-class 1:15

Notice that the above rules match both port 22 and the Minimize-Delay (0x10) ToS bit. The rules above use some additional elements, explained below:

-p tcp:
Specifies that only TCP packets will match the rule ("udp", "icmp" and others are accepted also). The -p directive must be included when using --dport or --sport.

--dport XX:
Specifies that only packets with a destination port number of "XX" will match the rule. Requires the use of -p.

--sport XX:
Specifies that only packets with a source port number of "XX" will match the rule. Requires the use of -p.

-m tos:
Load the Type of Service match module for iptables. The -m tos directive must be included when matching a ToS bit (--tos).

--tos XX:
Specifies that only packets with tos bit of "XX" will match the rule. Names such as "Minimize-Delay" are acceptable instead of binary values. Requires the use of -m tos.

Next, we will add rules for e-mail traffic class. We must match 3 ports: SMTP (25), POP (110) and IMAP (143):

#Mail traffic
iptables -A POSTROUTING -t mangle -o eth0 -p tcp --sport 25 -j CLASSIFY --set-class 1:30
iptables -A POSTROUTING -t mangle -o eth0 -p tcp --sport 110 -j CLASSIFY --set-class 1:30
iptables -A POSTROUTING -t mangle -o eth0 -p tcp --sport 143 -j CLASSIFY --set-class 1:30
iptables -A POSTROUTING -t mangle -o Serial0 -p tcp --dport 25 -j CLASSIFY --set-class 1:30
iptables -A POSTROUTING -t mangle -o Serial0 -p tcp --dport 110 -j CLASSIFY --set-class 1:30
iptables -A POSTROUTING -t mangle -o Serial0 -p tcp --dport 143 -j CLASSIFY --set-class 1:30

Finally, we will add rules to match FTP traffic from our security camera at 192.168.1.6. The rule will match both the IP address and port numbers (20 and 21):

iptables -A POSTROUTING -t mangle -o eth0 -p tcp -d 192.168.1.6 --sport 20:21 -j CLASSIFY --set-class 1:35
iptables -A POSTROUTING -t mangle -o Serial0 -p tcp -s 192.168.1.6 --dport 20:21 -j CLASSIFY --set-class 1:35

Please note the use of port ranges ("20:21" or "20,21", which is equivalent). Be careful to match the correct source or destination port. When the FTP replies to the security camera, it will reply from ports 20 or 21 to the 192.168.1.6 IP address. Accordingly, we have used the "-d" and "--sport" directives. Save your firewall configuration. It is not necessary to instate the rules immediately, but you will need to instate the rules before the configuration will take effect.

Troubleshooting/Observation

You can use the router's rule and packet counters (Option 6 on the firewall menu) to display the CLASSIFY rules and to verify that packets are successfully matching the rules you have entered. You will find the rules listed under the mangle table. The router will prompt you to display mangle table rules. Alternately, you can use the "iptables -L -n -v -t mangle" command from the Bash shell to view the same output.

You can observe the Quality of Service rules for each interface from the Bash shell by using the command "tc -s class show dev XXX" where XXX is the device you want to view. The output will display all configured classes, including the root and parent leaf class as well as current throughput information for each class.

Remember to save your configurations to flash whenever you are finished configuring your dynamic routing setup!