MikroTik RouterOS Queues and Bandwidth Management
DRAFT
Document revision 15-May-2002
This document applies to the MikroTik RouterOS V2.5
The queuing can be used for limiting the bandwidth for certain IP addresses, protocols or ports. The queuing is performed for packets leaving the router through an interface. It means that the queues should always be configured on the outgoing interface regarding the traffic flow. If there is a desire to limit the traffic arriving at the router, then it should be done at the outgoing interface of some other router.
References on Class-Based Queuing (CBQ) can be found at http://www.aciri.org/floyd/cbq.html
The MikroTik RouterOS V2.5 has different queue setup compared to the previous versions. When migrating from V2.4 to V2.5, please note that:
Installation
The queue management feature is included in the 'system' software package.
No additional software package installation is needed for this feature.
Configuring Simple Queues
Simple queues can be used to set up bandwidth management for the
whole traffic leaving an interface, or for certain source and/or destination addresses.
For more sophisticated queue setup use the queue trees described further on.
To add simple queues, use the /queue simple add command:
[MikroTik] queue simple>
add dst-address=192.168.0.0/24 interface=ether1 limit-at=128000
[MikroTik] queue simple> print
Flags: X - disabled, I - invalid
0 name="" src-address=0.0.0.0/0 dst-address=192.168.0.0/24
interface=ether1 limit-at=128000 queue=default priority=8 bounded=yes
[MikroTik] queue simple>
Argument description:
name - descriptive name for the queue
src-address - Source IP address. Can be in the form a.b.c.d/n, that consists of the IP address, and the number of bits in the network mask
src-netmask - Source netmask in decimal form xxx.xxx.xxx.xxx
dst-address - Destination IP address. Can be in the form a.b.c.d/n, that consists of the IP address, and the number of bits in the network mask
dst-netmask - Destination netmask in decimal form xxx.xxx.xxx.xxx
interface - Interface which packet leaves. Queues work only for packets leaving the interface.
limit-at - Maximum stream bandwidth (bits/s). '0' means no limit (default for the interface).
queue - queue type. If you specify the queue type other than 'default', then it overrides the default queue type set for the interface under '/queue interface'. See the '/queue type' for available types.
priority - Flow priority (1..8)
bounded - Queue is bounded. If set to 'yes', the queue can not occupy bandwidth of other queues. If set to 'no', the queue would use over the allocated bandwidth whenever possible. Only in case when other queues (the actual queues) are getting too long and a connection is not being satisfied, then the 'not-bounded' queues would be limited at their allocated bandwidth.
To track how the rules are processed, see the bytes and packets counters for the queues:
[MikroTik] queue simple> .. tree print
Flags: X - disabled, I - invalid, D - dynamic
0 D name="" parent=ether1 flow="" limit-at=128000 max-burst=20
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=23543
packets=76
[MikroTik] queue simple>
Queue rules are processed in the order they appear in the /queue tree print list. If some packet matches the queue rule, then the queuing mechanism specified in that rule is applied to it, and no more rules are processed for that packet.
The queue types are used to specify some common argument values for queues. There are four default built-in queue types: default, ethernet-default, wireless-default, and synchronous-default. The built-in queue types cannot be removed. You can add your own queue types by specifying the argument values, for example:
[MikroTik] queue type>
add name=CUSTOMER-def kind=red red-min-threshold=0 red-burst=0
[MikroTik] queue type> print
0 name=default kind=none bfifo-limit=15000 pfifo-limit=10 red-limit=60
red-min-threshold=10 red-max-threshold=50 red-burst=20 sfq-perturb=5
sfq-allot=1514
1 name=ethernet-default kind=none bfifo-limit=15000 pfifo-limit=10
red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
sfq-perturb=5 sfq-allot=1514
2 name=wireless-default kind=sfq bfifo-limit=15000 pfifo-limit=10
red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
sfq-perturb=5 sfq-allot=1514
3 name=synchronous-default kind=red bfifo-limit=15000 pfifo-limit=10
red-limit=60 red-min-threshold=10 red-max-threshold=50 red-burst=20
sfq-perturb=5 sfq-allot=1514
4 name=CUSTOMER-def kind=red bfifo-limit=15000 pfifo-limit=10 red-limit=60
red-min-threshold=0 red-max-threshold=50 red-burst=0 sfq-perturb=5
sfq-allot=1514
[MikroTik] queue type>
Argument description:
name - (required) name for the queue type
kind - kind of the queuing algorithm used (bfifo / none / pfifo / red / sfq)
pfifo - Packet First-In First-Out – is the simplest queuing algorithm. The packets are served in the same order as they are received.
bfifo-limit - BFIFO queue limit. Maximum packet number that queue can hold.
bfifo - The same as pfifo, except that this algorithm is byte-based but not packet-based.
red - Random Early Detection – an algorithm for congestion avoidance in packet-switched networks.
sfq - Stochastic Fair Queuing.
none - (same as default) The queue type as it is by default for the specific interface.
pfifo-limit - PFIFO queue limit. Maximum byte number that queue can hold.
red-limit - RED queue limit
red-min-threshold - RED minimum threshold. Before this value is achieved no packets will be thrown away.
red-max-threshold - RED maximum threshold. When this value is achieved the queue will throw away the packets using maximum probability, where this probability is a function of the average queue size.
red-burst - RED burst. Number of packets allowed for bursts of packets when there are no packets in the queue. The minimum value that can be used here is equal to the value of 'red-min-threshold'.
sfq-perturb -
sfq-allot -
For small limitations (64kbps, 128kbps) RED is more preferable. For larger speeds PFIFO will be as good as RED. RED consumes more memory and consumes more CPU than PFIFO & BFIFO.
Setting Default Queue Type for the Interface
To change the default queue type for the interface,
use the '/queue interface set' command, e.g.:
[MikroTik] queue interface> print # INTERFACE QUEUE 0 ether1 ethernet-default 1 prism1 default [MikroTik] queue interface> set prism1 queue=wireless-default [MikroTik] queue interface> print # INTERFACE QUEUE 0 ether1 ethernet-default 1 prism1 wireless-default [MikroTik] queue interface>
Configuring Queue Trees
The queue trees should be used when you want to use sophisticated
bandwidth allocation based on protocols, ports, groups of IP addresses, etc.
If you have added a simple queue, it is listed as dynamic one under the '/queue tree print', e.g.:
[MikroTik] queue tree> .. simple print
Flags: X - disabled, I - invalid
0 name=A_Simple src-address=0.0.0.0/0 dst-address=192.168.0.0/24
interface=ether1 limit-at=128000 queue=default priority=8 bounded=yes
[MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
0 D name=A_Simple parent=ether1 flow="" limit-at=128000 max-burst=20
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=56234
packets=634
[MikroTik] queue tree>
Argument description:
name - descriptive name for the queue
parent - (required) name of the parent queue. The top-level parents are the available interfaces. Lower level parents can be other queues. Dynamic queues (created with the simple queue tool) cannot be used as parents.
flow - flow mark of the packets to be queued. Flow marks can be assigned to the packets under /ip firewall mangle when the packets enter the router through the incoming interface
limit-at - Maximum stream bandwidth (bits/s). '0' means no limit (default for the interface).
max-burst - Maximal number of packets allowed for bursts of packets when there are no packets in the queue. Set to '0' for no burst.
queue - queue type. See the '/queue type' for available types.
priority - Flow priority (1..8)
weight - Flow weight
allot - Number of bytes allocated for the bandwidth. Should not be less than the MTU for the interface.
bounded - Queue is bounded. If set to 'yes', the queue can not occupy bandwidth of other queues. If set to 'no', the queue would use over the allocated bandwidth whenever possible. Only in case when other queues (the actual queues) are getting too long and a connection is not being satisfied, then the 'not-bounded' queues would be limited at their allocated bandwidth.
bytes, packets - number of bytes and packets processed by this queue. The counters can be reset using the 'reset' command
IP packet flow through the router is given in the following diagram:
As wee see from the diagram, we should use the /ip firwall mangle to mark the packets of the incoming flow, and then apply the queues on them when the packets leave the router through the outgoing interface.
To mark the packets, use the mangle feature:
[MikroTik] ip firewall mangle>
add action=mangle mark-flow=abc-http protocol tcp src-port=80
[MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid
0 src-address=0.0.0.0/0:80 in-interface=all dst-address=0.0.0.0/0:0-65535
protocol=tcp tcp-options=any icmp-options=any:any
src-mac-address=00:00:00:00:00:00 limit-count=0 limit-burst=0
limit-time=0s action=mangle mark-flow=abc-http tcp-mss=dont-change
bytes=0 packets=0
[MikroTik] ip firewall mangle>
See the Firewall Filters and Network Address Translation (NAT) Manual for details on how to mark the packets.
You can add queue using the /queue tree add command:
[MikroTik] ip queue tree>
add name=HTTP parent=ether1 flow=abc-http limit-at=128000 \
max-burst=0 bounded=yes
[MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
0 D name=A_Simple parent=ether1 flow="" limit-at=128000 max-burst=20
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=0
packets=0
1 name=HTTP parent=ether1 flow=abc-http limit-at=128000 max-burst=0
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=0
packets=0
[MikroTik] queue tree>
Queue Applications
One of the ways to avoid network traffic ‘jams’ is usage of traffic shaping in large networks.
Traffic shaping and bandwidth allocation is implemented in the MikroTik RouterOS as queuing mechanism.
Thus, the network administrator is able to allocate a definite portion of the total bandwidth
and grant it to a particular network segment or interface.
Also the bandwidth of particular nodes can be limited by using this mechanism.
Further on, several examples of using bandwidth management are given arranged according to complexity:
Assume we want to emulate a 128k download and 64k upload line connecting IP network 192.168.0.0/24. The network is served through the Local interface of customer's router. The basic network setup is in the following diagram:
The IP addresses and routes of the MikroTik router are as follows:
[MikroTik] > ip address print
Flags: X - disabled, I - invalid, D - dynamic
# ADDRESS NETWORK BROADCAST INTERFACE
0 10.0.0.217/24 10.0.0.217 10.0.0.255 Public
1 192.168.0.254/24 192.168.0.0 192.168.0.255 Local
[MikroTik] > ip route print
Flags: X - disabled, I - invalid, D - dynamic, J - rejected,
C - connect, S - static, R - rip, O - ospf, B - bgp
# DST-ADDRESS G GATEWAY DISTANCE INTERFACE
0 S 0.0.0.0/0 r 10.0.0.1 1 Public
1 DC 192.168.0.0/24 r 0.0.0.0 0 Local
2 DC 10.0.0.0/24 r 0.0.0.0 0 Public
[MikroTik] >
Assume you want to limit the bandwidth to 128kbps on downloads and 64kbps on uploads for all hosts on the LAN. Bandwidth limitation is done by applying queues for outgoing interfaces regarding the traffic flow. It is enough to add two queues at the MikroTik router:
[MikroTik] queue simple>
add name=Down interface Local limit-at 128000
add name=UP interface Public limit-at 64000
[MikroTik] queue simple> print
Flags: X - disabled, I - invalid
0 name=Down src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Local
limit-at=128000 queue=default priority=8 bounded=yes
1 name=UP src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Public
limit-at=64000 queue=default priority=8 bounded=yes
[MikroTik] queue simple> .. tree print
Flags: X - disabled, I - invalid, D - dynamic
0 D name=Down parent=Local flow="" limit-at=128000 max-burst=20
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=60
packets=1
1 D name=UP parent=Public flow="" limit-at=64000 max-burst=20
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=4169
packets=30
[MikroTik] queue simple>
Leave all other parameters as set by default. The limit is approximately 128kbps going to the LAN and 64kbps leaving the client's LAN. Please note, that the queues have been added for the outgoing interfaces regarding the traffic flow.
To monitor the traffic flow through the interface while doing file transfer, use the '/interface monitor-traffic' command:
[MikroTik] interface> monitor-traffic Public
received-packets-per-second: 9
received-bits-per-second: 4.32kbps
sent-packets-per-second: 6
sent-bits-per-second: 65.58kbps
received-packets-per-second: 7
received-bits-per-second: 3.36kbps
sent-packets-per-second: 10
sent-bits-per-second: 65.15kbps
received-packets-per-second: 11
received-bits-per-second: 5.66kbps
sent-packets-per-second: 7
sent-bits-per-second: 52.70kbps
[MikroTik] interface>
If you want to exclude the server from being limited, add two queues for it with limit-at=0 (no limit) and move them to the top:
[MikroTik] queue simple>
add name=Serv_D interface=Local dst-address=192.168.0.17/32 limit-at=0
add name=Serv_U interface Public src-address=192.168.0.17/32 limit-at=0
[MikroTik] queue simple> print
Flags: X - disabled, I - invalid
0 name=Down src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Local
limit-at=128000 queue=default priority=8 bounded=yes
1 name=UP src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Public
limit-at=64000 queue=default priority=8 bounded=yes
2 name=Serv_D src-address=0.0.0.0/0 dst-address=192.168.0.17/32
interface=Local limit-at=0 queue=default priority=8 bounded=yes
3 name=Serv_U src-address=192.168.0.17/32 dst-address=0.0.0.0/0
interface=Public limit-at=0 queue=default priority=8 bounded=yes
[MikroTik] queue simple> move 2 0
[MikroTik] queue simple> move 3 1
[MikroTik] queue simple> print
Flags: X - disabled, I - invalid
0 name=Serv_D src-address=0.0.0.0/0 dst-address=192.168.0.17/32
interface=Local limit-at=0 queue=default priority=8 bounded=yes
1 name=Serv_U src-address=192.168.0.17/32 dst-address=0.0.0.0/0
interface=Public limit-at=0 queue=default priority=8 bounded=yes
2 name=Down src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Local
limit-at=128000 queue=default priority=8 bounded=yes
3 name=UP src-address=0.0.0.0/0 dst-address=0.0.0.0/0 interface=Public
limit-at=64000 queue=default priority=8 bounded=yes
[MikroTik] queue simple>
Example of Using Masquerading
If masquerading is used for the local address space 192.168.0.0/24 of
the client computers in the previous example setup,
then the outgoing traffic has masqueraded source address 10.0.0.217, i.e.,
the outgoing packets have external address of the router as the source.
If you use simple queues, as in the previous example, the queuing rule for incoming traffic should match the customer's local addresses, whereas the rule for outgoing traffic should match the router's external address as the source address. The previous example would work fine, but you cannot exclude the server from being limited.
To apply specific queuing for the server, use '/ip firewall mangle' to mark the packets originated from the server:
[MikroTik] ip firewall mangle>
add src-address=192.168.0.17/32 action=mangle mark-flow=Serv_Up
add in-interface=Local action=mangle mark-flow=Local-all
[MikroTik] ip firewall mangle> print
Flags: X - disabled, I - invalid
0 src-address=192.168.0.17/32:0-65535 in-interface=all
dst-address=0.0.0.0/0:0-65535 protocol=all tcp-options=any
icmp-options=any:any src-mac-address=00:00:00:00:00:00 limit-count=0
limit-burst=0 limit-time=0s action=mangle mark-flow=Serv_Up
tcp-mss=dont-change bytes=0 packets=0
1 src-address=0.0.0.0/0:0-65535 in-interface=Local
dst-address=0.0.0.0/0:0-65535 protocol=all tcp-options=any
icmp-options=any:any src-mac-address=00:00:00:00:00:00 limit-count=0
limit-burst=0 limit-time=0s action=mangle mark-flow=Local-all
tcp-mss=dont-change bytes=0 packets=0
[MikroTik] ip firewall mangle>
Add a queue to the queue tree, which uses the flow mark:
[MikroTik] queue tree>
add name=Server parent=Public flow=Serv_Up
add name=Workst parent=Public flow=Local-all \
limit-at=64000 bounded=yes max-burst=0
[MikroTik] queue tree> print
Flags: X - disabled, I - invalid, D - dynamic
0 name=Server parent=Public flow=Serv_Up limit-at=0 max-burst=20
queue=default priority=8 weight=1 allot=1514 bounded=no bytes=0
packets=0
1 name=Workst parent=Public flow=Local-all limit-at=64000 max-burst=0
queue=default priority=8 weight=1 allot=1514 bounded=yes bytes=0
packets=0
[MikroTik] queue tree>
Thus, we used queue trees for limiting the upload. Use the same simple queues as in the previous example for limiting the download.