Traffic Shaping will make your internet experience better... More fluent...
By: Cas Mollien Last modified on: 08 July 2008, 6:04h
OK, using the traffic shaper in m0n0wall... Where to begin... The documentation on how to set up and use traffic shaper is, at the time of this writing, insufficient to non-existant. Recently, I have done some searching and experimenting to find out how it actually works. As you will read, I currently use Asterisk as my home PBX solution. If you do not use Asterisk but you DO use a VoIP service, the same settings apply. If you do not use VoIP, you can leave VoIP out of the equation and the article will still help you out for your personal setup.
This article is a journal of my efforts.
The hardware: I am running m0n0wall (v1.2b8 at time of writing) on a Soekris net4801. It has proven to be stable, but a little high in price. My computers are Win2k, WinXP, Linux (CentOS for Asterisk PBX, Mepis on laptop) and PocketPC.
The Connection: I have a Bellsouth DSL (ADSL) connection with a published maximum speed of 1500/256.
The Challenge: Get my Asterisk PBX server to deliver reliable phone service (VoIP), while doing a newsgroup download. Surfing, streaming media and other regular but generally low-bandwidth tasks should not suffer from my download behaviour.
The Bottom Line: At this moment, I am very satisfied with the results. Running a full usenet (newsgroups) download will still allow me to make and receive phone calls without dropped packets. I do have to say that my config needs more tweaking to get it perfect. For example, streaming video does not work without interruptions. I guess this has something to do with the used ports, so I will figure this out in time.
The Learning Experience:
My startpoint was the following site: http://millfam.org/cms/content/view/21/2/. I would really suggest that you read this article, as it explains a lot of the basics. In short, the article and my testing tought me this:
- Pipes do not set priority! Pipes are used to give a certain ammount of bandwidth to specific traffic. If you would like to call this priority, fine, but in my mind priority means that packets for high priority traffic are picked out of the queue and put in front of lower priority packets. In m0n0wall's case, bandwidth is reserved for 'high priority' packets so they can use the (hopefully less busy) carpool lane, so to speak.
- If you use a total of 100 for the total priority in a pipe, it is save to talk about percentages. Meaning: If you have 2 queues, 80 and 20 respectively, traffic routed over queue 1 will be allowed to use 80% of the pipe when there are packets waiting for the other queue as well. If there are no packets in the other queue, the full pipe bandwidth may be used. To speak in automotive terms, 2 pipes of 50 on a 4-lane highway would mean that up to 2 lanes are available for carpool traffic if there are any carpoolers on the road. And yes, it is 'up to'. If there are only a few carpoolers, only 1 lane will be available to them, allowing regular traffic to use 3. (Remember that this is an analogy and that m0n0 can actually make 0.5 or 0.2 lanes available...)
- The last point also means that, eventhough the highway is at maximum capacity for non-carpoolers (traffic jam), a maximum of 2 lanes will allow carpoolers to keep moving. However, when the maximum of 2 lanes is reached for carpoolers while more carpoolers enter the highway, the carpoollane itself will become FULL. All cars (packets) will be in the same traffic jam! Now you may understand why this is not prioritizing traffic: If traffic would be prioritized, the cars on the carpool-lanes would be picked out and put in front of the traffic jam, allowing them to go first. With traffic shaping, this is not the case.
- Set the pipe bandwidth correctly, below the actual speed. Eventhough speeds are published by providers, one will amost never see these speeds reached. Find out your actual speeds (using DUMeter or the m0n0 traffic graph) and substract 10-20k from it for a start. I have had to adjust it downwards several times, both up and download, to get a reliable sound on my phone. This however, has not affected my download speed visibly.
- Traffic shaping is a one-sided deal. QoS (Quality Of Service) is ideally a two-sided deal. QoS actually prioritizes packets (see the previous point) on outbound AND INBOUND traffic. In the ideal situation, a QoS router is in place at every hop between your router and the provider, which quarantees the speed between the endpoints. Traffic shaping however, is done only locally and does not quarantee anything beyond the local endpoint (your own router). The incoming shaping works up to a point, but cannot be called reliable since you cannot control what the other endpoint sends to you. Inbound traffic shaping allows you to DROP certain 'low priority' traffic, saving bandwidth for important traffic. Priority and importance are used here as human interpretation, not as computer values.
- Inbound and outboud in the m0n0wall interface are used from m0n0wall's perspective. This means that the human interpreted 'incoming traffic' (downloads) is seen by m0n0 as 'Incoming on WAN' and 'Outgoing to LAN'. Therefore, shaping incoming traffic could mean that you have to create an 'Outgoing to LAN' rule.
- Small Packets do not (normally) cause traffic jams. They should be treated with the highest priority. As an example, ACK ('acknowledgement of receipt' to the sender, TCP only) packets should be sent as soon as possible to allow the next packet to be sent. Failing to prioritize ACK packets can result a highly unstable VoIP situation, since the sender is waiting for the acknowledgement to send the next packet. If the ACK is not received fast enough, the sender will consider the packet 'lost' and resend it. This is definately not what we want....
The Winding Road 1, Setting up Queues:
To keep things simple, I assumed 1 pipe, using the full available bandwidth of my internet connection (1275/210, the actual speed of my connection minus a bit). More pipes is possible but it will disallow you to actually use the full potential of your connection if 'the other pipe' is idle. But hey, if I had a 6000/1000 available, I would set up a 200/200 pipe for VoIP alone! :-)
First, I made a list of all the things I regularly do on the internet and gave them a priority. This priority is not so much based on what I personally feel is important to me, but is based on what 'can wait forever', 'can wait for now' or 'cannot wait'. For example, opening a web-page in 2 seconds or in 3 is not very disturbing. But having a 1 second delay in a phone conversation is! Waiting 1 second longer for a newsgroup download is not even noticable... With the buffering of streaming media, a 1 second delay could be un-noticable, but a 10 second delay would stop the media and start buffering...
And so, I came out to 5 upload priorities:
Priority #1 Upload - VoIP and Small Pkg
Priority #2 Upload - Streaming Media
Priority #3 Upload - Browsing
Bulk Upload - Upload for Internal services (FTP/Mail/WWW)
Hated Upload - NNTP, FTP, P2P, Etc...
and 4 download priorities:
Priority #1 Download - VoIP and Small Pkg
Priority #2 Download - Streaming and ICMP
Bulk Download - Browsing, Internal Services
Your mileage may vary, but this works for me.
When assigning weight to these queues I used percentages, as this makes it easy to calculate necessary bandwidth. Therefore, the total weight for a pipe should be 100.
To make the calculation, you have to approach it from a perspective that every queue has maximum packets waiting, eventhough this is a situation that may almost never occur.
The good part about VoIP is that it uses a fixed bandwidth (well, most codecs do). An other known factor is that UPLOAD is the bottleneck. Using the standard codecs (ULAW/ALAW), the constant bandwidth is approximately 80k per conversation. With my bandwidth, 256k upstream published but about 220k in reality, this would mean that I can get 2 simultanous VoIP conversations (160k) reliably. This means that at least 72 percent of my upload needs to be allocated to VoIP streams. I took 75 percent to include small packets.
The remaining 25 percent is to be divided between the other 4 queues.
One thing was clear to me: on both upload and download, when I need my bandwidth for anything but usenet, I want it available. Therefore, the 'Hated Download' and 'Hated Upload' are set to 1. This means that if every queue is used, usenet will only get 1% of the available pipe (dropping download speed to almost zero).
I gave Streaming Media Upload 15%, Browsing Upload 6% and Bulk upload 3%.
As for download queues, it is very important that non-VoIP packets are dropped to make room for my conversations. With 1% gone to 'Hated Download', the remaining 99% were split as follows:
VoIP: 15 (15% of my actual 1275k is enough for 2 ULAW conversations)
The Winding Road 2, Rules:
After setting up the initial pipes and queues, dropping packets into them is quite simple. You only need to know the port numbers that are being used and the queue you want to drop them in.
When creating rules, I kept the following points in mind:
- Rules are parsed top-to-bottom. When a packet matches a rule, parsing stops. This means that the order in which the rules are set, is important.
- Since the WAN interface is the bottleneck (LAN runs at 100mbps), all rules are created on interface WAN. I, like most people, do not need LAN-based shaping.
Since it is hard to go into all rules and what they do, you can find my current rules here as a text-file. You might even be able to copy some or all of it into the m0n0wall config file and change it to your needs, but this is at your own risk.
Some rules are self-explanatory, such as VoIP port 5060, VoIP port 16384/16484 and Usenet port 119.
Others are not, such as ACK, ESP, GRE, ICMP, etc.... All these packets are are dropped in the high priority queues. Especially the ACK-rule is important, as described earlier in this article.
I will not get into what application uses which port, as this is something that can be found on several websites. Furthermore, the traffic shaper text file should give you a pretty decent idea of my setup, including ports and descriptions.