In this article we will briefly look at what RTP is and how it is used to stream VoIP audio. The article then considers how certain network transmission characteristics may introduce jitter or packet loss and the measures that are used in VoIP equipment to mitigate the effects. Other phenomenon which have a bearing on the audio quality on VoIP calls, along with the features used on VoIP equipment to overcome them, are also briefly discussed.

RTP

RTP is Real-time Transport Protocol. It is a general purpose protocol for the streaming of audio, video or any similar data over IP networks. In a VoIP call, each RTP packet carries a small sample of audio (typically 20 or 30ms) which is constructed by the sending device from analogue signals picked up by the microphone in the phone’s handset.

Within the RTP protocol, each packet must be numbered and time-stamped. This has to be done by the source device – the one that is sending the packets.

The presence of sequence numbers and time stamps allows the receiving device to inspect the packet headers and determine if the packets are arriving in the correct sequence, with constant or varying delay or if any are missing.

RTCP

RTCP is RTP Control Protocol. The protocol is used alongside RTP to provide reporting of the quality of the RTP stream being received at the far end of a connection. The RTCP packets are sent from time to time in the reverse direction of the RTP packets. RTCP packets contain data describing the quality of the RTP stream being received. They are sent to the sending equipment so it can know how good or bad the audio quality is at the other end of the line. Asterisk has some limited capabilities for users to view audio quality information at the command line. For example, you can try the commands “sip show channelstats” and “rtcp set stats on|off”. The following article talks a bit about call quality in the context of RTCP reports.
http://www.voip-info.org/wiki/view/Asterisk+RTCP

Jitter and packet loss

Jitter is all about the timing and the sequence of the arriving RTP packets. If they arrive in a nice steady stream at regular intervals in the correct sequence then you have low jitter. If they arrive in bursts interspersed with gaps, or if they arrive out of sequence, then you have high jitter. Jitter happens when the RTP packet stream traverses the network (LAN, WAN or Internet) because it has to share network capacity with other data. The following diagram illustrates how jitter can be created.

By the way, QoS (DSCP) is a way of marking packets so the intermediate network equipment is aware of their relative importance. QoS packet tagging allows the network equipment to prioritise one type of packet over another. For example, pushing newly received RTP packets through to the output interface in preference to other data packets, even if the other packets arrived first.

The following diagram illustrates what our original stream of RTP packets might look like after they have traversed the network, become jittered and arrived at the receiving equipment.

The variation in packet delay is generally referred to as jitter, although a more accurate description of this phenomenon is Packet Delay Variation (PDV).
http://en.wikipedia.org/wiki/Packet_delay_variation

The sequence numbering of RTP packets allows a receiving device at the far end to check if the packets are still in the correct sequence or if any are missing. Packets can get out of sequence if they take different routes over the network. Packets can be dropped if there is network congestion somewhere along the route or if there are network errors.

Jitter buffers

A jitter buffer is used at the receiving equipment to store incoming RTP packets, re-align them in terms of timing and check they are in the correct order. If some arrive slightly out-of-sequence then, provided it is large enough, the jitter buffer can put them back into the right sequence. However, for this to work the receiving device must delay the audio very slightly while it checks and reassembles the packet stream.

If a packet was dropped (or simply does not arrive in time) then the receiving device has somehow to “fill in” the gap using a process known as Packet Loss Concealment or PLC. Packet loss needs to be less than 1% if it is not to have too great an impact on call audio quality. Greater than 3% would certainly be noticeable as a degradation of quality.

Even if the RTP packets remain in the correct sequence and there is zero packet loss, large variations in the end-to-end transmission time for the packets may cause degradation of audio quality that can only really be fixed through the use of a jitter buffer.

Jitter buffers in Asterisk

Jitter buffering is not enabled in the default Asterisk configuration files. Enabling them is not as simple as you would hope because their activation is conditional on a number of different factors. First, you must enable the jitter buffers in the conf file relevant to the appropriate leg of your bridged calls. Typically this means in chan_dahdi.conf where you are using Asterisk to bridge between SIP and TDM circuits (you would think it would be in sip.conf, but apparently not). For SIP-to-SIP calls, Asterisk will often just let the jittered packets be forwarded as received, leaving it for the downstream end-point to de-jitter the stream. If you want to learn more about activation of Asterisk jitter buffers and PLC, look for chapter 15 in asterisk.pdf which you will find in the source install sub-directory /doc/tex.

Latency

Latency is simply a measure of the delay and it is measured in millisecond. Less than 140ms is almost undetectable to the human ear. Somewhere between 150ms and 200ms it begins to become perceptible and as the latency gets greater so it becomes more noticeable and more annoying.

There are several potential causes for latency (one of which is the use of large jitter buffers). Conversion between different codecs and the technology required to join SIP to TDM (or vice versa) will introduce small delays. There will also be a measurable time delay for packets to traverse the network – ping can be used to give a rough indication of the round trip time (time for a packet to get there and back), but ping is only a crude measure because network latency is influenced by packet type, packet size, QoS settings and by how congested the network is at any given moment.

Echo

Echo occurs when a user hears their own speech coming back to them and the total latency (including the return path) exceeds 150ms. If the time delay is low enough then the sound of your own voice does not cause much of a problem even at relatively high amplitude. In fact, some return sound in the earpiece of a phone is generally a good thing because it makes the handset feel “live”. This acceptable feedback is called sidetone in the telephony industry.

There is a lot to say about echo and echo cancellation so I will publish a more comprehensive discussion of the topic in a future article.

Silence suppression, VAD and CNG

Silence suppression is a mechanism primarily designed to reduce network bandwidth demands, allowing VoIP equipment to send far less RTP data when the caller is not talking. The mechanism is defined in RFC3389. The source device uses Voice Activity Detection (VAD) to detect when the caller is speaking. During pauses in the speech it does not send audio samples in the RTP packets, but instead sends a special instruction showing that silence started or ended.

Ideally, the receiving device then needs to be able to regenerate suitable background noise to replace the missing audio – a mechanism called Comfort Noise Generation (CNG). Without CNG, the listener might find it very disconcerting to hear complete silence when the person at the other end of the line is not talking.

Echo suppressors and Noise cancelling microphones

Don’t confuse silence suppression and echo suppressors as they are not the same thing. An echo suppressor is used to reduce or prevent acoustic feedback on a speaker-phone. It does this by automatically reducing the microphone sensitivity whenever sound is coming out of the speaker. Conversely, it should reduce the speaker volume when there is no speech being played through the speaker. In effect it makes a speaker-phone operate in such a way that either the caller’s voice can be heard from the speaker or the local user’s voice is being picked up and transmitted by the microphone – never both at the same time.

A noise cancelling microphone is subtly different again. It automatically reduces its sensitivity when the user is not talking – in terms of gain characteristics it makes quiet things quieter and loud things louder. The idea is that you would use a noise cancelling microphone in a noisy environment like a call centre. When the agent stops speaking, the microphone rapidly adjusts itself to a lower sensitivity so it won’t pick up too much background noise. As soon as the agent starts speaking agin, the sensitivity will increase back to normal thereby making both the agent’s voice and the background noise louder.

MOS

MOS is a measure of audio quality that directly relates to the caller experience. It stands for Mean Opinion Score and was originally based on subjective opinions of people using a phone in controlled conditions. Now, in the VoIP world, it is used as a pseudo-objective measure allowing different levels of audio quality and speech clarity to be compared. A MOS of 3.0 is fair and of 4.0 is good. Wikipedia has a good article on this topic and the link is given below. It is interesting to note that the G729a codec can only deliver a MOS of about 3.9 whereas using G711 it can be as high as 4.5.
http://en.wikipedia.org/wiki/Mean_opinion_score

Further reading

An excellent overview covering many of the issues discussed here is given in an extract from a Cisco Press book here:
http://www.ciscopress.com/articles/article.asp?p=357102

Feedback – but not acoustic!

If you found this article useful, please take a moment to vote using the coloured buttons below. If the article did not meet your expectations, still feel free to vote, but also please leave me a comment so I know what to change or how to improve it later.

Mediaproxy 2.5.2 is a Python application from AG-Projects which is available as a free download as well as being available as a commercial product from AG-Projects. It is used in combination with the Mediaproxy module of OpenSIPS.

Mediaproxy 2 has several dependencies and can be quite tricky to install. The INSTALL instructions that come with the package are very helpful, but unfortunately they are aimed primarily at installers who either have Debian or Ubuntu Linux distributions. If you are using CentOS (or Red Hat), especially the 64 bit versions, then getting Mediaproxy 2 to run can be difficult.

This article is essentially an update for an earlier article showing how to install Mediaproxy 2.3.8 on CentOS 5. The installation process on CentOS 6 is quite different so I decided to leave the older article as it was and write up the process for CentOS 6 as a new article. Readers can choose whichever description is most suitable.

Here are the sequential instructions for installing the libraries needed for Mediaproxy 2.5.2. A number of packages that previously had to be installed using source tarballs are now installed using yum, making the process much easier.

Install some development and system packages using YUM

Development tools
yum groupinstall “Development Tools”

Development libraries and headers for some existing packages
yum install iptables-devel.x86_64
yum install libgpg-error-devel.x86_64
yum install python-devel.x86_64

libnfnetlink and libnetfilter_conntrack

yum install libnfnetlink-devel.x86_64
yum install libnetfilter_conntrack-devel.x86_64

libgcrypt

yum install libgcrypt-devel

Install some Python packages

python-zopeinterface

yum install python-zope-interface

python-cjson

This module can be installed using YUM, but first you will have to activate the rpmforge repository. The best way to do this is using the instructions here:
http://wiki.centos.org/AdditionalResources/Repositories/RPMForge

It is also possible to install the required python-cjson package from a source tarball as follows:

Suggested version: python-cjson-1.0.5.tar.gz
Locating a download site: http://pypi.python.org/pypi/python-cjson/1.0.5
Unzip: tar -xf python-cjson-1.0.5.tar.gz
Change to the sub-directory created when you unzipped the tarball
Build and install the library:
./setup.py build
./setup.py install

The following python packages are best installed from source tarballs. Details are given below:

python-application

Suggested version: python-application-1.3.0.tar.gz
Locating a download site: http://pypi.python.org/pypi/python-application/1.3.0
Unzip: tar -xf python-application-1.3.0.tar.gz
Change to the sub-directory created when you unzipped the tarball
Build and install the library:
./setup.py build
./setup.py install

python-gnutls

Suggested version: python-gnutls-1.2.4.tar.gz
Locating a download site: http://pypi.python.org/pypi/python-gnutls/1.2.4
Unzip: tar -xf python-gnutls-1.2.4.tar.gz
Change to the sub-directory created when you unzipped the tarball
Build and install the library:
./setup.py build
./setup.py install

Install the Twisted Python package

Install the core Twisted package from Twisted Matrix Labs
Suggested version: Twisted-11.0.0.tar.bz2
Locating a download site: http://twistedmatrix.com/trac
Unzip: tar -xf Twisted-11.0.0.tar.bz2
Change to the sub-directory created when you unzipped the tarball
Build and install the library:
./setup.py build
./setup.py install

Note also that version 8.2.0 of Twisted is available in the default YUM repositories. It is easier to install using YUM, but I prefer to install the later version from the source tarball because it matches the version used in the Debian install packages provided by AG-Projects.

A Checklist of dependencies and versions

On the system used for testing, the list of packages (and their versions) required by Mediaproxy looked like this prior to installation of the mediaproxy source code:

Install Mediaproxy 2

Mediaproxy 2
Suggested version: mediaproxy-2.5.2.tar.gz
Locating a download site: http://download.ag-projects.com/MediaProxy
Suggested location to copy and then unzip the tarball: /usr/local
Unzip: tar -xf mediaproxy-2.5.2.tar.gz
Change to the sub-directory created when you unzipped the tarball
Build and install the library (system wide):
./setup.py build
./setup.py install

Build and install the library (standalone in the local directory):
./build_inplace

And finally…

Make sure iptables is installed, running and correctly configured
Mediaproxy depends on iptables to relay IP packets through your server. Check that it is installed and running.

You can see the list of rules using the following command:
iptables -L -v -n

Note: In CentOS 6 you may well see a default rule preventing packet forwarding. If you leave this rule in place, the mediaproxy relay will start but it will not be able to relay any RTP. You will therefore need to change or delete this rule.

Set ip forwarding with immediate effect
echo “1″ > /proc/sys/net/ipv4/ip_forward

Set ip forwarding permanently
Edit the file /etc/sysctl.conf
Change the line “net.ipv4.ip_forward = 0″
to “net.ipv4.ip_forward = 1″

config.ini
Create a config.ini file from the sample template provided (config.ini.sample) and copy it to the appropriate directory. Using the system wide version, put it in /etc/mediaproxy. For standalone build you can also put it in the local directory where media-relay and media-dispatcher are located. If a file exists in both locations, the local one takes precedence

OpenSIPS and Asterisk are both open source projects and both are used for Voice over IP. However, they perform quite different roles, have different capabilities and different strengths and weaknesses. This article reviews how they are so different and considers what role each product can play in the infrastructure of an Internet Telephony Service Provider solution.

Fundamental differences

A succinct, but slightly technical, distinction between OpenSIPS and Asterisk is that OpenSIPS is essentially a SIP Proxy Server while Asterisk is essentially a Media Server (for a detailed explanation of these terms click here). This means that OpenSIPS is not normally the final end-point for a voice call; instead it relays the call control signals from one server to another. On the other hand, Asterisk does terminate calls and, even if it appears to relay a call onward to another destination, it does so be creating a new call and linking the audio streams to make the two calls appear as one – this behaviour is referred to as a “Back-to-Back User Agent” or B2BUA for short.

There are several other important differences including:

• OpenSIPS has no inbuilt media capabilities – it cannot record or play speech or music
• Asterisk has media capabilities which means it can be used as an IVR (with menus of options for callers to select using their keypad) or as a voicemail server etc.
• Asterisk has many built-in features that allow it to act as an IP-PBX or business telephone system. It can also act as a conference server. OpenSIPS does not have these capabilities.
• Asterisk can act as a gateway between different telephony technologies (including ISDN) whereas OpenSIPS is only a SIP Server.

So, you may think OpenSIPS sounds like a poor relation of Asterisk with fewer capabilities, but in fact these products are just not designed to do the same job. Where OpenSIPS scores is in its robustness, flexibility, adherence to SIP standards and, above all, its speed and call capacity which are in a completely different league to those of Asterisk.

A point-by-point comparison between OpenSIPS and Asterisk was written in 2008 in an article that I believe should be attributed to Flavio Goncalves (although the same text has since been copied and re-published many times by others pretending it is their own work). Here is a link to the article.

When to choose Asterisk

Asterisk can interconnect with legacy telephony technologies such as analogue (FXS ports where handsets are to be connected and FXO where a line from the Telco needs to be connected) and ISDN. Combined with its built-in PBX features, this makes it very suitable for customer premises as a replacement for, or add-on to, an existing PBX or business telephone system. Its voicemail, IVR and conferencing capabilities make it a popular choice for hosted service providers and its ability to bridge different telephone technologies means it can also be used as a PSTN gateway, for example where a service provider wants a resilient infrastructure with multiple disparate paths connecting their services to the public telephone network.

For many applications, especially as corporate end-user equipment or CPE, Asterisk is very competent. In these situations, I would generally recommend using a distribution that includes a web GUI interface (FreePBX, PBX-in-a-flash, Elastix or similar). However, Asterisk has some weaknesses and limitations that become more apparent as your solution is scaled up. For a telephony service provider, the call capacity of a single Asterisk server – typically up to 200 simultaneous calls – is likely to be too low. In a complex network environment, its slightly simplistic solutions for near-end and far-end NAT traversal may not work for some situations or for some makes of IP phone or ATA’s. It is a popular target for hackers, who know all its loopholes and weaknesses. From experience, I also have doubts about its ability to stand up to denial of service attacks. It does not support multiple SIP registrations to the same account.

When to choose OpenSIPS

OpenSIPS is frequently used by Internet telephony service providers (ITSP’s) as a “front door” – a connection point for a wide range of SIP devices and SIP trunks. In the right hands, it can be configured to cope with a range of different network architectures and can identify and fix non-standard SIP implementations as well as being able to fix the SIP headers for remote devices operating behind NAT. It can be used to load balance incoming calls to a group of media servers- possibly Asterisk servers. For outbound traffic, it can select different carriers using least cost routing (LCR) combined with failover in case the preferred route is unavailable or already at maximum capacity. Unlike Asterisk, it correctly handles multiple registrations to the same account.

The flexibility of OpenSIPS derives from its use of a configuration file containing a control script written like a computer program – the language is similar to C. This allows the application to inspect the contents of each SIP request down to a very low level, looking at headers, source address, contact address, user-agent, call ID, etc. The SIP requests can be modified, forwarded, forked, dropped or sent a response. In this way it is easy for OpenSIPS to distinguish between normal traffic and illegal requests or requests designed to identify open ports on SIP servers (e.g. friendly-scanner). This ability makes OpenSIPS even more suitable as a “front door” for access to to your other SIP-based servers – it has the ability to recognise and drop the most common port-scanning requests, the capacity and robustness to withstand DoS attacks and the flexibility to let you add rules for special cases and exceptions. What I am describing is, almost, a SIP firewall. Viewed in this light, OpenSIPS is a direct alternative to the so-called Session Border Controllers (SBC’s) that are increasingly being promoted as a “must-have” component in corporate VoIP solutions.

When combined with Mediaproxy from AG-Projects, OpenSIPS is able to solve problems of far-end NAT traversal in a more refined manner than Asterisk. The structure and content of external data referenced by OpenSIPS at run time (e.g. tables in a MySQL database) is better defined than the equivalent within Asterisk. OpenSIPS adheres more closely to the RFC’s that define how SIP should behave and it can support SIP over TCP or UDP as well as secure SIP over TLS. However, on the minus side, OpenSIPS cannot do codec translation (transcoding) or use telephony cards to interface to ISDN, analogue etc, it does not support IAX or H323, and it has no media capabilities for recording, playing voice files, mixing, conferencing, text-to-speech or speech recognition.

Teaching OpenSIPS new tricks

As a product, OpenSIPS does not stand still. Version 1.7 was released in the summer of 2011 and further releases are promised soon. By virtue of the Dialog module, it is now fully state aware which has allowed many improvements including better Call Data Records (CDR’s) and the ability to limit the number of simultaneous calls allowed to a particular end point. The developers have also added a B2BUA module. In part the justification for this was to allow improved topology hiding – the process whereby the addresses of some of your telephony servers can be hidden from the external users. This may be of benefit where OpenSIPS is required to act as a kind of SIP firewall or SBC, but it seems somewhat out of character to me, especially as one of the original oft-quoted differences between Asterisk and OpenSIPS was that Asterisk, unlike OpenSIPS, is a B2BUA.

Working together

In my experience, there are great benefits to be gained from using a combination of OpenSIPS and Asterisk. They complement each other well, especially for the design of a secure, scalable, high volume hosted telephony solution. OpenSIPS can be used as the main portal and can load balance incoming SIP requests to multiple Asterisk boxes. In this role, OpenSIPS is also able to protect the Asterisk servers from the majority of port scanning and password guessing intrusions. Another – or possibly the same – OpenSIPS server can route outbound SIP requests to carriers using LCR.  The outbound routing can be augmented with automatic probing of routes to monitor for faults so that traffic can be redirected to another destination when a fault is detected.

Our experience has shown that OpenSIPS combined with Asterisk presents some new and interesting challenges. For example, you have to decide if registrations are to be made to OpenSIPS only, Asterisk only or to both. Furthermore, authentication challenges and their responses, when initiated by Asterisk, must be allowed to pass through OpenSIPS in a transparent manner. Other issues we have solved at Smartvox include proper handling of keep-alive packets, SIP session timers, routing and SDP address fixing for servers behind NAT or in a DMZ, address substitutions for mixed carrier connections (some carriers using private peering and others on the Internet), fixing of non-symmetric SIP ports for Cisco handsets, Asterisk SIP timer issues, activation of message waiting lamps and presence. We are always happy to gain new clients and take on new challenges, so please get in touch to discuss your project.

There are a number of open source applications available that are used to build IP Telephony solutions. OpenSIPS may not be as well-known as Asterisk, but it is widely used by service providers as a core part of their infrastructure because of its robustness, speed and capacity.

In this article I will review the history of OpenSIPS, explain what it is, what features it offers and its core operational roles. This will include a summary of the many optional modules available within the OpenSIPS project. In a later article, I plan to explore different networking scenarios and examine in more detail the roles that OpenSIPS can play in the infrastructure of an Internet Telephony Service Provider.

Overview and history

OpenSIPS is an open source SIP Proxy program that runs on Linux platforms. The development project strives to provide a product that has plenty of useful features and high performance, while adhering closely to published standards. It is freely available as downloadable source code at www.opensips.org and is licensed under the GNU General Public License (GPL). Details of the license terms can be found here and project documentation is located here.

The history of the project is not simple and it started life in 2001 as SIP Express Router, or SER. In 2004 the team and copyrights were moved to iptel.org which was bought by TEKELEC in 2005. Nominally, the iptel SER project web site still exists today, but since 2006 it stagnated (the most recent News item is dated May 2007) and the feature set described there is now extremely out of date. In 2005 some members of the original core SER development team set up the OpenSER project which appeared to continue where SER left off. At the time, the OpenSER developers spoke of significant underlying structural improvements being made to allow better performance and easier on-going development. The Romanian-based company Voice System has actively supported the project since then, although in 2008 irreconcilable differences within the OpenSER development team resulted in it forking in two directions under different names – Kamailio and OpenSIPS. The split was not very amicable and consequently an online search is more likely to turn up invective than an objective analysis of the differences between the two branches. I cannot offer advice for those wishing to choose between Kamailio and OpenSIPS, but at the time the split occurred I had to pick one and my decision was to go with OpenSIPS. I cannot now remember why.

What does OpenSIPS do and how does it do it?

The primary purpose of any SIP Proxy server is to receive SIP requests, examine and classify them, check that they are valid and then retransmit them to an appropriate destination. Before retransmitting a request, OpenSIPS may alter the request by changing, adding or removing headers and/or changing the primary destination address – the so-called Request URI. It can intercept unwanted requests and return an appropriate rejection response (or it can simply drop unwanted requests if that is more appropriate). Permitted requests are relayed to a destination which most often is determined directly from the Request URI, but may also be another Proxy server. OpenSIPS is also able to fork requests to multiple destinations – either in parallel or one at a time.

A SIP Proxy server must also be able to handle the responses that are sent back to it. An example of a SIP request is the INVITE request which is used to set up a call. The called device will usually send a series of responses such as 100 Trying, 180 Ringing and finally 200 Ok to show that the call has been answered. The SIP Proxy must relay the appropriate responses back to the original client device that initiated the request, but it is also allowed to modify the responses if appropriate.

In addition to this role as a routing engine, OpenSIPS is also able to act as a Registrar server. In this role, it accepts and processes REGISTER requests sent to it from time to time by remote devices. These requests are used to update a location database which stores information about the network location and contact details for various SIP User Agent devices. [A SIP User Agent or “UA” is the jargon used to describe SIP equipment such as IP phones, softphones, IP-PBX’s etc]. OpenSIPS is then able to lookup details in its location database whenever it needs to route a request to a registered device.

OpenSIPS generally runs as a background service (or daemon) listening for incoming SIP requests. By default it listens for UDP requests on port 5060 on all network interfaces, but the defaults can be changed. The behaviour of OpenSIPS is determined by a script read from a configuration file. The format of the script is text rather like a programming language – in some respects the syntax quite closely resembles the C programming language. OpenSIPS must have a script before it can run and a default script is supplied as part of the install package. The file is called opensips.cfg and is typically stored in /etc/opensips. The script has a module initialisation section at the beginning, followed by one or more “route blocks” which in many ways resemble subroutines in a programming language. The programmable nature of this control script is what gives OpenSIPS its enormous flexibility, but it also makes for quite a steep learning curve for beginners. The language supports a large number of core functions and variables plus functions exported by the optional modules, making it relatively easy to perform a wide range of manipulations on the SIP messages as they pass through.

It is very common to link the OpenSIPS control script to a MySQL database (other databases are also supported). This allows the script to reference information stored in tables that may be altered and read by other applications. For example, a list of user ID’s and passwords is stored in the subscriber table and, when a device registers, its location and contact details are stored in the location table.

Who uses OpenSIPS?

OpenSIPS is used by many prestigious VoIP service providers. These service providers have no reason to make it known, but with the help of a simple SIP ping utility I was able to identify the following (in no particular order): Google Voice, Magrathea, Voiptalk, OrbTalk, Gradwell, Voxbeam, Freespeech and Sipbroker. So if you choose OpenSIPS, you will be in good company.

How does OpenSIPS differ from Asterisk?

The inescapable question! This article is intended to be a review of OpenSIPS and not a comparison with Asterisk so I have relegated this discussion to a different article.

Features and capabilities

The original feature set of SER and OpenSER, while adequate and competent, was not wide ranging. The emphasis was more on performance and conformance than on features. However, the product has been under continual development for several years and now includes a wide range of optional modules. The range of features available through the optional modules is now extensive enough to make OpenSIPS a serious alternative to the SBC’s (Session Border Controllers) available commercially at eye-watering prices.

The following tables show a selection of the modules categorised according to their complexity and how essential they are to the core operation of the product.

Commonly required “core” modules

Very popular additional modules and options

Advanced and more complex modules and options

My previous two articles explored QoS tagging of voice data packets using ToS/DiffServ values and of Ethernet frames using CoS or Priority values. QoS is often advocated as an essential part of any self-respecting VoIP solution and there is no doubt it can make a big difference in the right circumstances. However, it would be a mistake to expect too much of QoS or to assume that it will always make a difference no matter what. To understand where and how it can help we need first to examine the underlying causes of network congestion – how and where it can happen. 

What causes network congestion?

A network is a mesh of interconnected nodes. The nodes are pieces of network equipment, like routers and Ethernet switches, and the interconnections could be fibre optic or Ethernet (copper) cables. 

Let’s be clear, the data being transmitted over a single copper pair or fibre-optic cable all travel at the same speed – you do not get frames overtaking each other! So the points at which network congestion can occur are the nodes. Congestion happens at a node when the rate of ingress of data exceeds the rate at which the data can be forwarded to the next destination, the next node. 

How does network equipment respond to congestion?

There are potentially several ways that the network equipment could deal with congestion and the actual method will depend in part on the capabilities of the equipment itself. For example, it might be possible for a downstream node to signal to the upstream node to stop sending for a while, to pause. This method is referred to as “flow control”. 

Another option, if the router is able to detect congestion in a downstream route, then it might be able to send data via a different, less congested route. This would require quite a sophisticated device that is able to learn about alternative routes and evaluate them to choose the preferred route under different circumstances of network congestion. It would need to be able to read and interpret the Explicit Congestion Notifications coming back to it in the Layer 3 packets. 

The mechanism that is of most interest to us now, however, is the prioritisation of packets or frames within the network equipment (the devices at the network nodes). As we will see, this is dependent on the presence of memory-based data buffers within the routing equipment, the management of virtual queues and algorithms that determine how data are assigned to and removed from the queues. 

Flow control

Flow control is generally only possible with a full duplex interconnection. A flow control mechanism may take the form of a ‘PAUSE Frame’ in Layer 2 or an ‘ICMP Source Quench request’ in Layer 3. The concept is illustrated below: 

However, flow control is not a complete solution and is not always possible. Consider, for example, the case of a media stream carrying voice or video – such data is time-critical and cannot therefore tolerate significant interruptions. Indeed, if flow control is not going to simply push the problem back onto other upstream nodes, then the flow control signal would have to be sent all the way back to the original source that is sending the data. That may not be possible and, even if it is, the source may simply not be able to pause – the ultimate source for an audio media stream is the person speaking and they are hardly likely to be controlled by a flow control signal in the network. 

Memory buffers

No matter which mechanisms are used to manage congestion, there is an inescapable necessity for the network equipment at each node to have some form of memory-based buffering. Buffering allows it to receive data and decide how to handle it before sending it on to the next node. Without buffering, any situation where data are being received at the equipment faster than they can be transmitted out the other end has only one option: the “excess” must be discarded (so called ‘packet loss’). 

All network equipment, beyond a simple hub, must therefore have some memory buffering. Furthermore, the core mechanism for discrimination and prioritisation of network traffic works by using these internal buffers to queue the data. 

On a very basic router or switch the memory buffer would be quite small without being sub-divided – it would behave as a simple queue where new data are added to the back of the queue while the older data at the front of the queue gets processed (so-called FIFO). On the more sophisticated and expensive boxes the buffers are larger and are sub-divided into multiple queues – as new data packets arrive they are assigned to the back of the most appropriate queue depending on their QoS settings. 

The software running on the router takes packets from the front of each queue in a pre-defined way that allows some queues to have a higher priority than others. The algorithms used to process these queues have to be clever enough to take account of preferences for low latency and risk of packet loss – basically trying to prioritise some queues while avoiding the risk that the lowest priority queues never get a look-in. If the buffers are getting full, flow control would be attempted. Failing that, when packets have to be discarded, the algorithms might look at the QoS settings to determine the drop priority and thereby choose which ones to drop. 

Limitations of memory-based buffering

While buffering of data is essential and is integral to the prioritisation of network traffic and the general management of bandwidth, it is not a panacea. In fact, the more that buffering is used, the greater the latency (delay). Furthermore, while buffering may help to avoid packet loss, it cannot prevent it if the ingress rate at a node exceeds the outflow rate for a long period of time. In effect, the buffering just helps to smooth out short-term peaks by time-shifting the excess input at one point in time to a quieter period a little later. 

The shaded area of the graph, where inbound transmission rates exceed the maximum outflow rate, equates to data that must be temporarily put into the buffer. Only when the inbound rate drops below the dotted red line is it possible for the equipment to empty its buffers. If the rate stays above the red line for too long, then the buffers would become full and packets would have to be dropped. 

The implications for practical application of VoIP QoS

We need to draw some conclusions from this exploration of network congestion issues. What does it mean in practical terms for VoIP system designers and IP-PBX installers? Based on the above points and also drawing upon my own experience of real-world installations, I would suggest that the following conclusions can be drawn: 

• Network congestion happens at nodes where the data ingress rate exceeds the outflow rate. If this ‘speed’ disparity exists for too long, then packet loss is likely to occur. If there is no congestion then QoS is more or less irrelevant.
• QoS influences how network congestion is handled. It can determine which packets are most likely to be dropped if buffers get full and it can influence the latency (delay) of certain data streams by prioritising one stream with respect to another.
• Network equipment may allow a fixed proportion of the overall bandwidth to be set aside for a particular QoS class of traffic – this type of bandwidth management will normally allow lower priority data to “borrow” some of the reserved bandwidth when the high priority traffic is not using it.
• Just using QoS settings to mark packets or frames as “high priority” as they leave your IP phone or IP-PBX, guarantees absolutely nothing about how they will be handled.
• Effective bandwidth management is possible within your LAN or corporate WAN where you have control over the topology of the network, the choice of equipment and its configuration. The network equipment must support traffic shaping or traffic prioritisation and be configured to recognise and use your chosen QoS tags such as DiffServ or CoS.
• Once network traffic goes outside your own network infrastructure it is quite likely that QoS tags will be overwritten or ignored. This is especially true when traffic leaves your premises to traverse the Internet using an ordinary broadband connection. If you need end-to-end real time QoS over a broadband Internet connection you will need to look closely at the packages being offered by different service providers and you must expect to pay a premium for a broadband connection that supports it.
• In most cases, QoS is far more relevant to transmissions in one direction than in the other. For example, if network congestion is happening at a node because the outbound connection is slower than the inward one then the chances are that for data travelling the other way there will be no problem. Furthermore, you may be able to set the QoS tags on packets you are sending, but you may not be able to set it on the ones you are receiving.
• Bandwidth management is a multi-faceted problem: there may be trade-offs between decreasing latency vs increasing packet loss; getting faster throughput of voice almost certainly means there will be times when there is slower throughput for other data and could even result in packet loss for other data.

Topics for further discussion and reading 

If time permits, I will write a follow-up article looking at how bandwidth management works in practice; in particular how prioritisation using QoS is generally implemented within a network switch or router. If you want further reading, a good starting point is wikipedia and Google – try looking up the following topics: dropped packets, latency, jitter, weighted fair queuing, QoS. 

I would also recommend this article about QoS, written by Berni Gardiner.

In part 1, we examined the Layer 2 QoS settings available on most VoIP equipment. In this second part, I will explore the Layer 3 parameters and offer practical suggestions for the values that should be assigned to them. We will briefly look at the history and structure of the ToS and DSCP fields and their place within the DiffServ packet prioritisation model.

Recap

I explained in part 1 how Layer 2 data can be tagged with a priority value (sometimes called CoS). The L2 priority value is stored in the 3-bit PCP field of the Ethernet frame and this field is part of an optional 4-byte section loosely referred to as the “VLAN tag”.

Completely separate to this, it is also possible to tag IP packets within Layer 3 in a way that allows routers and switches to recognise that some packets should be given a higher priority than others. The following diagram illustrates how the different mechanisms exist at different layers in the OSI model:

Layer 3 QoS settings

As the above diagram shows, the QoS mechanisms that operate at L3 are DiffServ, DSCP and ToS. These L3 mechanisms have the potential to operate over the Internet thereby giving them a wider reach than L2 (which normally only operates within the scope of a LAN or private network). Theoretically, L3 QoS prioritisation could be applied all the way through to the remote end-point in a VoIP connection. However, in practice you are unlikely to be able to guarantee that routing equipment outside your control will respect the QoS tagging that has been applied to the IP packets.

The ToS field and how it has changed over time

The original design for the L3 Internet Protocol (see RFC 791) defined a standard structure for Internet Datagram Headers that included an 8 bit “Type of Service” (ToS) field. Its purpose was to provide storage for QoS parameters to allow network equipment to prioritise certain traffic at times of high load. Three of the eight bits were set aside for an “IP Precedence” value (very similar to the PCP priority value described in part 1) and a further three bits were designated as facility flags to indicate if a packet had preference for low delay, high throughput or high reliability. The last two bits were reserved for future use.

In 1998 and 2001, new standards were published (see RFC 2474, RFC 2475 and RFC 3168) which redefined the 8-bit ToS field to allow its use for the so-called “Differentiated Services”. The name “ToS” continues to be used today, even though the original interpretation given in RFC 791 has been superseded by the newer Differentiated Services model and the name “DS field” should be used instead. The 8-bit DS field comprises the first 6 bits as a DSCP value and the last 2 bits which are used for “Explicit Congestion Notification” (ECN) data:
The above diagram is copied directly from RFC 3168 which describes ECN.

Differentiated Services and the DSCP value

While the basic concepts are straightforward, the practical implementation – especially the terminology and labelling associated with the Differentiated Services Field – is remarkably confusing. As a VoIP engineer, you probably just want to know what value you should set for one parameter on the VoIP equipment’s setup menus. You don’t want to have to read two articles in Wikipedia and a couple of RFC’s just to be able to set that one value.

Let’s try to unpick the various points of confusion as briefly as possible:

• DiffServ is a broad term covering the architecture and mechanisms of this particular approach to QoS. There are other models that can be used too – for example IntServ and RSVP.
• Differentiated Services Code Point (DSCP) is a 6-bit value within an 8-bit field in the IP header.
• The 8-bit field is sometimes called the ToS or TOS field. It may also be called the DS field or DSF.
• The first 3 bits of the DSCP field may be referred to as the “Class Selector”. The same bits in the ToS field definition (in RFC791) were assigned to the so-called “IP Precedence” value. Some values relevant to VoIP are shown below.
• The last 2 bits of the 8-bit field, used for congestion notification, are set dynamically by routers. You can assume 00 whenever you are obliged to explicitly assign a value to them.
• When specifying a value for DSCP on your VoIP equipment, be careful to check if it expects a 6-bit value or an 8-bit value, if it requires the value to be given in hex or in decimal or as a named constant.

Wow. So what is that last point saying – what do I mean by a “named constant”? The answer comes from the fact 6 bits allows for a lot of different values. The 6 bits can in fact be further sub-divided into elements that describe the Class Selector and the Drop Probability. To make these values more easily readable by humans, they are often described by an abbreviated name indicating the “Per Hop Behaviour”, the Class Selector value and Drop Probability. Here are some examples:

• AF31 where AF=Assured Forwarding, 3 indicates Class 3 and 1 is the Drop Probability
• EF Expedited Forwarding – recommended for RTP
• CS5 Class Selector 5

A selection of ToS “IP Precedence” values and their equivalent Class Selector value:

• IP Precedence 0 (CS0) Routine or Best Effort – typically used for data
• IP Precedence 3 (CS3) Flash – used for voice signalling (e.g. SIP)
• IP Precedence 5 (CS5) Critical – used for RTP

I am not going to attempt to provide a complete list or even a complete explanation of the DSCP values – there is plenty of existing material on the Internet that does this already. However, you may want to try the following links:

Excellent overview of QoS includes a table of DSCP values: http://www.rhyshaden.com/qos.htm

List of commonly used DSCP values as binary, decimal and named constant:
http://www.cisco.com/en/US/docs/switches/datacenter/nexus1000/sw/4_0/qos/configuration/guide/qos_6dscp_val.pdf

A handy table for converting between DSCP and ToS; giving values in binary, hex and decimal:
http://bytesolutions.com/Support/Knowledgebase/KB_Viewer/ArticleId/34/DSCP-TOS-CoS-Presidence-conversion-chart.aspx

What values should be used on VoIP equipment?

A sensible choice for SIP signalling messages is AF31. This requires the ToS field to be set to 0×68 (hex) or 104 (decimal). CS3 is also suitable for SIP.

A sensible choice for RTP and RTCP is EF. This requires the ToS field to be set to 0xB8 (hex) or 184 (decimal)

Unfortunately, different manufacturers require different formats for entry of the data.

Linksys

On my Linksys devices, the parameters are located in the Line tab in a section entitled “Network Settings” (you need to login as admin and select advanced viewing). The values are described as “SIP ToS/DiffServ” and “RTP ToS/DiffServ” and they encompass the whole 8-bit ToS field. They are entered as Hex values and the defaults are 0×68 for SIP and 0xb8 for RTP, which are equivalent to AF31 for SIP and EF for RTP.

Snom

On the Snom 360 the settings are available in the QoS/Security tab of the Advanced settings page. The Layer 3 settings are shown as two “TOS/Diffserv” values, one for RTP and another for SIP. The values are entered as decimal numbers which are internally converted to an 8-bit value for the whole ToS field. The default on the Snom for both SIP and RTP is 160 which is equivalent to 10100000 in binary. To see what DSCP value this equals, you must strip off the last two zeroes (the ECN field), leaving the binary value 101000. From tables we can determine that this binary value is called CS5 when using the named constants.

Aastra

On the Aastra IP phones, the layer 3 settings are shown on the web GUI within the Network Settings page under the heading “Type of Service DSCP”. The values are entered as decimal numbers, but unlike the previous manufacturers they represent only a 6-bit DSCP field (the 2-bit ECN field is not included). The default values are 26 for SIP and 46 for RTP/RTCP – equivalent to AF31 and EF.

Grandstream

The parameter is called “Layer 3 QoS” and it expects a value, entered as a decimal number, that is equivalent to the 6-bit DSCP field. The default on my GXV3140 is set to 48 which is equivalent to the named constant CS6 – i.e. Class Selector 6. It does not allow different values to be specified for SIP and RTP.

Asterisk settings

In recent versions of Asterisk, you can set the value for the layer 3 DSCP field by editing the sip.conf file. The parameter names refer to the ToS field, but the values you specify are named constants for the DSCP field, so even the developers who work on Asterisk have trouble understanding the terminology! The values are set using the following parameters (here shown with their default values):

tos_sip=cs3                    ; Sets TOS for SIP packets.
tos_audio=ef                   ; Sets TOS for RTP audio packets.
tos_video=af41                 ; Sets TOS for RTP video packets.
tos_text=af41                  ; Sets TOS for RTP text packets.

Further reading

If QoS is to make a difference, it requires a good deal more than just setting the DSCP, ToS or CoS values on the IP telephony device or PBX. If QoS is an important requirement in your VoIP projects, I strongly recommend that you read my follow-up article about QoS in practice. Just click this link to go to the article.

Feedback on this article

Please take a moment to use the coloured voting buttons below to provide some feedback to the author. If you liked the article, great. If it fell a little short of your expectations, please leave a brief comment or send me an email (info (at) smartvox.co.uk) so I know what needs changing. Thanks.

The QoS settings on VoIP phones and related equipment can be perplexing. Here, I will attempt to explain what parameters like CoS, ToS, DiffServ and DSCP really mean and offer practical suggestions for the values that should be assigned to them.

Part 1 of this article starts with a broad overview and then focuses on Layer 2 network QoS settings; i.e. the settings associated with the VLAN tag in 802.1 Ethernet protocols. If you are looking for information about Layer 3 settings such as DSCP, ToS and IP Precedence, then you may prefer to skip straight to part 2.

What is QoS?

QoS (Quality of Service) is a somewhat all-encompassing term and means different things to different people, even within the context of Voice over IP. The broader issues of audio quality on VoIP calls, echo cancellation and the mechanisms within end-points designed to compensate for packet loss and jitter are discussed in a different article here. The QoS settings that this article seeks to explain are all directly related to frame and packet tagging for prioritisation of network traffic. Even restricting ourselves to this specific area, there is enough material to fill at least one decent sized book. In order to keep it to a manageable scale for this article I’m going to have to brush over some of the details very quickly. I apologise in advance if some experts out there find my condensed descriptions over-simplistic or inaccurate.

The structure of Network Traffic

Traffic on a network is passed around as a series of discrete chunks of data called “frames”, “packets” or “messages” depending which level they exist at. The structure of frames and packets generally conforms to a common pattern in so far as they all could be described as an envelope containing (a) information about the packet itself and (b) the “payload”. The term “payload” is used to describe the core data that needs to be sent from A to B. Information about the packet or frame typically includes the source and destination address, but may also include other parameters such as a description of the packet’s importance or priority. It is these priority fields that I will be discussing here.

The actual structure of a frame or packet will conform to certain industry standards (without this, you would not be able to interconnect equipment from different manufacturers). For example, the IEEE 802 standards define what Ethernet frames look like. Internet Protocols such as IPv4 and IPv6 are also governed by standards. The original standards may or may not have included a field that could be used to define the priority of the data, but more on that later.

To understand the QoS parameters used for VoIP, it is vital to first understand the different network layers within which these frames and packets are operating. Some parameters are relevant within Layer 2 and others are relevant within Layer 3. The following diagram summarises it:

Layer 2 QoS settings

The early standards for Ethernet defined a relatively simple structure for each frame that did not include a field for priority. An optional additional field, 4 bytes long, was introduced by the IEEE 802.1 working group (see 802.1D, 802.1Q and 802.1p). This optional field may sometimes be loosely referred to as the “VLAN tag”. Two bytes of the VLAN tag are assigned for Tag Control Information comprising a 3-bit Priority Code Point, a 1-bit CFI flag and a 12-bit VLAN ID. The field that we are interested in is the Priority Code Point (PCP) because it is used for VoIP QoS tagging in layer 2.

Priority values that may be assigned to the PCP field are defined in the 802.1p standard. Values can be between 0 and 7, but remember this value is used for traffic classification and thus represents more than a simple linear scale. The audio stream, or RTP, for a VoIP call requires low latency and should be assigned a priority value of 5. The signalling traffic, or SIP, can tolerate higher latency so it is usually assigned a priority value of 3. Sometimes the layer 2 priority value is referred to as a CoS value (Class of Service) although this terminology has the potential to be ambiguous.

Cautionary note:

Layer 2 802.1p QoS settings are stored in the VLAN tag which is not always present by default. Please take care when enabling or setting a VLAN ID on your VoIP equipment as it may result in a complete loss of remote connectivity. You should be able to recover from such problems by disabling VLAN options using the phone’s built-in menus and LCD display, but in the worst case you may have to return it to default settings using a factory reset. If unsure, a VLAN ID of zero should (in theory) be recognised by non-VLAN enabled network equipment.

Aastra settings

On the Aastra IP phones, the layer 2 settings are shown on the web GUI within the Network Settings page under the heading VLAN. It will only allow you to modify the values if you tick the “VLAN Enable” box, which makes sense as the priority value is stored in the VLAN tag. You can set a different priority for the SIP messages, the RTP stream and the RTCP messages, but the default values already shown (3 for SIP, 5 for RTP and RTCP) will not normally need to be changed.

Linksys settings

On Cisco and Linksys phones, the layer 2 settings are shown on the Line tab under the heading Network Settings. Look for the fields labelled SIP CoS Value and RTP CoS value.

Snom settings

On my  Snom 360 phone, some settings are available in the QoS/Security tab of the Advanced settings page. It allows you to specify a Priority to be associated with each VLAN ID. There are also settings for VLAN ID and Priority for the Net Port and the PC Port, but you cannot set different priorities for SIP, RTP and RTCP.

Grandstream settings

Taking two different Grandstream phones, I found the QoS settings under the “Advanced Settings” tab on the older GXP phone and under “Maintenance” on the newer GXV phone. On the latter they are shown in the “Network Settings” sub-page. Identification is simple because Grandstream call them  ”Layer 2 QoS 802.1Q/VLAN Tag” and “Layer 2 QoS 802.1p Priority Value”. Nice unambiguous labelling.

Asterisk settings

In recent versions of Asterisk, you can set the value for the layer 2 PCP field by editing the sip.conf file. The values are set using the following parameters (here shown with their default values):

cos_sip=3
cos_audio=5
cos_video=4
cos_text=3

Further reading

In part 2, I look at QoS settings for prioritising network traffic using the Layer 3 IP protocol.

If QoS is to make a difference, it requires a good deal more than just setting the DSCP, ToS or CoS values on the IP telephony device or PBX. If QoS is an important requirement in your VoIP projects, I strongly recommend that you read my follow-up article about QoS in practice. Just click this link to go to the article.

Feedback on this article

Please take a moment to use the coloured voting buttons below to provide some feedback to the author. If you liked the article, great. If it fell a little short of your expectations, please leave a brief comment or send me an email (info (at) smartvox.co.uk) so I know what needs changing. Thanks.

Part 1 laid the foundations for creating and accessing a shared voicemail box. In this, part 2, I explain how the lamp on the BLF key is switched on and off to show there are messages waiting in the shared box. Note that this is separate from any existing MWI lamp used for personal voicemail. It uses a custom device state within Asterisk switched by an external application. The external application in this case is a bash script that is called by another part of Asterisk using the externnotify option of the voicemail module.

Setting externnotify to call an external application

Edit the file /etc/asterisk/voicemail.conf and search for the parameter externnotify. In the sample file it is usually commented out – remove the comment character. If it is not already in the file, just add it as a new line. It is located in the [general] section. We require it to be like this:

externnotify=/usr/bin/groupmwi.sh

Creating the script run by externnotify

Using your favourite editor, create a new file in /usr/bin and name it groupmwi.sh

Copy and paste the following lines into the file and save it.

#!/bin/sh
#
# Control script to change the status of a custom device state in Asterisk
# Called from Asterisk whenever a voicemail message is left
#
VM_CONTEXT=$1
EXTEN=$2
VM_COUNT=$3

ASTCMD="/usr/sbin/asterisk -rx"

if [ $EXTEN -ne 4444 ]; then
    exit 0
fi

if [[ $VM_COUNT = "0" ]]; then
    logger -t groupmwi "Switching OFF Group mailbox message"
    $ASTCMD "originate Local/s@groupmwi-off extension 4@default"
else
    logger -t groupmwi "Switching ON Group mailbox message"
    $ASTCMD "originate Local/s@groupmwi-on extension 4@default"
fi

exit 0

Set the file as executable using the Linux command chmod 755 /usr/bin/groupmwi.sh

What does the script do?

Writing scripts called by externnotify is already documented elsewhere, so I will not repeat it all here. Just check these links if you are interested in the general principles of operation:

http://www.voip-info.org/wiki/view/Asterisk+config+voicemail.conf

http://asterisk.mdaniel.net/?p=14

The specific script we are using here simply checks if there are any new messages in box 4444. If the box is not 4444, it exits and does nothing. For box 4444, it executes a small section of Asterisk dialplan code using the originate command. If there are messages in the box it executes some dialplan commands in a context called groupmwi-on and if there are no messages it executes the code in groupmwi-off. As well as executing the dialplan commands, it also writes a line of text to the system log file to assist with debugging (see the lines starting logger -t).

Adding the status switching code to your dialplan

Generally, Asterisk dialplans are stored in the file /etc/asterisk/extensions.conf. However, if you are using Trixbox or some other FreePBX variant, then you must not directly edit extensions.conf. Instead, you will need to edit the file /etc/asterisk/extensions_custom.conf. Just add the following code:

[groupmwi-on]
; Switch custom device state to show group voicemail has messages waiting
exten => _X.,1,Goto(s,1)
exten => s,1,Set(DEVICE_STATE(Custom:groupmwi)=INUSE)
exten => s,2,Noop(New state for groupmwi is ${DEVICE_STATE(Custom:groupmwi)})

[groupmwi-off]
; Switch custom device state to show group voicemail has no messages waiting
exten => _X.,1,Goto(s,1)
exten => s,1,Set(DEVICE_STATE(Custom:groupmwi)=NOT_INUSE)
exten => s,2,Noop(New state for groupmwi is ${DEVICE_STATE(Custom:groupmwi)})

See this companion article for more detailed information about using Custom Device States.

Testing the script

First make sure the subscription request from the programmable BLF key on the phone has been accepted by Asterisk. At the CLI, type the command “sip show subscriptions”. You should see something like this:

Peer         User Call ID  Extension       Last state Type            Mailbox
192.168.1.57 4444 3903f2c8 974444@from-int Idle       dialog-info+xml <none>
1 active SIP subscription

If you cannot see the subscription, try rebooting the IP phone. If it is still not there, check that the phone is registering ok and check the settings on the BLF key as explained in part 1. Make sure you defined the hint and added it in the right place – can you see it when you type “core show hints” at the CLI?

Assuming the subscription has worked ok, you can test that the bash script is able to turn on and turn off the BLF lamp, by calling the script directly from the Linux command prompt. To turn the lamp on:

/usr/bin/groupmwi.sh default 4444 2

To turn the lamp off:

/usr/bin/groupmwi.sh default 4444 0

And finally…

It should all be working now. The lamp comes on when new messages are waiting and goes off when they have been checked or deleted. You can access the voicemail box by pressing the BLF button and entering the mailbox password when prompted (I suggested 1471 as the password in part 1).

Now you can see how to control a BLF lamp using a custom device state that is switched by an external program, you will probably be able to think of many other useful applications using the same mechanism. Please remember to leave comments and to vote if you found the article useful. Remember too that Smartvox can provide expert assistance with setting up large or small Asterisk applications and with the OpenSIPS SIP proxy server and with the design of resilient VoIP and ITSP infrastructure – call John on +44 1727 221221 or email us at info(at)smartvox.co.uk.

It’s a requirement that people often seem to ask for - a single voicemail box, taking messages for a department, that can be easily monitored and accessed by several different users. A typical application would be to record out-of-hours messages which are then checked in the morning by any of a number of users, perhaps just depending who arrives first at the office.

While it is fairly easy to configure Asterisk or Trixbox to record the messages, it is less clear how to configure an indicator lamp to show messages are waiting on several IP extension phones. Oh yes, and of course we will want the programmable key to also work as a single-press access for retrieving messages from the voicemail box.

In part 1 of this 2-part article I describe how to create a shared mailbox, send your out-of-hours calls to it, configure a custom device state to control a programmable key’s lamp on an arbitrary number of IP phones and insert code into the dialplan to allow the BLF key to talk to Asterisk. In part 2, I explain how to switch the status of that custom device state to reflect whether there are any messages in a specific voicemail box.

Creating the shared voicemail box

In Trixbox or FreePBX

Just add a new dummy extension and make sure the Voicemail status is enabled. If you select “Generic SIP Device” when creating the extension, then be sure to assign an unused extension number to it and – very important – an obscure, unguessable string for the SIP password (shown as “secret” in the “Device Options” section). For example, use 8 randomly chosen characters and digits for “secret”. No one will ever need to use the SIP password, but if you use a weak one like 123 then it may render your Asterisk system vulnerable to hacking.

The voicemail password is different – it is located in the “Voicemail & Directory” section and should be set to something that all your users can remember while not being so obvious that anyone could guess it. For the examples shown here, I created a SIP extension 4444, set the Display Name to “Dept Voicemail” and in the Device Options section I kept the default setting for mailbox as “4444@default”. The voicemail password was set to 1471. Email notification is optional and is activated simply by inserting an address in “Email Address”.

In raw Asterisk

Edit the file /etc/asterisk/voicemail.conf and scroll down until you see the [default] section. In this section, add a definition for your departmental mailbox. The format for adding new voicemail boxes (as explained in the comments in the sample file) is as follows:

<mailbox>=<password>,<name>,<email>,<pager_email>,<options>

My examples use the following voicemail box definition (the email address and attach option are optional and control delivery of message alerts by email):

4444 => 1471,Dept Voicemail,group@mymaildomain.com,,attach=yes

Sending out-of-hours callers to the shared voicemail box

In Trixbox or FreePBX, it is simply a case of setting the destination for your inbound calls to Voicemail: <4444> Dept Voicemail. If you want to use a timer to send your inbound calls to this shared voicemail box, then take a look at the modules called Time Groups and Time Conditions. Time Groups defines the hours, days, etc and Time Conditions controls the routing of calls to different destinations at times defined in the selected Time Group.

If you are using raw Asterisk, you will need to edit the file /etc/asterisk/extensions.conf and find the appropriate context where inbound calls are handled. To send a call to the shared voicemail box, add a line like this:

exten => _X.,n,Voicemail(4444,u)

The option u, shown above, tells Asterisk to play the “unavailable announcement” to the caller, followed by instructions. For the complete list of announcement options go to
http://www.voip-info.org/wiki/view/Asterisk+cmd+VoiceMail

If you want to use timers to only route the call to this voicemail at certain times of day then check for details at http://www.voip-info.org/wiki/view/Asterisk+cmd+GotoIfTime

About custom device states

If you have not already done so, I now recommend that you read my earlier article about Asterisk’s custom device states. In it, I explain how you can check if your version of Asterisk supports custom device states and I also explain how a device state can be set from the dial plan and how it can be linked to a programmable BLF key using Asterisk hints. Click here to go to that article.

Setting up a custom device state and linking it to a hint

We are going to use a custom device state called groupmwi to show when there are messages waiting in the shared mailbox 4444. The phone’s BLF key tells Asterisk that it is interested in the state of a device by sending a subscribe request – Asterisk knows which device it is interested in because the BLF key also sends an ID string, usually numeric, to specify what it wants to subscribe to. Asterisk hints map these ID strings to objects that it knows about.

To proceed, we must first choose the ID string that will be mapped to our custom device state, groupmwi. The shared mailbox in my examples is 4444. The default feature code to access a voicemail box is *97. The ID string actually does two jobs – it links the BLF lamp to the custom device state, but it is also used to retrieve messages from the box when the BLF key is pressed. I therefore chose to set the ID string to 974444. So my hint looks like this:

exten => 974444,hint,Custom:groupmwi

As well as defining the hint, I will also include dialplan entries that allow access to the stored messages when the BLF key is pressed, because they go in the same section as the hint. So the code we need to add looks like this:

; Retrieve messages from the group message box by dialling 974444
exten => 974444,1,Answer
exten => 974444,2,VoicemailMain(4444@default)
; Add hint to link to the Custom Device State
exten => 974444,hint,Custom:groupmwi

Ok so far? – I hope. But where does this code need to be added? The answer depends if you are using Trixbox/FreePBX or raw asterisk:

For FreePBX based systems, you must edit the file /etc/asterisk/extensions_custom.conf and add the lines to the end of the file in a new context section called [ext-local-custom]:

For raw Asterisk, edit the file /etc/asterisk/extensions.conf and add the lines to the end of the default context (or whichever context you use for calls from extensions):

Setting up the BLF programmable key on the IP phone

On your IP phone, configure one of the programmable keys to be of type BLF and set the number (or value) field to 974444. This is all explained in more detail in the companion article mentioned earlier.

Changing the Custom Device State when voicemail arrives

All that remains now is to make the custom device state change to INUSE when there are messages waiting and NOT_INUSE when there are no messages. This is achieved by:

1. Configuring the Asterisk voicemail option externnotify to call a bash script whenever the number of messages changes in a voicemail box
2. Writing that bash script – its job is to check the input parameters sent from Asterisk and run an appropriate portion of dialplan code
3. Writing two special context sections in the dialplan which change the custom device state to INUSE or NOT_INUSE. These are used by the bash script.

Details are available in part 2.

Do you want to know how to use a custom device state to control the lamp on a programmable key of an IP phone? In this article I explain how to set up the hints and make any number of IP phones subscribe to a custom device state and how to switch the custom status from within the Asterisk dial plan.

In later articles I plan to show how this can be put to a practical use. I will explain how to switch the state from an external script and use this to provide a shared voicemail box linked to a programmable key on several extension phones.

Does your version of Asterisk support custom device states?

The first step is to make sure Asterisk supports custom device states. Version 1.2 and all earlier versions did not support this feature. Version 1.6 does support it. If you have version 1.4 then it might or might not support the feature – it did not come as standard, but was available as a kind of after-sales upgrade. The module we are looking for is func_devstate.so and you should check that it is available on your Asterisk server before proceeding.

You can check if the module is present by typing the command “module show like devstate” at the Asterisk CLI (Command Line Interface). If the module is installed, you should see the following output:

Module              Description                               Use Count
func_devstate.so    Gets or sets a device state in the dialp  0
1 modules loaded

A further check is to look for the Asterisk function that is exported by the devstate module. The function we are looking for is DEVICE_STATE. You can search for installed functions using a CLI command. To check for any function containing the word “STATE”, the command would be “core show functions like STATE”. You should see the following output:

Matching Custom Functions:
-------------------------------------------------------------------------------
DEVICE_STATE      DEVICE_STATE(device)                 Get or Set a device state
EXTENSION_STATE   EXTENSION_STATE(extension[@context]  Get an extension's state
2 matching custom functions installed.

Using the DEVICE_STATE function

Custom devices are dynamically created when the dial plan assigns a value to one of them using the Set command. Each new custom device takes whatever name you choose when assigning the value – there is no pre-existing master list of custom devices and no list has to be explicitly created or maintained. This has advantages (simplicity) and disadvantages (typing errors in device names will not be trapped). Note, however, that there is a pre-defined list of states that can be assigned to a device – see below.

From the Asterisk CLI, you can view information about the DEVICE_STATE function by typing the command “core show function DEVICE_STATE”. This shows you how to set a state for a custom device called “lamp1″ and it also shows a list of all the permitted state values that you may assign to a device. Depending on the make of IP phone you are using, different states will typically cause the lamp to go on continuously, to flash fast or slow or potentially even to change colour where that is an option. You should experiment to see what happens for different values, but remember what happens on a Snom may not be the same on a Grandstream, etc. The states available are:

UNKNOWN, NOT_INUSE, INUSE, BUSY, INVALID, UNAVAILABLE, RINGING, RINGINUSE, ONHOLD

For example, to assign the INUSE value to a custom device called OnLeave:

exten => _X.,n,Set(DEVICE_STATE(Custom:OnLeave)=INUSE)

Linking a device state to an Asterisk hint

Remote SIP devices, such as IP phones, use a subscribe-notify mechanism to monitor the status of a device on Asterisk. To do this they send a SUBSCRIBE request to Asterisk with an identification tag that must match an Asterisk hint. In other words, custom device states (and any other device state) can only be seen by using a hint in the dialplan. Hints are explained in more detail in my article here.

Adding the following to your dial plan would link the subscribe tag 6123 to the state of the custom device called OnLeave:

exten => 6123,hint,Custom:OnLeave

To be sure that the hint can be found, it is best to add it to the context that is normally used by the subscribing phone. This may be the “default” context or, if using Trixbox or FreePBX, it would usually be the “from-internal” context. To add a hint for your Trixbox or FreePBX extensions, you will have to edit the file /etc/asterisk/extensions_custom.conf and add a context called ext-local-custom and put the hint in that. It would look like this:

[ext-local-custom]
; Add hint to link to the Custom Device State
exten => 6123,hint,Custom:OnLeave

The “tag” value of 6123 that was used for the example above is, in part, an arbitrary number. It must be unique, at least so no two hints use the same number in the same context. It must match the value you set on the subscribing device (usually that means the number assigned to the BLF programmable key on your IP phone). However, for reasons that should become apparent as you read on, your choice for this number may be influenced by your wish to not only activate a lamp on your IP phone, but to also make the key next to that lamp do something relevant and useful.

Setting up the BLF programmable key on the IP phone

BLF stands for Busy Lamp Field and the terminology is inherited from older phone systems where the operator could view the status of every extension and trunk on a display board. The board would be populated with an array of lamps, each representing one extension or one trunk line on the telephone system. The manufacturers of modern IP phones use BLF to describe the option that is available on programmable keys to make them subscribe (via SIP) to monitor the status of other extensions or other devices. This is the how you can link a programmable key and its associated lamp to a device state in Asterisk.

Setting up the IP phone should be very straightforward provided your phone has a spare programmable BLF key. Not all phones have directly programmable keys – for example the Linksys SPA942 has four line keys, but they cannot be re-assigned using the web interface. One of my favourite budget IP phones for use as an Asterisk extension is the Aastra 6731i – it has a total of 8 programmable keys although two of these are pre-assigned and harder to change, but that still leaves 6. Snom refer to them as Function Keys, but they are the same thing.

Each make of phone is different, but generally it should be straightforward to use the web interface of your phone, find the page where programmable keys are set and choose the key you want to set. Set the type as BLF and set the number (or value) to match the tag number used in the hint – 6123 in the examples above. There is often another field for each key where you can select the account or line. In almost all cases you should set this to be the account that is used to register the phone with Asterisk. Don’t forget to save the new settings and reboot the phone if needed (it is not usually necessary).

That should be sufficient. The phone will send a SUBSCRIBE request to Asterisk. Asterisk will match it to the hint which in turn links it to the custom state. You can check what subscriptions are active at the Asterisk command line interface using the command “sip show subscriptions”. It is also possible to see a list of all the hints – i.e. what you can subscribe to – using the CLI command “core show hints”. The last column in the output from this command is a column showing how many “Watchers” are subscribed to this hint. It is perfectly acceptable for more than one IP phone to subscribe to the same hint at the same time.

If you press the button on a BLF programmed key, it will place a call to the number that you assigned to the key. Since there is considerable flexibility in your choice for this number and further flexibility in the Asterisk dial plans to respond to a dialled number, you should be able to configure the system so the action of the key is sensibly linked to the activation of the lamp. One example is for the lamp to show that there are messages waiting in a shared voicemail box and the key is used to access the messages in the box. Details of how to do this are now available in a 2-part article on this site – click here for part 1.