SIP User Credentials in the ITSP environment – part 3

In this, the third part of the article, we will look at how the ITSP Proxy server handles outbound calls, paying particular attention to the authentication requirements and to the handling of Caller ID.

You may also wish to refer back to part 1 where we reviewed the basic entities and concepts used to identify a user, a handset or service and how these entities are used to register devices. The same entities are also used to authenticate and identify users making outbound calls. Part 2 examined how OpenSIPS, or a similar SIP Proxy server, can be used as an ITSP front end server handling inbound calls.

Outbound call

An “outbound” call is one that the end-user makes from their phone to a remote destination. Within the context of our idealised ITSP Proxy server model, an outbound call would be shown on the diagram like this:

Outbound call handling in ITSP Proxy serverFigure 10

The diagram shows an outbound call from a Centrex-style handset (i.e. one registered as part of a hosted PBX service). It does not take a great leap of imagination to see also how an outbound call could be made from a handset via one of the IP-PBX’s and SIP trunks shown faintly on the right of the diagram.

Key issues relevant to handling an outbound call

The key topics for an ITSP’s Proxy server handling outbound calls are:

  • Authentication
  • Authorisation
  • Routing
  • Billing and reports
  • Caller ID

Issues of routing and billing are largely outside the scope of this article, so we will concentrate mainly on authentication and Caller ID. Please note that authorisation (not the same as authentication) is closely linked to routing and applies to questions such as “is this user allowed to call this number or use this route?”

However, before we get onto the primary topics of authentication and Caller ID, I want to briefly touch on the subject of numbering plans. If you get the design of the numbering plan right, it will make your life much easier with respect to every aspect of outbound call handling, especially route selection.

Numbering plans and SIP User ID

It is probably simplest to explain the concept of the “numbering plan” by way of an example. I am sure you will soon understand what I mean and be able to see why its design is so important and also how it is closely linked to the SIP User ID.

In figure 10 there are two IP handsets identified as “Hosted service account A” and “Hosted service account B”. For the sake of this discussion, let us assume that these two devices represent different extensions within the same organisation. This means that the users of these phones might expect to be able to call each other with, say, a 3-digit extension number. Let us suppose also that the Internet Telephony Service Provider plans to be able to support up to 9999 different organisations on his multi-tenant service. He may therefore decide to use 7-digit SIP User ID’s made up of a 4-digit customer number and a 3-digit extension number. With such a regime, it is relatively simple to convert a 3-digit internal number dialled by a user into a 7-digit SIP User ID based on the fact that the destination extension must be within the same organisation and so both will share the same 4-digit prefix.

Internal call on a multi-tenant hosted PBX serviceFigure 11

Numbering plans and Dial plans

The most important thing with any numbering plan is that, when a user dials a number, the intended destination can never be ambiguous. It would not be acceptable if, say, a 7-digit number dialled by a user could either be interpreted as a destination on the PSTN or as an internal SIP User ID. You must pay careful attention to the formats of the numbers that your users might expect to be able to use in countries where your services operate.

A good dial plan will allow your users to call directory enquiries, the operator, emergency services, information services (weather, talking clock, etc), cell phones, free phones, international numbers and local numbers as well as internal extensions and long distance numbers. If possible, it is always better to avoid the somewhat lazy and old fashioned PBX solution of “always press 9 for an outside line”.

Here is an example that illustrates the importance of avoiding ambiguity. It is based on the earlier example of 7-digit SIP User ID’s containing 3-digit extension numbers. If we allowed an extension to be allocated the 3-digit number 911 (or in the UK 999) there will be ambiguity: If someone dials 911 (or 999), was the intended destination the internal extension or the emergency services?

Let us now move on and examine one of the core topics for this article.

Authentication of outbound calls

Almost all SIP Proxy servers will need to carry out some form of authentication for outbound calls. Authentication is used to match a caller with a user account on your system – so you know who to send the bill to. Without it, users and hackers would be able to make expensive calls through the system without any payment. The service provider would almost certainly end up having to pay for the calls.

A SIP Proxy server will generally use one of the following two methods for authentication of a caller:

  1. User credentials
  2. Source IP address

Using OpenSIPS, a simple way to test if an INVITE request from a caller meets one of the above two requirements would be to use a piece of code like this:

if (check_address("3","$si","$sp","$proto")) {
    #
    # OUTBOUND CALL: From a SIP Trunk identified by source IP address
    #
    xlog("L_INFO", " Outbound call: R-URI=$ru From=$fu (SIP Trunk static IP $si)\n");
    $avp(authid) = "$si";
} else if (is_from_local()) {
    #
    # OUTBOUND CALL: Check the credentials (and the FROM)
    #
    if (!proxy_authorize("","subscriber")) {
	proxy_challenge("","0");
	exit;
    #
    # Optional to enforce From-ID = Auth-ID.
    #
    } else if(!db_check_from()) {
	# From hdr does not match. Reject it
    	sl_send_reply("403", "Forbidden, use From=ID");
	xlog("L_INFO", " DROPPING Request to $ru  From=$fu  Failed From check\n");
	exit;
    }
    $avp(authid) = $au;
    consume_credentials();
} else {
    	sl_send_reply("404", "Not Found");
	xlog("L_INFO", " DROPPING Request to $ru  Unknown Domain in From=$fu\n");
	exit;
};

The first check uses the function check_address which is part of the Permissions module in OpenSIPS. In this example, we have defined Group 3 to be the group containing a list of SIP Trunk IP source addresses. Any INVITE request coming from one of these source addresses will be allowed without the need for credentials.

INVITE’s from any other address are allowed through only if the From domain matches one defined in the domain table and if the calling device authenticates correctly when challenged (a process that is extremely similar to the authentication process for REGISTER requests described in part 1 of this article, also illustrated in figure 12 below).

Checking the From header on outbound calls

Note that, as part of the authentication process, the code shown above includes an additional test using the function db_check_from(). This function ensures that the username in the From header matches the Auth-ID used in the authentication digest credentials. Although the comments indicate that this supplementary check is optional, I recommend that you leave it in unless you (a) understand the implications of omitting it and (b) have a very good reason to leave it out.

Let’s consider the purpose of this test in more detail: First, what would happen if it was omitted? The answer is, it would allow a call to be made from any device provided the Auth-ID and password are correct. If, by chance, you were using data contained in the From header for billing, then there would be an extreme risk of the costs for a call being attributed to the wrong account. The From header also plays an important part in establishing the Caller ID, so it would be relatively easy for a mischievously minded person to make calls with an arbitrary Caller-ID.

If there was a pressing need to allow the SIP User ID and the Auth-ID to not be identical, then I would recommend inserting some additional code that checks for a known relationship between the Auth-ID and the username given in the From header. I have successfully used the alias_db_find() function for this purpose – it is part of the alias_db module. The code is not shown here for reasons of brevity, but is very similar to the code shown at the end of this article.

Caller ID on outbound calls

The Caller ID on outbound calls is initially based on the contents of the From header. The From header may comprise a “display name” (which most SIP devices load from a user-defined configuration parameter) and a URI based directly on the SIP User ID. Some IP phones, like the Aastra, allow you to define the Caller ID as a different value to the SIP User ID.

Here is an edited sample from an INVITE request. The IP phone is making an outbound call to 01727221221 and the SIP Proxy has the fictitious address of 123.45.67.89:

INVITE sip:01727221221@123.45.67.89:5060 SIP/2.0
Via: SIP/2.0/UDP 172.16.5.12:6175;branch=z9hG4bK-d8754z;rport
To: <sip:01727221221@123.45.67.89:5060>
From: "Toby Franklin" <sip:7301102@172.16.5.12:6175>;tag=bd52a002
Contact: <sip:7301102@172.16.5.12:6175;rinstance=1fedfda326>

In this example, the SIP User ID is 7301102 and the Display Name was set to Toby Franklin.

First, it is important to be aware of all the SIP headers that can have a role in establishing the Caller ID. In addition to the From header, extensions to the SIP protocol also define two additional headers that may be used to identify the visible Caller ID and the “network id” of the caller. These headers are Remote-Party-ID and P-Asserted-Identity. For brevity, I will refer to these as RPID and PAI headers. (By the way, if the caller wants their number to be withheld, this is normally indicated by the addition of a privacy tag on RPID or a separate Privacy header).

Generally, it is safe to assume that the average IP phone or softphone will at least include a From header in the INVITE request. The other headers may or may not be present and it is up to the SIP Proxy to determine the need for addition or removal of RPID and PAI headers.

When a call is being sent out to a destination on the PSTN, we will want the Caller ID to be a valid call-back number, not the SIP User ID. However, when the call is “internal” (extension-to-extension) we will want the Caller ID to be the caller’s extension number. So how can we fix the Caller ID in these two situations?

Implementation within OpenSIPS

If we now consider an OpenSIPS script, the section that handles outbound calls will first need to include primary logic to determine if the call is internal (going to another extension) or is going out onto the PSTN via a carrier. This will not only determine the content of the headers but also which headers are needed.

You can add a header using the append_hf function or remove a header using remove_hf. The function is_present_hf can be used to check if a header exists. These functions are all provided in the SIPMSGOPS module (pre-1.8, they were in TEXTOPS).

Caller-ID on Internal calls

For an internal call, if you used the 7-digit SIP User ID’s I described earlier, it may be sufficient to strip off the first 4 digits from the SIP User ID and rewrite the username element in the From header. Depending on the make/model of the destination device, it is possible that values given in an RPID or PAI header will take precedence over the data in the From header.

The design of your scheme for SIP User ID’s and numbering plans is critical to the ease of implementing simple Caller ID strategies on internal calls. Using a well-designed scheme, it should be possible to apply a hard-coded algorithm that converts the SIP User ID to their internal extension number. If your scheme does not allow this then some kind of table lookup will be required, perhaps using the ALIAS_DB module.

Caller-ID on calls to the PSTN

In my experience, the requirements for caller identification vary from one carrier to another when routing to the PSTN. For example, a carrier may take the username element of the From header, or the value given in the Remote-Party-ID header, as the value to display at the destination handset. Often the carrier will expect to receive the numeric caller id in a particular format such as North American Number Plan, E.164, International calls with (or without) a + prefix and, in the UK, with or without the leading zero. The support department for your carrier should be able to advise you about this, but it is best to check by making several test calls over different routes to a range of destination devices to make sure the caller number is displayed correctly.

And how can OpenSIPS get the correct Caller ID for a user making a call to the PSTN? The simplest solution is to use the rpid field of the subscriber table. This field allows a number to be associated with every user account. One caveat to mention here is that the primary key of the subscriber table contains the Auth-ID not the SIP User ID, so if you have some accounts where these differ, then the rpid field may not be the solution after all. The primary purpose of the rpid field is to populate an RPID header on outbound calls, but there is no reason why it could not also be used to overwrite the username in the From header. For details, see documentation for the AUTH module and especially the module parameter rpid_avp and the function append_rpid_hf().

Modifying the From header in OpenSIPS

Modification of the From header within OpenSIPS needs to be approached with some caution. Consider an example where SIP requests are sent from device A to a Proxy server and then on to a carrier. If the Proxy server modifies the From header on the request, it will also need to reverse the modification on all the responses coming back. Furthermore, whatever logic you insert in your OpenSIPS control script to modify From headers, must be repeated in all sections of the script that might subsequently handle additional requests in the same dialogue – for example, in the section that handles loose-routed requests.

OpenSIPS provides a handy function – uac_replace_from() – in the UAC module for changing the From header, but even this comes with caveats. For example, if there is any possibility of this function being called twice for the same branch, then it will completely break the SIP request formatting!

Outbound call handling in an ITSP SIP Proxy server – Recap

In this part of the article, we have examined several topics associated with outbound call handling in a SIP Proxy server and it may now be helpful review how these all come together with the help of a diagram:

Explanation for outbound call handling in an ITSP Proxy serverFigure 12

Figure 12 shows how an outbound call is sent to the SIP Proxy server as an INVITE request from the IP Phone. Unless authentication is being done by source IP address, the first INVITE request is normally rejected with “407 Proxy Authentication required” (a proxy challenge) and then a second request is sent that is similar to the first but with additional authorisation details (Auth ID and an encrypted password).

The Proxy may modify the From header or add new headers, such as Remote-Party-ID or P-Asserted-Identity, which are used by downstream servers to identify the Caller ID. A mechanism available in the AUTH module of OpenSIPS, takes values from the rpid field in the subscriber table and uses them to populate a new RPID header.

The SIP Proxy server will inspect the destination number in the R-URI in the INVITE request that it receives from the user device. It will use that number to determine the most suitable route for the call and it may also make changes to, or even replace, the number before relaying it onwards to the chosen gateway or carrier.

What if the SIP User ID, Auth ID and Caller ID are not identical?

So we finally come back to this annoying question, first posed in part 1 of this article. Of course, if you are able to keep the SIP User ID and Auth-ID identical then it is best to do so. As I will explain below, it should not be necessary to use mismatched User ID and Auth-ID if the remote device is an IP-PBX on a SIP trunk. Thus, it is only likely to be a problem when supporting a relatively simple device like an IP phone or an ATA.

Caller ID for IP-PBX’s and SIP Trunks

Most IP-PBX’s offer a wide range of “smart” options for setting the Caller ID and these are generally quite independent of the Auth-ID used to register the SIP trunk (if registration is even required).

As an example, let’s consider the options available in v2.9.0 of FreePBX: In addition to its own extension number, each extension can be given an explicit “CID Num Alias” for internal calls and an “Outbound CID” for external calls. Each outbound route can have its own CID, which may or may not override the one set for the extension, and can be flagged as an “Emergency Route” if required. Each trunk has options for the outbound CID configured using the fields “Outbound CallerID” and “CID Options”. Thus the trunk settings may override the extension and route settings in most cases other than emergencies where a parameter for an “Emergency CID” can be set in the extension configuration form. I think this makes the point – the options are extensive.

Simple SIP devices (not using SIP trunking)

On a simple device, it may be convenient to set the SIP User ID to a value based on (or even the same as) the DID. This makes for a simple solution to setting the Caller ID on outbound calls. However, for reasons already discussed in part 1, you might want the Auth-ID to be completely different.

When Auth-ID is not the same as SIP User ID, it will cause registrations to fail when the OpenSIPS script checks that the To header matches the SIP User ID. It can also cause authentication on outbound calls to fail if the script checks that the From header matches the Auth-ID.

One approach that can be used to overcome this problem is to pre-populate the dbaliases table with values mapping between the SIP User ID and the Auth-ID or vice-versa.  There are several possible approaches we could choose, but by way of illustration, I will show just one. In the example code below, records in the dbaliases table map Auth-ID to SIP User ID. The code tests for an exact match between the To header username and the aliased SIP User ID, but it is also possible to use a regular expression test thereby allowing one Auth-ID to match several similar SIP User ID’s.

The following snippet of code shows how a REGISTER request could be checked. It is similar to the code shown in part 1 of this article, except that some extra lines are inserted for the case where db_check_to() returns false. The extra code queries the dbaliases table, searching for a record with the value of Auth-ID ($au) in the alias_username field. The value returned in the variable called ‘sipid’ is then used to test the username in the To header. If it matches, then the registration is allowed through, otherwise the registration is rejected.

if (is_uri_host_local()) {
    if (!www_authorize("", "subscriber")) {
        www_challenge("", "1");
        exit;
    };

    if (!db_check_to()) {
        # To header doesn't match. Reject unless auth-id maps to To username
        $var(checkuri) = "sip:" + $au + "@mysipdomain.com";
        if (alias_db_find("dbaliases", "$var(checkuri)", "$var(sipid)", "d")) {
            if ($tU==$(var(sipid){uri.user})) {
                xlog("L_INFO", " Reg for auth-id $au aliased to $var(sipid) in To hdr\n");
            } else {
                sl_send_reply("403", "Forbidden 1");
                exit;
            }
        } else {
            sl_send_reply("403", "Forbidden 2");
            exit;
        }
    };

    if (!save("location")) {
        xlog("L_WARN", "**REGISTER: save failed for $ct.fields(uri)\n");
        sl_reply_error();
    } else {
        xlog("L_WARN", "  REGISTER rcvd $si:$sp  Contact=$ct.fields(uri)\n");
    };
    exit;

} else {
     sl_send_reply("403", "Forbidden 3");
};

Using this solution, it is possible that the dbaliases table could now contain two quite different sets of records. Some records describe the relationship between DID and SIP User ID (used for inbound calls and described in part 2) while others describe the relationship between Auth-ID and SIP User ID.

In conclusion

This three part article roamed around many issues, but the theme common to all of it was an examination of the relationships between SIP User ID, Auth-ID, DID and Caller-ID within the context of how we might use OpenSIPS as the Proxy server for a multi-tenant ITSP service. I hope that the complete article will work as an  OpenSIPS tutorial, giving some insight into the choices and decisions to be made when designing a multi-tenant ITSP service.

Even though matters of billing and routing were deliberately skimmed over, you cannot fail to have noticed that there are many inter-related issues that conspire to make the subject more complex than it first appears. Regrettably, there are no simple off-the-shelf solutions that can be expected to work in the majority of cases. Each ITSP service will have its own unique requirements and it is quite possible, within one ITSP service, that different carriers or routes used to reach the PSTN will require different headers to be present or the values in those headers to use different formats.

I would generally recommend that the Proxy server handling SIP trunks be separated from the one handling hosted PBX services, but this may not always be possible and would add cost too. If you are setting up a new service, allow plenty of thinking and design time for things like the numbering plan, the dial plan, the format of SIP User ID’s and the relationship between User ID and Auth-ID. Make sure your billing solution will always receive suitable values in all the fields of the Call Data Record. For example, if the billing solution identifies the account using an ID sent in the authentication credentials, what value will it receive when the caller has a SIP Trunk and authentication is by source IP? Try to anticipate how the solution will cope in the future as you support more devices and more users – also as you add features and functions – because changes are many times more expensive on a running system than they are on the drawing board or during construction of a new system.

As always, your comment and feedback welcome.

  What did you think of this article? Please vote by clicking a coloured button
(99%) (1%) (0%) (0%)

10 thoughts on “SIP User Credentials in the ITSP environment – part 3

  1. An excellent series of articles on OpenSIPS, I’m just gathering ideas to implement it myself and have just one question.

    How would you approach checking caller IDs from SIP trunks, by that I mean rather than a user being allowed to set any caller ID they want, having a list of caller IDs that a user is allowed to send and then validating or replacing the one sent.

    • Hi Darren

      My assumption was that the equipment connected to the SIP trunk would be responsible for any complex rules and would set the required Caller ID. That means OpenSIPS should only need to do quite simple validity checks. You could use the ALIAS_DB module if all that is required is to look up exact matches. If you want to add in a bit more intelligence, perhaps using pattern matching, then you could devolve the whole process to an external application and call it using exec_avp or it might be possible to write it within opensips.cfg doing ad-hoc SQL queries to tables containing the validity data using avp_db_query. OpenSIPS has built-in string handling capabilities – see documentation under the heading “transformations”.

      John

  2. Exactly, my client is asking to remove on that part, the “Remote-Party-ID” header which i assume is not in the “From” right?

    How can i add the prefix to the Remote-Party-ID header?

    Thanks so much!

    • The From header and Remote-Party-ID header are completely separate entities. There is almost always a From header, but RPID is more optional.

      I would use is_present_hf to test if the RPID header is already present. If it is, use remove_hf. Now you know it is not present, you can use append_hf to add it back with the uri formatted how you want it.

      • Wow,

        This is an accelerate opensips MBA!! (Thanks, really)

        the syntax seems like this right now:
        if (to_uri =~ “sip:901xxxxxx@”)
        {
        if !($fU=~”34.*”)
        {
        uac_replace_from(“”,”sip:34$fU@example.us”);
        remove_hf;
        append_hf(“”,”sip:34$fU@example.us”);
        }
        prefix(“+34”);
        rewritehost(“8.1.2.3”);
        route(66);
        }

        Which to me seems to be ok, but sure there’s something wrong…

        best

  3. Hi,

    Firstly thanks so much for this very interesting entry…

    I have a problem that i almost solved with this entry:

    I have a domestic number which is only available to be called from Spain which could be: 901xxxxxx
    And this number is sent to Germany via VoIP, my syntax is as follows:
    if (to_uri =~ “sip:9014xxxxxx@”) {
    if ( $fU != “34.*” )
    {
    uac_replace_from(“”,”sip:34$fU@example.us”);
    }
    prefix(“+34”);
    rewritehost(“8.1.2.3”);
    route(66);
    }

    But anyway the client is asking for replacing the party caller id to have also the “+34”, as i have to force anything, i do not know how can i also modify that field.

    thanks so much!!

    • $fU != “34.*” will not work because .* is a regular expression but != is not.
      Instead try !($fU=~”34.*”) or better to test for digits after the 34 !($fU=~”34[0-9]{6}”)

      prefix (“+34”) will insert +34 in front of the r-uri (destination number), not the caller ID. So you would probably end up with an R-URI like this: sip:+349014xxxxxx@8.1.2.3

      John

      • Thanks for the tip!
        does the “…{6}” “…and the following 6 digits”??
        On the other hand, it seemed to work, but maybe as you said, not as i intended.

        I managed to make the “from” and the “r_uri” to have the +34 but now what I’m really trying to find is the syntax to also add it to the caller_ID, how can i do that?

        Thanks so much, this is really helping!

        • Yes, [0-9]{6} means 6 digits. (BTW, you can use !~ as a test for not matching). If there is any doubt about which part of your code is executing, you should add xlog statements which write to the system log file and will help you to debug your code.

          To adjust “Caller ID” you must know which header is being used. Often, the Caller ID uses the From header. However, From headers have 2 parts – ‘display’ and ‘uri’. You are removing ‘display’ so maybe you need to use one of the following instead:
          uac_replace_from(“34$fU”,”sip:34$fU@example.us”);
          uac_replace_from(“sip:34$fU@example.us”);

          Another possibility would be to add a Remote-Party-ID header. It just depends which header is being used by the downstream servers.

Leave a Comment